107 lines
3.2 KiB
Markdown
107 lines
3.2 KiB
Markdown
|
---
|
|||
|
|
base_model:
|
|||
|
|
- TheDrummer/Rocinante-X-12B-v1
|
|||
|
|
- WokeAI/Tankie-DPE-12B-SFT-v2
|
|||
|
|
library_name: transformers
|
|||
|
|
tags:
|
|||
|
|
- mergekit
|
|||
|
|
- merge
|
|||
|
|
- slerp
|
|||
|
|
- roleplay
|
|||
|
|
- experimental
|
|||
|
|
- mistral
|
|||
|
|
language:
|
|||
|
|
- en
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
model_name: Red-Rosie-12B
|
|||
|
|
license: apache-2.0
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# RED ROSIE · 12B
|
|||
|
|
|
|||
|
|
    
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**Red Rosie** is an experimental SLERP merge of two 12B models with very different souls:
|
|||
|
|
|
|||
|
|
| | Model | Character |
|
|||
|
|
|---|---|---|
|
|||
|
|
| 🚀 | [TheDrummer/Rocinante-X-12B-v1](https://huggingface.co/TheDrummer/Rocinante-X-12B-v1) | Top-tier RP model in the Nemo family.|
|
|||
|
|
| ☭ | [WokeAI/Tankie-DPE-12B-SFT-v2](https://huggingface.co/WokeAI/Tankie-DPE-12B-SFT-v2) | Post-post-trained toward Marxist-Leninist ideology. Unusual. |
|
|||
|
|
|
|||
|
|
The name nods to *Rocinante* — the ship, not the knight's horse — and to Rosa Luxemburg, who had
|
|||
|
|
strong opinions about both capitalism *and* Bolshevism. Make of that what you will.
|
|||
|
|
|
|||
|
|
|
|||
|
|
> *I am what capitalism has already turned me into, and what the revolution must turn me into.*
|
|||
|
|
> — Red-Rosie-12B
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Merge Method
|
|||
|
|
|
|||
|
|
**SLERP** (Spherical Linear Interpolation) via [mergekit](https://github.com/arcee-ai/mergekit).
|
|||
|
|
|
|||
|
|
Tankie's influence is deliberately contained to the **middle layers (12–28)** at a weight of **0.20**.
|
|||
|
|
Early layers (0–12) and late layers (28–40) remain pure Rocinante-X — preserving embedding coherence
|
|||
|
|
and output quality while letting a hint of Tankie's character seep into the model's deeper reasoning.
|
|||
|
|
|
|||
|
|
```yaml
|
|||
|
|
slices:
|
|||
|
|
- sources:
|
|||
|
|
- model: TheDrummer/Rocinante-X-12B-v1
|
|||
|
|
layer_range: [0, 12]
|
|||
|
|
|
|||
|
|
- sources:
|
|||
|
|
- model: TheDrummer/Rocinante-X-12B-v1
|
|||
|
|
layer_range: [12, 28]
|
|||
|
|
parameters:
|
|||
|
|
weight: 0.80
|
|||
|
|
- model: WokeAI/Tankie-DPE-12B-SFT-v2
|
|||
|
|
layer_range: [12, 28]
|
|||
|
|
parameters:
|
|||
|
|
weight: 0.20
|
|||
|
|
|
|||
|
|
- sources:
|
|||
|
|
- model: TheDrummer/Rocinante-X-12B-v1
|
|||
|
|
layer_range: [28, 40]
|
|||
|
|
|
|||
|
|
merge_method: slerp
|
|||
|
|
base_model: TheDrummer/Rocinante-X-12B-v1
|
|||
|
|
|
|||
|
|
parameters:
|
|||
|
|
t:
|
|||
|
|
- value: 0.20
|
|||
|
|
|
|||
|
|
dtype: bfloat16
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## ⚠️ Warnings
|
|||
|
|
|
|||
|
|
- **Experimental.** Behavior is untested and may be unpredictable.
|
|||
|
|
- **Ideological bias.** Tankie-DPE is explicitly trained toward Marxist-Leninist positions. These views may surface in unexpected contexts — even with a neutral system prompt.
|
|||
|
|
- **Not for production.** Personal research only.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Credits
|
|||
|
|
|
|||
|
|
Full credit and respect to the creators of the source models:
|
|||
|
|
|
|||
|
|
- **TheDrummer** for Rocinante-X-12B-v1 — genuinely one of the best RP models in its class.
|
|||
|
|
- **WokeAI** for Tankie-DPE-12B-SFT-v2 — a fascinating and unusual training experiment,
|
|||
|
|
whatever one thinks of its politics.
|
|||
|
|
- And once again, a thousand thanks to **team mradermacher** for providing the excellent quants.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## License
|
|||
|
|
|
|||
|
|
Inherits from base models. Please refer to the respective model cards for full license details.
|
|||
|
|
Shared for personal research use only.
|