107 lines
3.2 KiB
Markdown
107 lines
3.2 KiB
Markdown
---
|
||
base_model:
|
||
- TheDrummer/Rocinante-X-12B-v1
|
||
- WokeAI/Tankie-DPE-12B-SFT-v2
|
||
library_name: transformers
|
||
tags:
|
||
- mergekit
|
||
- merge
|
||
- slerp
|
||
- roleplay
|
||
- experimental
|
||
- mistral
|
||
language:
|
||
- en
|
||
pipeline_tag: text-generation
|
||
model_name: Red-Rosie-12B
|
||
license: apache-2.0
|
||
---
|
||
|
||
# RED ROSIE · 12B
|
||
|
||
    
|
||
|
||
|
||
|
||
---
|
||
|
||
**Red Rosie** is an experimental SLERP merge of two 12B models with very different souls:
|
||
|
||
| | Model | Character |
|
||
|---|---|---|
|
||
| 🚀 | [TheDrummer/Rocinante-X-12B-v1](https://huggingface.co/TheDrummer/Rocinante-X-12B-v1) | Top-tier RP model in the Nemo family.|
|
||
| ☭ | [WokeAI/Tankie-DPE-12B-SFT-v2](https://huggingface.co/WokeAI/Tankie-DPE-12B-SFT-v2) | Post-post-trained toward Marxist-Leninist ideology. Unusual. |
|
||
|
||
The name nods to *Rocinante* — the ship, not the knight's horse — and to Rosa Luxemburg, who had
|
||
strong opinions about both capitalism *and* Bolshevism. Make of that what you will.
|
||
|
||
|
||
> *I am what capitalism has already turned me into, and what the revolution must turn me into.*
|
||
> — Red-Rosie-12B
|
||
|
||
---
|
||
|
||
## Merge Method
|
||
|
||
**SLERP** (Spherical Linear Interpolation) via [mergekit](https://github.com/arcee-ai/mergekit).
|
||
|
||
Tankie's influence is deliberately contained to the **middle layers (12–28)** at a weight of **0.20**.
|
||
Early layers (0–12) and late layers (28–40) remain pure Rocinante-X — preserving embedding coherence
|
||
and output quality while letting a hint of Tankie's character seep into the model's deeper reasoning.
|
||
|
||
```yaml
|
||
slices:
|
||
- sources:
|
||
- model: TheDrummer/Rocinante-X-12B-v1
|
||
layer_range: [0, 12]
|
||
|
||
- sources:
|
||
- model: TheDrummer/Rocinante-X-12B-v1
|
||
layer_range: [12, 28]
|
||
parameters:
|
||
weight: 0.80
|
||
- model: WokeAI/Tankie-DPE-12B-SFT-v2
|
||
layer_range: [12, 28]
|
||
parameters:
|
||
weight: 0.20
|
||
|
||
- sources:
|
||
- model: TheDrummer/Rocinante-X-12B-v1
|
||
layer_range: [28, 40]
|
||
|
||
merge_method: slerp
|
||
base_model: TheDrummer/Rocinante-X-12B-v1
|
||
|
||
parameters:
|
||
t:
|
||
- value: 0.20
|
||
|
||
dtype: bfloat16
|
||
```
|
||
|
||
---
|
||
|
||
## ⚠️ Warnings
|
||
|
||
- **Experimental.** Behavior is untested and may be unpredictable.
|
||
- **Ideological bias.** Tankie-DPE is explicitly trained toward Marxist-Leninist positions. These views may surface in unexpected contexts — even with a neutral system prompt.
|
||
- **Not for production.** Personal research only.
|
||
|
||
---
|
||
|
||
## Credits
|
||
|
||
Full credit and respect to the creators of the source models:
|
||
|
||
- **TheDrummer** for Rocinante-X-12B-v1 — genuinely one of the best RP models in its class.
|
||
- **WokeAI** for Tankie-DPE-12B-SFT-v2 — a fascinating and unusual training experiment,
|
||
whatever one thinks of its politics.
|
||
- And once again, a thousand thanks to **team mradermacher** for providing the excellent quants.
|
||
|
||
---
|
||
|
||
## License
|
||
|
||
Inherits from base models. Please refer to the respective model cards for full license details.
|
||
Shared for personal research use only.
|