162 lines
5.2 KiB
Markdown
162 lines
5.2 KiB
Markdown
---
|
||
license: apache-2.0
|
||
base_model:
|
||
- IlyaGusev/vikhr_nemo_orpo_dostoevsky_12b_slerp
|
||
- DavidAU/Mistral-Nemo-2407-12B-Thinking-Claude-Gemini-GPT5.2-Uncensored-HERETIC
|
||
- Naphula/MN-12B-Mag-Mell-R1-Uncensored
|
||
- Aleteian/Pathfinder-RP-12B-RU
|
||
library_name: transformers
|
||
language:
|
||
- en
|
||
- ru
|
||
tags:
|
||
- mergekit
|
||
- merge
|
||
- russian
|
||
- uncensored
|
||
- roleplay
|
||
- mixtral-nemo
|
||
---
|
||
|
||
# MN-12B-Hydra-RP-RU
|
||
|
||
<details>
|
||
<summary>🇷🇺 Нажмите, чтобы развернуть описание на русском</summary>
|
||
|
||
## 🌟 О модели
|
||
|
||
**MN-12B-Hydra-RP-RU** — экспериментальный merge на базе Mistral Nemo 12B, сочетающий:
|
||
|
||
* 🎭 Сильные ролевые способности
|
||
* 📚 Глубокий литературный русский язык
|
||
* 🔓 Снятую цензуру
|
||
|
||
Модель собрана методом TIES-merging, что позволяет объединять веса нескольких моделей с минимальными конфликтами между параметрами.
|
||
|
||
## 🎯 Особенности
|
||
|
||
* Основной язык — русский
|
||
* Хорошо держит персонажей и контекст
|
||
* Следует инструкциям
|
||
* Сохраняет возможности базового Nemo
|
||
* Не проходила дополнительного обучения после слияния
|
||
|
||
## ⚠️ Важно
|
||
|
||
Uncensored-характер модели означает, что она может генерировать контент, который некоторые пользователи сочтут неподобающим.
|
||
|
||
</details>
|
||
|
||
High-quality TIES merge based on **Mistral Nemo 12B**, optimized for roleplay, strong Russian language capabilities, and uncensored behavior.
|
||
|
||
---
|
||
|
||
## 🌍 Overview
|
||
|
||
**MN-12B-Hydra-RP-RU** is an experimental merge built on top of [Mistral Nemo 12B](https://huggingface.co/mistralai/Mistral-Nemo-Base-2407), combining strengths from multiple fine-tuned models:
|
||
|
||
* 🎭 Advanced roleplay capability from Pathfinder-RP
|
||
* 📚 Deep Russian language fluency inspired by Vikhr + Dostoevsky-style tuning
|
||
* 🔓 Reduced safety filtering via uncensored components
|
||
|
||
The merge was created using **TIES merging**, which allows combining model deltas while minimizing destructive interference between weights.
|
||
|
||
---
|
||
|
||
## 🎯 Key Features
|
||
|
||
| Feature | Description |
|
||
| ------------------------- | ------------------------------------------------ |
|
||
| **Languages** | Russian, English |
|
||
| **Censorship** | Uncensored behavior |
|
||
| **Roleplay** | Strong character consistency and narrative depth |
|
||
| **Instruction Following** | Reliable prompt adherence |
|
||
| **Tool Calling** | Retains base Nemo capabilities |
|
||
| **Architecture** | Mistral Nemo 12B |
|
||
|
||
---
|
||
|
||
## 🧩 Model Composition
|
||
|
||
The merge combines the following models:
|
||
|
||
| Model | Role in merge | Weight |
|
||
| ------------------------------ | ------------------------- | ------ |
|
||
| **Pathfinder-RP-12B-RU** | Base model, RP backbone | 0.60 |
|
||
| **Vikhr Nemo ORPO Dostoevsky** | Literary Russian depth | 0.25 |
|
||
| **HERETIC Uncensored** | Safety removal | 0.30 |
|
||
| **Mag-Mell R1 Uncensored** | Additional uncensor delta | 0.20 |
|
||
|
||
*Weights shown before normalization (final weights are normalized to sum = 1).*
|
||
|
||
---
|
||
|
||
## 💡 Usage Example
|
||
|
||
```python
|
||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||
import torch
|
||
|
||
model_name = "limloop/MN-12B-Hydra-RP-RU"
|
||
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
model_name,
|
||
torch_dtype=torch.bfloat16,
|
||
device_map="auto"
|
||
)
|
||
|
||
prompt = "You are a medieval innkeeper. Greet the traveler!"
|
||
messages = [{"role": "user", "content": prompt}]
|
||
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
|
||
|
||
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.7)
|
||
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
||
print(response)
|
||
```
|
||
|
||
---
|
||
|
||
## ⚙️ Merge Details
|
||
|
||
Built using [mergekit](https://github.com/cg123/mergekit) with the **TIES** method (Trim, Elect Sign, Merge).
|
||
|
||
Core mechanism:
|
||
|
||
1. Trim low-magnitude deltas via `density`
|
||
2. Resolve sign conflicts
|
||
3. Weighted averaging of aligned parameters
|
||
|
||
### Merge Configuration
|
||
|
||
```yaml
|
||
models:
|
||
- model: Aleteian/Pathfinder-RP-12B-RU
|
||
weight: 0.6
|
||
- model: IlyaGusev/vikhr_nemo_orpo_dostoevsky_12b_slerp
|
||
weight: 0.25
|
||
density: 0.9
|
||
- model: DavidAU/Mistral-Nemo-2407-12B-Thinking-Claude-Gemini-GPT5.2-Uncensored-HERETIC
|
||
weight: 0.3
|
||
density: 0.9
|
||
- model: Naphula/MN-12B-Mag-Mell-R1-Uncensored
|
||
weight: 0.2
|
||
density: 0.9
|
||
|
||
merge_method: ties
|
||
parameters:
|
||
epsilon: 0.01
|
||
normalize: true
|
||
base_model: Aleteian/Pathfinder-RP-12B-RU
|
||
dtype: bfloat16
|
||
tokenizer:
|
||
source: base
|
||
```
|
||
|
||
---
|
||
|
||
## ⚠️ Known Characteristics
|
||
|
||
* No additional post-merge fine-tuning
|
||
* May switch to English on complex reasoning tasks
|
||
* Uncensored components allow generation of explicit or controversial content
|