44 lines
1.5 KiB
Markdown
44 lines
1.5 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
base_model: mistralai/Mistral-Nemo-Base-2407
|
||
|
|
tags:
|
||
|
|
- roleplay
|
||
|
|
- creative-writing
|
||
|
|
- mistral
|
||
|
|
- nemo
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
---
|
||
|
|
|
||
|
|
# In-Character RP 12B — v0.1
|
||
|
|
|
||
|
|
A roleplay / creative-writing finetune of **Mistral-Nemo-Base-2407 (12B)** focused on **staying in character**: villains stay villainous, guarded characters don't open up on command, and distinct personalities hold up across a conversation.
|
||
|
|
|
||
|
|
> **v0.1 is the SFT base.** The core differentiator — an anti-positivity-drift preference pass (diversity-preserving DPO) plus an anti-slop pass (FTPO) — lands in **v0.2**. This release establishes the base behavior and prose.
|
||
|
|
|
||
|
|
## Prompt format — ChatML
|
||
|
|
|
||
|
|
```
|
||
|
|
<|im_start|>system
|
||
|
|
{character card / persona}<|im_end|>
|
||
|
|
<|im_start|>user
|
||
|
|
{user message}<|im_end|>
|
||
|
|
<|im_start|>assistant
|
||
|
|
```
|
||
|
|
|
||
|
|
## Recommended sampler
|
||
|
|
|
||
|
|
`temperature 1.0` · `min_p 0.05` · `DRY (multiplier 0.8, base 1.75, allowed_length 2)` · `temperature_last`. Drop top_p; leave XTC off.
|
||
|
|
|
||
|
|
## Training
|
||
|
|
|
||
|
|
- **Base:** Mistral-Nemo-Base-2407 (12B, Apache-2.0) — a warm, lightly-aligned base well-suited to RP.
|
||
|
|
- **Method:** QLoRA (4-bit) + high-rank rsLoRA (r=256), embed/lm_head trained; ChatML imposed on the base tokenizer.
|
||
|
|
- **Data:** a cleaned RP / creative mix — LimaRP, PIPPA, CoSER, Gryphe Sonnet-3.5 character cards, WritingPrompts, Opus-Instruct, Synthetic-RP — filtered for refusals, positivity-drift, and slop.
|
||
|
|
- **v0.1 run:** ~16M tokens, seq 4096, single 48 GB GPU.
|
||
|
|
|
||
|
|
## Status
|
||
|
|
|
||
|
|
Research preview (v0.1). Uncensored creative model — use responsibly.
|