1.5 KiB
1.5 KiB
license, base_model, tags, language, pipeline_tag
| license | base_model | tags | language | pipeline_tag | |||||
|---|---|---|---|---|---|---|---|---|---|
| apache-2.0 | mistralai/Mistral-Nemo-Base-2407 |
|
|
text-generation |
In-Character RP 12B — v0.1
A roleplay / creative-writing finetune of Mistral-Nemo-Base-2407 (12B) focused on staying in character: villains stay villainous, guarded characters don't open up on command, and distinct personalities hold up across a conversation.
v0.1 is the SFT base. The core differentiator — an anti-positivity-drift preference pass (diversity-preserving DPO) plus an anti-slop pass (FTPO) — lands in v0.2. This release establishes the base behavior and prose.
Prompt format — ChatML
<|im_start|>system
{character card / persona}<|im_end|>
<|im_start|>user
{user message}<|im_end|>
<|im_start|>assistant
Recommended sampler
temperature 1.0 · min_p 0.05 · DRY (multiplier 0.8, base 1.75, allowed_length 2) · temperature_last. Drop top_p; leave XTC off.
Training
- Base: Mistral-Nemo-Base-2407 (12B, Apache-2.0) — a warm, lightly-aligned base well-suited to RP.
- Method: QLoRA (4-bit) + high-rank rsLoRA (r=256), embed/lm_head trained; ChatML imposed on the base tokenizer.
- Data: a cleaned RP / creative mix — LimaRP, PIPPA, CoSER, Gryphe Sonnet-3.5 character cards, WritingPrompts, Opus-Instruct, Synthetic-RP — filtered for refusals, positivity-drift, and slop.
- v0.1 run: ~16M tokens, seq 4096, single 48 GB GPU.
Status
Research preview (v0.1). Uncensored creative model — use responsibly.