3.5 KiB
base_model, library_name, tags, license
| base_model | library_name | tags | license | ||||
|---|---|---|---|---|---|---|---|
|
transformers |
|
cc-by-nc-4.0 |
Continued from the 4.3 lineage with Gemma staying securely in her role as judge rather than rewriting directly.
600+ steps RLVR GRPO in spicy roleplaying first-turns, with Gemma 4 31b as judge paying attention to writing craft; followed by DPO including self-rewrites of about half of the particularly bad trajectories from RLVR (until Gemma rated them well in comparison, and particular issues like a godmoded POV were fixed). Ran 5 seeds at low batch size to explore and merged via Karcher mean.
A mix of approaches including utilizing and not-utilizing the <feelings> tags were tried in RLVR. She's trying some interesting things with the tokens, including compound-word emotions that split the difference between word and phrase, but the ratings aren't notably better with the tags than without them - so I wouldn't consider her a reasoning model yet, and it's not in the chat template.
More Arsenic self-portraits:
Amethyst:
Aegis:
)
arsenic-v4.4-merged
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the Karcher Mean merge method.
Models Merged
The following models were included in the merge:
- /home/ann/Documents/Projects/training_models/outputs/baked_v44_craft_seed1
- /home/ann/Documents/Projects/training_models/outputs/baked_v44_craft_seed2
- /home/ann/Documents/Projects/training_models/outputs/baked_v44_craft_seed5
- /home/ann/Documents/Projects/training_models/outputs/baked_v44_craft_seed4
- /home/ann/Documents/Projects/training_models/outputs/baked_v44_craft_seed3
Configuration
The following YAML configuration was used to produce this model:
models:
- model: /home/ann/Documents/Projects/training_models/outputs/baked_v44_craft_seed1
- model: /home/ann/Documents/Projects/training_models/outputs/baked_v44_craft_seed2
- model: /home/ann/Documents/Projects/training_models/outputs/baked_v44_craft_seed3
- model: /home/ann/Documents/Projects/training_models/outputs/baked_v44_craft_seed4
- model: /home/ann/Documents/Projects/training_models/outputs/baked_v44_craft_seed5
merge_method: karcher
parameters:
normalize: true
int8_mask: true
tokenizer_source: /home/ann/Documents/Projects/training_models/outputs/baked_v44_craft_seed1
dtype: bfloat16