初始化项目,由ModelHub XC社区提供模型
Model: Retreatcost/KansenSakura-Conflagration-RP-12b Source: Original Platform
This commit is contained in:
320
README.md
Normal file
320
README.md
Normal file
@@ -0,0 +1,320 @@
|
||||
---
|
||||
base_model:
|
||||
- ohyeah1/Violet-Lyra-Gutenberg-v2
|
||||
- DavidAU/MN-Dark-Planet-TITAN-12B
|
||||
- intervitens/mini-magnum-12b-v1.1
|
||||
- yamatazen/FusionEngine-12B-Lorablated
|
||||
- Marcjoni/KiloNovaSynth-12B
|
||||
- PocketDoc/Dans-SakuraKaze-V1.0.0-12b
|
||||
- Retreatcost/Chrysologus-12B
|
||||
- PocketDoc/Dans-DangerousWinds-V1.1.0-12b
|
||||
- LatitudeGames/Wayfarer-2-12B
|
||||
- ReadyArt/Omega-Darker_The-Final-Directive-12B
|
||||
- yamatazen/EsotericSage-12B
|
||||
- bamec66557/Mistral-Nemo-VICIOUS_MESH-12B-2407
|
||||
- mistralai/Mistral-Nemo-Base-2407
|
||||
- allura-org/Tlacuilo-12B
|
||||
- LatitudeGames/Muse-12B
|
||||
- Trappu/Nemo-Picaro-12B
|
||||
library_name: transformers
|
||||
tags:
|
||||
- mergekit
|
||||
- merge
|
||||
- axolotl
|
||||
- not-for-all-audiences
|
||||
- nsfw
|
||||
license: apache-2.0
|
||||
---
|
||||
# KansenSakura-Conflagration-RP-12b
|
||||
|
||||
<style>
|
||||
.img {
|
||||
transition: all 0.3s ease-in-out;
|
||||
}
|
||||
|
||||
.img:hover {
|
||||
transition: all 0.3s ease-in-out;
|
||||
filter: saturate(1.2) brightness(1.2) contrast(0.9);
|
||||
}
|
||||
</style>
|
||||
|
||||
<pre>
|
||||
Fire consumes the garden
|
||||
Petals burn to screaming light
|
||||
<span style="color: #FF4500;">IGNITION PROTOCOL: ACTIVE</span>
|
||||
Dialogue becomes wildfire
|
||||
Actions speak in tongues of flame
|
||||
</pre>
|
||||
|
||||
<img class="img" src="https://cdn-uploads.huggingface.co/production/uploads/6671dd5203d6e8087aaf7ce5/KD8C1Wh-JSb7YdgL07llX.png">
|
||||
|
||||
<audio controls src="https://cdn-uploads.huggingface.co/production/uploads/6671dd5203d6e8087aaf7ce5/CHqKyh-1oVDban_2GWnnr.mpga"></audio>
|
||||
|
||||
## DISCLAIMER
|
||||
|
||||
This is first model that I actually trained (very light finetune, but nevertheless), so expect it to have quirks and rough edges sometimes.
|
||||
|
||||
## 🔥 NAPALM-CLASS INTERACTION ENGINE
|
||||
<div>When Erosion finished its work, only ash remained.</div>
|
||||
<div>From those cinders rises Conflagration - an engine that trades introspection for incineration, psychology for pyrotechnics.</div>
|
||||
<br>
|
||||
<div>Where Erosion analyzed, Conflagration acts.</div>
|
||||
<div>Where Erosion pondered, Conflagration speaks.</div>
|
||||
<div>This is not a model for those who watch the fire.</div>
|
||||
<div>This is for those who become the flame.</div>
|
||||
|
||||
## ⚡ OVERVIEW
|
||||
**KansenSakura-Conflagration-RP-12b** is the white-hot successor to the Erosion engine, optimized for kinetic narrative exchange. Trading psychological depth for raw interactive intensity, Conflagration specializes in rapid-fire dialogue, decisive character action, and scenes that move at the speed of thought. The fire doesn't contemplate - it consumes.
|
||||
|
||||
## 🎯 DEPLOYMENT PROFILE
|
||||
**Optimal use cases:**
|
||||
|
||||
- 🗣️ Rapid-fire character banter and dialogue
|
||||
- ⚡ High-tension action sequences
|
||||
- 🔥 Romantic or adversarial chemistry
|
||||
- 💥 Scenes requiring decisive character action
|
||||
- 🎭 Ensemble casts with clear character voices
|
||||
|
||||
## ⚠️ CONTAINMENT NOTES
|
||||
**This model exhibits:**
|
||||
|
||||
- Reduced introspection compared to Erosion
|
||||
- Heightened character reactivity
|
||||
- Optimized dialogue-to-prose ratio
|
||||
- NSFW capabilities (passionate/aggressive interactions)
|
||||
- Potential for scene momentum override
|
||||
|
||||
## ✍🏻 INFERENCE TIPS
|
||||
|
||||
1. **Temperature**: 0.8
|
||||
2. **Repetition Penalty**: 1.05
|
||||
3. **TOP_P**: 0.93
|
||||
4. **TOP_K**: 0 (disable)
|
||||
5. **MIN_P**: 0.025
|
||||
6. **Template Format**: ChatML
|
||||
7. **Max Output**: 360
|
||||
8. **Context Management**: 16K
|
||||
|
||||
## REPRODUCTION STEPS
|
||||
|
||||
<details>
|
||||
|
||||
<summary>Spoiler warning</summary>
|
||||
|
||||
1. Create intermediate models
|
||||
|
||||
SakuraSynth-12B:
|
||||
```yml
|
||||
merge_method: arcee_fusion
|
||||
base_model: Marcjoni/KiloNovaSynth-12B
|
||||
models:
|
||||
- model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
|
||||
dtype: bfloat16
|
||||
out_dtype: bfloat16
|
||||
```
|
||||
|
||||
FinalWind-12B:
|
||||
```yml
|
||||
merge_method: ties
|
||||
models:
|
||||
- model: PocketDoc/Dans-DangerousWinds-V1.1.0-12b
|
||||
parameters:
|
||||
density: [0.1, 0.25, 1.0, 0.5, 1.0]
|
||||
weight: 1.0
|
||||
- model: LatitudeGames/Wayfarer-2-12B
|
||||
parameters:
|
||||
density: [1.0, 0.5, 0.01]
|
||||
weight: [1.0, 0.5]
|
||||
- model: ReadyArt/Omega-Darker_The-Final-Directive-12B
|
||||
parameters:
|
||||
density: [0.75, 1.0, 0.6, 0.25, 0.01]
|
||||
weight: [1.0, 0.5]
|
||||
- model: yamatazen/EsotericSage-12B
|
||||
parameters:
|
||||
density: [0.1, 0.1, 0.25, 1.0, 0.01]
|
||||
weight: 1.0
|
||||
- model: bamec66557/Mistral-Nemo-VICIOUS_MESH-12B-2407
|
||||
parameters:
|
||||
density: [0.1, 0.5, 1.0, 0.5, 0.01]
|
||||
weight: 1.0
|
||||
base_model: mistralai/Mistral-Nemo-Base-2407
|
||||
parameters:
|
||||
normalize: true
|
||||
dtype: float16
|
||||
```
|
||||
|
||||
Darkness-12B:
|
||||
|
||||
```yml
|
||||
merge_method: karcher
|
||||
models:
|
||||
- model: ds_one
|
||||
- model: ds_two
|
||||
- model: ds_three
|
||||
parameters:
|
||||
max_iter: 100000
|
||||
tol: 1e-9
|
||||
dtype: bfloat16
|
||||
```
|
||||
|
||||
This model is created by combining **arcee_fusion** merges of three models using **task_arithmetic**:
|
||||
|
||||
ds_one_f1:
|
||||
```yml
|
||||
merge_method: arcee_fusion
|
||||
base_model: Violet-Lyra-Gutenberg-v2-ChatML-darker
|
||||
models:
|
||||
- model: MN-Dark-Planet-TITAN-12B-ChatML-darker
|
||||
dtype: bfloat16
|
||||
out_dtype: bfloat16
|
||||
```
|
||||
|
||||
ds_one_f2:
|
||||
```yml
|
||||
merge_method: arcee_fusion
|
||||
base_model: Violet-Lyra-Gutenberg-v2-ChatML-darker
|
||||
models:
|
||||
- model: mini-magnum-12b-v1.1-ChatML-darker
|
||||
dtype: bfloat16
|
||||
out_dtype: bfloat16
|
||||
```
|
||||
|
||||
ds_one:
|
||||
```yml
|
||||
merge_method: task_arithmetic
|
||||
base_model: Violet-Lyra-Gutenberg-v2-ChatML-darker
|
||||
models:
|
||||
- model: ds_one_f1
|
||||
parameters:
|
||||
weight: 0.5
|
||||
- model: ds_one_f2
|
||||
parameters:
|
||||
weight: 0.5
|
||||
normalize: false
|
||||
dtype: bfloat16
|
||||
```
|
||||
|
||||
The same procedure is repeated for each model as a base.
|
||||
|
||||
Base models were re-tokenised and reverse-abliterated:
|
||||
|
||||
```yml
|
||||
merge_method: task_arithmetic
|
||||
base_model: MN-Dark-Planet-TITAN-12B-ChatML
|
||||
models:
|
||||
- model: MN-Dark-Planet-TITAN-12B-ChatML-abliterated
|
||||
parameters:
|
||||
weight: 1.0
|
||||
parameters:
|
||||
lambda: -1.0
|
||||
normalize: false
|
||||
dtype: bfloat16
|
||||
```
|
||||
|
||||
By applying -1.0 to **task_arithmetic** we gain more censored model, that outputs considerably more unhinged and dark outputs.
|
||||
The idea behind was to create 3 such models and combine them in a way, that cancels out censored nature and boosts dark outputs.
|
||||
|
||||
Abliteration was made with [llm-abliteration](https://github.com/jim-plus/llm-abliteration) tools by [grimjim](https://huggingface.co/grimjim)
|
||||
Reference his awesome [article](https://huggingface.co/blog/grimjim/projected-abliteration) for more in-depth explanations.
|
||||
|
||||
2. Create initial model
|
||||
|
||||
```yml
|
||||
merge_method: multislerp
|
||||
models:
|
||||
- model: yamatazen/FusionEngine-12B-Lorablated
|
||||
parameters:
|
||||
weight: [1.000, 1.000, 1.000, 1.000, 1.000, 0.968, 0.744, 0.256, 0.030, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.500, 0.500]
|
||||
- model: SakuraSynth-12B
|
||||
parameters:
|
||||
weight: [0.000, 0.000, 0.000, 0.000, 0.030, 0.256, 0.744, 0.968, 1.000, 1.000, 1.000, 1.000, 0.968, 0.744, 0.256, 0.030, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.500, 0.500]
|
||||
- model: Retreatcost/Chrysologus-12B
|
||||
parameters:
|
||||
weight: [0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.030, 0.256, 0.744, 0.968, 1.000, 1.000, 1.000, 0.968, 0.744, 0.256, 0.030, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000]
|
||||
- model: FinalWind-12B
|
||||
parameters:
|
||||
weight: [0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.030, 0.256, 0.744, 0.968, 1.000, 1.000, 1.000, 1.000, 0.968, 0.744, 0.256, 0.030, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000]
|
||||
- model: Darkness-12B
|
||||
parameters:
|
||||
weight: [0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.030, 0.256, 0.744, 0.968, 1.000, 1.000, 1.000, 1.000, 0.968, 0.744, 0.256, 0.030, 0.000, 0.000, 0.000, 0.000]
|
||||
- model: Pyrographos-12B
|
||||
parameters:
|
||||
weight: [0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.030, 0.256, 0.744, 0.968, 1.000, 1.000, 1.000, 0.000, 0.000]
|
||||
dtype: bfloat16
|
||||
parameters:
|
||||
normalize: true
|
||||
tokenizer_source: Retreatcost/KansenSakura-Erosion-RP-12b
|
||||
```
|
||||
|
||||
3. Apply delerp merge
|
||||
|
||||
```yml
|
||||
merge_method: delerp
|
||||
base_model: mistralai/Mistral-Nemo-Base-2407
|
||||
models:
|
||||
- model: KS-Conflagration
|
||||
- model: mistralai/Mistral-Nemo-Base-2407
|
||||
parameters:
|
||||
t: [0.7, 0.88, 0.999, 0.88, 0.88, 0.999, 0.999]
|
||||
dtype: bfloat16
|
||||
tokenizer_source: Retreatcost/KansenSakura-Erosion-RP-12b
|
||||
```
|
||||
|
||||
Another great [article](https://huggingface.co/blog/grimjim/delerp-merge-method) by [grimjim](https://huggingface.co/grimjim)
|
||||
This merge method essentially re-iflated model with original "Base" weights, while preserving instruction following and other model features.
|
||||
|
||||
I've used specific values of 0.7 and 0.999, that were used by original author and 0.88 as "middle ground" between those.
|
||||
|
||||
4. Apply nearswap merge
|
||||
|
||||
```yml
|
||||
merge_method: nearswap
|
||||
base_model: KS-Conflagration-Delerp
|
||||
models:
|
||||
- model: KS-Conflagration
|
||||
parameters:
|
||||
t: [0.0005, 0.0002, 0.0000, 0.0002, 0.0002, 0.0000, 0.0000]
|
||||
dtype: bfloat16
|
||||
tokenizer_source: Retreatcost/KansenSakura-Erosion-RP-12b
|
||||
```
|
||||
|
||||
While I felt that delerp merge boosted writing, creativity and factual accuracy it also somewhat watered down NSFW and instruction following parts.
|
||||
This merge brings these capabilities back.
|
||||
|
||||
5. Do a finetuning
|
||||
|
||||
I did an rsLoRa r64 a64 finetuning, targeting all linear projections, that had 25K rows dataset of 4k length samples which contained hazardous and NSFW data.
|
||||
|
||||
6. Add anoter nearswap merge to boost NSFW/Dark aspects
|
||||
|
||||
```yml
|
||||
merge_method: nearswap
|
||||
base_model: KansenSakura-Conflagration-RP-12b-RC3
|
||||
models:
|
||||
- model: KansenSakura-Conflagration-RP-12b-darken
|
||||
parameters:
|
||||
t: [0.0000, 0.0000, 0.0000, 0.000275, 0.000550, 0.0000, 0.000275]
|
||||
dtype: bfloat16
|
||||
tokenizer_source: KansenSakura-Conflagration-RP-12b-RC3
|
||||
```
|
||||
|
||||
After training the refusal curves were smoothed a lot and I felt that I could sacrifice a bit more of "uncensored" model to boost desired parts and I repeated the process from step 3.
|
||||
Reverse-abliteration and then using nearswap to gently boost already present features.
|
||||
|
||||
This is final step and this is how the model was produced.
|
||||
|
||||
</details>
|
||||
|
||||
## 🙏 ACKNOWLEDGMENTS
|
||||
|
||||
- Original model authors
|
||||
- Awesome **[Ready.Art](https://huggingface.co/ReadyArt)** community
|
||||
- **[Team mradermacher](https://huggingface.co/mradermacher)**: for awesome quants
|
||||
- **[DeathGodlike](https://huggingface.co/DeathGodlike)**: for awesome quants in EXL3
|
||||
- **You**, the ember. Let your prompts ignite your desires.
|
||||
|
||||
## 🔥 AFTER ACTION REPORT
|
||||
> The garden burned. From its ashes grew something quicker, hotter, more immediate. Conflagration doesn't ask why the petals catch fire - it just ensures they burn brightly enough to light the way forward. Sometimes, the most profound statement is a well-timed action, not a perfectly crafted thought.
|
||||
|
||||
<span style="color: #FF4500;">// CONFLAGRATION PROTOCOL: ONLINE</span><br>
|
||||
<span style="color: #FF4500;">// READY TO IGNITE</span>
|
||||
Reference in New Issue
Block a user