KansenSakura-Conflagration-RP-12b/README.md

---
base_model:
- ohyeah1/Violet-Lyra-Gutenberg-v2
- DavidAU/MN-Dark-Planet-TITAN-12B
- intervitens/mini-magnum-12b-v1.1
- yamatazen/FusionEngine-12B-Lorablated
- Marcjoni/KiloNovaSynth-12B
- PocketDoc/Dans-SakuraKaze-V1.0.0-12b
- Retreatcost/Chrysologus-12B
- PocketDoc/Dans-DangerousWinds-V1.1.0-12b
- LatitudeGames/Wayfarer-2-12B
- ReadyArt/Omega-Darker_The-Final-Directive-12B
- yamatazen/EsotericSage-12B
- bamec66557/Mistral-Nemo-VICIOUS_MESH-12B-2407
- mistralai/Mistral-Nemo-Base-2407
- allura-org/Tlacuilo-12B
- LatitudeGames/Muse-12B
- Trappu/Nemo-Picaro-12B
library_name: transformers
tags:
- mergekit
- merge
- axolotl
- not-for-all-audiences
- nsfw
license: apache-2.0
---
# KansenSakura-Conflagration-RP-12b

<style>
.img {
  transition: all 0.3s ease-in-out;
}
  
.img:hover {
  transition: all 0.3s ease-in-out;
  filter: saturate(1.2) brightness(1.2) contrast(0.9);
}
</style>

<pre>
Fire consumes the garden
Petals burn to screaming light
<span style="color: #FF4500;">IGNITION PROTOCOL: ACTIVE</span>
Dialogue becomes wildfire
Actions speak in tongues of flame
</pre>

<img class="img" src="https://cdn-uploads.huggingface.co/production/uploads/6671dd5203d6e8087aaf7ce5/KD8C1Wh-JSb7YdgL07llX.png">

<audio controls src="https://cdn-uploads.huggingface.co/production/uploads/6671dd5203d6e8087aaf7ce5/CHqKyh-1oVDban_2GWnnr.mpga"></audio>

## DISCLAIMER

This is first model that I actually trained (very light finetune, but nevertheless), so expect it to have quirks and rough edges sometimes.

## 🔥 NAPALM-CLASS INTERACTION ENGINE
<div>When Erosion finished its work, only ash remained.</div>
<div>From those cinders rises Conflagration - an engine that trades introspection for incineration, psychology for pyrotechnics.</div>
<br>
<div>Where Erosion analyzed, Conflagration acts.</div>
<div>Where Erosion pondered, Conflagration speaks.</div>
<div>This is not a model for those who watch the fire.</div>
<div>This is for those who become the flame.</div>

## ⚡ OVERVIEW
**KansenSakura-Conflagration-RP-12b** is the white-hot successor to the Erosion engine, optimized for kinetic narrative exchange. Trading psychological depth for raw interactive intensity, Conflagration specializes in rapid-fire dialogue, decisive character action, and scenes that move at the speed of thought. The fire doesn't contemplate - it consumes.

## 🎯 DEPLOYMENT PROFILE
**Optimal use cases:**

- 🗣️ Rapid-fire character banter and dialogue
- ⚡ High-tension action sequences
- 🔥 Romantic or adversarial chemistry
- 💥 Scenes requiring decisive character action
- 🎭 Ensemble casts with clear character voices

## ⚠️ CONTAINMENT NOTES
**This model exhibits:**

- Reduced introspection compared to Erosion
- Heightened character reactivity
- Optimized dialogue-to-prose ratio
- NSFW capabilities (passionate/aggressive interactions)
- Potential for scene momentum override

## ✍🏻 INFERENCE TIPS

1.  **Temperature**: 0.8
2.  **Repetition Penalty**: 1.05
3.  **TOP_P**: 0.93
4.  **TOP_K**: 0 (disable)
5.  **MIN_P**: 0.025
6.  **Template Format**: ChatML
7.  **Max Output**: 360
8.  **Context Management**: 16K

## REPRODUCTION STEPS

<details>

  <summary>Spoiler warning</summary>

1. Create intermediate models

SakuraSynth-12B:
```yml
merge_method: arcee_fusion
base_model: Marcjoni/KiloNovaSynth-12B
models:
  - model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
dtype: bfloat16
out_dtype: bfloat16
```

FinalWind-12B:
```yml
merge_method: ties
models:
  - model: PocketDoc/Dans-DangerousWinds-V1.1.0-12b
    parameters:
      density: [0.1, 0.25, 1.0, 0.5, 1.0]
      weight: 1.0
  - model: LatitudeGames/Wayfarer-2-12B
    parameters:
      density: [1.0, 0.5, 0.01]
      weight: [1.0, 0.5]
  - model: ReadyArt/Omega-Darker_The-Final-Directive-12B
    parameters:
      density: [0.75, 1.0, 0.6, 0.25, 0.01]
      weight: [1.0, 0.5]
  - model: yamatazen/EsotericSage-12B
    parameters:
      density: [0.1, 0.1, 0.25, 1.0, 0.01]
      weight: 1.0
  - model: bamec66557/Mistral-Nemo-VICIOUS_MESH-12B-2407
    parameters:
      density: [0.1, 0.5, 1.0, 0.5, 0.01]
      weight: 1.0
base_model: mistralai/Mistral-Nemo-Base-2407
parameters:
  normalize: true
dtype: float16
```

Darkness-12B:

```yml
merge_method: karcher
models:
  - model: ds_one
  - model: ds_two
  - model: ds_three
parameters:
  max_iter: 100000
  tol: 1e-9
dtype: bfloat16
```

This model is created by combining **arcee_fusion** merges of three models using **task_arithmetic**:

ds_one_f1:
```yml
merge_method: arcee_fusion
base_model: Violet-Lyra-Gutenberg-v2-ChatML-darker
models:
  - model: MN-Dark-Planet-TITAN-12B-ChatML-darker
dtype: bfloat16
out_dtype: bfloat16
```

ds_one_f2:
```yml
merge_method: arcee_fusion
base_model: Violet-Lyra-Gutenberg-v2-ChatML-darker
models:
  - model: mini-magnum-12b-v1.1-ChatML-darker
dtype: bfloat16
out_dtype: bfloat16
```

ds_one:
```yml
merge_method: task_arithmetic
base_model: Violet-Lyra-Gutenberg-v2-ChatML-darker
models:
  - model: ds_one_f1
    parameters: 
      weight: 0.5
  - model: ds_one_f2
    parameters: 
      weight: 0.5
normalize: false
dtype: bfloat16
```

The same procedure is repeated for each model as a base.

Base models were re-tokenised and reverse-abliterated:

```yml
merge_method: task_arithmetic
base_model: MN-Dark-Planet-TITAN-12B-ChatML
models:
  - model: MN-Dark-Planet-TITAN-12B-ChatML-abliterated
    parameters: 
      weight: 1.0
parameters:
  lambda: -1.0
normalize: false
dtype: bfloat16
```

By applying -1.0 to **task_arithmetic** we gain more censored model, that outputs considerably more unhinged and dark outputs.
The idea behind was to create 3 such models and combine them in a way, that cancels out censored nature and boosts dark outputs.

Abliteration was made with [llm-abliteration](https://github.com/jim-plus/llm-abliteration) tools by [grimjim](https://huggingface.co/grimjim)
Reference his awesome [article](https://huggingface.co/blog/grimjim/projected-abliteration) for more in-depth explanations.

2. Create initial model

```yml
  merge_method: multislerp
  models:
    - model: yamatazen/FusionEngine-12B-Lorablated
      parameters:
        weight: [1.000, 1.000, 1.000, 1.000, 1.000, 0.968, 0.744, 0.256, 0.030, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.500, 0.500]
    - model: SakuraSynth-12B
      parameters:
        weight: [0.000, 0.000, 0.000, 0.000, 0.030, 0.256, 0.744, 0.968, 1.000, 1.000, 1.000, 1.000, 0.968, 0.744, 0.256, 0.030, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.500, 0.500]
    - model: Retreatcost/Chrysologus-12B
      parameters:
        weight: [0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.030, 0.256, 0.744, 0.968, 1.000, 1.000, 1.000, 0.968, 0.744, 0.256, 0.030, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000]
    - model: FinalWind-12B
      parameters:
        weight: [0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.030, 0.256, 0.744, 0.968, 1.000, 1.000, 1.000, 1.000, 0.968, 0.744, 0.256, 0.030, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000]
    - model: Darkness-12B
      parameters:
        weight: [0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.030, 0.256, 0.744, 0.968, 1.000, 1.000, 1.000, 1.000, 0.968, 0.744, 0.256, 0.030, 0.000, 0.000, 0.000, 0.000]
    - model: Pyrographos-12B
      parameters:
        weight: [0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.030, 0.256, 0.744, 0.968, 1.000, 1.000, 1.000, 0.000, 0.000]
  dtype: bfloat16
  parameters:
    normalize: true
  tokenizer_source: Retreatcost/KansenSakura-Erosion-RP-12b
```

3. Apply delerp merge

```yml
merge_method: delerp
base_model: mistralai/Mistral-Nemo-Base-2407
models:
  - model: KS-Conflagration
  - model: mistralai/Mistral-Nemo-Base-2407
parameters:
  t: [0.7, 0.88, 0.999, 0.88, 0.88, 0.999, 0.999]
dtype: bfloat16
tokenizer_source: Retreatcost/KansenSakura-Erosion-RP-12b
```

Another great [article](https://huggingface.co/blog/grimjim/delerp-merge-method) by [grimjim](https://huggingface.co/grimjim)
This merge method essentially re-iflated model with original "Base" weights, while preserving instruction following and other model features. 

I've used specific values of 0.7 and 0.999, that were used by original author and 0.88 as "middle ground" between those.

4. Apply nearswap merge

```yml
merge_method: nearswap
base_model: KS-Conflagration-Delerp
models:
  - model: KS-Conflagration
parameters:
  t: [0.0005, 0.0002, 0.0000, 0.0002, 0.0002, 0.0000, 0.0000]
dtype: bfloat16
tokenizer_source: Retreatcost/KansenSakura-Erosion-RP-12b
```

While I felt that delerp merge boosted writing, creativity and factual accuracy it also somewhat watered down NSFW and instruction following parts.
This merge brings these capabilities back.

5. Do a finetuning

I did an rsLoRa r64 a64 finetuning, targeting all linear projections, that had 25K rows dataset of 4k length samples which contained hazardous and NSFW data.

6. Add anoter nearswap merge to boost NSFW/Dark aspects

```yml
merge_method: nearswap
base_model: KansenSakura-Conflagration-RP-12b-RC3
models:
  - model: KansenSakura-Conflagration-RP-12b-darken
parameters:
  t: [0.0000, 0.0000, 0.0000, 0.000275, 0.000550, 0.0000, 0.000275]
dtype: bfloat16
tokenizer_source: KansenSakura-Conflagration-RP-12b-RC3
```

After training the refusal curves were smoothed a lot and I felt that I could sacrifice a bit more of "uncensored" model to boost desired parts and I repeated the process from step 3.
Reverse-abliteration and then using nearswap to gently boost already present features.

This is final step and this is how the model was produced.

</details>

## 🙏 ACKNOWLEDGMENTS

- Original model authors
- Awesome **[Ready.Art](https://huggingface.co/ReadyArt)** community
- **[Team mradermacher](https://huggingface.co/mradermacher)**: for awesome quants
- **[DeathGodlike](https://huggingface.co/DeathGodlike)**: for awesome quants in EXL3
- **You**, the ember. Let your prompts ignite your desires.

## 🔥 AFTER ACTION REPORT
> The garden burned. From its ashes grew something quicker, hotter, more immediate. Conflagration doesn't ask why the petals catch fire - it just ensures they burn brightly enough to light the way forward. Sometimes, the most profound statement is a well-timed action, not a perfectly crafted thought.

<span style="color: #FF4500;">// CONFLAGRATION PROTOCOL: ONLINE</span><br>
<span style="color: #FF4500;">// READY TO IGNITE</span>