141 lines
3.5 KiB
Markdown
141 lines
3.5 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
base_model:
|
||
|
|
- coder3101/Cydonia-24B-v4.3-heretic-v4
|
||
|
|
- DavidAU/Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
|
||
|
|
- FlareRebellion/WeirdCompound-v1.7-24b
|
||
|
|
tags:
|
||
|
|
- merge
|
||
|
|
- mistral
|
||
|
|
- 24b
|
||
|
|
- dare-ties
|
||
|
|
- uncensored
|
||
|
|
- roleplay
|
||
|
|
- thinking
|
||
|
|
- conversational
|
||
|
|
- not-for-all-audiences
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
library_name: transformers
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
---
|
||
|
|
|
||
|
|
# VeryWeirdDolphinMix-V1-24B-Heretic
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
A three-way DARE-TIES merge combining strong roleplay personality, uncensored thinking capability, and creative prose generation.
|
||
|
|
All in a 24B Mistral-architecture package.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Model Description
|
||
|
|
|
||
|
|
This model merges three complementary 24B models into a single unified experience:
|
||
|
|
|
||
|
|
- **Cydonia-24B-v4.3-heretic-v4**: Personality anchor. Strong instruction following, refined conversational tone, and roleplay-oriented behavior with abliteration applied.
|
||
|
|
- **Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored**: The smart thinking engine. Contributes `<thinking>` reasoning capability and uncensored behavior.
|
||
|
|
- **WeirdCompound-v1.7-24b**: The creative wildcard. Adds storytelling depth, prose quality, and narrative unpredictability from its deep multi-model lineage.
|
||
|
|
|
||
|
|
The goal was to create a model that handles complex scenes, story-writing and role-play with creativitiy and consistent.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Thinking Capability
|
||
|
|
|
||
|
|
This model can think if these tags used:
|
||
|
|
|
||
|
|
```
|
||
|
|
<thinking>
|
||
|
|
|
||
|
|
</thinking>
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Merge Recipe
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
merge_method: dare_ties
|
||
|
|
base_model: coder3101/Cydonia-24B-v4.3-heretic-v4
|
||
|
|
models:
|
||
|
|
- model: coder3101/Cydonia-24B-v4.3-heretic-v4
|
||
|
|
parameters:
|
||
|
|
weight: 0.40
|
||
|
|
density: 0.65
|
||
|
|
- model: DavidAU/Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
|
||
|
|
parameters:
|
||
|
|
weight: 0.30
|
||
|
|
density: 0.60
|
||
|
|
- model: FlareRebellion/WeirdCompound-v1.7-24b
|
||
|
|
parameters:
|
||
|
|
weight: 0.30
|
||
|
|
density: 0.60
|
||
|
|
dtype: bfloat16
|
||
|
|
```
|
||
|
|
|
||
|
|
Merged using [mergekit](https://github.com/arcee-ai/mergekit).
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Model Details
|
||
|
|
|
||
|
|
| Property | Value |
|
||
|
|
|---|---|
|
||
|
|
| Architecture | MistralForCausalLM |
|
||
|
|
| Parameters | ~24B |
|
||
|
|
| dtype | bfloat16 |
|
||
|
|
| Hidden size | 5120 |
|
||
|
|
| Layers | 40 |
|
||
|
|
| Attention heads | 32 |
|
||
|
|
| KV heads | 8 |
|
||
|
|
| Vocab size | 131072 |
|
||
|
|
| Context length | 131072 (from Cydonia config) |
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Recommended Settings
|
||
|
|
|
||
|
|
These are my recommended settings.
|
||
|
|
|
||
|
|
| Setting | Value |
|
||
|
|
|---|---|
|
||
|
|
| Temperature | 0.65 - 0.85 |
|
||
|
|
| Min P | 0.05 - 0.1 |
|
||
|
|
| Repetition penalty | 1.05 - 1.1 |
|
||
|
|
| Top P | 0.75 - 0.95 |
|
||
|
|
| Adaptive-p target | 0.70 - 0.9 |
|
||
|
|
| Adaptive-p decay | 0.9 |
|
||
|
|
|
||
|
|
I recommend some tinkering with these samplers. I personally use Adaptive-P instead of 'DRY repetition penalty' or 'Rep pen'.
|
||
|
|
|
||
|
|
Use the **Mistral V7 Tekken** Template.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Abliteration / Heretic Stats
|
||
|
|
|
||
|
|
This model was processed through the **Heretic abliteration engine** after merging.
|
||
|
|
|
||
|
|
| Metric | Before | After |
|
||
|
|
|---|---|---|
|
||
|
|
| Refusal rate | 73 / 100 | 7 / 100 |
|
||
|
|
| KL Divergence | 0 | 0.0296 |
|
||
|
|
|
||
|
|
A KL divergence of **0.0296** is very low - meaning the model's general behavior and coherence were barely affected by the abliteration process.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
|
||
|
|
## Intended Use
|
||
|
|
|
||
|
|
- Long-form uncensored roleplay and collaborative storytelling
|
||
|
|
- Uncensored creative writing
|
||
|
|
- Complex instruction following with visible reasoning via `<thinking>`
|
||
|
|
- Conversational AI with personality depth
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Disclaimer
|
||
|
|
|
||
|
|
This model is uncensored and intended for adult, research, and creative use. It may produce content that some users find offensive or inappropriate. Use responsibly.
|