150 lines
3.7 KiB
Markdown
150 lines
3.7 KiB
Markdown
|
---
|
||
|
|
base_model:
|
||
|
|
- coder3101/Cydonia-24B-v4.3-heretic-v4
|
||
|
|
- FlareRebellion/WeirdCompound-v1.7-24b
|
||
|
|
- WasamiKirua/Sakura-24B-Spice
|
||
|
|
- ReadyArt/Broken-Tutu-24B-Unslop-v2.0
|
||
|
|
- zerofata/MS3.2-PaintedFantasy-v4.1-24B
|
||
|
|
- DavidAU/Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
|
||
|
|
library_name: transformers
|
||
|
|
tags:
|
||
|
|
- mergekit
|
||
|
|
- merge
|
||
|
|
- model-stock
|
||
|
|
- mistral
|
||
|
|
- 24b
|
||
|
|
- dare-ties
|
||
|
|
- uncensored
|
||
|
|
- roleplay
|
||
|
|
- thinking
|
||
|
|
- conversational
|
||
|
|
- not-for-all-audiences
|
||
|
|
license: apache-2.0
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
---
|
||
|
|
# EnceladusHyperStock
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
**EnceladusHyperStock-24B** is ***Experimental*** version of my previous multimerge model; 'VeryWeirdDolphinMix-V1-24B-Heretic'. And this doesnt mean this merge-model is better.
|
||
|
|
|
||
|
|
The goal was to create a model that handles complex scenes, story-writing and role-play with creativitiy and consistent.
|
||
|
|
|
||
|
|
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
||
|
|
|
||
|
|
---
|
||
|
|
## Thinking Capability
|
||
|
|
|
||
|
|
This model can think if these tags used:
|
||
|
|
|
||
|
|
```
|
||
|
|
<thinking>
|
||
|
|
|
||
|
|
</thinking>
|
||
|
|
```
|
||
|
|
---
|
||
|
|
|
||
|
|
## Recommended Settings
|
||
|
|
|
||
|
|
These are my recommended settings.
|
||
|
|
|
||
|
|
| Setting | Value |
|
||
|
|
|---|---|
|
||
|
|
| Temperature | 0.65 - 0.85 |
|
||
|
|
| Min P | 0.05 - 0.1 |
|
||
|
|
| Repetition penalty | 1.05 - 1.1 |
|
||
|
|
| Top P | 0.75 - 0.95 |
|
||
|
|
| Adaptive-p target | 0.70 - 0.9 |
|
||
|
|
| Adaptive-p decay | 0.9 |
|
||
|
|
|
||
|
|
Use the **Mistral V7 Tekken** Template.
|
||
|
|
|
||
|
|
|
||
|
|
I recommend some tinkering with these samplers. I personally use Adaptive-P instead of 'DRY repetition penalty' or 'Rep pen'.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Intended Use
|
||
|
|
|
||
|
|
- Long-form uncensored roleplay and collaborative storytelling
|
||
|
|
- Uncensored creative writing
|
||
|
|
- Complex instruction following with visible reasoning via `<thinking>`
|
||
|
|
- Conversational AI with personality depth
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Disclaimer
|
||
|
|
|
||
|
|
This model is **uncensored** and intended for adult, research, fictional story, and creative use. It may produce content that some users find offensive or inappropriate. Use responsibly.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Merge Details
|
||
|
|
### Merge Methods
|
||
|
|
|
||
|
|
This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) and [Dare-Ties](https://arxiv.org/abs/2306.01708) merge methods.
|
||
|
|
|
||
|
|
### Models Merged
|
||
|
|
|
||
|
|
The following models were included in the merge:
|
||
|
|
* R:\Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
|
||
|
|
* R:\Sakura-24B-Spice
|
||
|
|
* R:\Cydonia-24B-Heretic-v4
|
||
|
|
* R:\WeirdCompound
|
||
|
|
* R:\Broken-Tutu-24B-Unslop-v2.0
|
||
|
|
* R:\MergeOutput\EnceladusDare-ties(second step model)
|
||
|
|
|
||
|
|
### Configuration
|
||
|
|
|
||
|
|
The following YAML configuration(s) was used to produce this model:
|
||
|
|
---
|
||
|
|
- first step
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
merge_method: dare_ties
|
||
|
|
base_model: R:\WeirdCompound
|
||
|
|
models:
|
||
|
|
- model: R:\Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
|
||
|
|
parameters:
|
||
|
|
weight: 0.50
|
||
|
|
density: 0.55
|
||
|
|
- model: R:\Cydonia-24B-Heretic-v4
|
||
|
|
parameters:
|
||
|
|
weight: 0.45
|
||
|
|
density: 0.50
|
||
|
|
- model: R:\WeirdCompound
|
||
|
|
parameters:
|
||
|
|
weight: 0.45
|
||
|
|
density: 0.40
|
||
|
|
dtype: bfloat16
|
||
|
|
```
|
||
|
|
---
|
||
|
|
- second step (I've used Heretic in this step, 86/100 refusals to: 8/100 refusals with 0.0063 KL Divergence. Trial 92.)
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
merge_method: model_stock
|
||
|
|
base_model: R:\MergeOutput\EnceladusDare-ties
|
||
|
|
models:
|
||
|
|
- model: R:\Cydonia-24B-Heretic-v4
|
||
|
|
- model: R:\MS3.2-PaintedFantasy-v2-24B
|
||
|
|
- model: R:\Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
|
||
|
|
- model: R:\WeirdCompound
|
||
|
|
dtype: bfloat16
|
||
|
|
```
|
||
|
|
---
|
||
|
|
- last step
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
merge_method: model_stock
|
||
|
|
base_model: R:\MergeOutput\EnceladusModelStock-Heretic
|
||
|
|
models:
|
||
|
|
- model: R:\MergeOutput\EnceladusModelStock-Heretic
|
||
|
|
- model: R:\Sakura-24B-Spice
|
||
|
|
- model: R:\Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
|
||
|
|
- model: R:\WeirdCompound
|
||
|
|
- model: R:\Broken-Tutu-24B-Unslop-v2.0
|
||
|
|
- model: R:\Cydonia-24B-Heretic-v4
|
||
|
|
dtype: bfloat16
|
||
|
|
```
|