122 lines
4.3 KiB
Markdown
122 lines
4.3 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
base_model:
|
||
|
|
- anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
|
||
|
|
- TheDrummer/Magidonia-24B-v4.3
|
||
|
|
- TheDrummer/Precog-24B-v1
|
||
|
|
- zerofata/MS3.2-PaintedFantasy-v3-24B
|
||
|
|
- TheDrummer/Cydonia-24B-v4.3
|
||
|
|
- ReadyArt/4.2.0-Broken-Tutu-24b
|
||
|
|
- zerofata/MS3.2-PaintedFantasy-v2-24B
|
||
|
|
datasets:
|
||
|
|
- OccultAI/illuminati_imatrix_v1
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
library_name: transformers
|
||
|
|
tags:
|
||
|
|
- ramplus_tl
|
||
|
|
- merge
|
||
|
|
- mergekit
|
||
|
|
widget:
|
||
|
|
- text: "Ślimaki 24B v1.1 ramplus_tl"
|
||
|
|
output:
|
||
|
|
url: https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/5ZybvYIHF-URERiO7GbWJ.png
|
||
|
|
---
|
||
|
|
|
||
|
|
<audio controls src="https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/FyBiCGYn-xghTyfIrxr9I.mpga"></audio>
|
||
|
|
|
||
|
|
> [!CAUTION]
|
||
|
|
> <span style="color:red; font-weight:bold">⚠️ Warning:</span> This model can produce narratives and RP that contain violent and graphic erotic content. Adjust your system prompt accordingly, and use **Mistral Tekken** chat template.
|
||
|
|
>
|
||
|
|
|
||
|
|
# 🐌 Ślimaki-24B-v1.1-ramplus_tl
|
||
|
|
|
||
|
|
Experimental merge of the new arcee-mergekit method `ramplus_tl` using the same components as Maginum Cydoms and Slimaki.
|
||
|
|
|
||
|
|
It may have some refusals and thus require jailbreaks or ablations in order to uncensor it.
|
||
|
|
|
||
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/5ZybvYIHF-URERiO7GbWJ.png" style="float:right; width:300px; height:300px; padding:10px;">
|
||
|
|
|
||
|
|
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
||
|
|
|
||
|
|
## Merge Details
|
||
|
|
### Merge Methods
|
||
|
|
|
||
|
|
This model was merged using the following merge method:
|
||
|
|
- [Reinforced Agent Merging Plus (Tensor-Local)](https://arxiv.org/abs/2601.13572)
|
||
|
|
|
||
|
|
Note: This merge was heavily inspired by [Maginum Cydoms](https://huggingface.co/Casual-Autopsy/Maginum-Cydoms-24B)
|
||
|
|
|
||
|
|
This model was [requested](https://huggingface.co/24B-Suite/Mergedonia-Suite-24B-v1-GGUF/discussions/1) as a way to test Arcee's newest mergekit method on the Slimaki/Cydoms components.
|
||
|
|
|
||
|
|
### Configuration
|
||
|
|
```yaml
|
||
|
|
merge_method: ramplus_tl
|
||
|
|
base_model: B:\24B\models--anthracite-core--Mistral-Small-3.2-24B-Instruct-2506-Text-Only
|
||
|
|
models:
|
||
|
|
- model: B:\24B\models--anthracite-core--Mistral-Small-3.2-24B-Instruct-2506-Text-Only
|
||
|
|
- model: B:\24B\models--TheDrummer--Cydonia-24B-v4.3
|
||
|
|
- model: B:\24B\models--ReadyArt--4.2.0-Broken-Tutu-24b
|
||
|
|
- model: B:\24B\models--zerofata--MS3.2-PaintedFantasy-v2-24B
|
||
|
|
- model: B:\24B\models--TheDrummer--Magidonia-24B-v4.3
|
||
|
|
- model: B:\24B\models--TheDrummer--Precog-24B-v1
|
||
|
|
- model: B:\24B\models--zerofata--MS3.2-PaintedFantasy-v3-24B
|
||
|
|
parameters:
|
||
|
|
epsilon: 0.001 # Increased from 1e-5 to 1e-3 for denser SFT/DPO task vectors
|
||
|
|
r: 0.25 # Increased from 0.1 to 0.2-0.3 for better SFT behavior preservation
|
||
|
|
alpha: 0.4 # Increased from 0.2 to 0.4 for enhanced rescaling
|
||
|
|
dtype: float32
|
||
|
|
out_dtype: bfloat16
|
||
|
|
tokenizer:
|
||
|
|
source: base
|
||
|
|
chat_template: auto
|
||
|
|
name: 🐌 Ślimaki-24B-v1.1-ramplus_tl
|
||
|
|
```
|
||
|
|
|
||
|
|
## Recommended Settings
|
||
|
|
> In particular temp 1 and topnsigma 1.25 seems to help a lot with improving quality but sometimes you need to reswipe.
|
||
|
|
>
|
||
|
|
> Some models are more sensitive to Rep Pen settings, 1.12 is another, you may need to adjust higher for smaller models (like 1.4 if overcooked) or lower for smarter ones.
|
||
|
|
>
|
||
|
|
> Not sure about adaptive_p that is a newer setting added to kobold
|
||
|
|
|
||
|
|
**(bolded kobold non-defaults)**
|
||
|
|
- **Temp 1.0**
|
||
|
|
- **TopNSigma 1.25**
|
||
|
|
- **Min-P 0.1**
|
||
|
|
- **Repetition Penalty 1.08**
|
||
|
|
- **Top-P 1.0**
|
||
|
|
- Top-K 100
|
||
|
|
- Top-A 0
|
||
|
|
- Typical Sampling 1
|
||
|
|
- Tail-Free Sampling 1
|
||
|
|
- Presence Penalty 0
|
||
|
|
- Sampler Seed -1
|
||
|
|
- Rp.Range 360
|
||
|
|
- Rp.Slope 0.7
|
||
|
|
- Smoothing Factor 0
|
||
|
|
- Smoothing Curve 1
|
||
|
|
- DynaTemp 0
|
||
|
|
- Mirostat Mode OFF ("2" enhances creativity but also errors)
|
||
|
|
- Mirostat Tau 5
|
||
|
|
- Mirostat Eta 0.1
|
||
|
|
- **DRY Multiplier 0.8**
|
||
|
|
- DRY Base 1.75
|
||
|
|
- DRY A.Len 2
|
||
|
|
- DRY L.Len 320
|
||
|
|
- **XTC Threshold 0.1**
|
||
|
|
- **XTC Probability 0.08** (The "Anti-Cliche" Shield)
|
||
|
|
- **DynaTemp ON** (The "Poor Man's Fading Mirostat")
|
||
|
|
- **Minimum Temperature 0.65**
|
||
|
|
- **Maximum Temperature 1.35**
|
||
|
|
- **Temperature 1.0**
|
||
|
|
- **DynaTemp-Range 0.35**
|
||
|
|
- **DynaTemp-Exponent 1**
|
||
|
|
|
||
|
|
## New Settings
|
||
|
|
I have not tested these yet but some users on reddit have reported the following settings work extremely well
|
||
|
|
- min_p: 0.05
|
||
|
|
- adaptive_target: 0.9 (down to 0.65 for more creative)
|
||
|
|
- adaptive_decay: 0.99
|
||
|
|
- repetition_penalty_range: 0
|
||
|
|
- temp and topnsigma: 0.75
|