85 lines
1.8 KiB
Markdown
85 lines
1.8 KiB
Markdown
|
---
|
||
|
|
library_name: transformers
|
||
|
|
license: apache-2.0
|
||
|
|
tags:
|
||
|
|
- merge
|
||
|
|
- mergekit
|
||
|
|
- roleplay
|
||
|
|
- rp
|
||
|
|
base_model:
|
||
|
|
- mistralai/Mistral-Nemo-Base-2407
|
||
|
|
---
|
||
|
|
|
||
|
|
# KaidenRp2400_12b_v1
|
||
|
|
|
||
|
|
Merged using mergekit.
|
||
|
|
we first merge some models in merge1 then we megre another list of models in merge2 then we merged them together.
|
||
|
|
|
||
|
|
here are intermediate merges:
|
||
|
|
(kainatq/KaidenRp2400_12b_v1_m1](https://huggingface.co/kainatq/KaidenRp2400_12b_v1_m1)
|
||
|
|
[https://huggingface.co/kainatq/KaidenRp2400_12b_v1_m2](https://huggingface.co/kainatq/KaidenRp2400_12b_v1_m2)
|
||
|
|
## Usage:
|
||
|
|
follow [mistral-nemo-base-2407](https://huggingface.co/mistralai/Mistral-Nemo-Base-2407) (chatml) instructions.
|
||
|
|
## Merge Configuration
|
||
|
|
|
||
|
|
### Merge 1:
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
merge_method: dare_ties
|
||
|
|
base_model: mistralai/Mistral-Nemo-Base-2407
|
||
|
|
tokenizer_source: union
|
||
|
|
parameters:
|
||
|
|
density: 0.5
|
||
|
|
weight: 1.0
|
||
|
|
models:
|
||
|
|
- model: Gryphe/Pantheon-RP-1.5-12b-Nemo
|
||
|
|
parameters:
|
||
|
|
weight: 0.33
|
||
|
|
- model: kainatq/RP-king-12b-II
|
||
|
|
parameters:
|
||
|
|
weight: 0.33
|
||
|
|
- model: elinas/Chronos-Gold-12B-1.0
|
||
|
|
parameters:
|
||
|
|
weight: 0.34
|
||
|
|
|
||
|
|
```
|
||
|
|
|
||
|
|
### Merge 2:
|
||
|
|
```yaml
|
||
|
|
merge_method: dare_ties
|
||
|
|
base_model: mistralai/Mistral-Nemo-Base-2407
|
||
|
|
tokenizer_source: union
|
||
|
|
parameters:
|
||
|
|
density: 0.5
|
||
|
|
weight: 1.0
|
||
|
|
models:
|
||
|
|
- model: mergekit-community/MN-Sappho-g2-12B
|
||
|
|
parameters:
|
||
|
|
weight: 0.33
|
||
|
|
- model: nbeerbower/Nemoties-ChatML-12B
|
||
|
|
parameters:
|
||
|
|
weight: 0.33
|
||
|
|
- model: pbevan11/Mistral-Nemo-Baseline-SFT
|
||
|
|
parameters:
|
||
|
|
weight: 0.34
|
||
|
|
```
|
||
|
|
### final model:
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
merge_method: dare_ties
|
||
|
|
base_model: mistralai/Mistral-Nemo-Base-2407
|
||
|
|
tokenizer_source: union
|
||
|
|
parameters:
|
||
|
|
density: 0.5
|
||
|
|
weight: 1.0
|
||
|
|
models:
|
||
|
|
- model: kainatq/KaidenRp2400_12b_v1_m1
|
||
|
|
parameters:
|
||
|
|
weight: 0.40
|
||
|
|
- model: kainatq/KaidenRp2400_12b_v1_m2
|
||
|
|
parameters:
|
||
|
|
weight: 0.40
|
||
|
|
- model: mergekit-community/MN-Sappho-j-12B
|
||
|
|
parameters:
|
||
|
|
weight: 0.20
|
||
|
|
```
|