50 lines
1.5 KiB
Markdown
50 lines
1.5 KiB
Markdown
|
---
|
||
|
|
base_model:
|
||
|
|
- EldritchLabs/MN-12B-RP-Ink-Longform-MPOA
|
||
|
|
- EldritchLabs/Human-Like-Mistral-Nemo-Instruct-2407-MPOA
|
||
|
|
- MuXodious/Rocinante-X-12B-v1-absolute-heresy
|
||
|
|
- MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy
|
||
|
|
language: eng
|
||
|
|
library_name: transformers
|
||
|
|
license: apache-2.0
|
||
|
|
tags:
|
||
|
|
- mergekit
|
||
|
|
- merge
|
||
|
|
- mistral
|
||
|
|
- nemo
|
||
|
|
- karcher_stock
|
||
|
|
---
|
||
|
|
|
||
|
|
# 🐟 Piranha 12B v1a
|
||
|
|
|
||
|
|
A prototype test for Savage Sands. This one is partially censored (despite having all uncensored components) but very creative. Therefore, the merge is "failed" because it should not have refusals. It was good enough to upload however. It recommended a "Piranha Tank", hence the name.
|
||
|
|
|
||
|
|
I tested with unablated base_model, and it seems this increased refusals a bit more.
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
architecture: MistralForCausalLM
|
||
|
|
base_model: B:/12B/MuXodious--Mistral-Nemo-Instruct-2407-absolute-heresy # mistralai--Mistral-Nemo-Instruct-2407
|
||
|
|
models:
|
||
|
|
- model: B:/12B/EldritchLabs--MN-12B-RP-Ink-Longform-MPOA
|
||
|
|
parameters:
|
||
|
|
weight: 1.0
|
||
|
|
- model: B:/12B/EldritchLabs--Human-Like-Mistral-Nemo-Instruct-2407-MPOA
|
||
|
|
parameters:
|
||
|
|
weight: 1.0
|
||
|
|
- model: B:/12B/MuXodious-Rocinante-X-12B-v1-absolute-heresy # TheDrummer--Rocinante-X-12B-v1
|
||
|
|
parameters:
|
||
|
|
weight: 1.0
|
||
|
|
merge_method: karcher_stock # v12B
|
||
|
|
parameters:
|
||
|
|
filter_wise: true
|
||
|
|
max_iter: 25
|
||
|
|
min_iter: 10
|
||
|
|
tol: 1.0e-9
|
||
|
|
magnitude_mean_geometric: false # TRUE makes it 75% karcher
|
||
|
|
dtype: float32
|
||
|
|
out_dtype: bfloat16
|
||
|
|
tokenizer:
|
||
|
|
source: union
|
||
|
|
chat_template: auto
|
||
|
|
name: karcher_stock_12b_v1
|
||
|
|
```
|