57 lines
2.0 KiB
Markdown
57 lines
2.0 KiB
Markdown
---
|
|
base_model:
|
|
- mistralai/Mistral-Nemo-Instruct-2407
|
|
- LatitudeGames/Wayfarer-2-12B
|
|
- WokeAI/Tankie-DPE-12B-SFT-v2
|
|
library_name: transformers
|
|
tags:
|
|
- mergekit
|
|
- merge
|
|
- mistral
|
|
- nemo
|
|
- nuslerp
|
|
widget:
|
|
- text: "Protocol-Phantom-12B"
|
|
output:
|
|
url: https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/3kDEO5re1fpIasFVmJeeK.png
|
|
---
|
|
|
|
> [!CAUTION]
|
|
> <span style="color:red; font-weight:bold">⚠️ Warning:</span> This model can produce narratives and RP that contain violent and graphic erotic content. Adjust your system prompt accordingly, and use **ChatML** chat template.
|
|
>
|
|
|
|
# 🧞 Protocol Phantom 12B
|
|
|
|
|
|
|
|
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
|
|
|
This model was merged using the [nuslerp](https://github.com/arcee-ai/mergekit/blob/main/docs/merge_methods.md) method using **mistralai/Mistral-Nemo-Instruct-2407** as a base.
|
|
|
|
**Protocol Phantom** has some refusals and may require jailbreaks or ablation to fully uncensor.
|
|
|
|
Several methods were tested including `karcher` and `della`, but they produced broken output. Models such as Impish Bloodmoon, SakuraKaze, Irix, VelvetCafe, Arsenic Shahrazad, and others had to be omitted from the merge.
|
|
|
|
### Configuration
|
|
|
|
The following YAML configuration was used to produce this model:
|
|
|
|
```yaml
|
|
architecture: MistralForCausalLM
|
|
base_model: B:/12B/mistralai--Mistral-Nemo-Instruct-2407
|
|
models:
|
|
- model: B:/12B/LatitudeGames--Wayfarer-2-12B
|
|
parameters:
|
|
weight: 0.5
|
|
- model: B:/12B/WokeAI--Tankie-DPE-12b-SFT-v2
|
|
parameters:
|
|
weight: 0.5
|
|
merge_method: nuslerp
|
|
parameters:
|
|
nuslerp_flatten: false # Flattens tensors to treat them as high-dimensional vectors
|
|
nuslerp_row_wise: true # Set to true if you want to interpolate per-row instead of per-tensor
|
|
dtype: float32
|
|
out_dtype: bfloat16
|
|
tokenizer:
|
|
source: union
|
|
``` |