39 lines
706 B
Markdown
39 lines
706 B
Markdown
---
|
|
base_model:
|
|
- SvalTek/SOR-ColdBrew-12B-Base-Testing
|
|
library_name: transformers
|
|
tags:
|
|
- mergekit
|
|
- merge
|
|
---
|
|
# merge
|
|
|
|
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
|
|
|
### Configuration
|
|
|
|
The following YAML configuration was used to produce this model:
|
|
|
|
```yaml
|
|
|
|
name: SOR-ColdBrew-12B-Base-Test3
|
|
|
|
models:
|
|
- model: TheDrunkenSnail/Son-of-Rhodia
|
|
parameters:
|
|
weight: 0.4
|
|
- model: SvalTek/SOR-ColdBrew-12B-Base-Testing
|
|
parameters:
|
|
weight: 0.6
|
|
base_model: redrix/GodSlayer-12B-ABYSS
|
|
merge_method: nuslerp
|
|
dtype: bfloat16
|
|
chat_template: "chatml"
|
|
tokenizer:
|
|
source: union
|
|
parameters:
|
|
normalize: true
|
|
int8_mask: true
|
|
|
|
|
|
``` |