base_model, language, library_name, license, tags, widget
| base_model | language | library_name | license | tags | widget | |||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
transformers | apache-2.0 |
|
|
Note
⚠️ Note: This model requires Mistral Tekken chat template.
🌌 Riemannian Redshift 12B v1
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This is an experimental karcher merge of several high quality Vortex5 models. I used float32 precision and max_iter: 1000 to ensure the best bits were chosen for the Riemannian center. This merge took 5 hours using graph_v18 as an accelerant with 8GB VRAM.
This model was merged using the Karcher Mean merge method.
Models Merged
The following models were included in the merge:
- Vortex5/Maroon-Sunset-12B
- Vortex5/Azure-Starlight-12B
- Vortex5/Scarlet-Seraph-12B
- Vortex5/Amber-Starlight-12B
- Vortex5/Shining-Seraph-12B
- Vortex5/Red-Synthesis-12B
- Vortex5/Starlit-Shadow-12B
- Vortex5/Crimson-Constellation-12B
- Vortex5/Vermilion-Sage-12B
- Vortex5/Astral-Noctra-12B
Configuration
The following YAML configuration was used to produce this model:
models:
- model: B:/12B/models--Vortex5--Astral-Noctra-12B
- model: B:/12B/models--Vortex5--Azure-Starlight-12B
- model: B:/12B/models--Vortex5--Crimson-Constellation-12B
- model: B:/12B/models--Vortex5--Red-Synthesis-12B
- model: B:/12B/models--Vortex5--Shining-Seraph-12B
- model: B:/12B/models--Vortex5--Starlit-Shadow-12B
- model: B:/12B/models--Vortex5--Vermilion-Sage-12B
- model: B:/12B/models--Vortex5--Scarlet-Seraph-12B
- model: B:/12B/models--Vortex5--Maroon-Sunset-12B
- model: B:/12B/models--Vortex5--Amber-Starlight-12B
merge_method: karcher
parameters:
max_iter: 1000
tol: 1.0e-9
dtype: float32
out_dtype: bfloat16
tokenizer:
source: union
chat_template: auto
name: 🌌 Riemannian-Redshift-12B-v1