---
base_model:
- Vortex5/Astral-Noctra-12B
- Vortex5/Azure-Starlight-12B
- Vortex5/Crimson-Constellation-12B
- Vortex5/Red-Synthesis-12B
- Vortex5/Shining-Seraph-12B
- Vortex5/Starlit-Shadow-12B
- Vortex5/Vermilion-Sage-12B
- Vortex5/Scarlet-Seraph-12B
- Vortex5/Maroon-Sunset-12B
- Vortex5/Amber-Starlight-12B
language:
- en
library_name: transformers
license: apache-2.0
tags:
- karcher
- merge
- mergekit
- mistral
- nemo
widget:
  - text: "Riemannian-Redshift-12B-v1"
    output:
      url: https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/-5OsIfrWlUxoJZP4U83uq.png
---

> [!NOTE]
> <span style="color:red; font-weight:bold">⚠️ Note:</span> This model requires **Mistral Tekken** chat template.
>

# 🌌 Riemannian Redshift 12B v1

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

![Redshift](https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/-5OsIfrWlUxoJZP4U83uq.png)

## Merge Details
### Merge Method

This is an experimental `karcher` merge of several high quality [Vortex5](https://huggingface.co/Vortex5) models. I used `float32` precision and `max_iter: 1000` to ensure the best bits were chosen for the Riemannian center. This merge took 5 hours using [graph_v18](https://huggingface.co/spaces/Naphula/model_tools/blob/main/graph_v18.py) as an accelerant with 8GB VRAM.

This model was merged using the [Karcher Mean](https://en.wikipedia.org/wiki/Karcher_mean) merge method.

### Models Merged

The following models were included in the merge:
* [Vortex5/Maroon-Sunset-12B](https://huggingface.co/Vortex5/Maroon-Sunset-12B)
* [Vortex5/Azure-Starlight-12B](https://huggingface.co/Vortex5/Azure-Starlight-12B)
* [Vortex5/Scarlet-Seraph-12B](https://huggingface.co/Vortex5/Scarlet-Seraph-12B)
* [Vortex5/Amber-Starlight-12B](https://huggingface.co/Vortex5/Amber-Starlight-12B)
* [Vortex5/Shining-Seraph-12B](https://huggingface.co/Vortex5/Shining-Seraph-12B)
* [Vortex5/Red-Synthesis-12B](https://huggingface.co/Vortex5/Red-Synthesis-12B)
* [Vortex5/Starlit-Shadow-12B](https://huggingface.co/Vortex5/Starlit-Shadow-12B)
* [Vortex5/Crimson-Constellation-12B](https://huggingface.co/Vortex5/Crimson-Constellation-12B)
* [Vortex5/Vermilion-Sage-12B](https://huggingface.co/Vortex5/Vermilion-Sage-12B)
* [Vortex5/Astral-Noctra-12B](https://huggingface.co/Vortex5/Astral-Noctra-12B)

### Configuration

The following YAML configuration was used to produce this model:

```yaml
models:
  - model: B:/12B/models--Vortex5--Astral-Noctra-12B
  - model: B:/12B/models--Vortex5--Azure-Starlight-12B
  - model: B:/12B/models--Vortex5--Crimson-Constellation-12B
  - model: B:/12B/models--Vortex5--Red-Synthesis-12B
  - model: B:/12B/models--Vortex5--Shining-Seraph-12B
  - model: B:/12B/models--Vortex5--Starlit-Shadow-12B
  - model: B:/12B/models--Vortex5--Vermilion-Sage-12B
  - model: B:/12B/models--Vortex5--Scarlet-Seraph-12B
  - model: B:/12B/models--Vortex5--Maroon-Sunset-12B
  - model: B:/12B/models--Vortex5--Amber-Starlight-12B
merge_method: karcher
parameters:
  max_iter: 1000
  tol: 1.0e-9
dtype: float32
out_dtype: bfloat16
tokenizer:  
  source: union
chat_template: auto
name: 🌌 Riemannian-Redshift-12B-v1
```