初始化项目,由ModelHub XC社区提供模型
Model: Naphula/Riemannian-Redshift-12B-v1 Source: Original Platform
This commit is contained in:
86
README.md
Normal file
86
README.md
Normal file
@@ -0,0 +1,86 @@
|
||||
---
|
||||
base_model:
|
||||
- Vortex5/Astral-Noctra-12B
|
||||
- Vortex5/Azure-Starlight-12B
|
||||
- Vortex5/Crimson-Constellation-12B
|
||||
- Vortex5/Red-Synthesis-12B
|
||||
- Vortex5/Shining-Seraph-12B
|
||||
- Vortex5/Starlit-Shadow-12B
|
||||
- Vortex5/Vermilion-Sage-12B
|
||||
- Vortex5/Scarlet-Seraph-12B
|
||||
- Vortex5/Maroon-Sunset-12B
|
||||
- Vortex5/Amber-Starlight-12B
|
||||
language:
|
||||
- en
|
||||
library_name: transformers
|
||||
license: apache-2.0
|
||||
tags:
|
||||
- karcher
|
||||
- merge
|
||||
- mergekit
|
||||
- mistral
|
||||
- nemo
|
||||
widget:
|
||||
- text: "Riemannian-Redshift-12B-v1"
|
||||
output:
|
||||
url: https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/-5OsIfrWlUxoJZP4U83uq.png
|
||||
---
|
||||
|
||||
> [!NOTE]
|
||||
> <span style="color:red; font-weight:bold">⚠️ Note:</span> This model requires **Mistral Tekken** chat template.
|
||||
>
|
||||
|
||||
# 🌌 Riemannian Redshift 12B v1
|
||||
|
||||
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
||||
|
||||

|
||||
|
||||
## Merge Details
|
||||
### Merge Method
|
||||
|
||||
This is an experimental `karcher` merge of several high quality [Vortex5](https://huggingface.co/Vortex5) models. I used `float32` precision and `max_iter: 1000` to ensure the best bits were chosen for the Riemannian center. This merge took 5 hours using [graph_v18](https://huggingface.co/spaces/Naphula/model_tools/blob/main/graph_v18.py) as an accelerant with 8GB VRAM.
|
||||
|
||||
This model was merged using the [Karcher Mean](https://en.wikipedia.org/wiki/Karcher_mean) merge method.
|
||||
|
||||
### Models Merged
|
||||
|
||||
The following models were included in the merge:
|
||||
* [Vortex5/Maroon-Sunset-12B](https://huggingface.co/Vortex5/Maroon-Sunset-12B)
|
||||
* [Vortex5/Azure-Starlight-12B](https://huggingface.co/Vortex5/Azure-Starlight-12B)
|
||||
* [Vortex5/Scarlet-Seraph-12B](https://huggingface.co/Vortex5/Scarlet-Seraph-12B)
|
||||
* [Vortex5/Amber-Starlight-12B](https://huggingface.co/Vortex5/Amber-Starlight-12B)
|
||||
* [Vortex5/Shining-Seraph-12B](https://huggingface.co/Vortex5/Shining-Seraph-12B)
|
||||
* [Vortex5/Red-Synthesis-12B](https://huggingface.co/Vortex5/Red-Synthesis-12B)
|
||||
* [Vortex5/Starlit-Shadow-12B](https://huggingface.co/Vortex5/Starlit-Shadow-12B)
|
||||
* [Vortex5/Crimson-Constellation-12B](https://huggingface.co/Vortex5/Crimson-Constellation-12B)
|
||||
* [Vortex5/Vermilion-Sage-12B](https://huggingface.co/Vortex5/Vermilion-Sage-12B)
|
||||
* [Vortex5/Astral-Noctra-12B](https://huggingface.co/Vortex5/Astral-Noctra-12B)
|
||||
|
||||
### Configuration
|
||||
|
||||
The following YAML configuration was used to produce this model:
|
||||
|
||||
```yaml
|
||||
models:
|
||||
- model: B:/12B/models--Vortex5--Astral-Noctra-12B
|
||||
- model: B:/12B/models--Vortex5--Azure-Starlight-12B
|
||||
- model: B:/12B/models--Vortex5--Crimson-Constellation-12B
|
||||
- model: B:/12B/models--Vortex5--Red-Synthesis-12B
|
||||
- model: B:/12B/models--Vortex5--Shining-Seraph-12B
|
||||
- model: B:/12B/models--Vortex5--Starlit-Shadow-12B
|
||||
- model: B:/12B/models--Vortex5--Vermilion-Sage-12B
|
||||
- model: B:/12B/models--Vortex5--Scarlet-Seraph-12B
|
||||
- model: B:/12B/models--Vortex5--Maroon-Sunset-12B
|
||||
- model: B:/12B/models--Vortex5--Amber-Starlight-12B
|
||||
merge_method: karcher
|
||||
parameters:
|
||||
max_iter: 1000
|
||||
tol: 1.0e-9
|
||||
dtype: float32
|
||||
out_dtype: bfloat16
|
||||
tokenizer:
|
||||
source: union
|
||||
chat_template: auto
|
||||
name: 🌌 Riemannian-Redshift-12B-v1
|
||||
```
|
||||
Reference in New Issue
Block a user