---
base_model:
- MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy
- Retreatcost/Evertide-RX-12B
- yamatazen/Aurora-SCE-12B
- Vortex5/Elysian-Sunrise-12B
- Vortex5/Mythic-Fabulist-12B
- Vortex5/Prototype-X-12b
datasets:
- OccultAI/illuminati_imatrix_v1
language:
- en
library_name: transformers
license: apache-2.0
tags:
- merge
- mergekit
- creative
- roleplay
- creative-writing
- creative-roleplay
widget:
  - text: "Helix-SCE-12B"
    output:
      url: https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/-kNBE8W7i32WkLJIIp60i.jpeg
---

# 🧬 Helix SCE 12B

![helix-sce](https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/-kNBE8W7i32WkLJIIp60i.jpeg)

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

## Merge Details
### Merge Method

This model was merged using the [SCE](https://arxiv.org/abs/2408.07990) merge method using MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy as a base.

### Models Merged

The following models were included in the merge:
- MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy
- Retreatcost/Evertide-RX-12B
- yamatazen/Aurora-SCE-12B
- Vortex5/Elysian-Sunrise-12B
- Vortex5/Mythic-Fabulist-12B
- Vortex5/Prototype-X-12b

### Configuration

The following YAML configuration was used to produce this model:

```yaml
architecture: MistralForCausalLM
base_model: B:\12B\MuXodious--Mistral-Nemo-Instruct-2407-absolute-heresy
models:
  - model: B:\12B\Retreatcost--Evertide-RX-12B
  - model: B:\12B\yamatazen--Aurora-SCE-12B
  - model: B:\12B\Vortex5--Elysian-Sunrise-12B
  - model: B:\12B\Vortex5--Mythic-Fabulist-12B
  - model: B:\12B\Vortex5--Prototype-X-12b
merge_method: sce
dtype: float32
out_dtype: bfloat16
parameters:
  normalize: true
  select_topk: 0.5
tokenizer:
  source: union
chat_template: auto
```