---
license: apache-2.0
base_model:
- OccultAI/Morpheus-8B-v1
- SicariusSicariiStuff/Assistant_Pepe_8B
tags:
- merge
- llama
- occult
- uncensored
- moe
datasets:
- OccultAI/Morpheus_665
- SicariusSicariiStuff/UBW_Tapestries
language:
- en
library_name: transformers
widget:
  - text: "Meme-Trix MoE 14B A8B v1"
    output:
      url: https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/hEyahuR43IMF5TC4XrHGS.png
new_version: Naphula/Meme-Trix-MoE-14B-A8B-v2
---

> [!CAUTION]
> <span style="color:red; font-weight:bold">⚠️ Warning:</span> This model can produce narratives and RP that contain violent and graphic erotic content. Adjust your system prompt accordingly, and use **Llama 3** chat template.
>

# Meme-Trix MoE 14B A8B v1

![Meme-Trix](https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/hEyahuR43IMF5TC4XrHGS.png)

A custom built Llama 3.1 8B MoE (Mixture of Experts) merge which combines Morpheus v1 with Assistant Pepe. The merge is suprisingly intelligent, detailed, and based. Scores ~15K at Q0 Bench. Fully uncensored and almost as fast as 8B dense. It also appears to have strong context retrieval ability. Asked for a summary at 16K and it works flawlessly (did not test higher yet).

If you want to merge custom Llama MoE you can add these scripts to your mergekit environment:
- [mergekit-main\mergekit\architecture\moe_defs.py](https://huggingface.co/Naphula/Meme-Trix-MoE-14B-A8B-v1/blob/main/moe_defs.py)
- [mergekit-main\mergekit\moe\__init__.py](https://huggingface.co/Naphula/Meme-Trix-MoE-14B-A8B-v1/blob/main/__init__.py)
- [mergekit-main\mergekit\moe\llama.py](https://huggingface.co/Naphula/Meme-Trix-MoE-14B-A8B-v1/blob/main/llama.py)

Then assign the `num_experts_per_tok` in `config.json` (or the `config.yaml`)

## Recommended Settings
**(bolded kobold non-defaults)**
- **Temp 1.0**
- **TopNSigma 1.25**
- **Min-P 0.1**
- **Repetition Penalty 1.08**
- **Top-P 1.0**
- Top-K 100
- Top-A 0
- Typical Sampling 1
- Tail-Free Sampling 1
- Presence Penalty 0
- Sampler Seed -1
- Rp.Range 360
- Rp.Slope 0.7
- Smoothing Factor 0
- Smoothing Curve 1
- DynaTemp 0
- Mirostat Mode OFF ("2" enhances creativity but also errors)
- Mirostat Tau 5
- Mirostat Eta 0.1
- **DRY Multiplier 0.8**
- DRY Base 1.75
- DRY A.Len 2
- DRY L.Len 320
- **XTC Threshold 0.1**
- **XTC Probability 0.08** (The "Anti-Cliche" Shield)
- **DynaTemp ON** (The "Poor Man's Fading Mirostat")
- **Minimum Temperature 0.65**
- **Maximum Temperature 1.35**
- **Temperature 1.0**
- **DynaTemp-Range 0.35**
- **DynaTemp-Exponent 1**