124 lines
3.2 KiB
Markdown
124 lines
3.2 KiB
Markdown
|
---
|
||
|
|
license: llama3.1
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
library_name: transformers
|
||
|
|
inference: true
|
||
|
|
base_model:
|
||
|
|
- NousResearch/Hermes-3-Llama-3.1-8B
|
||
|
|
- NousResearch/DeepHermes-3-Llama-3-8B-Preview
|
||
|
|
- nvidia/Llama-3.1-Nemotron-Nano-8B-v1
|
||
|
|
- deepseek-ai/DeepSeek-R1-Distill-Llama-8B
|
||
|
|
tags:
|
||
|
|
- merge
|
||
|
|
- dare_ties
|
||
|
|
- llama
|
||
|
|
- llama-3.1
|
||
|
|
- mist
|
||
|
|
---
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
# MIST-1-8B
|
||
|
|
|
||
|
|
MIST-1-8B (formerly MIST-Mini) is the smallest and fastest model in the **MIST model family** by [olaverse](https://huggingface.co/olaverse). Built by blending 4 specialized Llama 3.1 8B models using DARE+TIES — delivering strong performance at maximum speed.
|
||
|
|
fast, thorough, great for everyday use
|
||
|
|
|
||
|
|
## MIST Model Family
|
||
|
|
|
||
|
|
| Model | Params | Speed | Status |
|
||
|
|
|---|---|---|---|
|
||
|
|
| **MIST-1-8B** | 8B | ~63 tok/s | ✅ Available |
|
||
|
|
| [MIST-1-70B](https://huggingface.co/olaverse/MIST-1-70B) | 70B | ~23 tok/s | ✅ Available |
|
||
|
|
| [MIST-1-140B](https://huggingface.co/olaverse/MIST-1-140B) | 140B | ~8 tok/s | ✅ Available |
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Key Strengths
|
||
|
|
|
||
|
|
- ⚡ **Fastest** — 63 tok/s on H200, great for real-time applications
|
||
|
|
- 🧠 **Strong Reasoning** — DeepSeek R1 distillation
|
||
|
|
- 💻 **Clean Code** — production-ready with comments
|
||
|
|
- 📐 **Math** — accurate step-by-step solving
|
||
|
|
- 🤝 **Helpful** — low refusal rate
|
||
|
|
- 📦 **Lightweight** — 15GB, runs on consumer GPUs
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Benchmark Results
|
||
|
|
|
||
|
|
| Task | Speed | Quality |
|
||
|
|
|---|---|---|
|
||
|
|
| Reasoning | 4.5s | ✅ Correct |
|
||
|
|
| Coding | 4.0s | ✅ Clean code |
|
||
|
|
| Math | 4.0s | ✅ Step-by-step |
|
||
|
|
| General | 4.0s | ✅ Accurate |
|
||
|
|
| Instruction | 4.0s | ✅ Precise |
|
||
|
|
|
||
|
|
**Average: 63 tok/s**
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## How to Use
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
||
|
|
"olaverse/MIST-Mini-8B",
|
||
|
|
torch_dtype="auto",
|
||
|
|
device_map="auto",
|
||
|
|
)
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained("olaverse/MIST-Mini-8B")
|
||
|
|
|
||
|
|
messages = [{"role": "user", "content": "Your question here"}]
|
||
|
|
text = tokenizer.apply_chat_template(
|
||
|
|
messages, tokenize=False, add_generation_prompt=True
|
||
|
|
)
|
||
|
|
inputs = tokenizer(text, return_tensors="pt").to("cuda")
|
||
|
|
outputs = model.generate(**inputs, max_new_tokens=512)
|
||
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
|
|
```
|
||
|
|
|
||
|
|
## Hardware Requirements
|
||
|
|
|
||
|
|
| Precision | VRAM Required |
|
||
|
|
|---|---|
|
||
|
|
| bfloat16 | 16GB (RTX 3090/4090) |
|
||
|
|
| 4-bit | 6GB (RTX 3060+) |
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Recommended Generation Settings
|
||
|
|
|
||
|
|
These settings were verified through testing. Without `repetition_penalty`
|
||
|
|
and `min_p` the model will ramble and not stop cleanly.
|
||
|
|
|
||
|
|
```python
|
||
|
|
outputs = model.generate(
|
||
|
|
**inputs,
|
||
|
|
max_new_tokens=1024,
|
||
|
|
do_sample=True,
|
||
|
|
temperature=0.7,
|
||
|
|
top_p=0.95,
|
||
|
|
min_p=0.05,
|
||
|
|
repetition_penalty=1.5,
|
||
|
|
eos_token_id=[128040, 128009, 128001],
|
||
|
|
pad_token_id=128001,
|
||
|
|
)
|
||
|
|
```
|
||
|
|
|
||
|
|
### Stop Tokens
|
||
|
|
This model's ChatML parents (`<|im_end|>`) survived the DARE+TIES merge
|
||
|
|
alongside Llama 3.1 native tokens. Use all three:
|
||
|
|
|
||
|
|
| Token | ID | Source |
|
||
|
|
|---|---|---|
|
||
|
|
| `<\|im_end\|>` | 128040 | Hermes/Nemotron parents |
|
||
|
|
| `<\|eot_id\|>` | 128009 | Llama 3.1 native |
|
||
|
|
| `<\|end_of_text\|>` | 128001 | Llama 3.1 native |
|
||
|
|
|
||
|
|
## License
|
||
|
|
|
||
|
|
[Llama 3.1 Community License](https://llama.meta.com/llama3/license/)
|