---
language:
- en
license: apache-2.0
tags:
- causal-lm
- llama
- sable
- small-language-model
pipeline_tag: text-generation
---

# 🖤 Sable-Mini-30M (Preview)

**Sable-Mini** is a 30M parameter causal language model trained on approximately **3–4B tokens**. It is the flagship of the Sable tiny model family, balancing compact size with capable text generation.

> ⚠️ **This is a preview release** — expect improvements in future versions.

## 📊 Benchmark Results

Evaluated with the [LM Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness):

| Benchmark | Accuracy | Metric |
| :--- | :---: | :---: |
| **BoolQ** | **57.5%** | `acc` |
| **PIQA** | **57.8%** | `acc_norm` |
| **WinoGrande** | **53.3%** | `acc` |
| **ARC-Easy** | **39.0%** | `acc_norm` |
| **HellaSwag** | **28.1%** | `acc_norm` |
| **Lambada** | **11.7%** | `acc` |

![Benchmark Graph](benchmark.png)

## 🚀 Quick Start

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("yourorg/sable-mini-30m-preview")
model = AutoModelForCausalLM.from_pretrained("yourorg/sable-mini-30m-preview")

inputs = tokenizer("The future of tiny models is", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## 🏗️ Model Details

| Property | Value |
|----------|-------|
| Parameters | 30M |
| Architecture | Llama-style decoder |
| Training tokens | ~3–4B |
| Context length | ← add yours |
| Vocab size | ← add yours |

## ⚠️ Limitations

- Preview release — not recommended for production use
- Limited knowledge due to small parameter count
- May produce biased or incorrect outputs

## 📜 License

Apache 2.0 — free for commercial and research use.

---

*Part of the Sable family 🖤 — tiny models, sharp minds.*