sable-mini-30m-preview/README.md

language, license, tags, pipeline_tag

language	license	tags	pipeline_tag
en	apache-2.0	causal-lm llama sable small-language-model	text-generation

🖤 Sable-Mini-30M (Preview)

Sable-Mini is a 30M parameter causal language model trained on approximately 3–4B tokens. It is the flagship of the Sable tiny model family, balancing compact size with capable text generation.

⚠️ This is a preview release — expect improvements in future versions.

📊 Benchmark Results

Evaluated with the LM Evaluation Harness:

Benchmark	Accuracy	Metric
BoolQ	57.5%	acc
PIQA	57.8%	acc_norm
WinoGrande	53.3%	acc
ARC-Easy	39.0%	acc_norm
HellaSwag	28.1%	acc_norm
Lambada	11.7%	acc

🚀 Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("yourorg/sable-mini-30m-preview")
model = AutoModelForCausalLM.from_pretrained("yourorg/sable-mini-30m-preview")

inputs = tokenizer("The future of tiny models is", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

🏗️ Model Details

Property	Value
Parameters	30M
Architecture	Llama-style decoder
Training tokens	~3–4B
Context length	← add yours
Vocab size	← add yours

⚠️ Limitations

• Preview release — not recommended for production use
• Limited knowledge due to small parameter count
• May produce biased or incorrect outputs

📜 License

Apache 2.0 — free for commercial and research use.

---

Part of the Sable family 🖤 — tiny models, sharp minds.