ReasonLite-0.6B/README.md

license, datasets

license	datasets
openrail	amd/ReasonLite-Dataset

GitHub | Dataset | Blog

ReasonLite is an ultra-lightweight math reasoning model. With only 0.6B parameters, it leverages high-quality data distillation to achieve performance comparable to models over 10× its size, such as Qwen3-8B, reaching 75.2 on AIME24 and extending the scaling law of small models.

• 🔥 Best-performing 0.6B math reasoning model
• 🔓 Fully open-source — weights, scripts, datasets, synthesis pipeline
• ⚙️ Distilled in two stages to balance efficiency and high performance, using 6.1M high-quality samples.

---

🚀 Model

The model is trained in two progressive distillation stages. First, short-CoT data is used to distill Qwen3-0.6B into ReasonLite-0.6B-Turbo, improving AIME24 accuracy from 11.0 → 57.1. Then, long-CoT data is used to obtain ReasonLite-0.6B, further boosting accuracy to 75.2.

Model	Description	AIME24	Link
amd/ReasonLite-0.6B-Turbo	Short CoT balancing performance and efficiency	57.1	🤗 HuggingFace
amd/ReasonLite-0.6B	Long CoT for high performance	75.2	🤗 HuggingFace

---

📊 Evaluation Results

Metrics

• avg@16 — average accuracy from 16 sampled answers
• pass@8 — probability at least one correct answer appears among 8 samples

Model	Parameters	AMC23 avg@16	AMC23 pass@8	AIME25 avg@16	AIME25 pass@8	AIME24 avg@16	AIME24 pass@8
Qwen2.5-14B	14B	58.3	82.3	12.3	32.3	12.7	32.4
Deepseek-qwen-14B	14B	93.9	98.7	50.2	71.0	65.0	83.0
Qwen3-0.6B	0.6B	52.7	85.0	16.0	33.0	11.0	31.5
Qwen3-1.7B	1.7B	83.4	96.3	36.0	55.1	47.3	73.9
Qwen3-4B	4B	96.1	100	63.5	85.4	72.7	85.1
Qwen3-8B	8B	94.8	100	68.3	84.2	74.6	85.0
Qwen3-14B	14B	98.6	98.7	71.5	84.1	78.3	88.4
DeepscaleR-1.5B	1.5B	83.8	95.0	29.0	48.9	40.4	69.0
POLARIS-1.7B-Preview	1.7B	92.2	97.4	52.3	80.2	65.0	76.7
OpenMath-Nemotron-1.5B	1.5B	88.8	96.7	39.8	65.8	61.5	81.3
ReasonLite-0.6B-Turbo	0.6B	81.6	99.3	42.7	69.2	57.1	79.6
ReasonLite-0.6B	0.6B	95.2	100	62.9	84.1	75.2	90.2

---

📚 Dataset

We collected 343K math problems from Polaris and OpenMathReasoning. Using GPT-OSS as the teacher, we generated 9.1M raw answers under medium and high reasoning modes. We then produced pseudo-labels via majority voting, and finally retained 6.1M samples.

Dataset	Description	Size	Link
amd/ReasonLite-Dataset	Short CoT	4.3M	🤗 HuggingFace
amd/ReasonLite-Dataset	Long Cot	1.8M	🤗 HuggingFace

---

📌 Citation

@misc{reasonlite2025,
  title    = {ReasonLite: An Ultra-Lightweight 0.6B Reasoning Model},
  author   = {An, Zihao and Chen, Chushi and Liu, Ziqiong and Li, Dong and Barsoum, Emad},
  year     = {2025},
  url      = {https://github.com/AMD-AGI/ReasonLite},
  note     = {Open-source project}
}