56 lines
1.5 KiB
Markdown
56 lines
1.5 KiB
Markdown
|
---
|
||
|
|
base_model: meta-llama/Llama-3.2-3B-Instruct
|
||
|
|
license: llama3.2
|
||
|
|
tags:
|
||
|
|
- sft
|
||
|
|
- math
|
||
|
|
- thinking
|
||
|
|
- llama
|
||
|
|
---
|
||
|
|
|
||
|
|
# Llama-3.2-3B-ThinkSFT
|
||
|
|
|
||
|
|
`Llama-3.2-3B-Instruct` fine-tuned on 43.5K explicit reasoning traces
|
||
|
|
from [OpenThoughts-114k](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k)
|
||
|
|
(math subset, thinking format). No continual pre-training.
|
||
|
|
|
||
|
|
**Pipeline:** Base → Thinking SFT
|
||
|
|
|
||
|
|
Released as part of:
|
||
|
|
**[When Can LLMs Learn to Reason with Weak Supervision?](https://salmanrahman.net/rlvr-weak-supervision)**
|
||
|
|
— Rahman, Shen, Mordvina, Palangi, Gabriel, Izmailov (2026)
|
||
|
|
|
||
|
|
## Training Details
|
||
|
|
|
||
|
|
| | |
|
||
|
|
|---|---|
|
||
|
|
| Init | `meta-llama/Llama-3.2-3B-Instruct` |
|
||
|
|
| Data | [OpenThoughts-114k](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k) math subset (43.5K examples) |
|
||
|
|
| Epochs | 3 |
|
||
|
|
| Sequence length | 8,192 |
|
||
|
|
| Effective batch size | 256 sequences |
|
||
|
|
| Learning rate | 1.5e-5, cosine decay, 10% warmup |
|
||
|
|
| Optimizer | AdamW, weight decay 0.01 |
|
||
|
|
| Precision | BF16 + Flash Attention 2 |
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
|
||
|
|
model = AutoModelForCausalLM.from_pretrained("pavelslab-nyu/Llama-3.2-3B-ThinkSFT")
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained("pavelslab-nyu/Llama-3.2-3B-ThinkSFT")
|
||
|
|
```
|
||
|
|
|
||
|
|
## Citation
|
||
|
|
|
||
|
|
```bibtex
|
||
|
|
@article{rahman2026when,
|
||
|
|
title = {When Can LLMs Learn to Reason with Weak Supervision?},
|
||
|
|
author = {Rahman, Salman and Shen, Jingyan and Mordvina, Anna and
|
||
|
|
Palangi, Hamid and Gabriel, Saadia and Izmailov, Pavel},
|
||
|
|
journal = {Preprint},
|
||
|
|
year = {2026}
|
||
|
|
}
|
||
|
|
```
|