---
library_name: transformers
base_model:
- Qwen/Qwen3-1.7B-Base
---

# Qwen3 1.7B Smoltalk

Qwen3 1.7B just SFT'd on [smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk).

Trained using TRL with:

- 4x H100 from lambda labs
- epochs: 2 (i.e. 3,300 steps using our batch size)
- effective batch size: 128 (4 per gpu, 8 grad accumulation steps)
- warmup ratio: 0.03
- weight decay: 0.01
- learning rate: Forgot, will come back to add later.
- learning rate scheduler: cosine
- final training loss: 0.6432


|Benchmark|Score|
|---------|-----|
|AIME25|0%|
|GPQA|24.8%|
|GSM8K|54.2%|
|IFBench|18.3%|
|IFEval|55%|
|MMLU-Pro|22.8%|
|Multi-IF|32.5%|

## Quick Start

```python
from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model="lino-levan/qwen-3-1.7b-smoltalk",
)
messages = [
    {"role": "user", "content": "What is 3 * 8?"},
]
output = pipe(messages)
print(output[0]["generated_text"][-1]["content"])
```