qwen3-1.7b-smoltalk/README.md

library_name, base_model

library_name	base_model
transformers	Qwen/Qwen3-1.7B-Base

Qwen3 1.7B Smoltalk

Qwen3 1.7B just SFT'd on smoltalk.

Trained using TRL with:

• 4x H100 from lambda labs
• epochs: 2 (i.e. 3,300 steps using our batch size)
• effective batch size: 128 (4 per gpu, 8 grad accumulation steps)
• warmup ratio: 0.03
• weight decay: 0.01
• learning rate: Forgot, will come back to add later.
• learning rate scheduler: cosine
• final training loss: 0.6432

Benchmark	Score
AIME25	0%
GPQA	24.8%
GSM8K	54.2%
IFBench	18.3%
IFEval	55%
MMLU-Pro	22.8%
Multi-IF	32.5%

Quick Start

from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model="lino-levan/qwen-3-1.7b-smoltalk",
)
messages = [
    {"role": "user", "content": "What is 3 * 8?"},
]
output = pipe(messages)
print(output[0]["generated_text"][-1]["content"])