98 lines
2.7 KiB
Markdown
98 lines
2.7 KiB
Markdown
|
---
|
||
|
|
language:
|
||
|
|
- zh
|
||
|
|
- en
|
||
|
|
license: apache-2.0
|
||
|
|
base_model: Qwen/Qwen3-0.6B-Base
|
||
|
|
library_name: transformers
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
tags:
|
||
|
|
- qwen3
|
||
|
|
- qwen3-0.6b
|
||
|
|
- sft
|
||
|
|
- trl
|
||
|
|
- smoltalk2
|
||
|
|
- instruction-tuned
|
||
|
|
datasets:
|
||
|
|
- HuggingFaceTB/smoltalk2
|
||
|
|
---
|
||
|
|
|
||
|
|
# Qwen3-0.6B SFT on SmolTalk2
|
||
|
|
|
||
|
|
This model is a supervised fine-tuned version of `Qwen/Qwen3-0.6B-Base`.
|
||
|
|
It was trained with TRL SFT on selected no-think splits from `HuggingFaceTB/smoltalk2`.
|
||
|
|
|
||
|
|
The chat template was copied from the Qwen3 Instruct tokenizer so the model can be used with the standard chat-message interface.
|
||
|
|
|
||
|
|
## Training
|
||
|
|
|
||
|
|
- Base model: `Qwen/Qwen3-0.6B-Base`
|
||
|
|
- Method: full-parameter SFT
|
||
|
|
- Framework: TRL / Transformers
|
||
|
|
- Dataset: `HuggingFaceTB/smoltalk2`, config `SFT`
|
||
|
|
- Splits:
|
||
|
|
- `smoltalk_smollm3_smol_magpie_ultra_no_think`
|
||
|
|
- `OpenHermes_2.5_no_think`
|
||
|
|
- `tulu_3_sft_personas_instruction_following_no_think`
|
||
|
|
- `smoltalk_smollm3_systemchats_30k_no_think`
|
||
|
|
- Max length: 2048
|
||
|
|
- Steps: 5000
|
||
|
|
- Learning rate: 2e-5, cosine schedule, warmup ratio 0.03
|
||
|
|
- Effective batch size: 64
|
||
|
|
- Gradient clipping: 1.0
|
||
|
|
|
||
|
|
## Evaluation
|
||
|
|
|
||
|
|
The table below compares the official Qwen3-0.6B Instruct model against the official Base model using `lm-eval`.
|
||
|
|
|
||
|
|
| Task | Base | Instruct | Delta |
|
||
|
|
| ---- | ---: | -------: | ----: |
|
||
|
|
| `ifeval` | 16.27% | 43.62% | +27.36pp |
|
||
|
|
| `truthfulqa_mc2` | 45.98% | 42.84% | -3.15pp |
|
||
|
|
| `mmlu` | 50.46% | 40.26% | -10.20pp |
|
||
|
|
| `arc_challenge` | 38.65% | 34.13% | -4.52pp |
|
||
|
|
| `hellaswag` | 53.67% | 47.29% | -6.38pp |
|
||
|
|
| `winogrande` | 58.56% | 56.35% | -2.21pp |
|
||
|
|
| `gsm8k_cot` | 55.50% | 4.55% | -50.95pp |
|
||
|
|
|
||
|
|
The locally trained SFT checkpoint used as the DPO reference scored as follows:
|
||
|
|
|
||
|
|
| Task | SFT score |
|
||
|
|
| ---- | --------: |
|
||
|
|
| `ifeval` | 39.19% |
|
||
|
|
| `truthfulqa_mc2` | 46.52% |
|
||
|
|
| `mmlu` | 49.35% |
|
||
|
|
| `arc_challenge` | 37.03% |
|
||
|
|
| `hellaswag` | 53.36% |
|
||
|
|
| `winogrande` | 57.93% |
|
||
|
|
| `gsm8k_cot` | 48.37% |
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
|
||
|
|
model_id = "Lixiaojian123/qwen3-0.6b-sft"
|
||
|
|
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
|
||
|
|
|
||
|
|
messages = [
|
||
|
|
{"role": "user", "content": "用三句话解释什么是监督微调。"},
|
||
|
|
]
|
||
|
|
|
||
|
|
inputs = tokenizer.apply_chat_template(
|
||
|
|
messages,
|
||
|
|
tokenize=True,
|
||
|
|
add_generation_prompt=True,
|
||
|
|
return_tensors="pt",
|
||
|
|
).to(model.device)
|
||
|
|
|
||
|
|
outputs = model.generate(inputs, max_new_tokens=256)
|
||
|
|
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
|
||
|
|
```
|
||
|
|
|
||
|
|
## Limitations
|
||
|
|
|
||
|
|
This is a small 0.6B parameter model. It may hallucinate, make reasoning mistakes, and fail on complex instructions. The model was trained primarily on no-think instruction data, so it is not intended to expose long chain-of-thought traces.
|