136 lines
4.0 KiB
Markdown
136 lines
4.0 KiB
Markdown
---
|
||
language:
|
||
- en
|
||
- zh
|
||
license: apache-2.0
|
||
library_name: transformers
|
||
pipeline_tag: text-generation
|
||
tags:
|
||
- qwen3
|
||
- causal-lm
|
||
- supervised-fine-tuning
|
||
- math
|
||
- reasoning
|
||
- code
|
||
- science
|
||
base_model: Qwen/Qwen3-4B-Base
|
||
model-index:
|
||
- name: Qwen3-4B-SFT
|
||
results:
|
||
- task:
|
||
type: text-generation
|
||
dataset:
|
||
name: AIME 2024
|
||
type: aime2024
|
||
metrics:
|
||
- name: accuracy
|
||
type: accuracy
|
||
value: 20.8
|
||
- task:
|
||
type: text-generation
|
||
dataset:
|
||
name: AIME 2025
|
||
type: aime2025
|
||
metrics:
|
||
- name: accuracy
|
||
type: accuracy
|
||
value: 19.4
|
||
- task:
|
||
type: text-generation
|
||
dataset:
|
||
name: AMC 2023
|
||
type: amc2023
|
||
metrics:
|
||
- name: accuracy
|
||
type: accuracy
|
||
value: 58.0
|
||
- task:
|
||
type: text-generation
|
||
dataset:
|
||
name: GPQA-Diamond
|
||
type: gpqa_diamond
|
||
metrics:
|
||
- name: accuracy
|
||
type: accuracy
|
||
value: 29.1
|
||
---
|
||
|
||
## Qwen3-4B-SFT:
|
||
|
||
Qwen3-4B-SFT is a reasoning-focused model derived from Qwen3-4B-Base via full-parameter fine-tuning on the verl framework.
|
||
|
||
There is a notable shortage of reproducible 'warm-start' SFT bases in open-source practice, this model bridges the gap between base models and reinforcement learning models. Optimally aligned for Chain-of-Thought (CoT) and instruction following, it serves as a robust warm-start for Reinforcement Learning.
|
||
|
||
| Dataset | Base (4B)† | Qwen3-4B-SFT (this model) | Improvement |
|
||
| :--- | :---: | :---: | :---: |
|
||
| **AIME 2024** | 11.25% | **20.8%** | +9.55% |
|
||
| **AIME 2025** | 6.46% | **19.4%** | +12.94% |
|
||
| **AMC 2023** | 31.09% | **58.0%** | +26.91% |
|
||
| **GPQA-Diamond** | 7.77% | **29.1%** | +21.33% |
|
||
|
||
† Base (4B) figures are taken from [ (arXiv:2602.10885)](https://arxiv.org/pdf/2602.10885).
|
||
|
||
- Dataset card used for SFT: https://huggingface.co/datasets/96kevinli29/SFT-Dataset
|
||
|
||
## Qwen3-style reasoning and instruction following
|
||
|
||
Minimal pattern (illustrative):
|
||
|
||
```text
|
||
<|im_start|>user
|
||
… Among options A–D, which is correct? Reason step by step and put the final letter in \boxed{}.
|
||
<|im_end|>
|
||
|
||
<|im_start|>assistant
|
||
<think>
|
||
Compare A vs B vs C vs D against the stem; eliminate …; D remains consistent with …
|
||
</think>
|
||
Step-by-step: … (short derivation in the visible channel)
|
||
Final answer: \boxed{D}
|
||
<|im_end|>
|
||
```
|
||
Use a large enough **`max_new_tokens`** on hard math so both the **reasoning block** and the **visible** `\boxed{…}` line fit before generation stops.
|
||
|
||
## Configuration Notes
|
||
|
||
- Template: Trained with the **Qwen chat template**; learns to end responses with `<|im_end|>` (151645).
|
||
- Suggested Configuration:
|
||
```json
|
||
{
|
||
"eos_token_id": 151645
|
||
}
|
||
```
|
||
You may adjust settings according to your training or deployment needs.
|
||
|
||
## Training Infrastructure
|
||
|
||
- Cluster: MeluXina Supercomputer (LuxProvide)
|
||
- Node Config: 4 NVIDIA-A100 GPUs per node.
|
||
- Final SFT Run: 12 Node-hours (16× A100 for 3 hours)
|
||
- Total R&D Investment: ~700 Node-hours (Includes data ablation, hyperparameter sweeps, and extensive benchmark evaluation.)
|
||
|
||
## Project Links
|
||
|
||
- Training code repository: https://github.com/96kevinli29/base-model-sft-verl
|
||
|
||
## Limitations
|
||
|
||
- Not optimized for factual correctness in all domains
|
||
- May still produce hallucinations or unsafe outputs
|
||
- Performance is sensitive to prompt style and decoding settings
|
||
|
||
## Citation
|
||
|
||
If you use this model, please cite **this checkpoint**, bibTeX for this release :
|
||
|
||
```bibtex
|
||
@misc{qwen3-4b-sft-2026,
|
||
title = {{Qwen3-4B-SFT}: Supervised Fine-Tuned {Qwen3}-4B for Reasoning},
|
||
author = {Hongyang Li, Xiao Li and {Sea-Fill Community}},
|
||
year = {2026},
|
||
publisher = {Hugging Face},
|
||
howpublished = {\url{https://huggingface.co/SeaFill2025/Qwen3-4B-SFT}},
|
||
note = {Checkpoint trained with verl; warm-start for pre-RL alignment research. Maintained by Sea-Fill Community.}
|
||
}
|
||
```
|