qwen3_30b_a3b_to_4b_onpolicy_5k_src30k-35k_cont/README.md

license, library_name, pipeline_tag, language, base_model, datasets, tags

license	library_name	pipeline_tag	language	base_model	datasets	tags
apache-2.0	transformers	text-generation	en	RockToken/qwen3-30b-a3b-to-4b-onpolicy-10k	RockToken/openthoughts_prompt_math_5k_src30k-35k	knowledge-distillation on-policy-distillation math qwen3

Qwen3-4B distilled from Qwen3-30B-A3B — On-policy 5k (continual, prompts 30k–35k)

Continual on-policy knowledge distillation of a Qwen3-4B student toward the Qwen3-30B-A3B (MoE) teacher, on 5,000 math prompts from openthoughts3_math_50k (positions 30k–35k, disjoint from prior training slices).

The student for this run is RockToken/qwen3-30b-a3b-to-4b-onpolicy-10k, so the effective training exposure of this checkpoint is:

1. Off-policy KD on 20k teacher responses (positions 0–20k of openthoughts3_math_50k)
2. On-policy KD on 10k prompts (positions 20k–30k) — produced the 10k model
3. On-policy KD (this run) on 5k prompts (positions 30k–35k) — this checkpoint

Stages 2 and 3 use the same on-policy pipeline; only the seed weights, prompt slice, and step count differ.

Models

Role	Model
Student	Qwen3-4B-Instruct-2507 (after stages 1–2 above)
Teacher	Qwen/Qwen3-30B-A3B-Instruct-2507 (MoE, ~3B active)

enable_thinking=False throughout (both stages).

Training data

• Source: openthoughts3_math_50k (math split of OpenThoughts-3)
• Slice used here: prompts 30k–35k → RockToken/openthoughts_prompt_math_5k_src30k-35k
• 5,000 single-user-turn prompts, mean length 261 chars, p99 845 chars

Training setup

Framework: KDFlow — FSDP2 + SGLang rollout, Ray-orchestrated GPU co-location with sleep/wakeup.

Hardware: 1× node, 4× H100 (80 GB), ~34 h wall-clock (~136 GPU-hours).

Key hyperparameters (on-policy stage; same as stage 2)

Group	Value
Backend	fsdp2, bf16, gradient ckpt on
Epochs	1
Train batch	4 (micro 1)
Learning rate	2e-6, cosine, warmup 5%
KD ratio	1.0
KD loss	reverse KL (rkl)
KD algorithm	vanilla_kd
Temperature (KD)	1.0
Rollout engine	SGLang, TP=2, 1 engine
Rollout batch	2 prompts × 4 samples/prompt
generate_max_len	8000
prompt_max_len	800 (total max_len 8800)
Sampling	temperature 1.0, top-p 1.0
Teacher	TP=4, sleep/wakeup enabled

Off-policy stage 1 hyperparameters (for context): batch 128, LR 2e-5, kd_ratio 0.5, kd_loss KL, max_len 16384, packing_samples on, ring attention.

Intended use

Research on distillation dynamics. Domain skew is math (OpenThoughts-3 math split).

Limitations

• Trained end-to-end on math prompts only; not tuned for chat, safety, or non-math domains.
• enable_thinking=False — this student does not emit <think> traces.
• Continual training may drift capabilities relative to the base checkpoint; see the wandb curves for loss dynamics.

Weights & Biases

legendary-music-16 (stage 2, 10k) and lemon-sound-17 / 2gvd60dw (this run, stage 3) under project KDFlow, entity zhao666.