---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
language:
- en
base_model: RockToken/qwen3-30b-a3b-to-4b-onpolicy-10k
datasets:
- RockToken/openthoughts_prompt_math_5k_src30k-35k
tags:
- knowledge-distillation
- on-policy-distillation
- math
- qwen3
---

# Qwen3-4B distilled from Qwen3-30B-A3B — On-policy 5k (continual, prompts 30k–35k)

Continual on-policy knowledge distillation of a Qwen3-4B student toward the Qwen3-30B-A3B (MoE) teacher, on 5,000 math prompts from `openthoughts3_math_50k` (positions 30k–35k, disjoint from prior training slices).

The student for this run is [RockToken/qwen3-30b-a3b-to-4b-onpolicy-10k](https://huggingface.co/RockToken/qwen3-30b-a3b-to-4b-onpolicy-10k), so the effective training exposure of this checkpoint is:

1. **Off-policy KD** on 20k teacher responses (positions 0–20k of `openthoughts3_math_50k`)
2. **On-policy KD** on 10k prompts (positions 20k–30k) — produced the 10k model
3. **On-policy KD (this run)** on 5k prompts (positions 30k–35k) — this checkpoint

Stages 2 and 3 use the same on-policy pipeline; only the seed weights, prompt slice, and step count differ.

## Models

| Role    | Model                                                  |
|---------|--------------------------------------------------------|
| Student | Qwen3-4B-Instruct-2507 (after stages 1–2 above)        |
| Teacher | Qwen/Qwen3-30B-A3B-Instruct-2507 (MoE, ~3B active)     |

`enable_thinking=False` throughout (both stages).

## Training data

- Source: `openthoughts3_math_50k` (math split of OpenThoughts-3)
- Slice used here: prompts 30k–35k → [RockToken/openthoughts_prompt_math_5k_src30k-35k](https://huggingface.co/datasets/RockToken/openthoughts_prompt_math_5k_src30k-35k)
- 5,000 single-user-turn prompts, mean length 261 chars, p99 845 chars

## Training setup

Framework: [KDFlow](https://github.com/wanng-ide/KDFlow) — FSDP2 + SGLang rollout, Ray-orchestrated GPU co-location with sleep/wakeup.

Hardware: 1× node, 4× H100 (80 GB), ~34 h wall-clock (~136 GPU-hours).

### Key hyperparameters (on-policy stage; same as stage 2)

| Group           | Value                          |
|-----------------|--------------------------------|
| Backend         | fsdp2, bf16, gradient ckpt on  |
| Epochs          | 1                              |
| Train batch     | 4 (micro 1)                    |
| Learning rate   | 2e-6, cosine, warmup 5%        |
| KD ratio        | 1.0                            |
| KD loss         | reverse KL (`rkl`)             |
| KD algorithm    | `vanilla_kd`                   |
| Temperature (KD)| 1.0                            |
| Rollout engine  | SGLang, TP=2, 1 engine         |
| Rollout batch   | 2 prompts × 4 samples/prompt   |
| `generate_max_len` | 8000                        |
| `prompt_max_len`   | 800  (total `max_len` 8800) |
| Sampling        | temperature 1.0, top-p 1.0     |
| Teacher         | TP=4, sleep/wakeup enabled     |

Off-policy stage 1 hyperparameters (for context): batch 128, LR 2e-5, `kd_ratio` 0.5, `kd_loss` KL, `max_len` 16384, packing_samples on, ring attention.

## Intended use

Research on distillation dynamics. Domain skew is math (OpenThoughts-3 math split).

## Limitations

- Trained end-to-end on math prompts only; not tuned for chat, safety, or non-math domains.
- `enable_thinking=False` — this student does not emit `<think>` traces.
- Continual training may drift capabilities relative to the base checkpoint; see the wandb curves for loss dynamics.

## Weights & Biases

`legendary-music-16` (stage 2, 10k) and `lemon-sound-17` / `2gvd60dw` (this run, stage 3) under project `KDFlow`, entity `zhao666`.