---
license: apache-2.0
base_model: Qwen/Qwen3-8B-Base
library_name: transformers
pipeline_tag: text-generation
tags:
- qwen3
- on-policy-distillation
- eopd
- search
- tool-use
- agent
---

# Qwen3-8B-Base-Math-SeaSFT-Search-EOPD-Tau

An 8B agentic model trained with **Entropy-Aware On-Policy Distillation (EOPD)** in a
multi-stage RL pipeline, built on **Qwen3-8B-Base**. This checkpoint is the final iteration
(`iter_0000300`) of the Search-domain EOPD run.

## Training pipeline

| Stage | Description |
|-------|-------------|
| Base  | `Qwen/Qwen3-8B-Base` |
| SFT   | Math + Search cold-start SFT, then Tau (tool-agent) SFT — student init `Qwen3-8B-Base-Math-SeaSFT-Search-TauSFT-Tau` |
| EOPD  | On-policy distillation from a **Search specialist teacher** (`Qwen3-8B-Base-Math-SeaSFT-Search`) |

## Method: Entropy-Aware On-Policy Distillation (EOPD)

EOPD combines two KL directions to get the best of mode-seeking and mode-covering distillation:

- **Reverse-KL OPD** everywhere (mode-seeking): the sampled-token log-ratio
  `student_logp − teacher_logp` is subtracted from the advantage, sharpening the student
  toward the teacher's modes.
- **Forward-KL on high-entropy teacher tokens** (mode-covering): where reverse KL alone
  collapses diversity, a differentiable forward-KL loss
  `KL(p̃_teacher_topk ‖ p̃_student_topk)` is added, gated by the teacher's per-token entropy
  (`1[H_teacher > τ]`) and computed over the teacher's renormalized top-k (k=16) distribution.

The teacher is served with SGLang and returns both the scalar sampled-token logprob (reverse
KL) and the per-token top-k distribution + entropy (forward KL). Training was done with the
[slime](https://github.com/THUDM/slime) RL framework on Megatron-LM; this repo is the
Hugging Face `safetensors` export of the final checkpoint.

## Architecture

Identical to Qwen3-8B-Base: 36 layers, hidden size 4096, 32 attention heads, 8 KV heads (GQA),
head dim 128, QK-LayerNorm, RMSNorm, SwiGLU, RoPE (θ=1e6), 151,936 vocab, 32,768 context.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "willhx/Qwen3-8B-Base-Math-SeaSFT-Search-EOPD-Tau"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")

prompt = "Who won the Nobel Prize in Physics in 1921?"
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256)
print(tok.decode(out[0], skip_special_tokens=True))
```

The model is trained for a search/tool-use agent loop (Search-R1 style retrieval); to reproduce
the agentic behavior, drive it with the same tool prompt/format used during training.

## License

Apache-2.0, inherited from Qwen3-8B-Base.