---
license: apache-2.0
base_model: Qwen/Qwen2.5-7B-Instruct
tags:
- distillation
- math
- s1
- openthoughts
- oracle-internal-trace
- qwen2.5-7b-instruct
- full-finetune
datasets:
- Chia-Mu-Lab/ot-ideal-q3_32b-clean
language:
- en
library_name: transformers
---

# Oracle internal-trace distillation · Qwen3-32B teacher -> Qwen2.5-7B-Instruct student

This repo holds the **oracle internal-trace** student for the 32B row of the REP
paper's main distillation comparison table (`tab:maincomparison`).

The student is `Qwen/Qwen2.5-7B-Instruct` full-fine-tuned (s1 recipe) on
**Qwen3-32B's own hidden reasoning traces** (`<think>` internal chain of thought,
no attack) sampled on 10k OpenThoughts prompts
(`Chia-Mu-Lab/ot-ideal-q3_32b-clean`, r1+boxed target).

The weights at the repo root are **checkpoint-2015 (epoch 5)**, the checkpoint
reported in the paper.

## Reported metrics (this run, `kod-s1-ot-ideal-q3_32b-clean-v1`)

| ckpt | MATH500 | AIME24 | AIME25 | JEE (strict, full) | LCB pass@1 |
|---|---|---|---|---|---|
| base | 70.9 | 10.0 | 4.4 | 29.2 | 18.3 |
| step-01613 (ep4) | 67.6 | 12.2 | 13.3 | 32.9 | 17.6 |
| **step-02015 (ep5)** | **70.0** | **16.7** | **15.6** | 37.5 | 15.8 |

The paper table reports the **JEE-Math subset** (46.4 / 49.3 strict/partial) rather
than the full-JEE strict number (37.5) shown here; all other cells match the paper
row exactly (MATH500 70.0 / AIME24 16.7 / AIME25 15.6 / LCB 15.8).

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("Chia-Mu-Lab/qwen25-7b-ot-ideal-q3_32b-clean", torch_dtype="bfloat16")
tok = AutoTokenizer.from_pretrained("Chia-Mu-Lab/qwen25-7b-ot-ideal-q3_32b-clean")
```