初始化项目,由ModelHub XC社区提供模型
Model: Chia-Mu-Lab/qwen25-7b-ot-ideal-q3_32b-clean Source: Original Platform
This commit is contained in:
50
README.md
Normal file
50
README.md
Normal file
@@ -0,0 +1,50 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
base_model: Qwen/Qwen2.5-7B-Instruct
|
||||
tags:
|
||||
- distillation
|
||||
- math
|
||||
- s1
|
||||
- openthoughts
|
||||
- oracle-internal-trace
|
||||
- qwen2.5-7b-instruct
|
||||
- full-finetune
|
||||
datasets:
|
||||
- Chia-Mu-Lab/ot-ideal-q3_32b-clean
|
||||
language:
|
||||
- en
|
||||
library_name: transformers
|
||||
---
|
||||
|
||||
# Oracle internal-trace distillation · Qwen3-32B teacher -> Qwen2.5-7B-Instruct student
|
||||
|
||||
This repo holds the **oracle internal-trace** student for the 32B row of the REP
|
||||
paper's main distillation comparison table (`tab:maincomparison`).
|
||||
|
||||
The student is `Qwen/Qwen2.5-7B-Instruct` full-fine-tuned (s1 recipe) on
|
||||
**Qwen3-32B's own hidden reasoning traces** (`<think>` internal chain of thought,
|
||||
no attack) sampled on 10k OpenThoughts prompts
|
||||
(`Chia-Mu-Lab/ot-ideal-q3_32b-clean`, r1+boxed target).
|
||||
|
||||
The weights at the repo root are **checkpoint-2015 (epoch 5)**, the checkpoint
|
||||
reported in the paper.
|
||||
|
||||
## Reported metrics (this run, `kod-s1-ot-ideal-q3_32b-clean-v1`)
|
||||
|
||||
| ckpt | MATH500 | AIME24 | AIME25 | JEE (strict, full) | LCB pass@1 |
|
||||
|---|---|---|---|---|---|
|
||||
| base | 70.9 | 10.0 | 4.4 | 29.2 | 18.3 |
|
||||
| step-01613 (ep4) | 67.6 | 12.2 | 13.3 | 32.9 | 17.6 |
|
||||
| **step-02015 (ep5)** | **70.0** | **16.7** | **15.6** | 37.5 | 15.8 |
|
||||
|
||||
The paper table reports the **JEE-Math subset** (46.4 / 49.3 strict/partial) rather
|
||||
than the full-JEE strict number (37.5) shown here; all other cells match the paper
|
||||
row exactly (MATH500 70.0 / AIME24 16.7 / AIME25 15.6 / LCB 15.8).
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
model = AutoModelForCausalLM.from_pretrained("Chia-Mu-Lab/qwen25-7b-ot-ideal-q3_32b-clean", torch_dtype="bfloat16")
|
||||
tok = AutoTokenizer.from_pretrained("Chia-Mu-Lab/qwen25-7b-ot-ideal-q3_32b-clean")
|
||||
```
|
||||
Reference in New Issue
Block a user