CRISP-Qwen3-8B-v1/README.md

license, library_name, pipeline_tag, language, tags, base_model, datasets

license	library_name	pipeline_tag	language	tags	base_model	datasets
apache-2.0	transformers	text-generation	en	math reasoning reasoning-compression self-distillation crisp	Qwen/Qwen3-8B	pb09204048/CRISP

CRISP-Qwen3-8B-v1

Qwen3-8B trained with CRISP (Compressed Reasoning via Iterative Self-Policy Distillation) using the v1 conciseness teacher. Step-99 checkpoint.

Paper: https://arxiv.org/abs/2603.05433

CRISP teaches a reasoning model to think concisely by distilling its own concise behavior back into itself: the teacher is the same model conditioned on a conciseness instruction, the student has no instruction, and training minimizes per-token reverse KL from student to teacher on the student's own rollouts (teacher refreshed every M=50 steps). No ground-truth answers, token budgets, or difficulty estimators enter the loss.

This checkpoint uses the v1 teacher prompt: v1 (uniform): "Solve concisely and correctly. Be direct — avoid unnecessary elaboration, redundant steps, or restating the problem."

Other CRISP checkpoints: Qwen3-8B (v2), Qwen3-14B (v2), DeepSeek-R1-Distill-Llama-8B (v2). Training data: pb09204048/CRISP.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("pb09204048/CRISP-Qwen3-8B-v1")
model = AutoModelForCausalLM.from_pretrained("pb09204048/CRISP-Qwen3-8B-v1", device_map="auto")

Benchmark results (Qwen3-8B)

Accuracy (mean@8, %) and token reduction (Red., % vs. base) at a 30K-token budget. Math is scored with a dual-path grader (Answer: or \boxed{}); GPQA-Diamond and MMLU use exact letter-match. This model is the CRISP (v1) row.

Setting	MATH-500	AIME 2024	AIME 2025	GPQA-D	MMLU
Base	95.7 / —	76.2 / —	70.4 / —	61.5 / —	81.9 / —
Concise prompt (v2)	94.2 / 21.5%	74.6 / 9.8%	63.7 / 5.3%	59.5 / 30.7%	82.8 / 26.8%
Concise prompt (v1)	95.6 / 38.9%	74.2 / 20.2%	62.1 / 13.9%	56.8 / 29.5%	83.0 / 26.8%
CRISP (v2)	95.7 / 31.6%	75.0 / 17.1%	65.8 / 17.5%	58.3 / 17.2%	81.2 / 22.4%
CRISP (v1)	95.7 / 56.9%	72.9 / 32.9%	58.8 / 28.4%	58.5 / 36.2%	80.9 / 44.7%

Citation

@article{sang2026crisp,
  title={Crisp: Compressed reasoning via iterative self-policy distillation},
  author={Sang, Hejian and Xu, Yuanda and Zhou, Zhengze and He, Ran and Wang, Zhipeng and Sun, Jiachen},
  journal={arXiv preprint arXiv:2603.05433},
  year={2026}
}