初始化项目,由ModelHub XC社区提供模型
Model: PAI/DistillQwen-ThoughtY-8B Source: Original Platform
This commit is contained in:
72
README.md
Normal file
72
README.md
Normal file
@@ -0,0 +1,72 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
---
|
||||
# DistillQwen-ThoughtY: Enhanced Chain-of-Thought Reasoning Models
|
||||
|
||||
## Key Contributions
|
||||
- **Advanced Reasoning Models**: DistillQwen-ThoughtY series (4B/8B/32B) outperform previous versions (ThoughtX) and Qwen3 in thinking mode, achieving state-of-the-art results on mathematical, scientific, and coding tasks.
|
||||
- **OmniThought-0528 Dataset**: New 365K high-quality Chain-of-Thought (CoT) dataset distilled from DeepSeek-R1-0528 (top-tier Chinese model) with:
|
||||
- Cognitive Difficulty (CD) and Reasoning Verbosity (RV) annotations
|
||||
- Multi-teacher integration (DeepSeek-R1, DeepSeek-R1-0528, QwQ-32B)
|
||||
|
||||
## Performance Highlights
|
||||
| Model | AIME2024 | MATH500 | GPQA Diamond | LiveCodeBench V2 | Avg. |
|
||||
|---------------------|----------|---------|--------------|------------------|------|
|
||||
| **DistillQwen-ThoughtY-4B** | **76.7** | **95.2** | **56.1** | **75.8** | **76.0** |
|
||||
| **DistillQwen-ThoughtY-8B** | **76.7** | **94.6** | **62.1** | **78.1** | **77.9** |
|
||||
| **DistillQwen-ThoughtY-32B** | **90.0** | **95.2** | 63.6 | **76.3** | **81.3** |
|
||||
| OpenThinker2-32B | 76.7 | 90.8 | **64.1** | 72.5 | 76.0 |
|
||||
| DistillQwen-ThoughtX-32B | 80.0 | 92.6 | 64.0 | 73.4 | 77.5 |
|
||||
|
||||
## Quick Start
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
"alibaba-pai/DistillQwen-ThoughtY-4B",
|
||||
device_map="auto"
|
||||
)
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
||||
|
||||
prompt = "Solve ∫x e^x dx. Show your reasoning step-by-step."
|
||||
messages = [
|
||||
{"role": "user", "content": prompt}
|
||||
]
|
||||
text = tokenizer.apply_chat_template(
|
||||
messages,
|
||||
tokenize=False,
|
||||
add_generation_prompt=True,
|
||||
enable_thinking=True # Switches between thinking and non-thinking modes. Default is True.
|
||||
)
|
||||
inputs = tokenizer([text], return_tensors="pt").to("cuda")
|
||||
outputs = model.generate(**inputs, max_new_tokens=32768)
|
||||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
## Resources
|
||||
- [Models on Hugging Face](https://huggingface.co/alibaba-pai)
|
||||
- [OmniThought-0528 Dataset](https://huggingface.co/datasets/alibaba-pai/OmniThought-0528)
|
||||
- [EasyDistill Framework](https://github.com/modelscope/easydistill)
|
||||
|
||||
|
||||
## Reference
|
||||
|
||||
For more detailed information about the model, we encourage you to refer to our paper:
|
||||
|
||||
- **Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations**
|
||||
Wenrui Cai, Chengyu Wang, Junbing Yan, Jun Huang, Xiangzhong Fang
|
||||
[arXiv:2505.10937](https://arxiv.org/abs/2505.10937)
|
||||
|
||||
You can cite the paper using the following citation format:
|
||||
|
||||
```bibtex
|
||||
@misc{cai2025reasoningomnithoughtlargecot,
|
||||
title={Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations},
|
||||
author={Wenrui Cai and Chengyu Wang and Junbing Yan and Jun Huang and Xiangzhong Fang},
|
||||
year={2025},
|
||||
eprint={2505.10937},
|
||||
archivePrefix={arXiv},
|
||||
primaryClass={cs.CL},
|
||||
url={https://arxiv.org/abs/2505.10937}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user