128 lines
3.6 KiB
Markdown
128 lines
3.6 KiB
Markdown
---
|
|
license: apache-2.0
|
|
|
|
language:
|
|
- en
|
|
|
|
base_model:
|
|
- deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
|
|
- Qwen/Qwen2.5-Math-7B
|
|
|
|
tags:
|
|
- merge
|
|
- sens-merging
|
|
- math
|
|
- reasoning
|
|
- qwen2.5
|
|
- deepseek-r1
|
|
|
|
pipeline_tag: text-generation
|
|
---
|
|
|
|
# Qwen2.5-Math-DeepSeekR1-Sens-7B
|
|
|
|
A 7B merged model created by applying Sensitivity-aware Model Merging (Sens Merging) to:
|
|
|
|
- deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
|
|
- Qwen/Qwen2.5-Math-7B
|
|
|
|
The goal of this model is to preserve the strong mathematical reasoning ability of DeepSeek-R1-Distill while significantly reducing reasoning verbosity and output token length.
|
|
|
|
---
|
|
|
|
## Highlights
|
|
|
|
- Average accuracy: 66.9%
|
|
- Average output tokens: 701
|
|
- Output tokens reduced by 75.2% compared to DeepSeek-R1-Distill-Qwen-7B
|
|
- Only 2.5 points lower average accuracy than DeepSeek-R1-Distill-Qwen-7B
|
|
|
|
This model provides an attractive trade-off between reasoning quality and inference cost.
|
|
|
|
---
|
|
|
|
## Base Models
|
|
|
|
| Model | Avg Accuracy | Avg Tokens |
|
|
|-----------------------------|-------------:|-----------:|
|
|
| DeepSeek-R1-Distill-Qwen-7B | 69.4 | 2826 |
|
|
| Qwen2.5-Math-7B | 45.3 | 755 |
|
|
| Sens Merge (λ=0.4) | 66.9 | 701 |
|
|
|
|
---
|
|
|
|
## Benchmark Results
|
|
|
|
| Benchmark | Distill | Qwen2.5-Math | Sens Merge (λ=0.4) |
|
|
|----------------|--------:|-------------:|--------------------:|
|
|
| College Math | 66.0 | 37.9 | 70.4 |
|
|
| GSM8K | 90.2 | 84.5 | 90.6 |
|
|
| MATH | 94.4 | 73.3 | 90.2 |
|
|
| Minerva Math | 41.5 | 13.6 | 36.0 |
|
|
| OlympiadBench | 55.0 | 17.3 | 47.2 |
|
|
| Avg Accuracy | 69.4 | 45.3 | 66.9 |
|
|
| Avg Tokens | 2826 | 755 | 701 |
|
|
|
|
---
|
|
|
|
## Motivation
|
|
|
|
Large reasoning models such as DeepSeek-R1-Distill often produce long chains of thought, which increases inference cost.
|
|
|
|
This model explores whether model merging can reduce reasoning verbosity without requiring additional training.
|
|
|
|
By merging a reasoning model (DeepSeek-R1-Distill-Qwen-7B) with a compact mathematical model (Qwen2.5-Math-7B) using Sensitivity-aware Model Merging, the merged model:
|
|
|
|
- Maintains competitive reasoning performance
|
|
- Produces significantly shorter outputs
|
|
- Requires no gradient-based fine-tuning
|
|
- Uses only a small calibration dataset
|
|
|
|
---
|
|
|
|
## Comparison with DPO
|
|
|
|
We additionally compared Sens Merging with a DPO-trained model:
|
|
|
|
| Model | Avg Accuracy | Avg Tokens |
|
|
|-----------------------------|-------------:|-----------:|
|
|
| DeepSeek-R1-Distill-Qwen-7B | 69.4 | 2826 |
|
|
| DPO | 68.55 | 2402 |
|
|
| Sens Merge (λ=0.4) | 66.9 | 701 |
|
|
|
|
Sens Merging achieves a much larger reduction in output length while remaining competitive in accuracy.
|
|
|
|
---
|
|
|
|
## Usage
|
|
|
|
```python
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
|
|
|
model_name = "quangdung/Qwen2.5-Math-DeepSeekR1-Sens-7B"
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|
model_name,
|
|
torch_dtype="auto",
|
|
device_map="auto"
|
|
)
|
|
|
|
prompt = "Solve: If x^2 + 5x + 6 = 0, find x."
|
|
|
|
messages = [{"role": "user", "content": prompt}]
|
|
|
|
text = tokenizer.apply_chat_template(
|
|
messages,
|
|
tokenize=False,
|
|
add_generation_prompt=True
|
|
)
|
|
|
|
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
|
|
|
outputs = model.generate(
|
|
**inputs,
|
|
max_new_tokens=512
|
|
)
|
|
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |