151 lines
4.6 KiB
Markdown
151 lines
4.6 KiB
Markdown
---
|
||
license: apache-2.0
|
||
base_model: Qwen/Qwen2.5-7B-Instruct
|
||
tags:
|
||
- qwen2.5
|
||
- reasoning
|
||
- chain-of-thought
|
||
- fine-tuned
|
||
- scientific-reasoning
|
||
- math
|
||
language:
|
||
- en
|
||
pipeline_tag: text-generation
|
||
---
|
||
|
||
# Qwen2.5-7B Scientific Reasoning
|
||
Fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on 525 chain-of-thought scientific reasoning examples.
|
||
|
||
## Model Description
|
||
This model has been fine-tuned to provide explicit step-by-step reasoning for problem-solving tasks. It excels at:
|
||
- Mathematical word problems
|
||
- Scientific reasoning
|
||
- Logic puzzles
|
||
- Multi-step calculations
|
||
- Transparent thought processes
|
||
|
||
## Training Details
|
||
- **Base Model:** Qwen2.5-7B-Instruct
|
||
- **Training Data:** 525 scientific reasoning examples with chain-of-thought reasoning
|
||
- **Training Method:** LoRA (Low-Rank Adaptation)
|
||
- Rank: 16
|
||
- Alpha: 32
|
||
- Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
|
||
- **Quantization:** 4-bit (during training)
|
||
- **Training Time:** 1 hour 17 minutes
|
||
- **Training Loss:** 1.19 → 0.47 (61% reduction)
|
||
- **Eval Loss:** 0.746
|
||
- **Epochs:** 3
|
||
- **Total Steps:** 189
|
||
|
||
## Key Features
|
||
✅ **Explicit Reasoning:** Model shows its thinking process with "Let me think through this:" prefix
|
||
✅ **Step-by-Step Solutions:** Breaks down complex problems into manageable steps
|
||
✅ **Self-Verification:** Often double-checks calculations and reasoning
|
||
✅ **Pedagogical:** Great for educational contexts and tutoring
|
||
✅ **100% Accuracy:** Achieved perfect accuracy on comparison benchmark tests
|
||
|
||
## Usage
|
||
|
||
### Using Transformers
|
||
|
||
```python
|
||
import torch
|
||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||
|
||
model_name = "howellx/qwen25-7b-scientific-reasoning"
|
||
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
model_name,
|
||
torch_dtype=torch.bfloat16,
|
||
device_map="auto",
|
||
trust_remote_code=True
|
||
)
|
||
|
||
messages = [
|
||
{"role": "system", "content": "You are a skilled problem solver. Break down complex problems into manageable steps, show your reasoning process, and provide clear answers."},
|
||
{"role": "user", "content": "A store sells apples for $3 per pound. If you buy 4 pounds and get a 15% discount, how much do you pay?"}
|
||
]
|
||
|
||
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||
inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
||
|
||
outputs = model.generate(
|
||
**inputs,
|
||
max_new_tokens=512,
|
||
temperature=0.7,
|
||
do_sample=True,
|
||
top_p=0.9,
|
||
)
|
||
|
||
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
||
print(response.split("assistant\n")[-1])
|
||
```
|
||
|
||
### Using Ollama (GGUF version)
|
||
The GGUF version is also available for use with Ollama/llama.cpp.
|
||
|
||
## Example Outputs
|
||
**Input:** "A store sells apples for $3 per pound. If you buy 4 pounds and get a 15% discount, how much do you pay?"
|
||
|
||
**Output:**
|
||
```
|
||
Let me think through this:
|
||
|
||
Step 1: Calculate total cost before discount
|
||
- Price per pound: $3
|
||
- Quantity purchased: 4 pounds
|
||
Total cost = $3 × 4 = $12
|
||
|
||
Step 2: Apply the 15% discount
|
||
Discount amount = Total cost × Discount rate
|
||
= $12 × 0.15 = $1.80
|
||
|
||
Step 3: Calculate final payment
|
||
Final payment = Total cost - Discount amount
|
||
= $12 - $1.80 = $10.20
|
||
|
||
Therefore, you pay $10.20.
|
||
```
|
||
|
||
## Comparison with Base Model
|
||
Compared to the base Qwen2.5-7B-Instruct model:
|
||
- ✅ Shows explicit "Let me think through this:" reasoning prefix
|
||
- ✅ More transparent step-by-step problem decomposition
|
||
- ✅ Includes self-verification and double-checking
|
||
- ✅ 2-3x longer responses but more educational
|
||
- ✅ Same accuracy, better pedagogical value
|
||
|
||
## Best Use Cases
|
||
- Educational tutoring and homework help
|
||
- Teaching problem-solving strategies
|
||
- Scientific and mathematical reasoning tasks
|
||
- Situations requiring transparent reasoning
|
||
- Applications where showing work is important
|
||
|
||
## Limitations
|
||
- Responses are more verbose (2-3x longer than base model)
|
||
- Optimized for reasoning tasks, may be overkill for simple queries
|
||
- Fine-tuned on English scientific reasoning examples
|
||
|
||
## Citation
|
||
If you use this model, please cite:
|
||
|
||
```bibtex
|
||
@misc{qwen25-scientific-reasoning,
|
||
author = {Justin Howell},
|
||
title = {Qwen2.5-7B Scientific Reasoning},
|
||
year = {2026},
|
||
publisher = {HuggingFace},
|
||
howpublished = {\url{https://huggingface.co/howellx/qwen25-7b-scientific-reasoning}}
|
||
}
|
||
```
|
||
|
||
## License
|
||
Apache 2.0 (same as base Qwen2.5 model)
|
||
|
||
## Acknowledgments
|
||
- Base model: [Qwen Team](https://huggingface.co/Qwen)
|
||
- Training framework: HuggingFace Transformers + PEFT
|
||
- Distillation pipeline: Custom Claude-based chain-of-thought generation
|