qwen25-7b-scientific-reasoning/README.md

---
license: apache-2.0
base_model: Qwen/Qwen2.5-7B-Instruct
tags:
  - qwen2.5
  - reasoning
  - chain-of-thought
  - fine-tuned
  - scientific-reasoning
  - math
language:
  - en
pipeline_tag: text-generation
---

# Qwen2.5-7B Scientific Reasoning
Fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on 525 chain-of-thought scientific reasoning examples.

## Model Description
This model has been fine-tuned to provide explicit step-by-step reasoning for problem-solving tasks. It excels at:
- Mathematical word problems
- Scientific reasoning
- Logic puzzles
- Multi-step calculations
- Transparent thought processes

## Training Details
- **Base Model:** Qwen2.5-7B-Instruct
- **Training Data:** 525 scientific reasoning examples with chain-of-thought reasoning
- **Training Method:** LoRA (Low-Rank Adaptation)
  - Rank: 16
  - Alpha: 32
  - Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- **Quantization:** 4-bit (during training)
- **Training Time:** 1 hour 17 minutes
- **Training Loss:** 1.19 → 0.47 (61% reduction)
- **Eval Loss:** 0.746
- **Epochs:** 3
- **Total Steps:** 189

## Key Features
✅ **Explicit Reasoning:** Model shows its thinking process with "Let me think through this:" prefix
✅ **Step-by-Step Solutions:** Breaks down complex problems into manageable steps
✅ **Self-Verification:** Often double-checks calculations and reasoning
✅ **Pedagogical:** Great for educational contexts and tutoring
✅ **100% Accuracy:** Achieved perfect accuracy on comparison benchmark tests

## Usage

### Using Transformers

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "howellx/qwen25-7b-scientific-reasoning"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

messages = [
    {"role": "system", "content": "You are a skilled problem solver. Break down complex problems into manageable steps, show your reasoning process, and provide clear answers."},
    {"role": "user", "content": "A store sells apples for $3 per pound. If you buy 4 pounds and get a 15% discount, how much do you pay?"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True,
    top_p=0.9,
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response.split("assistant\n")[-1])
```

### Using Ollama (GGUF version)
The GGUF version is also available for use with Ollama/llama.cpp.

## Example Outputs
**Input:** "A store sells apples for $3 per pound. If you buy 4 pounds and get a 15% discount, how much do you pay?"

**Output:**
```
Let me think through this:

Step 1: Calculate total cost before discount
- Price per pound: $3
- Quantity purchased: 4 pounds
Total cost = $3 × 4 = $12

Step 2: Apply the 15% discount
Discount amount = Total cost × Discount rate
= $12 × 0.15 = $1.80

Step 3: Calculate final payment
Final payment = Total cost - Discount amount
= $12 - $1.80 = $10.20

Therefore, you pay $10.20.
```

## Comparison with Base Model
Compared to the base Qwen2.5-7B-Instruct model:
- ✅ Shows explicit "Let me think through this:" reasoning prefix
- ✅ More transparent step-by-step problem decomposition
- ✅ Includes self-verification and double-checking
- ✅ 2-3x longer responses but more educational
- ✅ Same accuracy, better pedagogical value

## Best Use Cases
- Educational tutoring and homework help
- Teaching problem-solving strategies
- Scientific and mathematical reasoning tasks
- Situations requiring transparent reasoning
- Applications where showing work is important

## Limitations
- Responses are more verbose (2-3x longer than base model)
- Optimized for reasoning tasks, may be overkill for simple queries
- Fine-tuned on English scientific reasoning examples

## Citation
If you use this model, please cite:

```bibtex
@misc{qwen25-scientific-reasoning,
  author = {Justin Howell},
  title = {Qwen2.5-7B Scientific Reasoning},
  year = {2026},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/howellx/qwen25-7b-scientific-reasoning}}
}
```

## License
Apache 2.0 (same as base Qwen2.5 model)

## Acknowledgments
- Base model: [Qwen Team](https://huggingface.co/Qwen)
- Training framework: HuggingFace Transformers + PEFT
- Distillation pipeline: Custom Claude-based chain-of-thought generation