qwen-2-5-3b-instruct-using-openai-gsm8k-data-enhanced-with-deepseek-v3/README.md

---
license: mit
datasets:
- openai/gsm8k
- eagle0504/openai-gsm8k-enhanced-using-together-ai-deepseek-train8k-test1k-v1
language:
- en
base_model:
- Qwen/Qwen2.5-3B-Instruct
library_name: transformers
tags:
- fine-tuned
- qwen
- deepseek
- gsm8k
- reasoning
---

# Qwen 2.5-3B-Instruct Fine-Tuned on OpenAI GSM8K with DeepSeek Augmentation

## Model Overview

This model is a fine-tuned version of **Qwen/Qwen2.5-3B-Instruct**, optimized for mathematical reasoning tasks using the **OpenAI GSM8K** dataset. The fine-tuning process enhances the model's ability to generate step-by-step explanations for grade school math problems, incorporating **reasoning augmentation** through DeepSeek. The model improves upon GSM8K’s standard answers by integrating additional contextual reasoning derived from DeepSeek’s small model.

### Key Features:
- **Base Model**: Qwen 2.5-3B-Instruct
- **Fine-Tuned On**: OpenAI GSM8K dataset
- **Enhancement**: Answer augmentation with reasoning insights from **DeepSeek-V3-Small**
- **Improved Reasoning**: Model not only provides correct answers but also **augments** explanations with logical steps inspired by DeepSeek’s generative capabilities.

## Dataset & Training Details

- **Dataset**: OpenAI’s GSM8K (Grade School Math 8K), a collection of high-quality math problems designed to test problem-solving skills.
- **Data Curation**: We have provided enhanced OpenAI's GSM8K dataset with reasoning skills generated using DeepSeek-R1, see [here](https://huggingface.co/datasets/eagle0504/openai-gsm8k-enhanced-using-together-ai-deepseek-train8k-test1k-v1).
- **Enhancement**: After fine-tuning on GSM8K, additional reasoning layers were introduced using **DeepSeek-V3-Small**, leading to richer, more interpretable answers.
- **Training Objective**: Improve step-by-step mathematical reasoning and **enhance logical deductions** in model-generated responses.

I have adopted some code from Unsloth and here's an updated [notebook](https://colab.research.google.com/drive/1HV0YkyiTD55j1xLRBHwJ_q3ex82W5EXr?usp=sharing) on Colab. Please feel free to copy it and run it yourself. 

You will need:
- Huggingface token
- Together.AI API Key
- Unsloth package

## How to Use Model via Terminal (Mac)

**Goal** Run Qwen-2.5-3B Instruct on Your Mac Using `llama.cpp`

Yes! You can run **Qwen-2.5-3B Instruct** on your Mac using `llama.cpp`. Here’s a step-by-step guide assuming you are starting from a clean macOS installation with only `pyenv` installed.

### **Step 1: Install Homebrew (if not installed)**
Homebrew is required to install `llama.cpp`.

1. Open **Terminal** (`Cmd + Space`, type `Terminal`, and press **Enter**).
2. Run:
   ```sh
   /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
   ```
3. After installation, add Homebrew to your PATH:
   ```sh
   echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> ~/.zprofile
   eval "$(/opt/homebrew/bin/brew shellenv)"
   ```

---

### **Step 2: Install `llama.cpp` via Homebrew**
Run:
```sh
brew install llama.cpp
```

Once installed, you should be able to use `llama-cli`.

---

### **Step 3: Run Qwen-2.5-3B Instruct with `llama-cli`**
To run the model, execute:
```sh
llama-cli -hf eagle0504/qwen-2-5-3b-instruct-using-openai-gsm8k-data-enhanced-with-deepseek-v3:Q8_0
```

---

### **Step 4: Additional Configurations (If Needed)**
If you encounter issues or need finer control, you may want to:

#### **A. Verify Installation**
Check if `llama-cli` is installed:
```sh
llama-cli --version
```
If you see a version output, it’s installed correctly.

#### **B. Run with Explicit Model Path**
If the default Hugging Face loader doesn't work, you can manually download the model:
1. **Create a models directory:**
   ```sh
   mkdir -p ~/llama_models && cd ~/llama_models
   ```
2. **Download the GGUF model file** from [Hugging Face](https://huggingface.co/eagle0504/qwen-2-5-3b-instruct-using-openai-gsm8k-data-enhanced-with-deepseek-v3):
   ```sh
   wget https://huggingface.co/eagle0504/qwen-2-5-3b-instruct-using-openai-gsm8k-data-enhanced-with-deepseek-v3/blob/main/unsloth.Q8_0.gguf
   ```
3. **Run the model manually**:
   ```sh
   llama-cli -m ~/llama_models/Q8_0.gguf
   ```

---

### **Step 5: Test the Model**
Try prompting it:
```sh
llama-cli -m ~/llama_models/Q8_0.gguf -p "Explain quantum computing in simple terms."
```
or interactively:
```sh
llama-cli -m ~/llama_models/Q8_0.gguf --interactive
```

## How to Use Model via Python

You can load this model with `transformers`:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "eagle0504/qwen-2-5-3b-instruct-using-openai-gsm8k-data-enhanced-with-deepseek-v3"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Example prompt
prompt = "A farmer has 24 apples. He gives 6 to each of his 3 children. How many does he have left?"
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_length=200)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

## Expected Performance

Compared to the base **Qwen2.5-3B-Instruct**, this fine-tuned model:
- Provides **more detailed explanations** when answering GSM8K math problems.
- Improves **logical reasoning** by incorporating DeepSeek-style augmented reasoning.
- Generates **clearer step-by-step solutions**, making it useful for educational or tutoring applications.

## Model Directory

The model is hosted on **Hugging Face Hub**:
👉 **[eagle0504/qwen-2-5-3b-instruct-using-openai-gsm8k-data-enhanced-with-deepseek-v3](https://huggingface.co/eagle0504/qwen-2-5-3b-instruct-using-openai-gsm8k-data-enhanced-with-deepseek-v3)**

## License

This model is released under the **MIT License**, allowing open usage and modifications.

---

If you have any questions or suggestions for improvements, feel free to reach out!

🔗 [LinkedIn](https://www.linkedin.com/in/yiqiaoyin/)