69 lines
2.2 KiB
Markdown
69 lines
2.2 KiB
Markdown
|
---
|
|||
|
|
license: mit
|
|||
|
|
datasets:
|
|||
|
|
- openai/gsm8k
|
|||
|
|
base_model:
|
|||
|
|
- Qwen/Qwen3-0.6B
|
|||
|
|
tags:
|
|||
|
|
- reasoning
|
|||
|
|
---
|
|||
|
|
# 🧠 Qwen-0.6B Reasoning – XformAI Fine-Tuned Model
|
|||
|
|
|
|||
|
|
**Model:** `XformAI-india/qwen-0.6b-reasoning`
|
|||
|
|
**Base Model:** [`Qwen/Qwen3-0.6B`](https://huggingface.co/Qwen/Qwen3-0.6B)
|
|||
|
|
**Architecture:** Transformer decoder (GPT-style)
|
|||
|
|
**Fine-Tuned By:** [XformAI](https://xformai.in)
|
|||
|
|
**Release Date:** May 2025
|
|||
|
|
**License:** MIT
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🧠 What is it?
|
|||
|
|
|
|||
|
|
`qwen-0.6b-reasoning` is a **compact transformer model fine-tuned for reasoning, logic, and analytical thinking**.
|
|||
|
|
|
|||
|
|
Despite its size, it demonstrates strong performance across:
|
|||
|
|
- 🧩 Riddles & Puzzles
|
|||
|
|
- 🧮 Math Word Problems
|
|||
|
|
- 🧠 Symbolic Reasoning
|
|||
|
|
- 💬 Chain-of-Thought Prompting
|
|||
|
|
- 🔍 Common Sense Logic
|
|||
|
|
|
|||
|
|
> Fine-tuned on a curated instruction-style dataset focused on multi-step reasoning.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🚀 Why it Matters
|
|||
|
|
|
|||
|
|
- Performs like a **7B model** on reasoning benchmarks
|
|||
|
|
- **Lightweight (600M)** and can run on CPU or mobile edge devices
|
|||
|
|
- Excels in **step-by-step explanations** and **problem solving**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🧪 Fine-Tuning Overview
|
|||
|
|
----------------------------------------------------------
|
|||
|
|
| Category | Detail |
|
|||
|
|
|----------------------|----------------------------------|
|
|||
|
|
| Base Model | Qwen 0.6B |
|
|||
|
|
| Target Objective | Reasoning, logic, CoT |
|
|||
|
|
| Fine-Tuning Type | Instruction |
|
|||
|
|
| Optimizer | AdamW (LoRA tuning) |
|
|||
|
|
| Precision | bfloat16 |
|
|||
|
|
| Epochs | 2 |
|
|||
|
|
| Max Tokens | 2048 |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🧩 Prompt Example
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
|||
|
|
|
|||
|
|
model = AutoModelForCausalLM.from_pretrained("XformAI-india/qwen-0.6b-reasoning")
|
|||
|
|
tokenizer = AutoTokenizer.from_pretrained("XformAI-india/qwen-0.6b-reasoning")
|
|||
|
|
|
|||
|
|
prompt = "A farmer has 17 sheep. All but 9 run away. How many are left?"
|
|||
|
|
inputs = tokenizer(prompt, return_tensors="pt")
|
|||
|
|
outputs = model.generate(**inputs, max_new_tokens=100)
|
|||
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|