69 lines
2.2 KiB
Markdown
69 lines
2.2 KiB
Markdown
---
|
||
license: mit
|
||
datasets:
|
||
- openai/gsm8k
|
||
base_model:
|
||
- Qwen/Qwen3-0.6B
|
||
tags:
|
||
- reasoning
|
||
---
|
||
# 🧠 Qwen-0.6B Reasoning – XformAI Fine-Tuned Model
|
||
|
||
**Model:** `XformAI-india/qwen-0.6b-reasoning`
|
||
**Base Model:** [`Qwen/Qwen3-0.6B`](https://huggingface.co/Qwen/Qwen3-0.6B)
|
||
**Architecture:** Transformer decoder (GPT-style)
|
||
**Fine-Tuned By:** [XformAI](https://xformai.in)
|
||
**Release Date:** May 2025
|
||
**License:** MIT
|
||
|
||
---
|
||
|
||
## 🧠 What is it?
|
||
|
||
`qwen-0.6b-reasoning` is a **compact transformer model fine-tuned for reasoning, logic, and analytical thinking**.
|
||
|
||
Despite its size, it demonstrates strong performance across:
|
||
- 🧩 Riddles & Puzzles
|
||
- 🧮 Math Word Problems
|
||
- 🧠 Symbolic Reasoning
|
||
- 💬 Chain-of-Thought Prompting
|
||
- 🔍 Common Sense Logic
|
||
|
||
> Fine-tuned on a curated instruction-style dataset focused on multi-step reasoning.
|
||
|
||
---
|
||
|
||
## 🚀 Why it Matters
|
||
|
||
- Performs like a **7B model** on reasoning benchmarks
|
||
- **Lightweight (600M)** and can run on CPU or mobile edge devices
|
||
- Excels in **step-by-step explanations** and **problem solving**
|
||
|
||
---
|
||
|
||
## 🧪 Fine-Tuning Overview
|
||
----------------------------------------------------------
|
||
| Category | Detail |
|
||
|----------------------|----------------------------------|
|
||
| Base Model | Qwen 0.6B |
|
||
| Target Objective | Reasoning, logic, CoT |
|
||
| Fine-Tuning Type | Instruction |
|
||
| Optimizer | AdamW (LoRA tuning) |
|
||
| Precision | bfloat16 |
|
||
| Epochs | 2 |
|
||
| Max Tokens | 2048 |
|
||
|
||
---
|
||
|
||
## 🧩 Prompt Example
|
||
|
||
```python
|
||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||
|
||
model = AutoModelForCausalLM.from_pretrained("XformAI-india/qwen-0.6b-reasoning")
|
||
tokenizer = AutoTokenizer.from_pretrained("XformAI-india/qwen-0.6b-reasoning")
|
||
|
||
prompt = "A farmer has 17 sheep. All but 9 run away. How many are left?"
|
||
inputs = tokenizer(prompt, return_tensors="pt")
|
||
outputs = model.generate(**inputs, max_new_tokens=100)
|
||
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |