language, license, tags, base_model, pipeline_tag, base_model_relation
| language | license | tags | base_model | pipeline_tag | base_model_relation | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
apache-2.0 |
|
pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1 | text-generation | quantized |
⚠️ Note: This 0.6B version is undertrained and does not reliably follow Hindi instructions. For a working Hindi model, please use Qwen3-4B-Hindi-Instruct-v2 (GGUF here).
Qwen3-0.6B Hindi Instruct v1 — GGUF
The smallest Hindi-capable instruction model you can run locally — fits in 370MB, runs on any laptop, no GPU needed.
Fine-tuned from Qwen/Qwen3-0.6B on English to Hindi instruction pairs. Quantized to GGUF for local inference with llama.cpp, LM Studio, and Ollama.
Quick Start
Download the model using huggingface-cli:
huggingface-cli download pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1-GGUF Qwen3-0.6B-Hindi-v1-Q4_K_M.gguf --local-dir .
Run with llama.cpp:
./llama-cli -m Qwen3-0.6B-Hindi-v1-Q4_K_M.gguf -p "भारत की राजधानी क्या है?" -n 256
Or open directly in LM Studio — New Model — Search pankajpandey-dev
Available Versions
| File | Quantization | Size | RAM Needed | Best For |
|---|---|---|---|---|
| Qwen3-0.6B-Hindi-v1-Q2_K.gguf | Q2_K | 0.28 GB | 1 GB | Minimum hardware |
| Qwen3-0.6B-Hindi-v1-Q4_K_M.gguf | Q4_K_M | 0.37 GB | 1.5 GB | Recommended |
| Qwen3-0.6B-Hindi-v1-Q5_K_M.gguf | Q5_K_M | 0.41 GB | 2 GB | Better quality |
| Qwen3-0.6B-Hindi-v1-Q8_0.gguf | Q8_0 | 0.60 GB | 2.5 GB | Highest quality |
Not sure which to pick? Always start with Q4_K_M — best balance of speed, size, and quality.
Model Details
| Property | Value |
|---|---|
| Base Model | Qwen/Qwen3-0.6B |
| Parameters | 600M |
| Architecture | Qwen2 — fully supported by llama.cpp |
| Fine-tune Method | QLoRA with LoRA r=16 alpha=16 |
| Training Steps | 60 steps |
| Training Data | 2000 English to Hindi instruction pairs |
| Max Context | 2048 tokens |
| Languages | Hindi and English |
| License | Apache 2.0 — commercial use allowed |
Example Prompts
Hindi Question Answering:
User: भारत की राजधानी क्या है?
Model: भारत की राजधानी नई दिल्ली है।
Hindi Instructions:
User: मुझे चाय बनाने का तरीका बताओ।
Model: चाय बनाने के लिए पहले पानी गरम करें...
Mixed Language:
User: Python में for loop कैसे लिखते हैं?
Model: Python में for loop इस तरह लिखते हैं...
Compatibility
| Tool | Status |
|---|---|
| llama.cpp | Full support |
| LM Studio | Full support |
| Ollama | Full support |
| Jan | Full support |
| Open WebUI | Full support |
Recommended Settings
Temperature: 0.7
Top-P: 0.9
Top-K: 40
Repeat Penalty: 1.1
Context Length: 2048
Why This Model?
- Tiny — 370MB, one of the smallest Hindi-capable GGUF models available
- Fast — runs fully on CPU, no GPU required
- Hindi-first — specifically trained for Hindi instruction following
- Open — Apache 2.0, free for personal and commercial use
- Actively maintained — v2 coming soon with more data
Roadmap
- Done: v1 — Base Hindi fine-tune on Qwen3-0.6B
- Next: v2 — 10x larger dataset, improved Hindi fluency
- Next: v3 — better instruction following
- Next: Qwen3-1.7B-Hindi — bigger model, same niche
- Next: Live demo Space on HuggingFace
Related Repos
| Repo | Description |
|---|---|
| pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1 | Full precision model in safetensors format |
| pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1-GGUF | This repo — GGUF quantized versions |
About the Author
Made by pankajpandey-dev Building open-source Hindi AI models for India
Follow for weekly model updates and new Hindi LLM releases.
Found this useful? Please like this repo — it helps other Hindi speakers find it.