155 lines
4.3 KiB
Markdown
155 lines
4.3 KiB
Markdown
|
---
|
||
|
|
language:
|
||
|
|
- hi
|
||
|
|
- en
|
||
|
|
license: apache-2.0
|
||
|
|
tags:
|
||
|
|
- fine-tuned
|
||
|
|
- gguf
|
||
|
|
- hindi
|
||
|
|
- india
|
||
|
|
- instruction-tuned
|
||
|
|
- llama.cpp
|
||
|
|
- ollama
|
||
|
|
- quantized
|
||
|
|
- qwen3
|
||
|
|
base_model: pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
base_model_relation: quantized
|
||
|
|
---
|
||
|
|
> ⚠️ **Note:** This 0.6B version is undertrained and does not reliably follow Hindi instructions. For a working Hindi model, please use **[Qwen3-4B-Hindi-Instruct-v2](https://huggingface.co/pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2)** ([GGUF here](https://huggingface.co/pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2-GGUF)).
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
|
||
|
|
# Qwen3-0.6B Hindi Instruct v1 — GGUF
|
||
|
|
|
||
|
|
The smallest Hindi-capable instruction model you can run locally — fits in 370MB, runs on any laptop, no GPU needed.
|
||
|
|
|
||
|
|
Fine-tuned from Qwen/Qwen3-0.6B on English to Hindi instruction pairs.
|
||
|
|
Quantized to GGUF for local inference with llama.cpp, LM Studio, and Ollama.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Quick Start
|
||
|
|
|
||
|
|
Download the model using huggingface-cli:
|
||
|
|
|
||
|
|
huggingface-cli download pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1-GGUF Qwen3-0.6B-Hindi-v1-Q4_K_M.gguf --local-dir .
|
||
|
|
|
||
|
|
Run with llama.cpp:
|
||
|
|
|
||
|
|
./llama-cli -m Qwen3-0.6B-Hindi-v1-Q4_K_M.gguf -p "भारत की राजधानी क्या है?" -n 256
|
||
|
|
|
||
|
|
Or open directly in LM Studio — New Model — Search pankajpandey-dev
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Available Versions
|
||
|
|
|
||
|
|
| File | Quantization | Size | RAM Needed | Best For |
|
||
|
|
|------|-------------|------|-----------|---------|
|
||
|
|
| Qwen3-0.6B-Hindi-v1-Q2_K.gguf | Q2_K | 0.28 GB | 1 GB | Minimum hardware |
|
||
|
|
| Qwen3-0.6B-Hindi-v1-Q4_K_M.gguf | Q4_K_M | 0.37 GB | 1.5 GB | Recommended |
|
||
|
|
| Qwen3-0.6B-Hindi-v1-Q5_K_M.gguf | Q5_K_M | 0.41 GB | 2 GB | Better quality |
|
||
|
|
| Qwen3-0.6B-Hindi-v1-Q8_0.gguf | Q8_0 | 0.60 GB | 2.5 GB | Highest quality |
|
||
|
|
|
||
|
|
Not sure which to pick? Always start with Q4_K_M — best balance of speed, size, and quality.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Model Details
|
||
|
|
|
||
|
|
| Property | Value |
|
||
|
|
|----------|-------|
|
||
|
|
| Base Model | Qwen/Qwen3-0.6B |
|
||
|
|
| Parameters | 600M |
|
||
|
|
| Architecture | Qwen2 — fully supported by llama.cpp |
|
||
|
|
| Fine-tune Method | QLoRA with LoRA r=16 alpha=16 |
|
||
|
|
| Training Steps | 60 steps |
|
||
|
|
| Training Data | 2000 English to Hindi instruction pairs |
|
||
|
|
| Max Context | 2048 tokens |
|
||
|
|
| Languages | Hindi and English |
|
||
|
|
| License | Apache 2.0 — commercial use allowed |
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Example Prompts
|
||
|
|
|
||
|
|
Hindi Question Answering:
|
||
|
|
|
||
|
|
User: भारत की राजधानी क्या है?
|
||
|
|
Model: भारत की राजधानी नई दिल्ली है।
|
||
|
|
|
||
|
|
Hindi Instructions:
|
||
|
|
|
||
|
|
User: मुझे चाय बनाने का तरीका बताओ।
|
||
|
|
Model: चाय बनाने के लिए पहले पानी गरम करें...
|
||
|
|
|
||
|
|
Mixed Language:
|
||
|
|
|
||
|
|
User: Python में for loop कैसे लिखते हैं?
|
||
|
|
Model: Python में for loop इस तरह लिखते हैं...
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Compatibility
|
||
|
|
|
||
|
|
| Tool | Status |
|
||
|
|
|------|--------|
|
||
|
|
| llama.cpp | Full support |
|
||
|
|
| LM Studio | Full support |
|
||
|
|
| Ollama | Full support |
|
||
|
|
| Jan | Full support |
|
||
|
|
| Open WebUI | Full support |
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Recommended Settings
|
||
|
|
|
||
|
|
Temperature: 0.7
|
||
|
|
Top-P: 0.9
|
||
|
|
Top-K: 40
|
||
|
|
Repeat Penalty: 1.1
|
||
|
|
Context Length: 2048
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Why This Model?
|
||
|
|
|
||
|
|
- Tiny — 370MB, one of the smallest Hindi-capable GGUF models available
|
||
|
|
- Fast — runs fully on CPU, no GPU required
|
||
|
|
- Hindi-first — specifically trained for Hindi instruction following
|
||
|
|
- Open — Apache 2.0, free for personal and commercial use
|
||
|
|
- Actively maintained — v2 coming soon with more data
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Roadmap
|
||
|
|
|
||
|
|
- Done: v1 — Base Hindi fine-tune on Qwen3-0.6B
|
||
|
|
- Next: v2 — 10x larger dataset, improved Hindi fluency
|
||
|
|
- Next: v3 — better instruction following
|
||
|
|
- Next: Qwen3-1.7B-Hindi — bigger model, same niche
|
||
|
|
- Next: Live demo Space on HuggingFace
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Related Repos
|
||
|
|
|
||
|
|
| Repo | Description |
|
||
|
|
|------|-------------|
|
||
|
|
| pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1 | Full precision model in safetensors format |
|
||
|
|
| pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1-GGUF | This repo — GGUF quantized versions |
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## About the Author
|
||
|
|
|
||
|
|
Made by pankajpandey-dev
|
||
|
|
Building open-source Hindi AI models for India
|
||
|
|
|
||
|
|
Follow for weekly model updates and new Hindi LLM releases.
|
||
|
|
|
||
|
|
Found this useful? Please like this repo — it helps other Hindi speakers find it.
|