---
language:
- hi
- en
license: apache-2.0
tags:
- fine-tuned
- gguf
- hindi
- india
- instruction-tuned
- llama.cpp
- ollama
- quantized
- qwen3
base_model: pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1
pipeline_tag: text-generation
base_model_relation: quantized
---
> ⚠️ **Note:** This 0.6B version is undertrained and does not reliably follow Hindi instructions. For a working Hindi model, please use **[Qwen3-4B-Hindi-Instruct-v2](https://huggingface.co/pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2)** ([GGUF here](https://huggingface.co/pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2-GGUF)).

---


# Qwen3-0.6B Hindi Instruct v1 — GGUF

The smallest Hindi-capable instruction model you can run locally — fits in 370MB, runs on any laptop, no GPU needed.

Fine-tuned from Qwen/Qwen3-0.6B on English to Hindi instruction pairs.
Quantized to GGUF for local inference with llama.cpp, LM Studio, and Ollama.

---

## Quick Start

Download the model using huggingface-cli:

    huggingface-cli download pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1-GGUF Qwen3-0.6B-Hindi-v1-Q4_K_M.gguf --local-dir .

Run with llama.cpp:

    ./llama-cli -m Qwen3-0.6B-Hindi-v1-Q4_K_M.gguf -p "भारत की राजधानी क्या है?" -n 256

Or open directly in LM Studio — New Model — Search pankajpandey-dev

---

## Available Versions

| File | Quantization | Size | RAM Needed | Best For |
|------|-------------|------|-----------|---------|
| Qwen3-0.6B-Hindi-v1-Q2_K.gguf | Q2_K | 0.28 GB | 1 GB | Minimum hardware |
| Qwen3-0.6B-Hindi-v1-Q4_K_M.gguf | Q4_K_M | 0.37 GB | 1.5 GB | Recommended |
| Qwen3-0.6B-Hindi-v1-Q5_K_M.gguf | Q5_K_M | 0.41 GB | 2 GB | Better quality |
| Qwen3-0.6B-Hindi-v1-Q8_0.gguf | Q8_0 | 0.60 GB | 2.5 GB | Highest quality |

Not sure which to pick? Always start with Q4_K_M — best balance of speed, size, and quality.

---

## Model Details

| Property | Value |
|----------|-------|
| Base Model | Qwen/Qwen3-0.6B |
| Parameters | 600M |
| Architecture | Qwen2 — fully supported by llama.cpp |
| Fine-tune Method | QLoRA with LoRA r=16 alpha=16 |
| Training Steps | 60 steps |
| Training Data | 2000 English to Hindi instruction pairs |
| Max Context | 2048 tokens |
| Languages | Hindi and English |
| License | Apache 2.0 — commercial use allowed |

---

## Example Prompts

Hindi Question Answering:

    User: भारत की राजधानी क्या है?
    Model: भारत की राजधानी नई दिल्ली है।

Hindi Instructions:

    User: मुझे चाय बनाने का तरीका बताओ।
    Model: चाय बनाने के लिए पहले पानी गरम करें...

Mixed Language:

    User: Python में for loop कैसे लिखते हैं?
    Model: Python में for loop इस तरह लिखते हैं...

---

## Compatibility

| Tool | Status |
|------|--------|
| llama.cpp | Full support |
| LM Studio | Full support |
| Ollama | Full support |
| Jan | Full support |
| Open WebUI | Full support |

---

## Recommended Settings

    Temperature:    0.7
    Top-P:          0.9
    Top-K:          40
    Repeat Penalty: 1.1
    Context Length: 2048

---

## Why This Model?

- Tiny — 370MB, one of the smallest Hindi-capable GGUF models available
- Fast — runs fully on CPU, no GPU required
- Hindi-first — specifically trained for Hindi instruction following
- Open — Apache 2.0, free for personal and commercial use
- Actively maintained — v2 coming soon with more data

---

## Roadmap

- Done: v1 — Base Hindi fine-tune on Qwen3-0.6B
- Next: v2 — 10x larger dataset, improved Hindi fluency
- Next: v3 — better instruction following
- Next: Qwen3-1.7B-Hindi — bigger model, same niche
- Next: Live demo Space on HuggingFace

---

## Related Repos

| Repo | Description |
|------|-------------|
| pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1 | Full precision model in safetensors format |
| pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1-GGUF | This repo — GGUF quantized versions |

---

## About the Author

Made by pankajpandey-dev
Building open-source Hindi AI models for India

Follow for weekly model updates and new Hindi LLM releases.

Found this useful? Please like this repo — it helps other Hindi speakers find it.