Qwen3-4B-Hindi-Instruct-v2-GGUF/README.md

base_model, base_model_relation, language, license, library_name, pipeline_tag, tags

base_model	base_model_relation	language	license	library_name	pipeline_tag	tags
pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2	quantized	hi en	apache-2.0	gguf	text-generation	qwen3 hindi indic india gguf llama.cpp ollama lm-studio instruction-tuned

🇮🇳 Qwen3-4B Hindi Instruct v2 — GGUF

GGUF quantizations of Qwen3-4B-Hindi-Instruct-v2 — a Hindi instruction-tuned Qwen3-4B model. These run locally on CPU or GPU with llama.cpp, Ollama, and LM Studio — no Python or heavy setup needed.

Part of the Hindi LLM Series, focused on bringing Indic-language models to local and edge devices.

Available Quants

File	Quant	Size	Recommended for
Qwen3-4B-Hindi-v2.Q4_K_M.gguf	Q4_K_M	2.5 GB	Best balance — start here
Qwen3-4B-Hindi-v2.Q5_K_M.gguf	Q5_K_M	2.9 GB	Higher quality, slightly larger
Qwen3-4B-Hindi-v2.Q8_0.gguf	Q8_0	4.3 GB	Near-lossless, maximum quality

If unsure, download Q4_K_M — it's the best size-to-quality tradeoff for most machines.

How to Run

Ollama

huggingface-cli download pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2-GGUF Qwen3-4B-Hindi-v2.Q4_K_M.gguf --local-dir .
ollama create qwen3-hindi -f Modelfile
ollama run qwen3-hindi "भारत के बारे में एक रोचक तथ्य बताओ।"

llama.cpp

./llama-cli -m Qwen3-4B-Hindi-v2.Q4_K_M.gguf -p "भारत की राजधानी क्या है?" -cnv

LM Studio

Search for this repo in LM Studio, download the Q4_K_M file, and chat directly in the GUI.

About the Model

This is a Hindi instruction fine-tune of Qwen3-4B (LoRA via Unsloth, 10K Hindi instruction pairs), quantized to GGUF for efficient local inference. It handles both Hindi (Devanagari) and English.

For full model details and the original 16-bit weights, see the base model card.

License

Apache 2.0 — commercial use allowed.

---

Part of the 🇮🇳 Hindi LLM Series by pankajpandey-dev.