68 lines
2.2 KiB
Markdown
68 lines
2.2 KiB
Markdown
|
---
|
||
|
|
base_model: pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2
|
||
|
|
base_model_relation: quantized
|
||
|
|
language:
|
||
|
|
- hi
|
||
|
|
- en
|
||
|
|
license: apache-2.0
|
||
|
|
library_name: gguf
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
tags:
|
||
|
|
- qwen3
|
||
|
|
- hindi
|
||
|
|
- indic
|
||
|
|
- india
|
||
|
|
- gguf
|
||
|
|
- llama.cpp
|
||
|
|
- ollama
|
||
|
|
- lm-studio
|
||
|
|
- instruction-tuned
|
||
|
|
---
|
||
|
|
|
||
|
|
# 🇮🇳 Qwen3-4B Hindi Instruct v2 — GGUF
|
||
|
|
|
||
|
|
GGUF quantizations of **[Qwen3-4B-Hindi-Instruct-v2](https://huggingface.co/pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2)** — a Hindi instruction-tuned Qwen3-4B model. These run **locally on CPU or GPU** with llama.cpp, Ollama, and LM Studio — no Python or heavy setup needed.
|
||
|
|
|
||
|
|
Part of the **Hindi LLM Series**, focused on bringing Indic-language models to local and edge devices.
|
||
|
|
|
||
|
|
## Available Quants
|
||
|
|
|
||
|
|
| File | Quant | Size | Recommended for |
|
||
|
|
|------|-------|------|-----------------|
|
||
|
|
| `Qwen3-4B-Hindi-v2.Q4_K_M.gguf` | Q4_K_M | 2.5 GB | **Best balance** — start here |
|
||
|
|
| `Qwen3-4B-Hindi-v2.Q5_K_M.gguf` | Q5_K_M | 2.9 GB | Higher quality, slightly larger |
|
||
|
|
| `Qwen3-4B-Hindi-v2.Q8_0.gguf` | Q8_0 | 4.3 GB | Near-lossless, maximum quality |
|
||
|
|
|
||
|
|
If unsure, download **Q4_K_M** — it's the best size-to-quality tradeoff for most machines.
|
||
|
|
|
||
|
|
## How to Run
|
||
|
|
|
||
|
|
### Ollama
|
||
|
|
```bash
|
||
|
|
huggingface-cli download pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2-GGUF Qwen3-4B-Hindi-v2.Q4_K_M.gguf --local-dir .
|
||
|
|
ollama create qwen3-hindi -f Modelfile
|
||
|
|
ollama run qwen3-hindi "भारत के बारे में एक रोचक तथ्य बताओ।"
|
||
|
|
```
|
||
|
|
|
||
|
|
### llama.cpp
|
||
|
|
```bash
|
||
|
|
./llama-cli -m Qwen3-4B-Hindi-v2.Q4_K_M.gguf -p "भारत की राजधानी क्या है?" -cnv
|
||
|
|
```
|
||
|
|
|
||
|
|
### LM Studio
|
||
|
|
Search for this repo in LM Studio, download the Q4_K_M file, and chat directly in the GUI.
|
||
|
|
|
||
|
|
## About the Model
|
||
|
|
|
||
|
|
This is a Hindi instruction fine-tune of Qwen3-4B (LoRA via Unsloth, 10K Hindi instruction pairs), quantized to GGUF for efficient local inference. It handles both Hindi (Devanagari) and English.
|
||
|
|
|
||
|
|
For full model details and the original 16-bit weights, see the **[base model card](https://huggingface.co/pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2)**.
|
||
|
|
|
||
|
|
## License
|
||
|
|
|
||
|
|
Apache 2.0 — commercial use allowed.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
*Part of the 🇮🇳 Hindi LLM Series by [pankajpandey-dev](https://huggingface.co/pankajpandey-dev).*
|