pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct-v1-GGUF

Files

ModelHub XC 8204f81b73 初始化项目，由ModelHub XC社区提供模型

Model: pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct-v1-GGUF
Source: Original Platform

2026-06-22 15:33:18 +08:00

4.3 KiB

Raw Blame History

license, language, base_model, tags, library_name, pipeline_tag, base_model_relation

license

language

base_model

MiniCPM5-1B-Hindi-Instruct v1 — GGUF Quantizations

GGUF quantized versions of pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct for efficient local inference with llama.cpp, Ollama, LM Studio, and other GGUF-compatible runtimes.

Part of the 🇮🇳 Hindi LLM Series by @pankajpandey-dev.

Available Quantizations

File	Quant	Size	Recommended Use
`MiniCPM5-1B-Hindi-Instruct-Q3_K_M.gguf`	Q3_K_M	~560 MB	Mobile, low-RAM devices, fast inference
`MiniCPM5-1B-Hindi-Instruct-Q4_K_M.gguf`	Q4_K_M	~670 MB	Recommended — best size/quality balance
`MiniCPM5-1B-Hindi-Instruct-Q5_K_M.gguf`	Q5_K_M	~790 MB	Better quality, slightly larger
`MiniCPM5-1B-Hindi-Instruct-Q6_K.gguf`	Q6_K	~900 MB	Near-lossless quality
`MiniCPM5-1B-Hindi-Instruct-Q8_0.gguf`	Q8_0	~1.2 GB	Highest quality, essentially full precision

Quick Start

llama.cpp

huggingface-cli download pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct-v1-GGUF MiniCPM5-1B-Hindi-Instruct-Q4_K_M.gguf --local-dir .

./llama-cli -m MiniCPM5-1B-Hindi-Instruct-Q4_K_M.gguf \
    -p "नमस्ते! बारिश के दिन पर एक छोटी कविता लिखो।" \
    -n 256 --temp 0.7 --top-p 0.9 --repeat-penalty 1.1

Ollama

Create a Modelfile: FROM ./MiniCPM5-1B-Hindi-Instruct-Q4_K_M.gguf PARAMETER temperature 0.7 PARAMETER top_p 0.9 PARAMETER repeat_penalty 1.1

Then run:

ollama create hindi-minicpm5 -f Modelfile
ollama run hindi-minicpm5 "मशीन लर्निंग क्या है?"

LM Studio

Download any .gguf file from this repo
Open LM Studio → Local Models → load the file
Use chat template: ChatML (<|im_start|> / <|im_end|>)

Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama(
    model_path = "MiniCPM5-1B-Hindi-Instruct-Q4_K_M.gguf",
    n_ctx      = 2048,
    n_threads  = 4,
)

response = llm.create_chat_completion(
    messages = [
        {"role": "user", "content": "भारत के बारे में एक रोचक तथ्य बताइए।"}
    ],
    temperature = 0.7,
    top_p       = 0.9,
    max_tokens  = 256,
)
print(response["choices"][0]["message"]["content"])

Recommended Generation Parameters

temperature: 0.7 (range 0.5–0.9)
top_p: 0.9
repeat_penalty: 1.1
max_tokens: 256–512 depending on task

Choosing the Right Quant

Phone / Raspberry Pi / 2GB RAM: Q3_K_M or Q4_K_M
Laptop / desktop CPU: Q4_K_M or Q5_K_M (best default)
Quality-focused workflows: Q6_K or Q8_0
Research / reproducibility: Q8_0

Base Model & Training

These quants are derived from the full-precision merged 16-bit model at pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct, which was fine-tuned from openbmb/MiniCPM5-1B on AI4Bharat's Hindi instruction datasets.

See the main model card for full training details.

Acknowledgements

OpenBMB — MiniCPM5-1B base model
AI4Bharat — indic-instruct-data-v0.1 (anudesh, dolly)
ggerganov/llama.cpp — GGUF format & quantization tools

Citation

@misc{pandey2026minicpm5hindigguf,
  title  = {MiniCPM5-1B-Hindi-Instruct v1 GGUF},
  author = {Pankaj Pandey},
  year   = {2026},
  url    = {https://huggingface.co/pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct-v1-GGUF}
}

Part of an ongoing effort to bring strong open-source LLMs to Indian languages. Feedback welcome via the community tab.

4.3 KiB Raw Blame History Unescape Escape