初始化项目，由ModelHub XC社区提供模型
Model: pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct-v1-GGUF Source: Original Platform
This commit is contained in:
ModelHub XC
132
README.md
Normal file
132
README.md
Normal file
@@ -0,0 +1,132 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- hi
|
||||
- en
|
||||
base_model: pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct
|
||||
tags:
|
||||
- hindi
|
||||
- indic
|
||||
- gguf
|
||||
- quantized
|
||||
- llama.cpp
|
||||
- ollama
|
||||
- lm-studio
|
||||
- minicpm5
|
||||
- instruction-tuned
|
||||
library_name: gguf
|
||||
pipeline_tag: text-generation
|
||||
base_model_relation: quantized
|
||||
---
|
||||
|
||||
# MiniCPM5-1B-Hindi-Instruct v1 — GGUF Quantizations
|
||||
|
||||
GGUF quantized versions of [`pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct`](https://huggingface.co/pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct) for efficient local inference with [llama.cpp](https://github.com/ggerganov/llama.cpp), [Ollama](https://ollama.ai), [LM Studio](https://lmstudio.ai), and other GGUF-compatible runtimes.
|
||||
|
||||
Part of the [🇮🇳 Hindi LLM Series](https://huggingface.co/collections/pankajpandey-dev) by [@pankajpandey-dev](https://huggingface.co/pankajpandey-dev).
|
||||
|
||||
## Available Quantizations
|
||||
|
||||
| File | Quant | Size | Recommended Use |
|
||||
|------|-------|------|-----------------|
|
||||
| `MiniCPM5-1B-Hindi-Instruct-Q3_K_M.gguf` | Q3_K_M | ~560 MB | Mobile, low-RAM devices, fast inference |
|
||||
| `MiniCPM5-1B-Hindi-Instruct-Q4_K_M.gguf` | Q4_K_M | ~670 MB | **Recommended** — best size/quality balance |
|
||||
| `MiniCPM5-1B-Hindi-Instruct-Q5_K_M.gguf` | Q5_K_M | ~790 MB | Better quality, slightly larger |
|
||||
| `MiniCPM5-1B-Hindi-Instruct-Q6_K.gguf` | Q6_K | ~900 MB | Near-lossless quality |
|
||||
| `MiniCPM5-1B-Hindi-Instruct-Q8_0.gguf` | Q8_0 | ~1.2 GB | Highest quality, essentially full precision |
|
||||
|
||||
## Quick Start
|
||||
|
||||
### llama.cpp
|
||||
|
||||
```bash
|
||||
huggingface-cli download pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct-v1-GGUF MiniCPM5-1B-Hindi-Instruct-Q4_K_M.gguf --local-dir .
|
||||
|
||||
./llama-cli -m MiniCPM5-1B-Hindi-Instruct-Q4_K_M.gguf \
|
||||
-p "नमस्ते! बारिश के दिन पर एक छोटी कविता लिखो।" \
|
||||
-n 256 --temp 0.7 --top-p 0.9 --repeat-penalty 1.1
|
||||
```
|
||||
|
||||
### Ollama
|
||||
|
||||
Create a `Modelfile`:
|
||||
FROM ./MiniCPM5-1B-Hindi-Instruct-Q4_K_M.gguf
|
||||
PARAMETER temperature 0.7
|
||||
PARAMETER top_p 0.9
|
||||
PARAMETER repeat_penalty 1.1
|
||||
|
||||
Then run:
|
||||
|
||||
```bash
|
||||
ollama create hindi-minicpm5 -f Modelfile
|
||||
ollama run hindi-minicpm5 "मशीन लर्निंग क्या है?"
|
||||
```
|
||||
|
||||
### LM Studio
|
||||
|
||||
1. Download any `.gguf` file from this repo
|
||||
2. Open LM Studio → Local Models → load the file
|
||||
3. Use chat template: ChatML (`<|im_start|>` / `<|im_end|>`)
|
||||
|
||||
### Python (llama-cpp-python)
|
||||
|
||||
```python
|
||||
from llama_cpp import Llama
|
||||
|
||||
llm = Llama(
|
||||
model_path = "MiniCPM5-1B-Hindi-Instruct-Q4_K_M.gguf",
|
||||
n_ctx = 2048,
|
||||
n_threads = 4,
|
||||
)
|
||||
|
||||
response = llm.create_chat_completion(
|
||||
messages = [
|
||||
{"role": "user", "content": "भारत के बारे में एक रोचक तथ्य बताइए।"}
|
||||
],
|
||||
temperature = 0.7,
|
||||
top_p = 0.9,
|
||||
max_tokens = 256,
|
||||
)
|
||||
print(response["choices"][0]["message"]["content"])
|
||||
```
|
||||
|
||||
## Recommended Generation Parameters
|
||||
|
||||
- **temperature:** 0.7 (range 0.5–0.9)
|
||||
- **top_p:** 0.9
|
||||
- **repeat_penalty:** 1.1
|
||||
- **max_tokens:** 256–512 depending on task
|
||||
|
||||
## Choosing the Right Quant
|
||||
|
||||
- **Phone / Raspberry Pi / 2GB RAM:** Q3_K_M or Q4_K_M
|
||||
- **Laptop / desktop CPU:** Q4_K_M or Q5_K_M (best default)
|
||||
- **Quality-focused workflows:** Q6_K or Q8_0
|
||||
- **Research / reproducibility:** Q8_0
|
||||
|
||||
## Base Model & Training
|
||||
|
||||
These quants are derived from the full-precision merged 16-bit model at [`pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct`](https://huggingface.co/pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct), which was fine-tuned from [`openbmb/MiniCPM5-1B`](https://huggingface.co/openbmb/MiniCPM5-1B) on AI4Bharat's Hindi instruction datasets.
|
||||
|
||||
See the [main model card](https://huggingface.co/pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct) for full training details.
|
||||
|
||||
## Acknowledgements
|
||||
|
||||
- [OpenBMB](https://huggingface.co/openbmb) — MiniCPM5-1B base model
|
||||
- [AI4Bharat](https://huggingface.co/ai4bharat) — `indic-instruct-data-v0.1` (anudesh, dolly)
|
||||
- [ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp) — GGUF format & quantization tools
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@misc{pandey2026minicpm5hindigguf,
|
||||
title = {MiniCPM5-1B-Hindi-Instruct v1 GGUF},
|
||||
author = {Pankaj Pandey},
|
||||
year = {2026},
|
||||
url = {https://huggingface.co/pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct-v1-GGUF}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Part of an ongoing effort to bring strong open-source LLMs to Indian languages. Feedback welcome via the community tab.*
|
||||
Reference in New Issue
Block a user