83 lines
2.3 KiB
Markdown
83 lines
2.3 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
base_model: sarvamai/sarvam-1-v0.5
|
||
|
|
tags:
|
||
|
|
- gguf
|
||
|
|
- llama-cpp
|
||
|
|
- quantized
|
||
|
|
- ollama
|
||
|
|
- indic
|
||
|
|
- indian-languages
|
||
|
|
- hindi
|
||
|
|
- multilingual
|
||
|
|
- sarvam
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
- hi
|
||
|
|
- bn
|
||
|
|
- ta
|
||
|
|
- te
|
||
|
|
- kn
|
||
|
|
- ml
|
||
|
|
- mr
|
||
|
|
- pa
|
||
|
|
- gu
|
||
|
|
- or
|
||
|
|
---
|
||
|
|
|
||
|
|
# Sarvam-1-v0.5 GGUF
|
||
|
|
|
||
|
|
GGUF quantized versions of [sarvamai/sarvam-1-v0.5](https://huggingface.co/sarvamai/sarvam-1-v0.5) for local inference with **llama.cpp**, **Ollama**, **LM Studio**, and **GPT4All**.
|
||
|
|
|
||
|
|
Sarvam-1 is an Indian multilingual LLM built by [Sarvam AI](https://www.sarvam.ai/) — supporting **22 Indian languages** including Hindi, Bengali, Tamil, Telugu, Kannada, Malayalam, Marathi, Punjabi, Gujarati, and Odia. Based on Llama architecture with 3.1B parameters.
|
||
|
|
|
||
|
|
## Available Quantizations
|
||
|
|
|
||
|
|
| File | Quant | Size | RAM Needed | Use Case |
|
||
|
|
|------|-------|------|------------|----------|
|
||
|
|
| `sarvam-1-v0.5-Q8_0.gguf` | Q8_0 | 2.5 GB | ~4 GB | Best quality, near-lossless |
|
||
|
|
| `sarvam-1-v0.5-f16.gguf` | F16 | 4.7 GB | ~6 GB | Full precision, maximum quality |
|
||
|
|
|
||
|
|
## How to Use
|
||
|
|
|
||
|
|
### With llama.cpp
|
||
|
|
|
||
|
|
```bash
|
||
|
|
./llama-cli -m sarvam-1-v0.5-Q8_0.gguf -p "भारत की राजधानी क्या है?" -n 256
|
||
|
|
```
|
||
|
|
|
||
|
|
### With Ollama
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Create a Modelfile
|
||
|
|
echo 'FROM ./sarvam-1-v0.5-Q8_0.gguf' > Modelfile
|
||
|
|
ollama create sarvam -f Modelfile
|
||
|
|
ollama run sarvam
|
||
|
|
```
|
||
|
|
|
||
|
|
### With LM Studio
|
||
|
|
|
||
|
|
1. Download the Q8_0 file
|
||
|
|
2. Open LM Studio → Load Model → Select the file
|
||
|
|
3. Start chatting in English or any supported Indian language
|
||
|
|
|
||
|
|
## Model Details
|
||
|
|
|
||
|
|
- **Architecture:** Llama
|
||
|
|
- **Parameters:** 3.1B
|
||
|
|
- **Hidden Size:** 2048
|
||
|
|
- **Layers:** 28
|
||
|
|
- **Attention Heads:** 16
|
||
|
|
- **Context Length:** Check original model card
|
||
|
|
- **Languages:** English + 22 Indian languages (Hindi, Bengali, Tamil, Telugu, Kannada, Malayalam, Marathi, Punjabi, Gujarati, Odia, and more)
|
||
|
|
- **License:** Apache 2.0
|
||
|
|
|
||
|
|
## Original Model
|
||
|
|
|
||
|
|
Built by [Sarvam AI](https://www.sarvam.ai/) — India's leading AI research company. See the original model at [sarvamai/sarvam-1-v0.5](https://huggingface.co/sarvamai/sarvam-1-v0.5).
|
||
|
|
|
||
|
|
## Quantized by
|
||
|
|
|
||
|
|
[Shaswata Tripathy](https://shaswatatripathy.com) | [GitHub](https://github.com/ShaswataTripathy) | [Medium](https://medium.com/@tripathyshaswata) | [LinkedIn](https://linkedin.com/in/shaswata-tripathy) | [Hugging Face](https://huggingface.co/tripathyShaswata)
|