laabam-ai-3b-v1-gguf/README.md

---
license: other
license_name: agni-community
license_link: https://huggingface.co/laabamone/laabam-ai-3b-v1-gguf/blob/main/LICENSE-AGNI-COMMUNITY.md
language:
  - en
  - hi
  - te
  - kn
  - ta
library_name: gguf
pipeline_tag: text-generation
tags:
  - agni
  - agni-lite
  - laabam-ai
  - multilingual
  - indic
  - llama-cpp
  - quantized
  - conversational
  - code
  - tamil
  - hindi
  - telugu
  - kannada
base_model:
  - Qwen/Qwen2.5-3B-Instruct
---

<div align="center">

# 🔥 Agni Lite 3B v1 — GGUF

### Quantized for Ollama, llama.cpp, and LM Studio

**Run Agni on your laptop, phone, or edge device**

[Full Model (Safetensors)](https://huggingface.co/laabamone/laabam-ai-3b-v1) • [LoRA Adapter](https://huggingface.co/laabamone/laabam-ai-3b-v1-lora) • [Website](https://www.laabamone.com/)

</div>

---

## Available Files

| File | Quantization | Size | Description | Recommendation |
|------|-------------|------|-------------|----------------|
| `laabam-ai-3b-v1-Q4_K_M.gguf` | Q4_K_M | 1.8 GB | 4-bit quantized (medium) | ✅ **Recommended** — best balance |
| `laabam-ai-3b-v1-bf16.gguf` | BF16 | 5.8 GB | Full precision | Maximum quality, needs 8GB+ RAM |

### Which file should I use?

- **Q4_K_M (1.8 GB)** — Best for most users. Runs on 4GB+ RAM. Minimal quality loss.
- **BF16 (5.8 GB)** — Full precision. Use when you need maximum accuracy.

## Quick Start

### Ollama

```bash
# Download the model
huggingface-cli download laabamone/laabam-ai-3b-v1-gguf \
  laabam-ai-3b-v1-Q4_K_M.gguf --local-dir .

# Create a Modelfile
cat > Modelfile << 'EOF'
FROM ./laabam-ai-3b-v1-Q4_K_M.gguf

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

SYSTEM "You are Agni, a helpful multilingual AI assistant created by Laabam One Business Solutions. You support Hindi, Tamil, Telugu, Kannada, and English."

PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER stop "<|im_end|>"
EOF

# Create and run
ollama create agni-lite -f Modelfile
ollama run agni-lite "Write hello world in Python"
```

### llama.cpp

```bash
./llama-cli -m laabam-ai-3b-v1-Q4_K_M.gguf \
  -p "You are Agni, a helpful assistant.\n\nUser: Tell me about Tamil Nadu.\nAssistant:" \
  -n 256 --temp 0.7
```

### LM Studio

1. Download `laabam-ai-3b-v1-Q4_K_M.gguf`
2. Place in your LM Studio models directory
3. Load and chat

### Python (llama-cpp-python)

```python
from llama_cpp import Llama

llm = Llama(
    model_path="laabam-ai-3b-v1-Q4_K_M.gguf",
    n_ctx=1024,
    n_threads=8,
)

output = llm.create_chat_completion(messages=[
    {"role": "system", "content": "You are Agni, a helpful multilingual AI assistant."},
    {"role": "user", "content": "भारत के बारे में बताओ।"}
])

print(output["choices"][0]["message"]["content"])
```

### Tamil Example

```python
output = llm.create_chat_completion(messages=[
    {"role": "system", "content": "நீங்கள் அக்னி AI. தமிழில் பதிலளிக்கவும்."},
    {"role": "user", "content": "சிலப்பதிகாரம் பற்றி சொல்லுங்கள்."}
])
print(output["choices"][0]["message"]["content"])
```

## Model Details

| Detail | Value |
|--------|-------|
| **Model Name** | Agni Lite 3B v1 |
| **Developer** | Laabam One Business Solutions Pvt Ltd |
| **Architecture** | Qwen2.5 (3B parameters) |
| **Training Method** | QLoRA (r=16, alpha=32, 4-bit NF4) with Unsloth |
| **Training Data** | ~98,000 curated multilingual samples |
| **Quantization Tool** | llama.cpp (convert + quantize) |
| **License** | [Agni Community License](LICENSE-AGNI-COMMUNITY.md) |

## Capabilities

- 🇮🇳 **Indian Languages** — Native Hindi, Tamil, Telugu, Kannada support
- 💻 **Coding** — Python, JavaScript, Bash, SQL, and more
- 🧠 **Reasoning** — Math, logic, and step-by-step problem solving
- 🤖 **Agentic** — Function calling and tool use
- 🏢 **Business** — Customer support, ERP, and automation

## Limitations

- 3B parameter model — may struggle with very complex multi-step reasoning
- Quantized versions have slight quality reduction vs full-precision
- Indian language quality is improving; English is currently stronger
- May generate incorrect or fabricated information
- Not a substitute for professional medical, legal, or financial advice

## Citation

```bibtex
@misc{agni-lite-3b-v1-gguf,
  title={Agni Lite 3B v1: India's Multilingual AI Assistant (GGUF)},
  author={Laabam One Business Solutions Pvt Ltd},
  year={2026},
  url={https://huggingface.co/laabamone/laabam-ai-3b-v1-gguf}
}
```

---

<div align="center">

Built with 🔥 in India by [Laabam One Business Solutions Pvt Ltd](https://www.laabamone.com/)

[Website](https://www.laabamone.com/) • [Contact](mailto:support@laabamone.com)

</div>