261 lines
6.7 KiB
Markdown
261 lines
6.7 KiB
Markdown
---
|
|
language:
|
|
- ne
|
|
- en
|
|
license: apache-2.0
|
|
base_model: Qwen/Qwen2.5-7B-Instruct
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- sanu-ai
|
|
- nepal
|
|
- nepali
|
|
- gguf
|
|
- ollama
|
|
- llama-cpp
|
|
- quantized
|
|
- bilingual
|
|
- south-asia
|
|
- low-resource
|
|
---
|
|
|
|
<div align="center">
|
|
|
|
# SANU AI 7B v0.2 GGUF
|
|
|
|
### Run Nepal's First AI Locally — 3 Commands, 5 Minutes
|
|
|
|
**Ollama Ready | Q4_K_M (4.68 GB) | Q8_0 (8.10 GB)**
|
|
|
|
[](https://opensource.org/licenses/Apache-2.0)
|
|
[](https://huggingface.co/Haubaa/SANU-AI-7B-v0.2)
|
|
[](#quick-start)
|
|
[](#)
|
|
|
|
---
|
|
|
|
*From zero to chatting with Nepal's AI in under 5 minutes.*
|
|
|
|
</div>
|
|
|
|
---
|
|
|
|
## Quick Start (Copy-Paste Ready)
|
|
|
|
### Step 1: Install Ollama
|
|
```bash
|
|
# macOS / Linux
|
|
curl -fsSL https://ollama.com/install.sh | sh
|
|
|
|
# Windows — download from https://ollama.com/download
|
|
```
|
|
|
|
### Step 2: Download SANU AI
|
|
```bash
|
|
# Download the Q4_K_M model (4.68 GB — recommended for most users)
|
|
curl -L -o sanu.gguf https://huggingface.co/Haubaa/SANU-AI-7B-v0.2-GGUF/resolve/main/sanu-ai-v02-q4_k_m.gguf
|
|
```
|
|
|
|
### Step 3: Create & Run
|
|
```bash
|
|
cat > Modelfile << 'EOF'
|
|
FROM ./sanu.gguf
|
|
SYSTEM "You are SANU AI, Nepal's first agentic AI assistant. You speak Nepali and English fluently. You have deep knowledge of Nepal's history, culture, geography, and current affairs."
|
|
PARAMETER temperature 0.7
|
|
PARAMETER num_ctx 2048
|
|
TEMPLATE "<|im_start|>system\n{{ .System }}<|im_end|>\n<|im_start|>user\n{{ .Prompt }}<|im_end|>\n<|im_start|>assistant\n"
|
|
EOF
|
|
|
|
ollama create sanu -f Modelfile
|
|
ollama run sanu
|
|
```
|
|
|
|
**That's it. You're now chatting with Nepal's first AI.**
|
|
|
|
---
|
|
|
|
## Available Models
|
|
|
|
| Model | Size | RAM Required | Speed | Best For |
|
|
|-------|------|-------------|-------|----------|
|
|
| **Q4_K_M** (Recommended) | **4.68 GB** | 8 GB+ | Fast | Daily use, laptops, most GPUs |
|
|
| Q8_0 (High Quality) | 8.10 GB | 12 GB+ | Moderate | Maximum quality, research |
|
|
|
|
---
|
|
|
|
## What Can SANU Do?
|
|
|
|
### Talk in Nepali
|
|
```
|
|
>>> नमस्ते, कस्तो छ?
|
|
नमस्ते! मलाई नेपालका सबैभन्दा प्रशंसित AI सहयोजक SANU मा बुझाएका हुँ।
|
|
```
|
|
|
|
### Nepal Knowledge
|
|
```
|
|
>>> Tell me about Nepal
|
|
Nepal, officially the Federal Democratic Republic of Nepal, is a landlocked
|
|
country in South Asia bordered by India to the south and east, China (Tibet)
|
|
to the north... Kathmandu is its capital and largest city. The Himalayas form
|
|
part of Nepal's northern border and include Mount Everest, the world's
|
|
highest peak...
|
|
```
|
|
|
|
### Write Code
|
|
```
|
|
>>> Write a Python hello world that prints namaste in Nepali
|
|
**THINK:** To write a Python script that prints "Namaste" in Nepali...
|
|
```python
|
|
def print_namaste():
|
|
name = u'नमस्ते'
|
|
print(name)
|
|
print_namaste()
|
|
```
|
|
|
|
### Answer in Pure Nepali
|
|
```
|
|
>>> नेपालमा कति जिल्ला छन्?
|
|
नेपालको सङ्घीय शासन प्रणालीले ७५ जिल्लाहरू नियुक्त गर्दछ...
|
|
```
|
|
|
|
---
|
|
|
|
## v0.1 → v0.2: What Changed?
|
|
|
|
<table>
|
|
<tr>
|
|
<th align="center">What</th>
|
|
<th align="center">v0.1</th>
|
|
<th align="center">v0.2</th>
|
|
<th align="center">Improvement</th>
|
|
</tr>
|
|
<tr>
|
|
<td><b>Training Data</b></td>
|
|
<td>290 samples</td>
|
|
<td><b>10,096 samples</b></td>
|
|
<td>35x more data</td>
|
|
</tr>
|
|
<tr>
|
|
<td><b>Loss</b></td>
|
|
<td>1.3724</td>
|
|
<td><b>0.795</b></td>
|
|
<td>42% lower</td>
|
|
</tr>
|
|
<tr>
|
|
<td><b>Hindi Contamination</b></td>
|
|
<td>Severe</td>
|
|
<td><b>Minimal</b></td>
|
|
<td>Clean Nepali</td>
|
|
</tr>
|
|
<tr>
|
|
<td><b>Identity</b></td>
|
|
<td>Confused (says "Qwen")</td>
|
|
<td><b>"I am SANU AI"</b></td>
|
|
<td>Fixed</td>
|
|
</tr>
|
|
<tr>
|
|
<td><b>Nepal Knowledge</b></td>
|
|
<td>Surface-level</td>
|
|
<td><b>Deep (Wikipedia + News)</b></td>
|
|
<td>Massive upgrade</td>
|
|
</tr>
|
|
<tr>
|
|
<td><b>LoRA Rank</b></td>
|
|
<td>r=16</td>
|
|
<td><b>r=32</b></td>
|
|
<td>2x model capacity</td>
|
|
</tr>
|
|
<tr>
|
|
<td><b>Q4_K_M Size</b></td>
|
|
<td>4.36 GB</td>
|
|
<td><b>4.68 GB</b></td>
|
|
<td>Same ballpark</td>
|
|
</tr>
|
|
<tr>
|
|
<td><b>Cost</b></td>
|
|
<td>$0</td>
|
|
<td><b>$0</b></td>
|
|
<td>Still free</td>
|
|
</tr>
|
|
</table>
|
|
|
|
### Response Comparison: v0.1 vs v0.2
|
|
|
|
| Prompt | v0.1 | v0.2 |
|
|
|--------|------|------|
|
|
| तिमी को हो? | "मैं Qwen हूँ, Alibaba Cloud द्वारा बनाया गया..." (Hindi, wrong identity) | "म SANU AI हुँ, नेपालको पहिलो AI सहायक" (Nepali, correct) |
|
|
| Tell me about Nepal | Generic 2-3 sentences | Detailed: history, geography, politics, culture, economy |
|
|
| नमस्ते, कस्तो छ? | "नमस्ते! मैं ठीक हूँ, धन्यवाद!" (Hindi mixed) | Pure Nepali response with SANU identity |
|
|
| Write Python code | Basic output | Structured THINK/PLAN/CODE/VERIFY format |
|
|
| सगरमाथा बारेमा | "माउंट एवरेस्ट दुनिया का सबसे ऊंचा पर्वत है" (Hindi) | Detailed Nepali: coordinates, history, UNESCO heritage |
|
|
|
|
---
|
|
|
|
## Training Details
|
|
|
|
| Spec | Value |
|
|
|------|-------|
|
|
| Base Model | Qwen 2.5 7B Instruct |
|
|
| Method | QLoRA 4-bit |
|
|
| LoRA Rank | r=32, alpha=64 |
|
|
| Data | 10,096 Nepali samples (Wikipedia + News + Custom) |
|
|
| Loss | 0.795 |
|
|
| GPU | NVIDIA P100 (Kaggle free tier) |
|
|
| Time | ~6 hours |
|
|
| Cost | **$0** |
|
|
|
|
---
|
|
|
|
## System Requirements
|
|
|
|
| Component | Minimum | Recommended |
|
|
|-----------|---------|-------------|
|
|
| **RAM** | 8 GB | 16 GB |
|
|
| **GPU VRAM** | Not required | 6 GB+ (for GPU acceleration) |
|
|
| **Disk** | 5 GB (Q4) / 9 GB (Q8) | 10 GB |
|
|
| **OS** | Windows 10+ / macOS 12+ / Linux | Any modern OS |
|
|
|
|
---
|
|
|
|
## Roadmap
|
|
|
|
| Version | Data | Status |
|
|
|---------|------|--------|
|
|
| v0.1 | 290 samples | Released |
|
|
| **v0.2** | **10,096 samples** | **Current** |
|
|
| v0.3 | 50K+ samples | Coming Soon |
|
|
| v1.0 | 200K+ samples | In Planning |
|
|
|
|
---
|
|
|
|
## Why This Matters
|
|
|
|
**30 million Nepali speakers. 125+ languages in Nepal. Zero dedicated AI models — until now.**
|
|
|
|
Every global LLM treats Nepali as noise. They confuse it with Hindi. They know nothing about Dashain, dal bhat, or the Malla dynasty. SANU AI is different — trained on real Nepali text from real Nepali sources.
|
|
|
|
And it cost **$0** to build.
|
|
|
|
---
|
|
|
|
## Citation
|
|
|
|
```bibtex
|
|
@misc{sanu-ai-2026,
|
|
title={SANU AI: Nepal's First Agentic AI Foundation Model},
|
|
author={Haubaa},
|
|
year={2026},
|
|
publisher={HuggingFace},
|
|
url={https://huggingface.co/Haubaa/SANU-AI-7B-v0.2-GGUF}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
<div align="center">
|
|
|
|
**Built in Nepal 🇳🇵 | For Nepal | For the World**
|
|
|
|
*Training cost: $0 | Dreams: Priceless*
|
|
|
|
</div>
|