70 lines
2.3 KiB
Markdown
70 lines
2.3 KiB
Markdown
|
---
|
|||
|
|
license: apache-2.0
|
|||
|
|
base_model: Qwen/Qwen2.5-1.5B
|
|||
|
|
language:
|
|||
|
|
- en
|
|||
|
|
- sa
|
|||
|
|
tags:
|
|||
|
|
- hinduism
|
|||
|
|
- vedanta
|
|||
|
|
- sanskrit
|
|||
|
|
- philosophy
|
|||
|
|
- qwen2
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Śruti 1.5B
|
|||
|
|
|
|||
|
|
Śruti is a guide to Hindu / Sanātana / Vedic / Vedāntic philosophy — the Vedas, Upaniṣads,
|
|||
|
|
Bhagavad Gītā, Purāṇas, the six darśanas, Yoga, Tantra, and the Bhakti traditions. It answers
|
|||
|
|
in clear English and, where relevant, quotes the original Sanskrit (Devanāgarī with IAST
|
|||
|
|
transliteration) with translation and source.
|
|||
|
|
|
|||
|
|
Built by fine-tuning **Qwen2.5-1.5B** (Apache-2.0) with LoRA on hand-authored Q&A covering the
|
|||
|
|
core texts and schools, curated alongside a ~19M-token corpus of primary sources (Vedas,
|
|||
|
|
Upaniṣads, 13 Purāṇas, the epics, Dharmaśāstra, darśana literature, Yoga, Tantra, and modern
|
|||
|
|
expositors such as Dasgupta and Vivekananda).
|
|||
|
|
|
|||
|
|
## Usage
|
|||
|
|
|
|||
|
|
### transformers
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|||
|
|
|
|||
|
|
repo = "MohitM2/sruti-1.5b"
|
|||
|
|
tok = AutoTokenizer.from_pretrained(repo)
|
|||
|
|
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto")
|
|||
|
|
|
|||
|
|
messages = [
|
|||
|
|
{"role": "system", "content": "You are Śruti, a knowledgeable and reverent guide to Hindu philosophy."},
|
|||
|
|
{"role": "user", "content": "What is the ātman according to Vedānta?"},
|
|||
|
|
]
|
|||
|
|
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
|
|||
|
|
out = model.generate(inputs, max_new_tokens=400, temperature=0.7, top_p=0.9)
|
|||
|
|
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### Ollama (GGUF)
|
|||
|
|
|
|||
|
|
Quantized GGUF files are in the [`gguf/`](./tree/main/gguf) folder (Q4_K_M ≈ 0.9 GB,
|
|||
|
|
Q8_0 ≈ 1.5 GB). With a Modelfile using the Qwen2.5 chat template:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
ollama create sruti -f Modelfile
|
|||
|
|
ollama run sruti "What is the atman according to Vedanta?"
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Training
|
|||
|
|
|
|||
|
|
- **Base:** Qwen/Qwen2.5-1.5B
|
|||
|
|
- **Method:** LoRA (r=64, α=128) on all attention + MLP projections, merged into the base
|
|||
|
|
- **Data:** hand-authored instruction Q&A on Hindu philosophy with Sanskrit citations
|
|||
|
|
- **Precision:** fp16; trained on a single NVIDIA T4/P100 (Kaggle)
|
|||
|
|
|
|||
|
|
## Limitations
|
|||
|
|
|
|||
|
|
A 1.5B-parameter model: it can misquote, oversimplify contested doctrinal points, or
|
|||
|
|
hallucinate citations despite training against it. Verify Sanskrit quotations and sources
|
|||
|
|
before scholarly use. Not a substitute for study with primary texts or qualified teachers.
|