70 lines
2.3 KiB
Markdown
70 lines
2.3 KiB
Markdown
---
|
||
license: apache-2.0
|
||
base_model: Qwen/Qwen2.5-1.5B
|
||
language:
|
||
- en
|
||
- sa
|
||
tags:
|
||
- hinduism
|
||
- vedanta
|
||
- sanskrit
|
||
- philosophy
|
||
- qwen2
|
||
pipeline_tag: text-generation
|
||
---
|
||
|
||
# Śruti 1.5B
|
||
|
||
Śruti is a guide to Hindu / Sanātana / Vedic / Vedāntic philosophy — the Vedas, Upaniṣads,
|
||
Bhagavad Gītā, Purāṇas, the six darśanas, Yoga, Tantra, and the Bhakti traditions. It answers
|
||
in clear English and, where relevant, quotes the original Sanskrit (Devanāgarī with IAST
|
||
transliteration) with translation and source.
|
||
|
||
Built by fine-tuning **Qwen2.5-1.5B** (Apache-2.0) with LoRA on hand-authored Q&A covering the
|
||
core texts and schools, curated alongside a ~19M-token corpus of primary sources (Vedas,
|
||
Upaniṣads, 13 Purāṇas, the epics, Dharmaśāstra, darśana literature, Yoga, Tantra, and modern
|
||
expositors such as Dasgupta and Vivekananda).
|
||
|
||
## Usage
|
||
|
||
### transformers
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
repo = "MohitM2/sruti-1.5b"
|
||
tok = AutoTokenizer.from_pretrained(repo)
|
||
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto")
|
||
|
||
messages = [
|
||
{"role": "system", "content": "You are Śruti, a knowledgeable and reverent guide to Hindu philosophy."},
|
||
{"role": "user", "content": "What is the ātman according to Vedānta?"},
|
||
]
|
||
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
|
||
out = model.generate(inputs, max_new_tokens=400, temperature=0.7, top_p=0.9)
|
||
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
|
||
```
|
||
|
||
### Ollama (GGUF)
|
||
|
||
Quantized GGUF files are in the [`gguf/`](./tree/main/gguf) folder (Q4_K_M ≈ 0.9 GB,
|
||
Q8_0 ≈ 1.5 GB). With a Modelfile using the Qwen2.5 chat template:
|
||
|
||
```
|
||
ollama create sruti -f Modelfile
|
||
ollama run sruti "What is the atman according to Vedanta?"
|
||
```
|
||
|
||
## Training
|
||
|
||
- **Base:** Qwen/Qwen2.5-1.5B
|
||
- **Method:** LoRA (r=64, α=128) on all attention + MLP projections, merged into the base
|
||
- **Data:** hand-authored instruction Q&A on Hindu philosophy with Sanskrit citations
|
||
- **Precision:** fp16; trained on a single NVIDIA T4/P100 (Kaggle)
|
||
|
||
## Limitations
|
||
|
||
A 1.5B-parameter model: it can misquote, oversimplify contested doctrinal points, or
|
||
hallucinate citations despite training against it. Verify Sanskrit quotations and sources
|
||
before scholarly use. Not a substitute for study with primary texts or qualified teachers.
|