4.3 KiB
license, base_model, language, library_name, pipeline_tag, tags
| license | base_model | language | library_name | pipeline_tag | tags | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| apache-2.0 | Qwen/Qwen2.5-3B-Instruct |
|
transformers | text-generation |
|
Nyaya-3B-v3 — Indian Legal Information Model
⚖️ Not legal advice. Nyaya is a legal information / guidance model, not a legal advisor. The practice of law in India is reserved to enrolled advocates under the Advocates Act, 1961. Always consult a licensed advocate for anything consequential. Free legal aid is available via NALSA / DLSA (Legal Services Authorities Act, 1987).
Nyaya answers everyday legal questions from Indian citizens in English, Hindi, and Hinglish, cites specific sections of current law (BNS / BNSS / BSA, post-July-2024, with IPC↔BNS bridging), and signals when a real lawyer is needed.
- Base model:
Qwen/Qwen2.5-3B-Instruct(Apache-2.0) - Method: LoRA SFT (bf16, no quantization) using RAFT — teacher answers regenerated under the inference-time RAG prompt, including deliberate retrieval-miss demonstrations. Merged to full weights for release.
- Designed to run with retrieval (RAG). The gains below come with a dense retriever supplying statute passages in context; used bare, it behaves close to the base model.
Intended use & limitations
- Use for: plain-language legal information, "how do I…" procedural guidance, old→new law (IPC↔BNS) mapping, terminology.
- Do not use for: actual legal advice, court filings, or any decision without a licensed advocate. Coverage is limited to the acts in the statute DB; it can be wrong or incomplete outside them.
Evaluation (Nyaya-Eval-v0, 500 frozen questions)
Primary metric is citation accuracy verified against the statute DB, not loss. Numbers below are strict exact-match on the frozen eval, with dense RAG (k=8):
| Setting | Strict acc. | Lenient acc. |
|---|---|---|
| Base + dense RAG | 3.4% | — |
| Nyaya-3B-v3 + dense RAG | 3.8% | 5.5% |
On the held-out dataset eval, citation pass-rate is 90.2% across 1068 examples.
Honest status: strict exact-match on the frozen benchmark is still low and the
project's human-eval ship gate has not been passed. This is an early research
release, not a validated "best" legal model. See the repo's docs/ROADMAP.md
go/no-go gates.
How to use
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("NyayaLabs98/nyaya-3b-v3")
model = AutoModelForCausalLM.from_pretrained("NyayaLabs98/nyaya-3b-v3", device_map="auto")
system = (
"You are Nyaya, an Indian legal information model. You provide accurate, "
"plain-language legal guidance for Indian citizens, cite specific sections of "
"current law (BNS/BNSS/BSA and other acts in force), clearly state uncertainty, "
"and recommend consulting a licensed advocate for anything consequential. "
"You provide legal information, not legal advice."
)
# For best results, prepend retrieved statute passages to the user turn (RAG).
messages = [
{"role": "system", "content": system},
{"role": "user", "content": "Police FIR nahi likh rahi, kya karu?"},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=512)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
Training curves & ablations
Training details
See training/ in this repo for the exact config, metric history, and all
figures (PNG + source CSV): train_v3.yaml, training_history_v3.json,
rag_eval_rescored.json, checkpoint_evals.json, figures/.
License & attribution
Model weights released under Apache-2.0, inheriting the base model's license.
Statutory text is Government of India material (India Code / legislative.gov.in).
See the repository NOTICE for data provenance.