---
license: apache-2.0
base_model: Qwen/Qwen2.5-3B-Instruct
language:
- en
- hi
library_name: transformers
pipeline_tag: text-generation
tags:
- legal
- india
- indian-law
- bns
- retrieval-augmented-generation
- qwen2.5
---

# Nyaya-3B-v3 — Indian Legal Information Model

> **⚖️ Not legal advice.** Nyaya is a legal *information / guidance* model, **not a
> legal advisor**. The practice of law in India is reserved to enrolled advocates
> under the Advocates Act, 1961. Always consult a licensed advocate for anything
> consequential. Free legal aid is available via NALSA / DLSA (Legal Services
> Authorities Act, 1987).

Nyaya answers everyday legal questions from Indian citizens in **English, Hindi,
and Hinglish**, cites specific sections of **current law** (BNS / BNSS / BSA,
post-July-2024, with IPC↔BNS bridging), and signals when a real lawyer is needed.

- **Base model:** [`Qwen/Qwen2.5-3B-Instruct`]( https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) (Apache-2.0)
- **Method:** LoRA SFT (bf16, no quantization) using **RAFT** — teacher answers
  regenerated under the inference-time RAG prompt, including deliberate
  retrieval-miss demonstrations. Merged to full weights for release.
- **Designed to run with retrieval (RAG).** The gains below come *with* a dense
  retriever supplying statute passages in context; used bare, it behaves close
  to the base model.

## Intended use & limitations

- **Use for:** plain-language legal information, "how do I…" procedural guidance,
  old→new law (IPC↔BNS) mapping, terminology.
- **Do not use for:** actual legal advice, court filings, or any decision without
  a licensed advocate. Coverage is limited to the acts in the statute DB; it can
  be wrong or incomplete outside them.

## Evaluation (Nyaya-Eval-v0, 500 frozen questions)

Primary metric is **citation accuracy verified against the statute DB**, not loss.
Numbers below are strict exact-match on the frozen eval, with dense RAG (k=8):

| Setting | Strict acc. | Lenient acc. |
|---|---|---|
| Base + dense RAG | 3.4% | — |
| **Nyaya-3B-v3 + dense RAG** | **3.8%** | **5.5%** |

On the held-out *dataset* eval, citation pass-rate is **90.2%** across
1068 examples.

**Honest status:** strict exact-match on the frozen benchmark is still low and the
project's human-eval ship gate has **not** been passed. This is an early research
release, **not** a validated "best" legal model. See the repo's `docs/ROADMAP.md`
go/no-go gates.

## How to use

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("NyayaLabs98/nyaya-3b-v3")
model = AutoModelForCausalLM.from_pretrained("NyayaLabs98/nyaya-3b-v3", device_map="auto")

system = (
    "You are Nyaya, an Indian legal information model. You provide accurate, "
    "plain-language legal guidance for Indian citizens, cite specific sections of "
    "current law (BNS/BNSS/BSA and other acts in force), clearly state uncertainty, "
    "and recommend consulting a licensed advocate for anything consequential. "
    "You provide legal information, not legal advice."
)
# For best results, prepend retrieved statute passages to the user turn (RAG).
messages = [
    {"role": "system", "content": system},
    {"role": "user", "content": "Police FIR nahi likh rahi, kya karu?"},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=512)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
```

## Training curves & ablations

![Program progression](training/figures/program_progression.png)
![RAG ablation](training/figures/rag_ablation.png)
![Checkpoint frozen-eval](training/figures/checkpoint_frozen_eval.png)
![Before/after behaviour](training/figures/behaviour_before_after.png)

## Training details

See `training/` in this repo for the exact config, metric history, and all
figures (PNG + source CSV): `train_v3.yaml`, `training_history_v3.json`,
`rag_eval_rescored.json`, `checkpoint_evals.json`, `figures/`.

## License & attribution

Model weights released under **Apache-2.0**, inheriting the base model's license.
Statutory text is Government of India material (India Code / legislative.gov.in).
See the repository `NOTICE` for data provenance.