初始化项目，由ModelHub XC社区提供模型
Model: MrRoyaleAce/nyaya-7b Source: Original Platform
This commit is contained in:
ModelHub XC
319
README.md
Normal file
319
README.md
Normal file
@@ -0,0 +1,319 @@
|
||||
---
|
||||
language:
|
||||
- en
|
||||
- hi
|
||||
license: apache-2.0
|
||||
base_model: mistralai/Mistral-7B-Instruct-v0.3
|
||||
tags:
|
||||
- legal
|
||||
- indian-legal
|
||||
- information-extraction
|
||||
- nlp
|
||||
- qlora
|
||||
- peft
|
||||
- finetuned
|
||||
- mistral
|
||||
- json-extraction
|
||||
- legal-nlp
|
||||
datasets:
|
||||
- d0r1h/ILSum
|
||||
- law-ai/InLegalNLP
|
||||
pipeline_tag: text-generation
|
||||
library_name: transformers
|
||||
model-index:
|
||||
- name: nyaya-7b
|
||||
results:
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Structured Legal Information Extraction
|
||||
metrics:
|
||||
- type: f1
|
||||
value: 0.425
|
||||
name: Statute F1
|
||||
- type: accuracy
|
||||
value: 0.64
|
||||
name: Outcome Accuracy
|
||||
- type: other
|
||||
value: 0.427
|
||||
name: Hallucination Rate (lower is better)
|
||||
- type: other
|
||||
value: 0.86
|
||||
name: JSON Validity Rate
|
||||
---
|
||||
|
||||
# 🏛️ Nyaya-7B — Indian Legal Judgment Parser
|
||||
|
||||
<p align="center">
|
||||
<img src="https://img.shields.io/badge/Base%20Model-Mistral--7B--Instruct--v0.3-blue" />
|
||||
<img src="https://img.shields.io/badge/Method-QLoRA%20(4--bit%20NF4)-orange" />
|
||||
<img src="https://img.shields.io/badge/Domain-Indian%20Legal%20NLP-green" />
|
||||
<img src="https://img.shields.io/badge/License-Apache%202.0-red" />
|
||||
<img src="https://img.shields.io/badge/Language-English%20%7C%20Hindi-purple" />
|
||||
</p>
|
||||
|
||||
**Nyaya-7B** is a domain-adapted, instruction-finetuned version of [Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3), trained on **10,000+ Indian Supreme Court and High Court judgments** to extract structured legal information into clean, validated JSON — at **zero API cost**, fully offline.
|
||||
|
||||
> **"Nyaya" (न्याय)** means *justice* in Sanskrit and Hindi.
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What it does
|
||||
|
||||
Given raw Indian court judgment text, Nyaya-7B extracts a full structured JSON covering:
|
||||
|
||||
| Field | Description |
|
||||
|---|---|
|
||||
| `case_name` | Petitioner v. Respondent |
|
||||
| `citation` | AIR / SCC / SCR citation |
|
||||
| `court` | Full court name (Supreme Court, High Court, etc.) |
|
||||
| `year` | Year of judgment |
|
||||
| `petitioner` / `respondent` | Party names |
|
||||
| `subject_matter` | Criminal / Civil / Constitutional / Tax / ... |
|
||||
| `statutes_cited` | List of Acts + Sections + descriptions |
|
||||
| `precedents_cited` | AIR/SCC citations with case names |
|
||||
| `legal_issues` | Issues framed by the court |
|
||||
| `holding` | Court's decision and reasoning (1–3 sentences) |
|
||||
| `outcome` | dismissed / allowed / disposed / remanded / modified |
|
||||
|
||||
---
|
||||
|
||||
## 📊 Benchmark Results
|
||||
|
||||
Evaluated on a **50-case held-out test set** of Indian SC/HC judgments, benchmarked head-to-head against Gemini 2.5 Flash:
|
||||
|
||||
| Metric | Gemini 2.5 Flash | **Nyaya-7B** | Winner |
|
||||
|---|---|---|---|
|
||||
| **Statute F1** | 0.227 | **0.425** | 🏆 Nyaya-7B (+87%) |
|
||||
| **Outcome Accuracy** | 0.20 | **0.64** | 🏆 Nyaya-7B (+220%) |
|
||||
| **Hallucination Rate** ↓ | 0.775 | **0.427** | 🏆 Nyaya-7B (−45%) |
|
||||
| **JSON Validity** | 0.98 | 0.86 | Gemini Flash |
|
||||
| **Field Coverage** | 0.893 | 0.855 | Gemini Flash |
|
||||
| **Cost per Judgment** | ~$0.0001 | **$0.00** | 🏆 Nyaya-7B |
|
||||
|
||||
> Nyaya-7B achieves **3.2× higher outcome classification accuracy** and **45% lower hallucination rate** than Gemini 2.5 Flash, while running entirely offline at zero cost.
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
pip install transformers torch accelerate bitsandbytes
|
||||
```
|
||||
|
||||
### Load & Run (GPU, 4-bit quantized)
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, BitsAndBytesConfig
|
||||
import torch, json
|
||||
|
||||
MODEL_ID = "mrroyaleace/nyaya-7b" # replace with your HuggingFace repo
|
||||
|
||||
bnb_config = BitsAndBytesConfig(
|
||||
load_in_4bit=True,
|
||||
bnb_4bit_quant_type="nf4",
|
||||
bnb_4bit_compute_dtype=torch.float16,
|
||||
)
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
|
||||
tokenizer.pad_token = tokenizer.eos_token
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
MODEL_ID,
|
||||
quantization_config=bnb_config,
|
||||
device_map="auto",
|
||||
)
|
||||
|
||||
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
|
||||
```
|
||||
|
||||
### Extract structured data from a judgment
|
||||
|
||||
```python
|
||||
judgment_text = """
|
||||
IN THE SUPREME COURT OF INDIA
|
||||
Criminal Appeal No. 1234 of 2022
|
||||
|
||||
State of Punjab ...Appellant
|
||||
Versus
|
||||
Gurpreet Singh ...Respondent
|
||||
|
||||
JUDGMENT
|
||||
|
||||
The appellant challenges the High Court's order acquitting the respondent
|
||||
of charges under Section 302 IPC read with Section 34 IPC...
|
||||
"""
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": "You are Nyaya, a specialized Indian legal extraction model."},
|
||||
{"role": "user", "content": f"Extract structured data from this judgment and return JSON:\n\n{judgment_text}"}
|
||||
]
|
||||
|
||||
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||||
|
||||
raw_output = pipe(
|
||||
prompt,
|
||||
max_new_tokens=512,
|
||||
do_sample=False,
|
||||
return_full_text=False,
|
||||
pad_token_id=tokenizer.eos_token_id,
|
||||
)[0]["generated_text"]
|
||||
|
||||
result = json.loads(raw_output.strip())
|
||||
print(result)
|
||||
```
|
||||
|
||||
### Expected Output
|
||||
|
||||
```json
|
||||
{
|
||||
"case_name": "State of Punjab v. Gurpreet Singh",
|
||||
"citation": null,
|
||||
"court": "Supreme Court of India",
|
||||
"year": 2022,
|
||||
"petitioner": "State of Punjab",
|
||||
"respondent": "Gurpreet Singh",
|
||||
"subject_matter": "Criminal",
|
||||
"statutes_cited": [
|
||||
{"act": "Indian Penal Code", "section": "302", "description": "Punishment for murder"},
|
||||
{"act": "Indian Penal Code", "section": "34", "description": "Acts done by several persons in furtherance of common intention"}
|
||||
],
|
||||
"precedents_cited": [],
|
||||
"legal_issues": [
|
||||
"Whether the High Court was justified in acquitting the respondent under Section 302 IPC?"
|
||||
],
|
||||
"holding": "The Supreme Court examined the evidence and found the High Court's reasoning sound...",
|
||||
"outcome": "dismissed"
|
||||
}
|
||||
```
|
||||
|
||||
### CPU Inference (no GPU required)
|
||||
|
||||
```python
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
MODEL_ID,
|
||||
torch_dtype=torch.float32,
|
||||
device_map="cpu",
|
||||
low_cpu_mem_usage=True,
|
||||
)
|
||||
# Note: CPU inference is significantly slower (~5–15 min per judgment)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Training Details
|
||||
|
||||
| Parameter | Value |
|
||||
|---|---|
|
||||
| **Base model** | mistralai/Mistral-7B-Instruct-v0.3 |
|
||||
| **Fine-tuning method** | QLoRA (Quantized Low-Rank Adaptation) |
|
||||
| **Quantization** | 4-bit NF4 with double quantization |
|
||||
| **LoRA rank** | 16 |
|
||||
| **LoRA alpha** | 32 |
|
||||
| **LoRA dropout** | 0.05 |
|
||||
| **Target modules** | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
||||
| **Training samples** | ~10,000 Indian SC/HC judgment pairs |
|
||||
| **Epochs** | 3 |
|
||||
| **Effective batch size** | 16 (batch 2 × grad_accum 8) |
|
||||
| **Learning rate** | 2e-4 |
|
||||
| **LR scheduler** | Cosine |
|
||||
| **Optimizer** | paged_adamw_8bit |
|
||||
| **Max sequence length** | 2048 tokens |
|
||||
| **Hardware** | Kaggle T4 × 2 (32 GB VRAM total) |
|
||||
| **Training time** | ~8 hours |
|
||||
| **Compute dtype** | float16 |
|
||||
|
||||
### Training Infrastructure
|
||||
|
||||
- Fine-tuned on **Kaggle T4 × 2** GPU notebooks
|
||||
- Monitored with **Weights & Biases** (JSON validity rate logged per epoch)
|
||||
- Adapter merged into base model weights with `merge_and_unload()` for zero inference overhead
|
||||
|
||||
---
|
||||
|
||||
## 📚 Training Data
|
||||
|
||||
The model was trained on ~2,000 Indian court judgment pairs curated and labeled from:
|
||||
|
||||
- **[ILSum](https://huggingface.co/datasets/d0r1h/ILSum)** — Indian Legal Summarization dataset (Supreme Court judgments)
|
||||
- **[InLegalNLP](https://huggingface.co/datasets/law-ai/InLegalNLP)** — Indian Legal NLP benchmark corpus
|
||||
|
||||
Labels were auto-generated using **Gemini 2.5 Flash** as a labelling oracle on the raw judgment texts, following the canonical extraction schema, then validated for JSON structure and field completeness.
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Limitations
|
||||
|
||||
- **Not for legal advice:** This model extracts structured information only. It does not provide legal opinions or advice. Always consult a qualified lawyer for legal matters.
|
||||
- **Pre-1950 judgments:** May perform poorly on archaic legal language from older judgments.
|
||||
- **Hindi/regional language text:** Primarily trained on English-language judgments; performance degrades on mixed-language or vernacular text.
|
||||
- **Scanned/handwritten PDFs:** Model accepts only clean text input — OCR preprocessing is required for scanned documents.
|
||||
- **Citation hallucination:** Significantly reduced (42.7% vs 77.5% baseline), but the model can still occasionally generate plausible-but-incorrect section numbers. Always validate critical citations against primary sources.
|
||||
- **Novel statutes:** Statutes not well-represented in training data (e.g., recent 2023–24 acts) may have lower extraction accuracy.
|
||||
|
||||
---
|
||||
|
||||
## ✅ Intended Use
|
||||
|
||||
- ⚖️ Legal research and document processing automation
|
||||
- 🤖 Paralegal workflow tools and legal analytics dashboards
|
||||
- 📖 Academic research on Indian legal NLP
|
||||
- 🔍 Building legal search and knowledge graph systems
|
||||
- 📊 Bulk digitization of case records
|
||||
|
||||
## ❌ Out-of-Scope Use
|
||||
|
||||
- Providing legal advice to individuals
|
||||
- Making or influencing judicial decisions
|
||||
- Use in actual legal proceedings without qualified human review
|
||||
- Any high-stakes decision-making without validation
|
||||
|
||||
---
|
||||
|
||||
## 📄 Output Schema
|
||||
|
||||
```python
|
||||
{
|
||||
"case_name": str, # "Petitioner v. Respondent"
|
||||
"citation": str | None, # "AIR 1997 SC 3986" or null
|
||||
"court": str, # Full court name
|
||||
"year": int | None, # 4-digit year
|
||||
"petitioner": str,
|
||||
"respondent": str,
|
||||
"subject_matter": str | None, # Criminal | Civil | Constitutional | ...
|
||||
"statutes_cited": [{"act": str, "section": str, "description": str}],
|
||||
"precedents_cited": [{"citation": str, "case_name": str | None}],
|
||||
"legal_issues": [str],
|
||||
"holding": str, # 1-3 sentence summary
|
||||
"outcome": str # dismissed | allowed | disposed | remanded | modified
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔗 Related Resources
|
||||
|
||||
- 🐙 **GitHub:** [nyaya-legal-ai](https://github.com/shubace/nyaya-legal-ai) — Full project source code including FastAPI backend, LangGraph pipeline, and Streamlit UI
|
||||
- 📊 **Base Model:** [mistralai/Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3)
|
||||
- 📁 **Dataset (ILSum):** [d0r1h/ILSum](https://huggingface.co/datasets/d0r1h/ILSum)
|
||||
|
||||
---
|
||||
|
||||
## 📝 Citation
|
||||
|
||||
If you use Nyaya-7B in your research or applications, please cite:
|
||||
|
||||
```bibtex
|
||||
@misc{nyaya7b2026,
|
||||
title = {Nyaya-7B: A QLoRA Fine-tuned LLM for Indian Legal Judgment Parsing},
|
||||
author = {Shubham Suman},
|
||||
year = {2026},
|
||||
url = {https://huggingface.co/mrroyaleace/nyaya-7b},
|
||||
note = {Fine-tuned from mistralai/Mistral-7B-Instruct-v0.3 on 2,000+ Indian SC/HC judgments}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Built with ❤️ for the Indian legal research community. Nyaya-7B is open-source and free to use under the Apache 2.0 license.*
|
||||
Reference in New Issue
Block a user