--- language: - en - hi license: apache-2.0 base_model: mistralai/Mistral-7B-Instruct-v0.3 tags: - legal - indian-legal - information-extraction - nlp - qlora - peft - finetuned - mistral - json-extraction - legal-nlp datasets: - d0r1h/ILSum - law-ai/InLegalNLP pipeline_tag: text-generation library_name: transformers model-index: - name: nyaya-7b results: - task: type: text-generation name: Structured Legal Information Extraction metrics: - type: f1 value: 0.425 name: Statute F1 - type: accuracy value: 0.64 name: Outcome Accuracy - type: other value: 0.427 name: Hallucination Rate (lower is better) - type: other value: 0.86 name: JSON Validity Rate --- # ๐Ÿ›๏ธ Nyaya-7B โ€” Indian Legal Judgment Parser

**Nyaya-7B** is a domain-adapted, instruction-finetuned version of [Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3), trained on **10,000+ Indian Supreme Court and High Court judgments** to extract structured legal information into clean, validated JSON โ€” at **zero API cost**, fully offline. > **"Nyaya" (เคจเฅเคฏเคพเคฏ)** means *justice* in Sanskrit and Hindi. --- ## ๐ŸŽฏ What it does Given raw Indian court judgment text, Nyaya-7B extracts a full structured JSON covering: | Field | Description | |---|---| | `case_name` | Petitioner v. Respondent | | `citation` | AIR / SCC / SCR citation | | `court` | Full court name (Supreme Court, High Court, etc.) | | `year` | Year of judgment | | `petitioner` / `respondent` | Party names | | `subject_matter` | Criminal / Civil / Constitutional / Tax / ... | | `statutes_cited` | List of Acts + Sections + descriptions | | `precedents_cited` | AIR/SCC citations with case names | | `legal_issues` | Issues framed by the court | | `holding` | Court's decision and reasoning (1โ€“3 sentences) | | `outcome` | dismissed / allowed / disposed / remanded / modified | --- ## ๐Ÿ“Š Benchmark Results Evaluated on a **50-case held-out test set** of Indian SC/HC judgments, benchmarked head-to-head against Gemini 2.5 Flash: | Metric | Gemini 2.5 Flash | **Nyaya-7B** | Winner | |---|---|---|---| | **Statute F1** | 0.227 | **0.425** | ๐Ÿ† Nyaya-7B (+87%) | | **Outcome Accuracy** | 0.20 | **0.64** | ๐Ÿ† Nyaya-7B (+220%) | | **Hallucination Rate** โ†“ | 0.775 | **0.427** | ๐Ÿ† Nyaya-7B (โˆ’45%) | | **JSON Validity** | 0.98 | 0.86 | Gemini Flash | | **Field Coverage** | 0.893 | 0.855 | Gemini Flash | | **Cost per Judgment** | ~$0.0001 | **$0.00** | ๐Ÿ† Nyaya-7B | > Nyaya-7B achieves **3.2ร— higher outcome classification accuracy** and **45% lower hallucination rate** than Gemini 2.5 Flash, while running entirely offline at zero cost. --- ## ๐Ÿš€ Quick Start ### Installation ```bash pip install transformers torch accelerate bitsandbytes ``` ### Load & Run (GPU, 4-bit quantized) ```python from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, BitsAndBytesConfig import torch, json MODEL_ID = "mrroyaleace/nyaya-7b" # replace with your HuggingFace repo bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, ) tokenizer = AutoTokenizer.from_pretrained(MODEL_ID) tokenizer.pad_token = tokenizer.eos_token model = AutoModelForCausalLM.from_pretrained( MODEL_ID, quantization_config=bnb_config, device_map="auto", ) pipe = pipeline("text-generation", model=model, tokenizer=tokenizer) ``` ### Extract structured data from a judgment ```python judgment_text = """ IN THE SUPREME COURT OF INDIA Criminal Appeal No. 1234 of 2022 State of Punjab ...Appellant Versus Gurpreet Singh ...Respondent JUDGMENT The appellant challenges the High Court's order acquitting the respondent of charges under Section 302 IPC read with Section 34 IPC... """ messages = [ {"role": "system", "content": "You are Nyaya, a specialized Indian legal extraction model."}, {"role": "user", "content": f"Extract structured data from this judgment and return JSON:\n\n{judgment_text}"} ] prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) raw_output = pipe( prompt, max_new_tokens=512, do_sample=False, return_full_text=False, pad_token_id=tokenizer.eos_token_id, )[0]["generated_text"] result = json.loads(raw_output.strip()) print(result) ``` ### Expected Output ```json { "case_name": "State of Punjab v. Gurpreet Singh", "citation": null, "court": "Supreme Court of India", "year": 2022, "petitioner": "State of Punjab", "respondent": "Gurpreet Singh", "subject_matter": "Criminal", "statutes_cited": [ {"act": "Indian Penal Code", "section": "302", "description": "Punishment for murder"}, {"act": "Indian Penal Code", "section": "34", "description": "Acts done by several persons in furtherance of common intention"} ], "precedents_cited": [], "legal_issues": [ "Whether the High Court was justified in acquitting the respondent under Section 302 IPC?" ], "holding": "The Supreme Court examined the evidence and found the High Court's reasoning sound...", "outcome": "dismissed" } ``` ### CPU Inference (no GPU required) ```python model = AutoModelForCausalLM.from_pretrained( MODEL_ID, torch_dtype=torch.float32, device_map="cpu", low_cpu_mem_usage=True, ) # Note: CPU inference is significantly slower (~5โ€“15 min per judgment) ``` --- ## ๐Ÿ”ง Training Details | Parameter | Value | |---|---| | **Base model** | mistralai/Mistral-7B-Instruct-v0.3 | | **Fine-tuning method** | QLoRA (Quantized Low-Rank Adaptation) | | **Quantization** | 4-bit NF4 with double quantization | | **LoRA rank** | 16 | | **LoRA alpha** | 32 | | **LoRA dropout** | 0.05 | | **Target modules** | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | | **Training samples** | ~10,000 Indian SC/HC judgment pairs | | **Epochs** | 3 | | **Effective batch size** | 16 (batch 2 ร— grad_accum 8) | | **Learning rate** | 2e-4 | | **LR scheduler** | Cosine | | **Optimizer** | paged_adamw_8bit | | **Max sequence length** | 2048 tokens | | **Hardware** | Kaggle T4 ร— 2 (32 GB VRAM total) | | **Training time** | ~8 hours | | **Compute dtype** | float16 | ### Training Infrastructure - Fine-tuned on **Kaggle T4 ร— 2** GPU notebooks - Monitored with **Weights & Biases** (JSON validity rate logged per epoch) - Adapter merged into base model weights with `merge_and_unload()` for zero inference overhead --- ## ๐Ÿ“š Training Data The model was trained on ~2,000 Indian court judgment pairs curated and labeled from: - **[ILSum](https://huggingface.co/datasets/d0r1h/ILSum)** โ€” Indian Legal Summarization dataset (Supreme Court judgments) - **[InLegalNLP](https://huggingface.co/datasets/law-ai/InLegalNLP)** โ€” Indian Legal NLP benchmark corpus Labels were auto-generated using **Gemini 2.5 Flash** as a labelling oracle on the raw judgment texts, following the canonical extraction schema, then validated for JSON structure and field completeness. --- ## โš ๏ธ Limitations - **Not for legal advice:** This model extracts structured information only. It does not provide legal opinions or advice. Always consult a qualified lawyer for legal matters. - **Pre-1950 judgments:** May perform poorly on archaic legal language from older judgments. - **Hindi/regional language text:** Primarily trained on English-language judgments; performance degrades on mixed-language or vernacular text. - **Scanned/handwritten PDFs:** Model accepts only clean text input โ€” OCR preprocessing is required for scanned documents. - **Citation hallucination:** Significantly reduced (42.7% vs 77.5% baseline), but the model can still occasionally generate plausible-but-incorrect section numbers. Always validate critical citations against primary sources. - **Novel statutes:** Statutes not well-represented in training data (e.g., recent 2023โ€“24 acts) may have lower extraction accuracy. --- ## โœ… Intended Use - โš–๏ธ Legal research and document processing automation - ๐Ÿค– Paralegal workflow tools and legal analytics dashboards - ๐Ÿ“– Academic research on Indian legal NLP - ๐Ÿ” Building legal search and knowledge graph systems - ๐Ÿ“Š Bulk digitization of case records ## โŒ Out-of-Scope Use - Providing legal advice to individuals - Making or influencing judicial decisions - Use in actual legal proceedings without qualified human review - Any high-stakes decision-making without validation --- ## ๐Ÿ“„ Output Schema ```python { "case_name": str, # "Petitioner v. Respondent" "citation": str | None, # "AIR 1997 SC 3986" or null "court": str, # Full court name "year": int | None, # 4-digit year "petitioner": str, "respondent": str, "subject_matter": str | None, # Criminal | Civil | Constitutional | ... "statutes_cited": [{"act": str, "section": str, "description": str}], "precedents_cited": [{"citation": str, "case_name": str | None}], "legal_issues": [str], "holding": str, # 1-3 sentence summary "outcome": str # dismissed | allowed | disposed | remanded | modified } ``` --- ## ๐Ÿ”— Related Resources - ๐Ÿ™ **GitHub:** [nyaya-legal-ai](https://github.com/shubace/nyaya-legal-ai) โ€” Full project source code including FastAPI backend, LangGraph pipeline, and Streamlit UI - ๐Ÿ“Š **Base Model:** [mistralai/Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) - ๐Ÿ“ **Dataset (ILSum):** [d0r1h/ILSum](https://huggingface.co/datasets/d0r1h/ILSum) --- ## ๐Ÿ“ Citation If you use Nyaya-7B in your research or applications, please cite: ```bibtex @misc{nyaya7b2026, title = {Nyaya-7B: A QLoRA Fine-tuned LLM for Indian Legal Judgment Parsing}, author = {Shubham Suman}, year = {2026}, url = {https://huggingface.co/mrroyaleace/nyaya-7b}, note = {Fine-tuned from mistralai/Mistral-7B-Instruct-v0.3 on 2,000+ Indian SC/HC judgments} } ``` --- *Built with โค๏ธ for the Indian legal research community. Nyaya-7B is open-source and free to use under the Apache 2.0 license.*