Qwen3-4B-Indian-Law/README.md

---
license: apache-2.0
language:
- en
tags:
- legal
- law
- indian-law
- legal-assistant
- qwen3
- unsloth
- lora
- instruction-tuning
- question-answering
- legal-reasoning
datasets:
- Prarabdha/indian-legal-supervised-fine-tuning-data
- viber1/indian-law-dataset
base_model:
- unsloth/Qwen3-4B
pipeline_tag: text-generation
library_name: transformers
---

# Qwen3-4B Indian Law

A domain-adapted legal assistant fine-tuned from **Qwen3-4B** on a large corpus of Indian legal texts, statutory provisions, constitutional law, criminal law, evidence law, procedural law, and court judgments.

The model is designed to assist with:

- Indian legal question answering
- Statutory interpretation
- Constitution-related queries
- Criminal law and procedure
- Legal reasoning
- Case law understanding
- Legal research assistance
- Judgment summarization
- Legal education and training

---

# Model Overview

| Item | Value |
|--------|--------|
| Base Model | unsloth/Qwen3-4B |
| Fine-Tuning Method | LoRA + QLoRA |
| Framework | Unsloth |
| Context Length | 4096 |
| Precision | BF16 |
| LoRA Rank | 32 |
| LoRA Alpha | 32 |
| Optimizer | AdamW 8-bit |
| Learning Rate | 2e-4 |
| Scheduler | Cosine |
| Epochs | 2 |
| Effective Batch Size | 32 |
| Domain | Indian Legal Knowledge |

---

# Training Dataset

The training corpus was created by combining multiple publicly available Indian legal datasets together with a large judgment corpus.

The objective was to expose the model to:

- Legal question answering
- Statutory provisions
- Constitutional law
- Criminal law
- Procedural law
- Evidence law
- Court judgments
- Legal summarization
- Legal reasoning

---

# Dataset Composition

## 1. Indian Legal Supervised Fine-Tuning Dataset

Source:

```text
Prarabdha/indian-legal-supervised-fine-tuning-data
```

Characteristics:

- Large-scale legal instruction dataset
- Context → Question → Answer format
- Derived from Indian court judgments
- Designed for legal reasoning and legal QA

Original Size:

```text
6,055,371 samples
```

To prevent over-representation and memorization, a subset was selected during dataset balancing.

Contribution:

```text
≈ 250,000 samples
```

Example:

```text
Context:
Delhi Development Authority v. Kanwar Kumar Mehta

Question:
Was the High Court justified in calculating interest on escalation charges?

Answer:
Yes. The High Court's decision was held justified on equitable grounds.
```

---

## 2. Indian Law Instruction Dataset

Source:

```text
viber1/indian-law-dataset
```

Characteristics:

- Legal instruction-response pairs
- Covers Indian legal concepts
- General legal knowledge
- Legal terminology

Samples:

```text
24,607
```

Example:

```text
Question:
What is the difference between a petition and a plaint?

Answer:
A petition is a formal request seeking relief, whereas a plaint is the written statement initiating a civil suit.
```

---

## 3. Constitution of India QA Dataset

Custom processed dataset containing question-answer pairs generated from constitutional provisions.

Coverage:

- Fundamental Rights
- Directive Principles
- Union and State relations
- Parliament
- Judiciary
- Constitutional amendments

Samples:

```text
4,082
```

Example:

```text
Question:
What is India according to the Constitution?

Answer:
India, that is Bharat, shall be a Union of States.
```

---

## 4. Indian Penal Code (IPC) Dataset

Custom processed IPC question-answer corpus.

Coverage:

- Definitions
- Offences
- Punishments
- Criminal liability
- General exceptions

Samples:

```text
2,267
```

Example:

```text
Question:
What is the title and extent of operation of the Indian Penal Code?

Answer:
The title is the Indian Penal Code and it extends to offences committed within India and certain offences committed outside India.
```

---

## 5. Code of Criminal Procedure (CrPC) Dataset

Custom processed question-answer dataset generated from CrPC provisions.

Coverage:

- Investigation
- Arrest
- Bail
- Trial procedures
- Appeals
- Criminal courts

Samples:

```text
8,194
```

Example:

```text
Question:
What is the short title and commencement of the CrPC?

Answer:
The Code of Criminal Procedure, 1973.
```

---

## 6. IndicLegalQA

Legal question-answer dataset derived from Indian Supreme Court judgments.

Coverage:

- Case law
- Judicial reasoning
- Legal interpretation

Samples:

```text
10,002
```

Example:

```text
Question:
Who was the respondent in Union of India?

Answer:
Maj. Gen. Manomoy Ganguly.
```

---

## 7. Bharatiya Nyaya Sanhita (BNS)

Structured dataset generated from the Bharatiya Nyaya Sanhita, 2023.

Coverage:

- Criminal offences
- Punishments
- Definitions
- Modern criminal law provisions

Source Structure:

```text
Chapter
Section
Section Name
Description
```

---

## 8. Bharatiya Sakshya Adhiniyam (BSA)

Structured dataset generated from the Bharatiya Sakshya Adhiniyam, 2023.

Coverage:

- Evidence law
- Documentary evidence
- Digital evidence
- Witness testimony

Source Structure:

```text
Chapter
Section
Section Name
Description
```

---

## 9. Indian Court Judgments Corpus

Largest component of the training data.

Sources include:

- Supreme Court judgments
- High Court judgments
- CourtNIC archives
- JUDIS archives

Documents processed:

```text
16,726 judgment files
```

Coverage:

- Constitutional law
- Civil law
- Criminal law
- Taxation
- Property law
- Administrative law
- Service law

Training samples were automatically converted into:

```text
Context → Question → Answer
```

instruction format.

---

# Dataset Balancing

The original corpus was heavily dominated by judgment-derived samples.

Without balancing:

```text
451,756 samples
```

Distribution:

```text
Judgment-heavy
```

To improve generalization across statutory and constitutional law, a balancing procedure was applied.

Final balanced dataset:

```text
304,930 samples
```

Approximate distribution:

| Category | Samples |
|-----------|-----------|
| General Legal QA | 190,744 |
| Court Judgments | 66,368 |
| Constitution | 32,346 |
| CrPC | 8,719 |
| IPC | 6,698 |
| BNS | 50 |
| BSA | 5 |

This balancing significantly reduced bias toward judgment memorization while preserving broad legal coverage.

---

# Training Configuration

The model was fine-tuned using LoRA adapters on top of Qwen3-4B.

## LoRA Configuration

```python
r=32
lora_alpha=32
lora_dropout=0.0
```

Target Modules:

```python
q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj
```

---

## Optimization

```python
Learning Rate: 2e-4
Weight Decay: 0.01
Warmup Ratio: 0.03
Scheduler: Cosine
Optimizer: AdamW 8-bit
```

---

## Training

```python
Epochs: 2
Max Sequence Length: 4096
Batch Size: 8
Gradient Accumulation: 4
Effective Batch Size: 32
Precision: BF16
```

---

# Example Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "goasty/Qwen3-4B-Indian-Law"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

prompt = """
What is Article 21 of the Constitution of India?
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

---

# Intended Use

Suitable for:

- Legal research assistance
- Educational purposes
- Law students
- Legal document analysis
- Statutory interpretation
- Legal Q&A systems
- Retrieval-Augmented Generation (RAG)

---

# Limitations

- Not a substitute for licensed legal counsel.
- May generate legally incorrect or outdated interpretations.
- Should not be relied upon for litigation or legal advice without expert review.
- Training data contains historical judgments and statutes which may have been amended or overruled.

---

# Acknowledgements

This work builds upon:

- Qwen Team
- Unsloth
- Hugging Face Datasets Community
- Indian Legal Open Data Contributors
- Supreme Court and High Court public legal records

---

# Citation

```bibtex
@misc{qwen3_indian_law,
  title={Qwen3-4B Indian Law},
  author={Aditya},
  year={2026},
  note={Fine-tuned Qwen3-4B model for Indian legal reasoning and question answering}
}
```