初始化项目，由ModelHub XC社区提供模型

Model: goasty/Qwen3-4B-Indian-Law Source: Original Platform
2026-06-25 13:23:37 +08:00
commit 812470fe09
14 changed files with 152846 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,527 @@
+---
+license: apache-2.0
+language:
+- en
+tags:
+- legal
+- law
+- indian-law
+- legal-assistant
+- qwen3
+- unsloth
+- lora
+- instruction-tuning
+- question-answering
+- legal-reasoning
+datasets:
+- Prarabdha/indian-legal-supervised-fine-tuning-data
+- viber1/indian-law-dataset
+base_model:
+- unsloth/Qwen3-4B
+pipeline_tag: text-generation
+library_name: transformers
+---
+
+# Qwen3-4B Indian Law
+
+A domain-adapted legal assistant fine-tuned from **Qwen3-4B** on a large corpus of Indian legal texts, statutory provisions, constitutional law, criminal law, evidence law, procedural law, and court judgments.
+
+The model is designed to assist with:
+
+- Indian legal question answering
+- Statutory interpretation
+- Constitution-related queries
+- Criminal law and procedure
+- Legal reasoning
+- Case law understanding
+- Legal research assistance
+- Judgment summarization
+- Legal education and training
+
+---
+
+# Model Overview
+
+| Item | Value |
+|--------|--------|
+| Base Model | unsloth/Qwen3-4B |
+| Fine-Tuning Method | LoRA + QLoRA |
+| Framework | Unsloth |
+| Context Length | 4096 |
+| Precision | BF16 |
+| LoRA Rank | 32 |
+| LoRA Alpha | 32 |
+| Optimizer | AdamW 8-bit |
+| Learning Rate | 2e-4 |
+| Scheduler | Cosine |
+| Epochs | 2 |
+| Effective Batch Size | 32 |
+| Domain | Indian Legal Knowledge |
+
+---
+
+# Training Dataset
+
+The training corpus was created by combining multiple publicly available Indian legal datasets together with a large judgment corpus.
+
+The objective was to expose the model to:
+
+- Legal question answering
+- Statutory provisions
+- Constitutional law
+- Criminal law
+- Procedural law
+- Evidence law
+- Court judgments
+- Legal summarization
+- Legal reasoning
+
+---
+
+# Dataset Composition
+
+## 1. Indian Legal Supervised Fine-Tuning Dataset
+
+Source:
+
+```text
+Prarabdha/indian-legal-supervised-fine-tuning-data
+```
+
+Characteristics:
+
+- Large-scale legal instruction dataset
+- Context → Question → Answer format
+- Derived from Indian court judgments
+- Designed for legal reasoning and legal QA
+
+Original Size:
+
+```text
+6,055,371 samples
+```
+
+To prevent over-representation and memorization, a subset was selected during dataset balancing.
+
+Contribution:
+
+```text
+≈ 250,000 samples
+```
+
+Example:
+
+```text
+Context:
+Delhi Development Authority v. Kanwar Kumar Mehta
+
+Question:
+Was the High Court justified in calculating interest on escalation charges?
+
+Answer:
+Yes. The High Court's decision was held justified on equitable grounds.
+```
+
+---
+
+## 2. Indian Law Instruction Dataset
+
+Source:
+
+```text
+viber1/indian-law-dataset
+```
+
+Characteristics:
+
+- Legal instruction-response pairs
+- Covers Indian legal concepts
+- General legal knowledge
+- Legal terminology
+
+Samples:
+
+```text
+24,607
+```
+
+Example:
+
+```text
+Question:
+What is the difference between a petition and a plaint?
+
+Answer:
+A petition is a formal request seeking relief, whereas a plaint is the written statement initiating a civil suit.
+```
+
+---
+
+## 3. Constitution of India QA Dataset
+
+Custom processed dataset containing question-answer pairs generated from constitutional provisions.
+
+Coverage:
+
+- Fundamental Rights
+- Directive Principles
+- Union and State relations
+- Parliament
+- Judiciary
+- Constitutional amendments
+
+Samples:
+
+```text
+4,082
+```
+
+Example:
+
+```text
+Question:
+What is India according to the Constitution?
+
+Answer:
+India, that is Bharat, shall be a Union of States.
+```
+
+---
+
+## 4. Indian Penal Code (IPC) Dataset
+
+Custom processed IPC question-answer corpus.
+
+Coverage:
+
+- Definitions
+- Offences
+- Punishments
+- Criminal liability
+- General exceptions
+
+Samples:
+
+```text
+2,267
+```
+
+Example:
+
+```text
+Question:
+What is the title and extent of operation of the Indian Penal Code?
+
+Answer:
+The title is the Indian Penal Code and it extends to offences committed within India and certain offences committed outside India.
+```
+
+---
+
+## 5. Code of Criminal Procedure (CrPC) Dataset
+
+Custom processed question-answer dataset generated from CrPC provisions.
+
+Coverage:
+
+- Investigation
+- Arrest
+- Bail
+- Trial procedures
+- Appeals
+- Criminal courts
+
+Samples:
+
+```text
+8,194
+```
+
+Example:
+
+```text
+Question:
+What is the short title and commencement of the CrPC?
+
+Answer:
+The Code of Criminal Procedure, 1973.
+```
+
+---
+
+## 6. IndicLegalQA
+
+Legal question-answer dataset derived from Indian Supreme Court judgments.
+
+Coverage:
+
+- Case law
+- Judicial reasoning
+- Legal interpretation
+
+Samples:
+
+```text
+10,002
+```
+
+Example:
+
+```text
+Question:
+Who was the respondent in Union of India?
+
+Answer:
+Maj. Gen. Manomoy Ganguly.
+```
+
+---
+
+## 7. Bharatiya Nyaya Sanhita (BNS)
+
+Structured dataset generated from the Bharatiya Nyaya Sanhita, 2023.
+
+Coverage:
+
+- Criminal offences
+- Punishments
+- Definitions
+- Modern criminal law provisions
+
+Source Structure:
+
+```text
+Chapter
+Section
+Section Name
+Description
+```
+
+---
+
+## 8. Bharatiya Sakshya Adhiniyam (BSA)
+
+Structured dataset generated from the Bharatiya Sakshya Adhiniyam, 2023.
+
+Coverage:
+
+- Evidence law
+- Documentary evidence
+- Digital evidence
+- Witness testimony
+
+Source Structure:
+
+```text
+Chapter
+Section
+Section Name
+Description
+```
+
+---
+
+## 9. Indian Court Judgments Corpus
+
+Largest component of the training data.
+
+Sources include:
+
+- Supreme Court judgments
+- High Court judgments
+- CourtNIC archives
+- JUDIS archives
+
+Documents processed:
+
+```text
+16,726 judgment files
+```
+
+Coverage:
+
+- Constitutional law
+- Civil law
+- Criminal law
+- Taxation
+- Property law
+- Administrative law
+- Service law
+
+Training samples were automatically converted into:
+
+```text
+Context → Question → Answer
+```
+
+instruction format.
+
+---
+
+# Dataset Balancing
+
+The original corpus was heavily dominated by judgment-derived samples.
+
+Without balancing:
+
+```text
+451,756 samples
+```
+
+Distribution:
+
+```text
+Judgment-heavy
+```
+
+To improve generalization across statutory and constitutional law, a balancing procedure was applied.
+
+Final balanced dataset:
+
+```text
+304,930 samples
+```
+
+Approximate distribution:
+
+| Category | Samples |
+|-----------|-----------|
+| General Legal QA | 190,744 |
+| Court Judgments | 66,368 |
+| Constitution | 32,346 |
+| CrPC | 8,719 |
+| IPC | 6,698 |
+| BNS | 50 |
+| BSA | 5 |
+
+This balancing significantly reduced bias toward judgment memorization while preserving broad legal coverage.
+
+---
+
+# Training Configuration
+
+The model was fine-tuned using LoRA adapters on top of Qwen3-4B.
+
+## LoRA Configuration
+
+```python
+r=32
+lora_alpha=32
+lora_dropout=0.0
+```
+
+Target Modules:
+
+```python
+q_proj
+k_proj
+v_proj
+o_proj
+gate_proj
+up_proj
+down_proj
+```
+
+---
+
+## Optimization
+
+```python
+Learning Rate: 2e-4
+Weight Decay: 0.01
+Warmup Ratio: 0.03
+Scheduler: Cosine
+Optimizer: AdamW 8-bit
+```
+
+---
+
+## Training
+
+```python
+Epochs: 2
+Max Sequence Length: 4096
+Batch Size: 8
+Gradient Accumulation: 4
+Effective Batch Size: 32
+Precision: BF16
+```
+
+---
+
+# Example Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_name = "goasty/Qwen3-4B-Indian-Law"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto"
+)
+
+prompt = """
+What is Article 21 of the Constitution of India?
+"""
+
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=256
+)
+
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+---
+
+# Intended Use
+
+Suitable for:
+
+- Legal research assistance
+- Educational purposes
+- Law students
+- Legal document analysis
+- Statutory interpretation
+- Legal Q&A systems
+- Retrieval-Augmented Generation (RAG)
+
+---
+
+# Limitations
+
+- Not a substitute for licensed legal counsel.
+- May generate legally incorrect or outdated interpretations.
+- Should not be relied upon for litigation or legal advice without expert review.
+- Training data contains historical judgments and statutes which may have been amended or overruled.
+
+---
+
+# Acknowledgements
+
+This work builds upon:
+
+- Qwen Team
+- Unsloth
+- Hugging Face Datasets Community
+- Indian Legal Open Data Contributors
+- Supreme Court and High Court public legal records
+
+---
+
+# Citation
+
+```bibtex
+@misc{qwen3_indian_law,
+  title={Qwen3-4B Indian Law},
+  author={Aditya},
+  year={2026},
+  note={Fine-tuned Qwen3-4B model for Indian legal reasoning and question answering}
+}
+```