llama-3.2-3b-legal-vn/README.md

---
base_model: unsloth/llama-3.2-3b-instruct-unsloth-bnb-4bit
tags:
- text-generation-inference
- transformers
- unsloth
- llama
license: apache-2.0
language:
- en
---

## Model Description

- **Developed by:** bqbbao6
- **License:** apache-2.0
- **Finetuned from model :** unsloth/llama-3.2-3b-instruct-unsloth-bnb-4bit
- **Fine-tuning Method:** LoRA (Low-Rank Adaptation) via Unsloth
- 
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)

---

## Intended Use

This model is designed for:

* Legal assistants and chatbots specialized in Vietnamese law.
* Routing systems that need to distinguish between general conversation and legal inquiries.
* Educational tools for students studying Vietnamese Civil and Administrative law.

## Training Data

The model was trained on a curated dataset of **3,199 samples**, including:

* **Router Data:** Pairs of user queries and their corresponding intent labels.
* **Legal Contexts:** Structured data extracted from Vietnamese legal documents (Decrees, Circulars, and Laws) formatted for RAG (Retrieval-Augmented Generation).

## Usage 
#### Installation

To run this model efficiently, it is recommended to use the `unsloth` library for faster inference and lower memory usage.

```bash
pip install unsloth
pip install --no-deps xformers trl peft accelerate bitsandbytes

```

#### Loading the Model

You can load the model using the following Python code:

```python
from unsloth import FastLanguageModel
import torch

model_id = "bqbbao6/llama-3.2-3b-legal-vn" # Your Model ID

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = model_id,
    max_seq_length = 2048,
    load_in_4bit = True, # Use 4-bit for faster inference
)
FastLanguageModel.for_inference(model) # Enable 2x faster inference

```

#### Sample Inference Code

Here is a complete example of how to use the model for a legal inquiry:

```python
# Prepare the conversation
messages = [
    {"role": "system", "content": "Bạn là một chuyên gia pháp luật Việt Nam."},
    {"role": "user", "content": "Người dân có quyền tham gia quản lý nhà nước không?"},
]

# Apply Chat Template
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize = True,
    add_generation_prompt = True,
    return_tensors = "pt",
).to("cuda")

# Generate response
outputs = model.generate(
    input_ids = inputs, 
    max_new_tokens = 512,
    use_cache = True
)

# Decode the output
response = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
print(response)

```

### Examples
##### 1. Intent Classification (Router)

**Input:**

```text
"Cho tôi hỏi thủ tục làm căn cước công dân gắn chip hiện nay như thế nào?"

```

**Output:**

```text
legal

```

##### 2. Legal Question Answering (RAG)

**Input:**

```text
Ngữ cảnh: Theo Nghị quyết số 98/2023/QH15, UBND thành phố Thủ Đức có thẩm quyền phê duyệt nhiệm vụ và đồ án quy hoạch phân khu, quy hoạch chi tiết các đồ án trên địa bàn.

Câu hỏi: UBND thành phố Thủ Đức có quyền phê duyệt quy hoạch phân khu không?

```

**Output:**

```text
Có. Theo Nghị quyết số 98/2023/QH15, UBND thành phố Thủ Đức có thẩm quyền phê duyệt nhiệm vụ và đồ án quy hoạch phân khu trên địa bàn thành phố thuộc thẩm quyền của UBND Thành phố Hồ Chí Minh trước đây.

```

## Limitations and Biases

* The model's knowledge is limited to the documents provided during training. It may not reflect the most recent legal amendments unless updated.
* Users should verify the information with official legal portals as the model might occasionally generate inaccurate legal interpretations (hallucinations).

---