Files

ModelHub XC a8a2e1e79a 初始化项目，由ModelHub XC社区提供模型

Model: bqbbao6/Qwen2.5-7B-legal-vn
Source: Original Platform

2026-07-20 21:08:09 +08:00

2.7 KiB

Raw Permalink Blame History

base_model, tags, license, language

base_model

Uploaded finetuned model

Developed by: bqbbao6
License: apache-2.0
Finetuned from model : unsloth/Qwen2.5-7B-Instruct-bnb-4bit

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.

Vietnamese Legal Qwen2.5 (7B Instruct - 4-bit Quantized)

Model: bqbbao6/Qwen2.5-7B-legal-vn

Version 3B: bqbbao6/Qwen2.5-3B-legal-vn

Model Description

This model is a fine-tuned version of Qwen2.5-7B-Instruct, optimized using Unsloth and quantized to 4-bit (bitsandbytes) .

The primary fine-tuning objective focuses on Vietnamese Legal Domain Mastery and Context-Based Question Answering (RAG). Unlike general-purpose LLMs that reply strictly from pre-trained parametric memory, this model is specifically aligned to prioritize and reason directly over the provided context (legal articles, decrees, and circulars), drastically reducing hallucination and ensuring high-fidelity legal consulting.

Key Enhancements:

Strict Context Adherence: Tuned to extract facts and formulate arguments heavily based on the input context—making it perfect for Retrieval-Augmented Generation (RAG) pipelines.
Legal Formalism: Adopts the authoritative, formal, and precise tone required in the Vietnamese administrative and legal sectors.
Hardware Efficiency: Operates smoothly within a ~6GB VRAM footprint during inference, leaving ample head-room for long contexts and tool-calling structures.

Model Details

Property	Value
Base Model	Qwen/Qwen2.5-7B-Instruct
Parameters	7 Billion
Quantization	4-bit (bitsandbytes / bnb-4bit)
Fine-tuning Method	QLoRA (Rank 16, Alpha 32)
Primary Task	Context-Driven Legal Q&A / Legal Agent

Deployment & Inference (vLLM)

To host this model using vLLM on standard cloud environments.

Start the vLLM Server:

!python -m vllm.entrypoints.openai.api_server \
    --model unsloth/Qwen2.5-7B-Instruct-bnb-4bit \
    --max-model-len 2048 \ # Can be changed
    --dtype float16 \
    --api-key 'your-api-key-here' \
    --max-num-seqs 16 \
    --trust-remote-code \
    --gpu-memory-utilization 0.85 \ # Can be changed
    --enforce-eager \
    --enable-auto-tool-choice \
    --tool-call-parser hermes \
    --port 8000 &

2.7 KiB Raw Permalink Blame History