初始化项目,由ModelHub XC社区提供模型
Model: GuruVarshini/supportlm-qwen2.5-3b Source: Original Platform
This commit is contained in:
76
README.md
Normal file
76
README.md
Normal file
@@ -0,0 +1,76 @@
|
||||
---
|
||||
language:
|
||||
- hi
|
||||
- ta
|
||||
- en
|
||||
tags:
|
||||
- customer-support
|
||||
- hinglish
|
||||
- tanglish
|
||||
- fine-tuned
|
||||
- qwen2.5
|
||||
- qlora
|
||||
base_model: Qwen/Qwen2.5-3B-Instruct
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# SupportLM — Hinglish/Tanglish Customer Support Model
|
||||
|
||||
A fine-tuned Qwen2.5-3B model for Indian e-commerce customer support
|
||||
in code-switched language (Hinglish and Tanglish).
|
||||
|
||||
## What it does
|
||||
|
||||
Given a customer support message in Hinglish or Tanglish, the model:
|
||||
1. Classifies the issue type into one of 8 categories
|
||||
2. Extracts the order ID if present in the message
|
||||
3. Generates an empathetic response in the same language the customer used
|
||||
|
||||
## Benchmark Results
|
||||
|
||||
Evaluated on 159 held-out test examples never seen during training.
|
||||
|
||||
| Metric | GPT-4o-mini | SFT Model | DPO Model |
|
||||
|--------|-------------|-----------|-----------|
|
||||
| Classification accuracy | 74.8% | **79.9%** | 78.0% |
|
||||
| Order ID extraction | 98.7% | **99.4%** | 99.4% |
|
||||
| Language match rate | 96.2% | 99.4% | **100.0%** |
|
||||
|
||||
SFT model chosen for deployment — best classification accuracy.
|
||||
|
||||
## Issue Types
|
||||
|
||||
order_status, return_request, payment_issue, delivery_complaint,
|
||||
product_defect, cancellation_request, wrong_item, discount_query
|
||||
|
||||
## Training Details
|
||||
|
||||
- Base model: Qwen/Qwen2.5-3B-Instruct
|
||||
- Training data: 1277 synthetic Hinglish/Tanglish examples
|
||||
- Training method: QLoRA (r=16, alpha=32) + DPO (beta=0.1)
|
||||
- Hardware: Google Colab T4 GPU (free tier)
|
||||
- Training time: ~40 minutes total (SFT + DPO)
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
import json
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained("GuruVarshini/supportlm-qwen2.5-3b")
|
||||
tokenizer = AutoTokenizer.from_pretrained("GuruVarshini/supportlm-qwen2.5-3b")
|
||||
|
||||
message = "bhai mere order ka kya hua yaar 3 din ho gaye"
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": "You are a customer support agent for an Indian e-commerce platform. Given a customer message, respond with a JSON object containing issue_type, order_id, and response."},
|
||||
{"role": "user", "content": message}
|
||||
]
|
||||
|
||||
input_ids = tokenizer.apply_chat_template(
|
||||
messages,
|
||||
return_tensors="pt",
|
||||
add_generation_prompt=True
|
||||
)
|
||||
output = model.generate(input_ids, max_new_tokens=200)
|
||||
print(tokenizer.decode(output[0][input_ids.shape[1]:], skip_special_tokens=True))
|
||||
Reference in New Issue
Block a user