79 lines
3.8 KiB
Markdown
79 lines
3.8 KiB
Markdown
|
---
|
||
|
|
library_name: transformers
|
||
|
|
tags:
|
||
|
|
- text-generation
|
||
|
|
- qwen2
|
||
|
|
- conversational
|
||
|
|
- fine-tuned
|
||
|
|
- ticket-classification
|
||
|
|
- structured-extraction
|
||
|
|
license: apache-2.0
|
||
|
|
language:
|
||
|
|
- hi
|
||
|
|
- en
|
||
|
|
base_model:
|
||
|
|
- Qwen/Qwen2.5-1.5B-Instruct
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
---
|
||
|
|
|
||
|
|
# Model Card for Model ID
|
||
|
|
|
||
|
|
# 🚀 Production-Grade SLM Structured Data Extractor
|
||
|
|
### **Fine-Tuned Qwen-1.5B-Instruct for Zero-Yapping Strict JSON Schema Compliance**
|
||
|
|
|
||
|
|
[](https://huggingface.co/spaces/Anshrajsingh/slm-finetunning)
|
||
|
|
[](https://huggingface.co/Anshrajsingh/qwen2.5-1.5b-ticket-classifier)
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 📌 Business Case & Problem Statement
|
||
|
|
In enterprise production environments, **unstructured text** (customer support tickets, invoices, logs) must be mapped to structured databases (SQL/NoSQL) with 100% deterministic reliability.
|
||
|
|
|
||
|
|
While massive commercial LLMs (like GPT-4o) achieve high accuracy, they introduce major bottlenecks for high-throughput narrow tasks:
|
||
|
|
1. **High API Costs:** Processing millions of tokens daily is economically unsustainable.
|
||
|
|
2. **High Latency:** Cloud API round-trips slow down real-time automated routing workflows.
|
||
|
|
3. **Data Privacy Risks:** Transmitting sensitive client information (PII) to third-party APIs violates data compliance laws (GDPR/HIPAA).
|
||
|
|
4. **Formatting Violations ("Yapping"):** General-purpose base models frequently violate strict formatting constraints by adding conversational filler (e.g., *"Here is the JSON you requested..."*).
|
||
|
|
|
||
|
|
### **The Solution**
|
||
|
|
This project demonstrates **Cost-Conscious AI Engineering** by fine-tuning a **1.5 Billion parameter Small Language Model (SLM)** to perform production-grade, schema-bound classification with near-zero latency and fraction of a cent compute cost, making it entirely deployable on the edge or low-cost commodity GPUs.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 📊 Performance & ROI Benchmark
|
||
|
|
|
||
|
|
We evaluated the model on a hidden test dataset of customer support tickets across 4 categorical schema keys: `category`, `urgency`, `sentiment`, and `action_required`.
|
||
|
|
|
||
|
|
| Metric | Base Model (Qwen2.5-1.5B-Instruct) | Fine-Tuned SLM (LoRA Adapted) |
|
||
|
|
| :--- | :--- | :--- |
|
||
|
|
| **Valid JSON Generation Rate** | 100% (Wrapped in Markdown) | **100% (Pure Strict JSON)** |
|
||
|
|
| **Schema Compliance** | 0.0% (Generated arbitrary tags) | **95.6% (Strictly adheres to enum keys)** |
|
||
|
|
| **Exact Match Accuracy** | 0.0% | **91.3%** |
|
||
|
|
| **Formatting Filler (Yapping)** | High | **0.0% (Starts with `{` ends with `}`)** |
|
||
|
|
| **Deployment Suitability** | General Chat | **Production-Ready Automated API** |
|
||
|
|
|
||
|
|
### 💰 Estimated Cost Breakdown (At Scale)
|
||
|
|
*Assumed volume: 1 Million Tickets/Month (~300M Tokens processed)*
|
||
|
|
* **Commercial LLM API (e.g., GPT-4o Class):** ~$750 - $1,500 / month
|
||
|
|
* **This Fine-Tuned SLM (Hosted on Single Low-End Instance / Serverless GPU):** **<$25 / month**
|
||
|
|
* **Net Business Savings:** **~96.5% Cost Reduction** with 100% data privacy.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 🛠️ Technical Implementation & Architecture
|
||
|
|
|
||
|
|
### **1. Technology Stack**
|
||
|
|
* **Base Model:** `Qwen/Qwen2.5-1.5B-Instruct`
|
||
|
|
* **Fine-Tuning Technique:** Parameter-Efficient Fine-Tuning (PEFT) using **QLoRA (4-bit quantization)**
|
||
|
|
* **Infrastructure:** Trained via PyTorch and Hugging Face `Trainer` pipeline on a single consumer-grade T4 GPU.
|
||
|
|
* **Inference UI:** Gradio application deployed seamlessly on Hugging Face Spaces.
|
||
|
|
|
||
|
|
### **2. Target Features & Schema Constraints**
|
||
|
|
The model was fine-tuned using custom-engineered synthetic dataset structures following the **ChatML** format to map arbitrary inputs directly into this immutable JSON schema:
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"category": "billing | technical | refund | account | general",
|
||
|
|
"urgency": "low | medium | high",
|
||
|
|
"sentiment": "positive | negative | neutral",
|
||
|
|
"action_required": "auto_reply | escalate | close"
|
||
|
|
}
|