75 lines
1.7 KiB
Markdown
75 lines
1.7 KiB
Markdown
|
---
|
||
|
|
library_name: transformers
|
||
|
|
base_model: Qwen/Qwen2.5-0.5B-Instruct
|
||
|
|
tags:
|
||
|
|
- text-generation
|
||
|
|
- json
|
||
|
|
- classification
|
||
|
|
- catllm
|
||
|
|
license: apache-2.0
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
---
|
||
|
|
|
||
|
|
# CatLLM JSON Formatter
|
||
|
|
|
||
|
|
A fine-tuned Qwen2.5-0.5B-Instruct model that converts messy LLM classification
|
||
|
|
output into valid [cat-llm](https://github.com/chrissoria/cat-llm) JSON format.
|
||
|
|
|
||
|
|
## Task
|
||
|
|
|
||
|
|
Given a list of numbered categories and raw (possibly malformed) classification
|
||
|
|
output from another LLM, this model produces clean JSON:
|
||
|
|
|
||
|
|
```json
|
||
|
|
{"1": "0", "2": "1", "3": "0", ...}
|
||
|
|
```
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
This model is used automatically by cat-llm when `json_formatter=True`:
|
||
|
|
|
||
|
|
```python
|
||
|
|
import catllm as cat
|
||
|
|
|
||
|
|
results = cat.classify(
|
||
|
|
input_data=df["responses"],
|
||
|
|
categories=["Positive", "Negative", "Neutral"],
|
||
|
|
api_key="your-key",
|
||
|
|
json_formatter=True, # enables the formatter fallback
|
||
|
|
)
|
||
|
|
```
|
||
|
|
|
||
|
|
Install the formatter dependencies: `pip install cat-llm[formatter]`
|
||
|
|
|
||
|
|
## Training
|
||
|
|
|
||
|
|
- **Base model:** Qwen/Qwen2.5-0.5B-Instruct
|
||
|
|
- **Method:** LoRA (r=16, alpha=32) merged into base weights
|
||
|
|
- **Training data:** 8,000 synthetic examples covering 26+ messy output formats,
|
||
|
|
with the category count spanning N=2..50 so the formatter reliably emits
|
||
|
|
large (28- and 48-key) JSON objects, not just small ones.
|
||
|
|
- **Epochs:** 2
|
||
|
|
- **Metrics:** evaluated separately on low-N (<=12 categories) and high-N
|
||
|
|
(>=25 categories) buckets; see the repository's eval gate for current numbers.
|
||
|
|
|
||
|
|
## Prompt Format
|
||
|
|
|
||
|
|
The model uses the Qwen chat template with:
|
||
|
|
|
||
|
|
**System:** JSON formatter instructions (built into cat-llm)
|
||
|
|
|
||
|
|
**User:**
|
||
|
|
```
|
||
|
|
Categories:
|
||
|
|
1. Category A
|
||
|
|
2. Category B
|
||
|
|
...
|
||
|
|
|
||
|
|
Raw classification output:
|
||
|
|
{messy output here}
|
||
|
|
```
|
||
|
|
|
||
|
|
**Assistant:** `{"1":"0","2":"1",...}`
|