dhanushmekaka/qwen25-1.5b-invoice-extraction-v3
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: dhanushmekaka/qwen25-1.5b-invoice-extraction-v3
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-11 06:58:17 +08:00
commit b671f5607d
12 changed files with 151955 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

133
README.md Normal file
View File

@@ -0,0 +1,133 @@
# qwen25-1.5b-invoice-extraction-v3
A QLoRA fine-tune of **Qwen2.5-1.5B-Instruct** that extracts structured financial terms from SEC
contract / commitment-letter text and returns them as a fixed-schema JSON object.
## Model Description
- **Base model:** `Qwen/Qwen2.5-1.5B-Instruct`
- **Fine-tuning method:** QLoRA (4-bit NF4), LoRA r=64, alpha=64, applied to all attention and MLP
projection layers, trained with [Unsloth](https://github.com/unslothai/unsloth)
- **Task:** Given a contract excerpt, extract every financial value present and label it with a
`term_type` from a fixed vocabulary, returned as strict JSON.
- **Trained on:** `TheTokenFactory/sec-contracts-financial-extraction-instructions` (3,430 examples),
using a two-stage curriculum (core financial fields first, then the full dataset).
## Output Format
```json
{
"financial_values": [
{"value": "US$3,050,000,000", "term_type": "credit_facility"},
{"value": "8.5%", "term_type": "interest_rate"}
]
}
```
`term_type` is restricted to: `interest_rate`, `equity_grant`, `bonus`, `credit_facility`,
`salary`, `threshold`, `fee`, `loan_amount`, `compensation`, `retirement_benefit`, `other`,
`purchase_price`, `severance`.
## Intended Use
Drafting / first-pass extraction of financial terms from contracts, commitment letters, and
similar legal-financial documents, as an aid for a human reviewer — **not** as a standalone
source of truth for financial or legal decisions.
## How to Use
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "dhanushmekaka/qwen25-1.5b-invoice-extraction-v3"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
messages = [
{"role": "system", "content": "You are a strict legal extraction AI... Output strictly as JSON."},
{"role": "user", "content": "<contract excerpt here>"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```
## Evaluation
Measured on a 50-example held-out split of the training dataset:
| Metric | Score |
|---|---|
| Exact Match Accuracy (full JSON object) | 16.0% |
| JSON Output Validity | 94.0% |
| Field-Level Extraction Accuracy | 44.9% |
| Avg. Inference Latency | 7.07 s |
| Avg. Throughput | 17.4 tok/s |
| `term_type` | Gold count | Precision | Recall |
|---|---|---|---|
| bonus | 9 | 66.7% | 44.4% |
| compensation | 4 | 0.0% | 0.0% |
| credit_facility | 1 | n/a | 0.0% |
| equity_grant | 3 | 40.0% | 66.7% |
| fee | 7 | 100.0% | 42.9% |
| interest_rate | 6 | 75.0% | 50.0% |
| loan_amount | 2 | 66.7% | 100.0% |
| purchase_price | 2 | 100.0% | 50.0% |
| retirement_benefit | 1 | n/a | 0.0% |
| salary | 7 | 50.0% | 57.1% |
| threshold | 7 | 42.9% | 42.9% |
## Limitations
This is a **v3, still-converging checkpoint** — training was stopped by a wall-clock budget at
90.8% of the planned steps (epoch 5.45 of 6), with eval loss still decreasing. The numbers above
should be read as a snapshot of an in-progress model, not a final result:
- **Exact Match (16%)** is a strict all-or-nothing metric over the whole JSON object; a single
wrong value anywhere in a multi-field extraction counts as a full miss. Field-Level Accuracy
(44.9%) is a better read on per-value correctness.
- **Rare `term_type`s** (`retirement_benefit`, `purchase_price`, `severance`, `compensation`,
`other`) have limited training coverage (under 250 combined examples vs. 850 for
`interest_rate` alone), and their precision/recall above is computed on single-digit eval
counts — treat those figures as directional, not reliable.
- **JSON validity (94%)** means roughly 1 in 17 outputs needs a parsing fallback or retry in
production.
- The 50-example eval set is small; metrics may shift with a larger held-out set.
**Recommended before production use:** complete the remaining training epoch, expand the eval set,
and add human review for `term_type`s with fewer than ~100 training examples.
## Training Data
`TheTokenFactory/sec-contracts-financial-extraction-instructions` — SEC contract excerpts
(commitment letters, credit agreements, employment/compensation terms) paired with gold JSON
financial-term annotations.
## License
Apache 2.0 (inherits from the Qwen2.5-1.5B-Instruct base model — verify against the base model's
license terms before commercial use).---
base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
tags:
- text-generation-inference
- transformers
- unsloth
- qwen2
license: apache-2.0
language:
- en
---
# Uploaded finetuned model
- **Developed by:** dhanushmekaka
- **License:** apache-2.0
- **Finetuned from model :** unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
This qwen2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)