qwen25-1.5b-invoice-extraction-v3/README.md

qwen25-1.5b-invoice-extraction-v3

A QLoRA fine-tune of Qwen2.5-1.5B-Instruct that extracts structured financial terms from SEC contract / commitment-letter text and returns them as a fixed-schema JSON object.

Model Description

• Base model: Qwen/Qwen2.5-1.5B-Instruct
• Fine-tuning method: QLoRA (4-bit NF4), LoRA r=64, alpha=64, applied to all attention and MLP projection layers, trained with Unsloth
• Task: Given a contract excerpt, extract every financial value present and label it with a term_type from a fixed vocabulary, returned as strict JSON.
• Trained on: TheTokenFactory/sec-contracts-financial-extraction-instructions (3,430 examples), using a two-stage curriculum (core financial fields first, then the full dataset).

Output Format

{
  "financial_values": [
    {"value": "US$3,050,000,000", "term_type": "credit_facility"},
    {"value": "8.5%", "term_type": "interest_rate"}
  ]
}

term_type is restricted to: interest_rate, equity_grant, bonus, credit_facility, salary, threshold, fee, loan_amount, compensation, retirement_benefit, other, purchase_price, severance.

Intended Use

Drafting / first-pass extraction of financial terms from contracts, commitment letters, and similar legal-financial documents, as an aid for a human reviewer — not as a standalone source of truth for financial or legal decisions.

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "dhanushmekaka/qwen25-1.5b-invoice-extraction-v3"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

messages = [
    {"role": "system", "content": "You are a strict legal extraction AI... Output strictly as JSON."},
    {"role": "user", "content": "<contract excerpt here>"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Evaluation

Measured on a 50-example held-out split of the training dataset:

Metric	Score
Exact Match Accuracy (full JSON object)	16.0%
JSON Output Validity	94.0%
Field-Level Extraction Accuracy	44.9%
Avg. Inference Latency	7.07 s
Avg. Throughput	17.4 tok/s

term_type	Gold count	Precision	Recall
bonus	9	66.7%	44.4%
compensation	4	0.0%	0.0%
credit_facility	1	n/a	0.0%
equity_grant	3	40.0%	66.7%
fee	7	100.0%	42.9%
interest_rate	6	75.0%	50.0%
loan_amount	2	66.7%	100.0%
purchase_price	2	100.0%	50.0%
retirement_benefit	1	n/a	0.0%
salary	7	50.0%	57.1%
threshold	7	42.9%	42.9%

Limitations

This is a v3, still-converging checkpoint — training was stopped by a wall-clock budget at 90.8% of the planned steps (epoch 5.45 of 6), with eval loss still decreasing. The numbers above should be read as a snapshot of an in-progress model, not a final result:

• Exact Match (16%) is a strict all-or-nothing metric over the whole JSON object; a single wrong value anywhere in a multi-field extraction counts as a full miss. Field-Level Accuracy (44.9%) is a better read on per-value correctness.
• Rare term_types (retirement_benefit, purchase_price, severance, compensation, other) have limited training coverage (under 250 combined examples vs. 850 for interest_rate alone), and their precision/recall above is computed on single-digit eval counts — treat those figures as directional, not reliable.
• JSON validity (94%) means roughly 1 in 17 outputs needs a parsing fallback or retry in production.
• The 50-example eval set is small; metrics may shift with a larger held-out set.

Recommended before production use: complete the remaining training epoch, expand the eval set, and add human review for term_types with fewer than ~100 training examples.

Training Data

TheTokenFactory/sec-contracts-financial-extraction-instructions — SEC contract excerpts (commitment letters, credit agreements, employment/compensation terms) paired with gold JSON financial-term annotations.

License

Apache 2.0 (inherits from the Qwen2.5-1.5B-Instruct base model — verify against the base model's license terms before commercial use).--- base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit tags:

• text-generation-inference
• transformers
• unsloth
• qwen2 license: apache-2.0 language:
• en

---

Uploaded finetuned model

• Developed by: dhanushmekaka
• License: apache-2.0
• Finetuned from model : unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.