---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- llama
- fine-tuned
- legal
- finance
- qa
base_model: thesreedath/slm-125m-base
---

# slm-125m-finetuned-qa

A 125.8M-parameter LLaMA-architecture model, fine-tuned from
[thesreedath/slm-125m-base](https://huggingface.co/thesreedath/slm-125m-base) on a
grounded Q&A instruction dataset covering legal, financial, and general text.

## Training data

7,521 chat-format (system/user/assistant) examples generated from a legal/SEC/web
corpus using Gemini (`gemini-3.1-flash-lite`) as teacher, filtered by an LLM-as-judge
pass for grounding/faithfulness, deduplicated by embedding similarity, covering four
task types: grounded QA, summarization, extraction, and rewriting.

## Training

Full fine-tune (not LoRA), 3 epochs, 1x H100, dynamic per-batch padding, loss masked to
the assistant response only. 354 steps, ~1.6M real tokens processed, 69 seconds total.
Validation perplexity 9.13 on the Q&A task.

## Usage

Prompts use the same special-token format as the base model:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("sahuabinash/slm-125m-finetuned-qa")
model = AutoModelForCausalLM.from_pretrained("sahuabinash/slm-125m-finetuned-qa")

prompt = "<|bos|><|system|>Answer the question using only the given passage.<|user|>What is a plaintiff?<|assistant|>"
inputs = tok(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=80)
print(tok.decode(out[0], skip_special_tokens=True))
```