support-125M-slm-sft/README.md

language, license, tags, datasets, pipeline_tag, library_name

language	license	tags	datasets	pipeline_tag	library_name
en	mit	slm llama from-scratch it-support call-centre instruction-tuned sft raft retrieval-augmented	applegrew/support-125M-slm-base	text-generation	transformers

Support 125M SLM - SFT (Instruction-Tuned + RAFT)

A 125M parameter Llama-style language model trained from scratch on ~2.6B tokens of IT support data, then instruction-tuned on ~17K examples including RAFT (Retrieval-Augmented Fine-Tuning) data.

Built from the pretrained base model (applegrew/support-125M-slm-base).

Capabilities

Capability	Example
IT troubleshooting	"My VPN keeps disconnecting" → step-by-step help
Casual chat	"Hi" → "Hello! How can I help you today?"
Follow-ups	"That didn't work" → "Let's try another approach"
Grounded answers (RAFT)	Given a KB article, answers only from it
Refusals (RAFT)	"Not in the context" → "I don't have enough information to answer"

Training Data (17,002 records)

Source	Pairs	Method	Cost
StackExchange (filtered IT)	10,372	Direct extraction	$0
Ubuntu IRC + Gemini	2,821	Teacher distillation	$1.50
Casual interactions	732	Seed + Gemini expansion	$0.04
RAFT dataset	3,077	Gemini 3.5 Flash-Lite	$0.40
Total	17,002		~$1.94

RAFT Pairs Breakdown

Type	Count	Behavior taught
Answerable (grounded)	2,306	Answer strictly from provided context
Unanswerable (refusal)	771	Say "not enough information"

Training Details

Setting	Value
Epochs	3
Learning rate	2e-4 (cosine decay)
Batch size	8
Loss masking	Assistant tokens only
Hardware	1x H100
Best val loss	2.099
Val perplexity	8.83
Total SFT cost	~$2.20

Usage

Plain chat

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("applegrew/support-125M-slm-sft")
tokenizer = AutoTokenizer.from_pretrained("applegrew/support-125M-slm-sft")

chat = "<|bos|><|system|>\nYou are a helpful IT support technician.<|eos|>\n<|user|>\nMy VPN keeps disconnecting every 5 minutes<|eos|>\n<|assistant|>\n"
inputs = tokenizer(chat, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))

RAG / Grounded (RAFT) style

context = "WiFi drops on Ubuntu 22.04. Run 'iwconfig', check Power Management, disable with 'sudo iwconfig wlan0 power off'."
question = "Why does my wifi keep dropping?"

chat = (
    "<|bos|><|system|>\nYou are a helpful IT support technician. Answer using ONLY the "
    "provided context. If the answer is not in the context, say you do not have enough "
    "information to answer.<|eos|>\n"
    f"<|user|>\n\n{context}\n\n\nQuestion: {question}<|eos|>\n<|assistant|>\n"
)

Chat Template

<|bos|><|system|>
{system_prompt}<|eos|>
<|user|>
{question}<|eos|>
<|assistant|>
{answer}<|eos|>

Only the assistant's answer contributes to the loss during training.

Web Demo

Try it live: https://vercel-liart-xi.vercel.app

Project Summary

Phase	Cost	Description
Data pipeline	$0	Clean → dedup → tokenize 2.6B IT tokens
Tokenizer	$0	16K BPE trained on corpus
Pretraining (6 epochs)	$31	125M Llama, 8x H100
SFT data	~$1.94	Gemini distillation + filtering + RAFT
SFT training (3 rounds)	~$2.20	1x H100
Total	~$35