138 lines
5.4 KiB
Markdown
138 lines
5.4 KiB
Markdown
---
|
||
license: apache-2.0
|
||
base_model: Qwen/Qwen3-0.6B
|
||
library_name: transformers
|
||
pipeline_tag: text-generation
|
||
language:
|
||
- en
|
||
tags:
|
||
- tool-calling
|
||
- function-calling
|
||
- qwen3
|
||
- lora
|
||
- sft
|
||
- trl
|
||
- zynthetix
|
||
---
|
||
|
||
# AL1-model-B — Qwen3-0.6B fine-tuned for native tool-calling
|
||
|
||
A compact (0.6B-parameter) language model fine-tuned to **call tools reliably** while staying a
|
||
capable general-chat assistant. It takes the Apache-2.0 [`Qwen/Qwen3-0.6B`](https://huggingface.co/Qwen/Qwen3-0.6B)
|
||
base and applies a single **LoRA supervised fine-tune (SFT)** targeting native, Hermes-style
|
||
`<tool_call>` emission over a small set of file/shell tools — then merges the adapter back into
|
||
the base for a self-contained model.
|
||
|
||
Part of the **Zynthetix AL-1** two-model lab (Model A = from-scratch NumPy LM; **Model B = improve an
|
||
open-source small model**, this repo). Code: https://github.com/GruheshKurra/AL1-model-B
|
||
|
||
---
|
||
|
||
## Results — before → after
|
||
|
||
Tool-calling is scored by **exact match** against a held-out 12-case set: `name_acc` = correct tool
|
||
name chosen, `full_acc` = correct name **and** arguments. Both evaluated with greedy decoding, fp32,
|
||
on the same device (no blending with chat scores).
|
||
|
||
| Metric | Base `Qwen3-0.6B` | **AL1-model-B (SFT)** | Δ |
|
||
|---|---|---|---|
|
||
| Tool `name_acc` | 0.500 | **1.000** | +0.500 |
|
||
| Tool `full_acc` | 0.500 | **0.917** (11/12) | +0.417 |
|
||
| Chat coherence (10 probes) | strong | strong, terser | no regression |
|
||
|
||
**Why the base scored only 0.5:** it *refused* to call `run_bash` and `edit_file` (0/2 each) and
|
||
mis-handled one `list_dir` — its arguments were fine *when* it chose to call. The SFT data therefore
|
||
over-weighted `edit_file` + `run_bash` + explicit anti-refusal examples, which closed the entire gap
|
||
(`name_acc` → 1.0 = a valid tool is now always selected).
|
||
|
||
> The merged fp16 model reproduces the adapter score exactly (`full_acc` 0.917) — merging lost nothing.
|
||
|
||
---
|
||
|
||
## The tools
|
||
|
||
Five file/shell tools, described to the model via the OpenAI-style `tools=` schema and the Qwen3
|
||
native chat template (Hermes `<tool_call>` tags):
|
||
|
||
| Tool | Purpose | Required args |
|
||
|---|---|---|
|
||
| `read_file` | Read a file's contents | `path` |
|
||
| `write_file` | Create/overwrite a file | `path`, `content` |
|
||
| `edit_file` | Replace an exact substring | `path`, `old_string`, `new_string` |
|
||
| `list_dir` | List a directory | `path` |
|
||
| `run_bash` | Run a shell command | `command` |
|
||
|
||
The model **emits** tool name + arguments; wiring those to a real executor (or an MCP client) is a
|
||
serving-layer concern, deliberately out of scope for the model itself.
|
||
|
||
---
|
||
|
||
## Usage
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
import torch
|
||
|
||
MODEL = "karthik-2905/AL1-model-B"
|
||
tok = AutoTokenizer.from_pretrained(MODEL)
|
||
model = AutoModelForCausalLM.from_pretrained(MODEL, dtype=torch.float16).to("mps") # or "cuda"/"cpu"
|
||
|
||
TOOLS = [{"type": "function", "function": {
|
||
"name": "read_file",
|
||
"description": "Read and return the contents of a file.",
|
||
"parameters": {"type": "object",
|
||
"properties": {"path": {"type": "string"}}, "required": ["path"]}}}] # + the other 4
|
||
|
||
msgs = [{"role": "user", "content": "Show me what's in config.py"}]
|
||
text = tok.apply_chat_template(msgs, tools=TOOLS, tokenize=False,
|
||
add_generation_prompt=True, enable_thinking=False)
|
||
ids = tok(text, return_tensors="pt").to(model.device)
|
||
out = model.generate(**ids, max_new_tokens=128, do_sample=False)
|
||
print(tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True))
|
||
# -> <tool_call>{"name": "read_file", "arguments": {"path": "config.py"}}</tool_call>
|
||
```
|
||
|
||
---
|
||
|
||
## How it was built
|
||
|
||
| Stage | Choice |
|
||
|---|---|
|
||
| Base | `Qwen/Qwen3-0.6B` (Apache-2.0) — GQA + RoPE + SwiGLU + RMSNorm |
|
||
| PEFT | LoRA `r=16`, `α=32`, dropout 0.05, on all 7 attention+MLP projections |
|
||
| Trainer | TRL `SFTTrainer`, `assistant_only_loss=True`, `max_length=512`, packing off |
|
||
| Schedule | lr `3e-4` (cosine, 3% warmup), 3 epochs, effective batch 32, bf16 |
|
||
| Hardware | RunPod **RTX A6000** (train), Apple Silicon / MPS (merge + eval) |
|
||
| Merge | `merge_and_unload()` → standalone fp16 |
|
||
|
||
**LR sweep** (10% data, 1 epoch) picked `3e-4` (eval_loss: `1e-4`=1.099, `2e-4`=0.999, `3e-4`=0.975).
|
||
The full 3-epoch run overfits by *eval_loss* after ~epoch 1 (0.91 → 0.99), but generation quality and
|
||
tool exact-match were best at 3 epochs — the **task metric beat the proxy loss**, so 3 epochs shipped.
|
||
|
||
### Data
|
||
|
||
- **1,423 train / 74 val** examples, chat-formatted to the Qwen3 template, ~**2:1 tool:chat** ratio.
|
||
- Tool examples teach the Hermes `<tool_call>` format across all 5 tools (over-weighting the two the
|
||
base refused); chat examples guard against chat regression.
|
||
- Train/val split is dedup'd and **leakage-checked against the eval set**.
|
||
|
||
### Evaluation
|
||
|
||
- **Tool track (trust anchor):** exact-match over a 12-case set, scored by `mlb/eval_tools.py`.
|
||
- **Chat track (regression guard):** 10 probes, currently eyeballed for coherence (LLM-judge = future work).
|
||
|
||
---
|
||
|
||
## Limitations
|
||
|
||
- **Small model (0.6B):** capable within its size, not a substitute for a large instruct model.
|
||
- **Chat quality is eyeballed,** not yet LLM-judge scored — no formal regression number.
|
||
- **No general-capability guard** (MMLU/GSM8K) was run; broad-knowledge regression is unmeasured.
|
||
- **4-bit quantization untested** — a 0.6B model can degrade more than the usual −1–2% at 4-bit.
|
||
|
||
---
|
||
|
||
## License
|
||
|
||
**Apache-2.0**, inherited from the `Qwen/Qwen3-0.6B` base.
|