AL1-model-B/README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-0.6B
library_name: transformers
pipeline_tag: text-generation
language:
- en
tags:
- tool-calling
- function-calling
- qwen3
- lora
- sft
- trl
- zynthetix
---

# AL1-model-B — Qwen3-0.6B fine-tuned for native tool-calling

A compact (0.6B-parameter) language model fine-tuned to **call tools reliably** while staying a
capable general-chat assistant. It takes the Apache-2.0 [`Qwen/Qwen3-0.6B`](https://huggingface.co/Qwen/Qwen3-0.6B)
base and applies a single **LoRA supervised fine-tune (SFT)** targeting native, Hermes-style
`<tool_call>` emission over a small set of file/shell tools — then merges the adapter back into
the base for a self-contained model.

Part of the **Zynthetix AL-1** two-model lab (Model A = from-scratch NumPy LM; **Model B = improve an
open-source small model**, this repo). Code: https://github.com/GruheshKurra/AL1-model-B

---

## Results — before → after

Tool-calling is scored by **exact match** against a held-out 12-case set: `name_acc` = correct tool
name chosen, `full_acc` = correct name **and** arguments. Both evaluated with greedy decoding, fp32,
on the same device (no blending with chat scores).

| Metric | Base `Qwen3-0.6B` | **AL1-model-B (SFT)** | Δ |
|---|---|---|---|
| Tool `name_acc` | 0.500 | **1.000** | +0.500 |
| Tool `full_acc` | 0.500 | **0.917** (11/12) | +0.417 |
| Chat coherence (10 probes) | strong | strong, terser | no regression |

**Why the base scored only 0.5:** it *refused* to call `run_bash` and `edit_file` (0/2 each) and
mis-handled one `list_dir` — its arguments were fine *when* it chose to call. The SFT data therefore
over-weighted `edit_file` + `run_bash` + explicit anti-refusal examples, which closed the entire gap
(`name_acc` → 1.0 = a valid tool is now always selected).

> The merged fp16 model reproduces the adapter score exactly (`full_acc` 0.917) — merging lost nothing.

---

## The tools

Five file/shell tools, described to the model via the OpenAI-style `tools=` schema and the Qwen3
native chat template (Hermes `<tool_call>` tags):

| Tool | Purpose | Required args |
|---|---|---|
| `read_file` | Read a file's contents | `path` |
| `write_file` | Create/overwrite a file | `path`, `content` |
| `edit_file` | Replace an exact substring | `path`, `old_string`, `new_string` |
| `list_dir` | List a directory | `path` |
| `run_bash` | Run a shell command | `command` |

The model **emits** tool name + arguments; wiring those to a real executor (or an MCP client) is a
serving-layer concern, deliberately out of scope for the model itself.

---

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

MODEL = "karthik-2905/AL1-model-B"
tok = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(MODEL, dtype=torch.float16).to("mps")  # or "cuda"/"cpu"

TOOLS = [{"type": "function", "function": {
    "name": "read_file",
    "description": "Read and return the contents of a file.",
    "parameters": {"type": "object",
        "properties": {"path": {"type": "string"}}, "required": ["path"]}}}]  # + the other 4

msgs = [{"role": "user", "content": "Show me what's in config.py"}]
text = tok.apply_chat_template(msgs, tools=TOOLS, tokenize=False,
                              add_generation_prompt=True, enable_thinking=False)
ids = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True))
# -> <tool_call>{"name": "read_file", "arguments": {"path": "config.py"}}</tool_call>
```

---

## How it was built

| Stage | Choice |
|---|---|
| Base | `Qwen/Qwen3-0.6B` (Apache-2.0) — GQA + RoPE + SwiGLU + RMSNorm |
| PEFT | LoRA `r=16`, `α=32`, dropout 0.05, on all 7 attention+MLP projections |
| Trainer | TRL `SFTTrainer`, `assistant_only_loss=True`, `max_length=512`, packing off |
| Schedule | lr `3e-4` (cosine, 3% warmup), 3 epochs, effective batch 32, bf16 |
| Hardware | RunPod **RTX A6000** (train), Apple Silicon / MPS (merge + eval) |
| Merge | `merge_and_unload()` → standalone fp16 |

**LR sweep** (10% data, 1 epoch) picked `3e-4` (eval_loss: `1e-4`=1.099, `2e-4`=0.999, `3e-4`=0.975).
The full 3-epoch run overfits by *eval_loss* after ~epoch 1 (0.91 → 0.99), but generation quality and
tool exact-match were best at 3 epochs — the **task metric beat the proxy loss**, so 3 epochs shipped.

### Data

- **1,423 train / 74 val** examples, chat-formatted to the Qwen3 template, ~**2:1 tool:chat** ratio.
- Tool examples teach the Hermes `<tool_call>` format across all 5 tools (over-weighting the two the
  base refused); chat examples guard against chat regression.
- Train/val split is dedup'd and **leakage-checked against the eval set**.

### Evaluation

- **Tool track (trust anchor):** exact-match over a 12-case set, scored by `mlb/eval_tools.py`.
- **Chat track (regression guard):** 10 probes, currently eyeballed for coherence (LLM-judge = future work).

---

## Limitations

- **Small model (0.6B):** capable within its size, not a substitute for a large instruct model.
- **Chat quality is eyeballed,** not yet LLM-judge scored — no formal regression number.
- **No general-capability guard** (MMLU/GSM8K) was run; broad-knowledge regression is unmeasured.
- **4-bit quantization untested** — a 0.6B model can degrade more than the usual −1–2% at 4-bit.

---

## License

**Apache-2.0**, inherited from the `Qwen/Qwen3-0.6B` base.