6.4 KiB
license, language, base_model, base_model_relation, pipeline_tag, library_name, datasets, tags
| license | language | base_model | base_model_relation | pipeline_tag | library_name | datasets | tags | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| apache-2.0 |
|
Qwen/Qwen3-4B-Instruct-2507 | finetune | text-generation | transformers |
|
|
switchboard-router-4b
Most prompts don't need a frontier model, the same way a plaster doesn't need a surgeon. Switchboard 4B reads a prompt and answers with one line of JSON: the cheapest tier that can handle it. Open weights, Apache 2.0.
Receipts, on 10,000 held-out prompts:
| Switchboard 4B | Random routing | |
|---|---|---|
| Cost vs. always Claude Opus 5.5 | 67.0% less | |
| Sent to a model too small | 5.0% | 10.2% |
Every raw answer is on the evaluation page.
This is the larger sibling of switchboard-router-0.5b. Same training data, same answer format, same test set. On 10,000 held-out prompts it saves more and sends fewer hard prompts to a cheap model.
Try the 0.5B in your browser: Switchboard demo · Run the 4B locally: GGUF for Ollama
Largest size: switchboard-router-32b, tested on the same 10,000 prompts. It sends fewer hard prompts to a cheap model than the 4B and saves a little less.
Run it
With Ollama (4.28 GB download, a GPU is recommended):
ollama run hf.co/switchboardllm/switchboard-router-4b-GGUF "Prove there are infinitely many primes of the form 4k+3"
{"tier":"frontier","domain":"math"}
With Transformers:
from transformers import AutoModelForCausalLM, AutoTokenizer
import json
repo = "switchboardllm/switchboard-router-4b"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="auto")
msgs = [{"role": "user", "content": "Unscramble the following words: eglar, ngareo, kobo"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=24, do_sample=False)
print(json.loads(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True)))
# {'tier': 'frontier', 'domain': 'general'}
With Python, using the switchboard-router package:
from switchboard_router import Router
router = Router(backend="ollama", backend_model="hf.co/switchboardllm/switchboard-router-4b-GGUF")
router.pick_model("Write a haiku about rain")
# 'anthropic/claude-haiku-5.5'
Use greedy decoding (do_sample=False, or temperature 0).
What it answers
| Field | Values |
|---|---|
tier |
small, mid or frontier: the cheapest class of model expected to answer the prompt well |
domain |
code, math, writing or general: a hint, from keyword labels |
If the output isn't valid JSON, send the prompt to frontier. A failed routing call should cost money, never quality.
4B vs 0.5B
Both models on the same 10,000 held-out prompts they never saw in training, in their natural mix:
| 0.5B (v0.1) | 4B | 4B minus 0.5B (95% CI) | |
|---|---|---|---|
| Cost saved vs. always using Claude Opus 5.5 | 64.5% | 67.0% | +2.5 pts (+1.8 to +3.3) |
| Hard prompts sent to frontier | 59.1% | 63.2% | +4.1 pts (+1.4 to +6.7) |
| Sent too low (needed frontier, sent cheaper) | 5.5% | 5.0% | -0.5 pts (-0.9 to -0.2) |
| Prompts served well (proxy) | 94.5% | 95.0% | |
| Ranking AUC (hard vs. rest) | 0.787 | 0.827 | |
| Tier accuracy | 60.3% | 64.0% | |
| Invalid JSON outputs | 0 | 0 | |
| Time per decision (RTX 5070 Ti, Transformers) | 288 ms | 489 ms |
Confidence intervals come from a paired bootstrap over the 10,000 prompts (2,000 resamples), and none of them include zero. The gain is real but moderate: the ceiling is set by the labels, and a perfect router on this data would save 76.2%.
How it's measured. Each prompt in routellm/gpt4_dataset comes with a 2023 Mixtral-8x7B answer scored 1-5 by a GPT-4 judge. A prompt counts as served well if Switchboard sent it to frontier, or sent it lower and that answer scored at least 4. Cost uses Surplus output prices per 1M tokens: Claude Haiku 5.5 ($0.50) for small, Claude Sonnet 5.5 ($10) for mid, Claude Opus 5.5 ($20) for frontier. Method and every 0.5B answer are on the evaluation page.
Builds
| Build | Size | Runs with | Same decision as this model |
|---|---|---|---|
| This repo (safetensors, BF16) | 8.04 GB | Transformers, vLLM | reference |
| GGUF Q8_0 | 4.28 GB | Ollama, llama.cpp | 339 of 347 test prompts |
Q4_K_M agreed on only 302 of 347, so it is not published. The few disagreements left in Q8_0 are borderline prompts: the BF16 model itself changes 5 of 347 decisions between batched and one-at-a-time inference.
The model
| Base | Qwen3-4B-Instruct-2507 (Apache-2.0) |
| Method | LoRA, rank 32, on all attention and MLP projections, merged into BF16 weights |
| Training data | routellm/gpt4_dataset (Apache-2.0, LMSYS), 63,749 rows, the same set as 0.5B v0.1 |
| Labels | Mixtral score 5 → small, 4 → mid, 1-3 → frontier; domain from keywords |
| Training | 1 epoch in 184 minutes on one RTX 5070 Ti. Best checkpoint by eval loss (0.0849). Loss on the JSON answer only. |
| Prompt format | The 0.5B chat template, so both models are drop-in replacements for each other |
| Prompt length | Trained on prompts cut to 512 tokens |
Limitations
- It still misroutes some prompts: 5.0% of held-out prompts were sent to a cheaper tier than they needed.
- Labels come from 2023 models and a GPT-4 judge, not from people. Mostly English prompts.
- It reads one prompt at a time, not the whole conversation, and only the first 512 tokens.
- A prompt can try to talk it into a cheaper tier. Don't use it as a security boundary.
domaincomes from keyword labels and is only a hint.- Switchboard is an independent project and is not affiliated with Anthropic, Alibaba, Robinhood, Surplus or the Switchboard oracle network.
License
Apache-2.0, inherited from Qwen3-4B-Instruct-2507 and routellm/gpt4_dataset.
