ModelHub XC d2d5e73f12 初始化项目,由ModelHub XC社区提供模型
Model: switchboardllm/switchboard-router-4b
Source: Original Platform
2026-11-15 14:17:33 +08:00

license, language, base_model, base_model_relation, pipeline_tag, library_name, datasets, tags
license language base_model base_model_relation pipeline_tag library_name datasets tags
apache-2.0
en
Qwen/Qwen3-4B-Instruct-2507 finetune text-generation transformers
routellm/gpt4_dataset
router
llm-routing
cost-optimization
switchboard

Switchboard

switchboard-router-4b

Most prompts don't need a frontier model, the same way a plaster doesn't need a surgeon. Switchboard 4B reads a prompt and answers with one line of JSON: the cheapest tier that can handle it. Open weights, Apache 2.0.

Receipts, on 10,000 held-out prompts:

Switchboard 4B Random routing
Cost vs. always Claude Opus 5.5 67.0% less
Sent to a model too small 5.0% 10.2%

Every raw answer is on the evaluation page.

This is the larger sibling of switchboard-router-0.5b. Same training data, same answer format, same test set. On 10,000 held-out prompts it saves more and sends fewer hard prompts to a cheap model.

Try the 0.5B in your browser: Switchboard demo · Run the 4B locally: GGUF for Ollama

Largest size: switchboard-router-32b, tested on the same 10,000 prompts. It sends fewer hard prompts to a cheap model than the 4B and saves a little less.

Run it

With Ollama (4.28 GB download, a GPU is recommended):

ollama run hf.co/switchboardllm/switchboard-router-4b-GGUF "Prove there are infinitely many primes of the form 4k+3"
{"tier":"frontier","domain":"math"}

With Transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer
import json

repo = "switchboardllm/switchboard-router-4b"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="auto")

msgs = [{"role": "user", "content": "Unscramble the following words: eglar, ngareo, kobo"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=24, do_sample=False)
print(json.loads(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True)))
# {'tier': 'frontier', 'domain': 'general'}

With Python, using the switchboard-router package:

from switchboard_router import Router

router = Router(backend="ollama", backend_model="hf.co/switchboardllm/switchboard-router-4b-GGUF")
router.pick_model("Write a haiku about rain")
# 'anthropic/claude-haiku-5.5'

Use greedy decoding (do_sample=False, or temperature 0).

What it answers

Field Values
tier small, mid or frontier: the cheapest class of model expected to answer the prompt well
domain code, math, writing or general: a hint, from keyword labels

If the output isn't valid JSON, send the prompt to frontier. A failed routing call should cost money, never quality.

4B vs 0.5B

Both models on the same 10,000 held-out prompts they never saw in training, in their natural mix:

0.5B (v0.1) 4B 4B minus 0.5B (95% CI)
Cost saved vs. always using Claude Opus 5.5 64.5% 67.0% +2.5 pts (+1.8 to +3.3)
Hard prompts sent to frontier 59.1% 63.2% +4.1 pts (+1.4 to +6.7)
Sent too low (needed frontier, sent cheaper) 5.5% 5.0% -0.5 pts (-0.9 to -0.2)
Prompts served well (proxy) 94.5% 95.0%
Ranking AUC (hard vs. rest) 0.787 0.827
Tier accuracy 60.3% 64.0%
Invalid JSON outputs 0 0
Time per decision (RTX 5070 Ti, Transformers) 288 ms 489 ms

Confidence intervals come from a paired bootstrap over the 10,000 prompts (2,000 resamples), and none of them include zero. The gain is real but moderate: the ceiling is set by the labels, and a perfect router on this data would save 76.2%.

How it's measured. Each prompt in routellm/gpt4_dataset comes with a 2023 Mixtral-8x7B answer scored 1-5 by a GPT-4 judge. A prompt counts as served well if Switchboard sent it to frontier, or sent it lower and that answer scored at least 4. Cost uses Surplus output prices per 1M tokens: Claude Haiku 5.5 ($0.50) for small, Claude Sonnet 5.5 ($10) for mid, Claude Opus 5.5 ($20) for frontier. Method and every 0.5B answer are on the evaluation page.

Builds

Build Size Runs with Same decision as this model
This repo (safetensors, BF16) 8.04 GB Transformers, vLLM reference
GGUF Q8_0 4.28 GB Ollama, llama.cpp 339 of 347 test prompts

Q4_K_M agreed on only 302 of 347, so it is not published. The few disagreements left in Q8_0 are borderline prompts: the BF16 model itself changes 5 of 347 decisions between batched and one-at-a-time inference.

The model

Base Qwen3-4B-Instruct-2507 (Apache-2.0)
Method LoRA, rank 32, on all attention and MLP projections, merged into BF16 weights
Training data routellm/gpt4_dataset (Apache-2.0, LMSYS), 63,749 rows, the same set as 0.5B v0.1
Labels Mixtral score 5 → small, 4 → mid, 1-3 → frontier; domain from keywords
Training 1 epoch in 184 minutes on one RTX 5070 Ti. Best checkpoint by eval loss (0.0849). Loss on the JSON answer only.
Prompt format The 0.5B chat template, so both models are drop-in replacements for each other
Prompt length Trained on prompts cut to 512 tokens

Limitations

  • It still misroutes some prompts: 5.0% of held-out prompts were sent to a cheaper tier than they needed.
  • Labels come from 2023 models and a GPT-4 judge, not from people. Mostly English prompts.
  • It reads one prompt at a time, not the whole conversation, and only the first 512 tokens.
  • A prompt can try to talk it into a cheaper tier. Don't use it as a security boundary.
  • domain comes from keyword labels and is only a hint.
  • Switchboard is an independent project and is not affiliated with Anthropic, Alibaba, Robinhood, Surplus or the Switchboard oracle network.

License

Apache-2.0, inherited from Qwen3-4B-Instruct-2507 and routellm/gpt4_dataset.

S
Description
Model synced from source: switchboardllm/switchboard-router-4b
Readme
13 MiB
0 Stars 1 Watchers 0 Forks
Languages
Jinja 100%