初始化项目，由ModelHub XC社区提供模型

Model: AryanNsc/qwen3-0.6b-tool-router Source: Original Platform
2026-07-03 14:08:18 +08:00
commit 34a5ded4f2
12 changed files with 152018 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,127 @@
+---
+license: apache-2.0
+tags:
+  - tool-calling
+  - edge-inference
+library_name: transformers
+---
+
+# qwen3-0.6b-tool-router
+
+**A low-latency, schema-strict tool/function calling model optimized for edge-device inference.**
+
+
+## Overview
+
+`qwen3-0.6b-tool-router` is a **verticalized Small Language Model (SLM)** derived from **Qwen3-0.6B**, purpose-built for **tool and function routing** under strict JSON schemas.
+
+Unlike general-purpose chat or instruction-following models, this model is optimized to run as a **deterministic router** in agentic systems, especially in **resource-constrained edge environments** (e.g., CPUs, embedded GPUs, mobile accelerators).
+
+Its sole responsibility is to reliably map **natural language queries → structured tool calls**, with **minimal latency** and **zero tolerance for hallucinated tools**.
+
+
+### Key Properties
+
+- **Model Size:** 0.6B parameters  
+- **No Chain-of-Thought:** Disabled to reduce token count and parsing cost  
+- **Strict JSON Output:** Designed for direct machine consumption  
+- **Low Memory Footprint:** QLoRA fine-tuning, edge-friendly quantization  
+- **Fast Cold Start:** Ideal for on-device or near-device inference  
+
+This makes it well-suited for:
+- On-device assistants
+- Local agent routers
+- Offline-capable systems
+- Privacy-sensitive deployments
+
+### BFCL Results
+
+| Category                  | Score |
+|---------------------------|-------|
+| **Non-Live Parallel AST** | **83.50%** |
+| **Multi-Turn Base**       | **90.42%** |
+| **Live Simple AST**       | **62.86%** |
+| **Live Parallel AST**     | **52.00%** |
+| **Relevance Detection**   | **90.89%** |
+
+```python
+import json
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+MODEL_PATH = "AryanNsc/qwen3-0.6b-tool-router"
+
+# Load tokenizer & model
+tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH, trust_remote_code=True)
+tokenizer.padding_side = "left"
+tokenizer.pad_token = tokenizer.eos_token
+
+model = AutoModelForCausalLM.from_pretrained(
+    MODEL_PATH,
+    device_map="auto",
+    torch_dtype="auto",
+    trust_remote_code=True
+)
+
+# Define a tool
+tools = [{
+    "name": "get_weather",
+    "description": "Get weather for a city",
+    "parameters": {
+        "type": "object",
+        "properties": {
+            "city": {"type": "string"}
+        },
+        "required": ["city"]
+    }
+}]
+
+# Build system prompt with tools
+system_prompt = (
+    "You may call one or more functions.\n\n"
+    "<tools>\n"
+    + "\n".join(json.dumps(t) for t in tools)
+    + "\n</tools>\n\n"
+    "Return the function call inside <tool_call></tool_call> tags."
+)
+
+messages = [
+    {"role": "system", "content": system_prompt},
+    {"role": "user", "content": "What's the weather in Tokyo?"}
+]
+
+# Apply chat template
+prompt = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True,
+    enable_thinking=False
+)
+
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+# Generate
+with torch.no_grad():
+    outputs = model.generate(
+        **inputs,
+        max_new_tokens=256,
+        do_sample=False,
+        pad_token_id=tokenizer.pad_token_id
+    )
+
+# Decode only the generated tokens
+generated = outputs[:, inputs.input_ids.shape[1]:]
+text = tokenizer.decode(generated[0], skip_special_tokens=True)
+
+print(text)
+```
+
+## Why This Model for Edge Inference?
+
+Edge environments demand:
+- Small model size
+- Predictable latency
+- Deterministic outputs
+- Minimal parsing overhead
+
+This model was explicitly trained to satisfy those constraints.