64 lines
2.7 KiB
Markdown
64 lines
2.7 KiB
Markdown
|
---
|
|||
|
|
license: apache-2.0
|
|||
|
|
base_model: Qwen/Qwen3-0.6B
|
|||
|
|
language:
|
|||
|
|
- zh
|
|||
|
|
tags:
|
|||
|
|
- tool-calling
|
|||
|
|
- function-calling
|
|||
|
|
- knowledge-distillation
|
|||
|
|
- qwen3
|
|||
|
|
- edge
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Qwen3-0.6B · 端侧工具调用蒸馏版
|
|||
|
|
|
|||
|
|
把 **Qwen3-8B** 教师模型的**工具调用（tool/function calling）+ 思维链**能力，通过**序列级知识蒸馏 + 全量 SFT** 蒸馏进 **Qwen3-0.6B** 学生模型，面向 **CPU / 端侧本地部署**。场景为端侧智能音频设备控制（7 个自定义工具）。
|
|||
|
|
|
|||
|
|
📦 完整代码 / 数据 / 评测 / 复现：**https://github.com/dengdengf/qwen3-tool-distill**
|
|||
|
|
|
|||
|
|
## 结果（自建执行式评测集，227 条，防污染）
|
|||
|
|
|
|||
|
|
| 指标 | 训练前 base(0.6B) | **本模型(蒸馏后)** | 教师(8B) |
|
|||
|
|
|---|---|---|---|
|
|||
|
|
| 工具调用精确匹配（名+参） | 59.0% | **94.3%** | 97.8% |
|
|||
|
|
| 工具选择准确率 | 79.4% | **94.9%** | 98.3% |
|
|||
|
|
| 参数 schema 合规率 | 96.6% | **99.4%** | 100% |
|
|||
|
|
| 越界请求·正确拒调用 | 98.1% | **100%** | 100% |
|
|||
|
|
|
|||
|
|
- 训练带来 **+35.2pt**（59.0→94.3），达教师 **96.4%** 水平；相较教师**体积压缩 13.7×**（16.4GB→1.19GB）。
|
|||
|
|
|
|||
|
|
## 用法
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|||
|
|
|
|||
|
|
model_id = "yifan02/qwen3-0.6b-tool-distill"
|
|||
|
|
tok = AutoTokenizer.from_pretrained(model_id)
|
|||
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16").cuda()
|
|||
|
|
|
|||
|
|
tools = [{"type": "function", "function": {
|
|||
|
|
"name": "set_volume",
|
|||
|
|
"description": "设置指定音频设备的音量大小(0-100)。",
|
|||
|
|
"parameters": {"type": "object", "properties": {
|
|||
|
|
"device": {"type": "string", "enum": ["living_room","bedroom","kitchen","study","all"]},
|
|||
|
|
"level": {"type": "integer", "minimum": 0, "maximum": 100}},
|
|||
|
|
"required": ["device","level"]}}}] # 完整 7 个工具见 GitHub 仓库
|
|||
|
|
|
|||
|
|
msgs = [{"role":"system","content":"你是一个智能音频助手，负责控制家里的音频设备。"},
|
|||
|
|
{"role":"user","content":"把客厅音量调到40"}]
|
|||
|
|
text = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True, tokenize=False)
|
|||
|
|
out = model.generate(**tok(text, return_tensors="pt").to("cuda"), max_new_tokens=512)
|
|||
|
|
print(tok.decode(out[0], skip_special_tokens=True))
|
|||
|
|
# -> <think>...</think> <tool_call>{"name":"set_volume","arguments":{"device":"living_room","level":40}}</tool_call>
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 训练细节
|
|||
|
|
|
|||
|
|
序列级知识蒸馏（教师生成「思维链+工具调用」轨迹，拒绝采样保留 95.7%）+ 全量 SFT；bf16、有效 batch 32、lr 1e-5、3 epoch、max_len 2048、按 eval_loss 选最优 checkpoint。单张 RTX 4090。
|
|||
|
|
|
|||
|
|
## 局限
|
|||
|
|
|
|||
|
|
评测数据为程序化合成、分布规整；测试集 n=227；多工具（一句两调用）仍落后教师 ~17pt（0.6B 容量上限）。详见 GitHub 仓库。
|