dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

fix(critical): disable thinking for tool_call requests — fixes d03_tool_call FAIL

Browse Source 
Root cause: When tool_choice=auto + tools present, the model enters
<think>...</think> mode by default. On BI-V100 hardware, decode is slow
enough that thinking consumes the entire max_tokens budget, and the model
finishes (finish=stop) before ever emitting <tool_call> XML.

Sub168 reference: d03 in 2.12s with tools=1, finish=tool_calls
Our sub509: d03 in 49.04s with tools=0, finish=stop — FAIL

Fix: Two-layer defense:
1. protocol.py normalize_messages: when tools active + tool_choice=auto
   and thinking not explicitly set, auto-set enable_thinking=False
2. qwen3coder_tool_parser.py adjust_request: same logic as defense-in-depth
3. baseline.muh synced with actual computility-run.yaml
This commit is contained in:
Claude
2026-08-07 07:45:28 +00:00
parent 812c374f7a
commit e0344b1730
3 changed files with 40 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
qwen3_6_scripts/protocol.py
View File

@@ -421,13 +421,30 @@ class ChatCompletionRequest(OpenAIBaseModel):
# The competition evaluator sends thinking={enable:true/false} (OpenAI API).
# Qwen3's chat template expects enable_thinking=True/False in kwargs.
thinking = data.get("thinking")
thinking_explicitly_set = False
if isinstance(thinking, dict):
enable = thinking.get("enable")
if enable is not None:
thinking_explicitly_set = True
ctk = data.get("chat_template_kwargs") or {}
ctk["enable_thinking"] = bool(enable)
data["chat_template_kwargs"] = ctk
# CRITICAL: When tools are present with tool_choice=auto and thinking
# is NOT explicitly requested, disable thinking to preserve token budget
# for tool call XML generation. Without this, the model spends all
# tokens on <think>...</think> and finishes before emitting <tool_call>.
# This matches the competition reference (sub168: d03 in 2.12s).
if not thinking_explicitly_set:
has_tools = data.get("tools") is not None and len(data.get("tools", [])) > 0
tc = data.get("tool_choice")
tool_choice_active = (tc == "auto" or (tc is None and has_tools)
or isinstance(tc, dict))
if has_tools and tool_choice_active:
ctk = data.get("chat_template_kwargs") or {}
ctk["enable_thinking"] = False
data["chat_template_kwargs"] = ctk
messages = data.get("messages")
if not isinstance(messages, list):
return data