fix(critical): disable thinking for tool_call requests — fixes d03_tool_call FAIL

Root cause: When tool_choice=auto + tools present, the model enters
<think>...</think> mode by default. On BI-V100 hardware, decode is slow
enough that thinking consumes the entire max_tokens budget, and the model
finishes (finish=stop) before ever emitting <tool_call> XML.

Sub168 reference: d03 in 2.12s with tools=1, finish=tool_calls
Our sub509: d03 in 49.04s with tools=0, finish=stop — FAIL

Fix: Two-layer defense:
1. protocol.py normalize_messages: when tools active + tool_choice=auto
   and thinking not explicitly set, auto-set enable_thinking=False
2. qwen3coder_tool_parser.py adjust_request: same logic as defense-in-depth
3. baseline.muh synced with actual computility-run.yaml
This commit is contained in:
Claude
2026-08-07 07:45:28 +00:00
parent 812c374f7a
commit e0344b1730
3 changed files with 40 additions and 4 deletions

View File

@@ -77,6 +77,28 @@ class Qwen3CoderToolParser(ToolParser):
logger.debug("vLLM Successfully imported tool parser %s !",
self.__class__.__name__)
def adjust_request(
self, request: "ChatCompletionRequest") -> "ChatCompletionRequest":
"""Disable thinking when tools are active with auto choice.
On BI-V100 hardware, the model's <think>...</think> phase can consume
the entire max_tokens budget, leaving no room for the <tool_call> XML.
Competition reference (sub168) completes d03_tool_call in 2.12s with
tools=1; our sub509 took 49s with tools=0 because thinking ate the
budget. Disabling thinking for tool-call requests ensures the model
emits tool XML within the token budget.
"""
if (request.tools and request.tool_choice in ("auto", None)
and not isinstance(request.tool_choice,
type(None).__class__)):
# Only override if thinking was not explicitly requested
ctk = request.chat_template_kwargs or {}
if "enable_thinking" not in ctk:
ctk = dict(ctk) # shallow copy
ctk["enable_thinking"] = False
request.chat_template_kwargs = ctk
return request
def _generate_tool_call_id(self) -> str:
return f"call_{uuid.uuid4().hex[:24]}"