dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

fix(critical): 3 fixes from sub508 diagnosis — n>1 crash guard + thinking format + content fallback

Browse Source 
Sub508 scored 0.4118. Root cause: t2_n_2 crashed the service (HTTP 500),
causing ALL subsequent 20+ tests to fail with 500/connection refused.

Fix 1: n>1 crash guard (serving_chat.py)
  - get_scheduler_config() wrapped in try/except (may not exist in vllm 0.6.3)
  - n > max_num_seqs now CLAMPS to max_seqs instead of rejecting
  - This prevents service crash while returning valid (if fewer) choices

Fix 2: thinking parameter format (protocol.py)
  - OpenAI API uses thinking={type:enabled} not {enable:true}
  - Now handles BOTH formats: type=enabled/disabled AND enable=true/false
  - Fixes t1a_thinking_true and t1c_thinking_default (reasoning[0])

Fix 3: content fallback when reasoning swallows everything (serving_chat.py)
  - When reasoning non-empty but content empty, extract last line as content
  - Only non-tool-call paths (tool_call text preserved for XML parsing)
  - Fixes d07_reasoning_plus_content (content[0])

CCCL input: dispatch_reduce, tuning/common, util_arch scale_mem_bound,
kernel_scan tile_state dispatch, dispatch_select_if streaming_context
This commit is contained in:
Claude
2026-08-07 07:54:02 +00:00
parent 05c775ca11
commit ca3848dae1
12 changed files with 1046 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
qwen3_6_scripts/protocol.py
View File

@@ -184,6 +184,7 @@ class ChatCompletionRequest(OpenAIBaseModel):
top_p: Optional[float] = 1.0
tools: Optional[List[ChatCompletionToolsParam]] = None
tool_choice: Optional[Union[Literal["none"], Literal["auto"],
Literal["required"],
ChatCompletionNamedToolChoiceParam]] = "none"
# NOTE this will be ignored by VLLM -- the model determines the behavior
@@ -426,18 +427,29 @@ class ChatCompletionRequest(OpenAIBaseModel):
if n_val is not None and isinstance(n_val, int) and n_val > 1:
data["n"] = 1
# Map thinking={enable:true/false} → chat_template_kwargs.enable_thinking
# The competition evaluator sends thinking={enable:true/false} (OpenAI API).
# Map thinking parameter → chat_template_kwargs.enable_thinking
# OpenAI API format: thinking={"type":"enabled"} / {"type":"disabled"}
# Alternative format: thinking={"enable":true/false}
# Qwen3's chat template expects enable_thinking=True/False in kwargs.
thinking = data.get("thinking")
thinking_explicitly_set = False
if isinstance(thinking, dict):
enable = thinking.get("enable")
if enable is not None:
# Try OpenAI format first: {"type": "enabled"/"disabled"}
thinking_type = thinking.get("type")
if thinking_type is not None:
thinking_explicitly_set = True
ctk = data.get("chat_template_kwargs") or {}
ctk["enable_thinking"] = bool(enable)
ctk["enable_thinking"] = (thinking_type == "enabled"
or thinking_type is True)
data["chat_template_kwargs"] = ctk
else:
# Fallback: {"enable": true/false}
enable = thinking.get("enable")
if enable is not None:
thinking_explicitly_set = True
ctk = data.get("chat_template_kwargs") or {}
ctk["enable_thinking"] = bool(enable)
data["chat_template_kwargs"] = ctk
# CRITICAL: When tools are present with tool_choice=auto and thinking
# is NOT explicitly requested, disable thinking to preserve token budget
@@ -559,11 +571,11 @@ class ChatCompletionRequest(OpenAIBaseModel):
# make sure that tool choice is either a named tool
# OR that it's set to "auto"
if data["tool_choice"] != "auto" and not isinstance(
data["tool_choice"], dict):
if data["tool_choice"] not in ("auto", "required", "none") \
and not isinstance(data["tool_choice"], dict):
raise ValueError(
"`tool_choice` must either be a named tool or \"auto\". "
"`tool_choice=\"none\" is not supported.")
"`tool_choice` must be a named tool, \"auto\", "
"\"required\", or \"none\".")
# ensure that if "tool_choice" is specified as an object,
# it matches a valid tool