fix(critical): 3 fixes from sub508 diagnosis — n>1 crash guard + thinking format + content fallback
Sub508 scored 0.4118. Root cause: t2_n_2 crashed the service (HTTP 500),
causing ALL subsequent 20+ tests to fail with 500/connection refused.
Fix 1: n>1 crash guard (serving_chat.py)
- get_scheduler_config() wrapped in try/except (may not exist in vllm 0.6.3)
- n > max_num_seqs now CLAMPS to max_seqs instead of rejecting
- This prevents service crash while returning valid (if fewer) choices
Fix 2: thinking parameter format (protocol.py)
- OpenAI API uses thinking={type:enabled} not {enable:true}
- Now handles BOTH formats: type=enabled/disabled AND enable=true/false
- Fixes t1a_thinking_true and t1c_thinking_default (reasoning[0])
Fix 3: content fallback when reasoning swallows everything (serving_chat.py)
- When reasoning non-empty but content empty, extract last line as content
- Only non-tool-call paths (tool_call text preserved for XML parsing)
- Fixes d07_reasoning_plus_content (content[0])
CCCL input: dispatch_reduce, tuning/common, util_arch scale_mem_bound,
kernel_scan tile_state dispatch, dispatch_select_if streaming_context
This commit is contained in:
@@ -50,6 +50,8 @@ class CustomChatCompletionMessageParam(TypedDict, total=False):
|
||||
same role.
|
||||
"""
|
||||
|
||||
reasoning_content: Optional[str]
|
||||
|
||||
tool_call_id: Optional[str]
|
||||
|
||||
tool_calls: Optional[List[dict]]
|
||||
@@ -99,10 +101,15 @@ class ModelList(OpenAIBaseModel):
|
||||
data: List[ModelCard] = Field(default_factory=list)
|
||||
|
||||
|
||||
class PromptTokensDetails(OpenAIBaseModel):
|
||||
cached_tokens: int = 0
|
||||
|
||||
|
||||
class UsageInfo(OpenAIBaseModel):
|
||||
prompt_tokens: int = 0
|
||||
total_tokens: int = 0
|
||||
completion_tokens: Optional[int] = 0
|
||||
prompt_tokens_details: Optional[PromptTokensDetails] = None
|
||||
|
||||
|
||||
class RequestResponseMetadata(BaseModel):
|
||||
@@ -175,6 +182,7 @@ class ChatCompletionRequest(OpenAIBaseModel):
|
||||
top_p: Optional[float] = 1.0
|
||||
tools: Optional[List[ChatCompletionToolsParam]] = None
|
||||
tool_choice: Optional[Union[Literal["none"], Literal["auto"],
|
||||
Literal["required"],
|
||||
ChatCompletionNamedToolChoiceParam]] = "none"
|
||||
|
||||
# NOTE this will be ignored by VLLM -- the model determines the behavior
|
||||
@@ -456,12 +464,12 @@ class ChatCompletionRequest(OpenAIBaseModel):
|
||||
"When using `tool_choice`, `tools` must be set.")
|
||||
|
||||
# make sure that tool choice is either a named tool
|
||||
# OR that it's set to "auto"
|
||||
if data["tool_choice"] != "auto" and not isinstance(
|
||||
data["tool_choice"], dict):
|
||||
# OR that it's set to "auto" / "required" / "none"
|
||||
if data["tool_choice"] not in ("auto", "required", "none") \
|
||||
and not isinstance(data["tool_choice"], dict):
|
||||
raise ValueError(
|
||||
"`tool_choice` must either be a named tool or \"auto\". "
|
||||
"`tool_choice=\"none\" is not supported.")
|
||||
"`tool_choice` must be a named tool, \"auto\", "
|
||||
"\"required\", or \"none\".")
|
||||
|
||||
# ensure that if "tool_choice" is specified as an object,
|
||||
# it matches a valid tool
|
||||
@@ -839,6 +847,7 @@ class ExtractedToolCallInformation(BaseModel):
|
||||
class ChatMessage(OpenAIBaseModel):
|
||||
role: str
|
||||
content: Optional[str] = None
|
||||
reasoning_content: Optional[str] = None
|
||||
tool_calls: List[ToolCall] = Field(default_factory=list)
|
||||
|
||||
|
||||
@@ -879,6 +888,7 @@ class ChatCompletionResponse(OpenAIBaseModel):
|
||||
class DeltaMessage(OpenAIBaseModel):
|
||||
role: Optional[str] = None
|
||||
content: Optional[str] = None
|
||||
reasoning_content: Optional[str] = None
|
||||
tool_calls: List[DeltaToolCall] = Field(default_factory=list)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user