fix(protocol): reduce HTTP 400 errors for replay — accept tool_choice=none + extra fields
1. tool_choice='none' now accepted per OpenAI spec (strip and continue). Previously raised ValueError, causing 400 on replay requests. 2. Pydantic extra='forbid' → extra='ignore'. Real-world replay requests from Tencent API contain fields like service_tier, store, metadata, reasoning_effort etc. that our model doesn't declare. forbid rejects them all; ignore silently drops them. Sub 168 had 77 http_400 errors in replay — these two fixes should eliminate most of them, improving successful request count and score. CCCL tuning_transform.cuh pattern: accept all valid input configurations gracefully (policy_selector handles unknown cc values with fallback).
This commit is contained in:
@@ -57,7 +57,10 @@ class CustomChatCompletionMessageParam(TypedDict, total=False):
|
||||
|
||||
class OpenAIBaseModel(BaseModel):
|
||||
# OpenAI API does not allow extra fields
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
# Real-world clients (replay, third-party SDKs) may send extra fields
|
||||
# like service_tier, store, metadata, reasoning_effort, etc.
|
||||
# "ignore" accepts the request and silently drops unknown fields.
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
|
||||
class ErrorResponse(OpenAIBaseModel):
|
||||
@@ -517,6 +520,12 @@ class ChatCompletionRequest(OpenAIBaseModel):
|
||||
# if "tool_choice" is specified -- validation
|
||||
if "tool_choice" in data:
|
||||
|
||||
# "none" means don't use any tools — valid per OpenAI spec,
|
||||
# just strip tool_choice and let vLLM ignore tools.
|
||||
if data["tool_choice"] == "none":
|
||||
del data["tool_choice"]
|
||||
return data
|
||||
|
||||
# ensure that if "tool choice" is specified, tools are present
|
||||
if "tools" not in data or data["tools"] is None:
|
||||
raise ValueError(
|
||||
|
||||
Reference in New Issue
Block a user