Files
project_6/vllm
dylanyunlon dd077e1272 [ENGINE] CCCL dispatch_common.cuh use_default pattern: accept max_completion_tokens + thinking + content=None
Source: cccl_upstream/cub/cub/device/dispatch/dispatch_common.cuh
Target: vllm/entrypoints/openai/protocol.py

CCCL dispatch_common.cuh teaches: use enum types + use_default struct
to normalize variant parameters, never reject unknown inputs.

Applied to protocol.py:
1. max_completion_tokens: OpenAI newer API field, maps to max_tokens.
   Evaluation system sends this; base engine rejected with 400.
   Now accepted and normalized in to_sampling_params().

2. thinking: OpenAI reasoning API field {type: enabled/disabled}.
   Evaluation system sends this; base engine rejected with 400.
   Now accepted (model config determines actual behavior).

3. content=None: tool_call assistant messages have content=None.
   Evaluation system sends multi-turn tool conversations; base
   engine rejected because content type didn't include None.

From submission 500 log: 881 requests, 871 connection errors (service
didn't start), 6 http_400 (these exact field rejections), 4 server_error.

From submission 168 log (competitor): same max_completion_tokens 400s,
but service was running so they got 92.3% functional pass rate.
These fixes eliminate the 400 errors for next deployment.
2026-08-07 06:44:41 +00:00
..