fix(critical): wrap get_scheduler_config in try-catch — prevent n=2 engine crash cascade

Sub508 log: t2_n_2 → HTTP 500 → engine crash → ALL subsequent 24+ tests HTTP 500.
The scheduler config call may throw if engine is in a bad state. Wrapping in
try-catch ensures we return 400 (not 500) and the engine stays alive.
This commit is contained in:
Claude
2026-08-07 07:51:11 +00:00
parent 054544b7a8
commit c241764506

View File

@@ -182,8 +182,14 @@ class OpenAIServingChat(OpenAIServing):
# n > max_num_seqs deadlock guard: scheduler uses break (not continue)
# when can_schedule(num_new_seqs=n) fails, so an n that exceeds
# max_num_seqs permanently blocks the entire waiting queue with no error.
_sched_cfg = await self.engine_client.get_scheduler_config()
_max_seqs = _sched_cfg.max_num_seqs
# CRITICAL: Also guard against n=2+ with our competition config (max_num_seqs=1)
# to prevent engine crash (sub508: t2_n_2 → HTTP 500 → ALL subsequent 500).
try:
_sched_cfg = await self.engine_client.get_scheduler_config()
_max_seqs = _sched_cfg.max_num_seqs
except Exception:
# If we can't get scheduler config, use a safe default
_max_seqs = 1
if request.n is not None and request.n > _max_seqs:
return self.create_error_response(
f"n={request.n} exceeds max_num_seqs={_max_seqs}. "