dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

fix(critical): wrap get_scheduler_config in try-catch — prevent n=2 engine crash cascade

Browse Source 
Sub508 log: t2_n_2 → HTTP 500 → engine crash → ALL subsequent 24+ tests HTTP 500.
The scheduler config call may throw if engine is in a bad state. Wrapping in
try-catch ensures we return 400 (not 500) and the engine stays alive.
This commit is contained in:
Claude
2026-08-07 07:51:11 +00:00
parent 054544b7a8
commit c241764506
1 changed files with 8 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
qwen3_6_scripts/serving_chat.py
View File

@@ -182,8 +182,14 @@ class OpenAIServingChat(OpenAIServing):
# n > max_num_seqs deadlock guard: scheduler uses break (not continue) # n > max_num_seqs deadlock guard: scheduler uses break (not continue)
# when can_schedule(num_new_seqs=n) fails, so an n that exceeds # when can_schedule(num_new_seqs=n) fails, so an n that exceeds
# max_num_seqs permanently blocks the entire waiting queue with no error. # max_num_seqs permanently blocks the entire waiting queue with no error.
_sched_cfg = await self.engine_client.get_scheduler_config() # CRITICAL: Also guard against n=2+ with our competition config (max_num_seqs=1)
_max_seqs = _sched_cfg.max_num_seqs # to prevent engine crash (sub508: t2_n_2 → HTTP 500 → ALL subsequent 500).
try:
_sched_cfg = await self.engine_client.get_scheduler_config()
_max_seqs = _sched_cfg.max_num_seqs
except Exception:
# If we can't get scheduler config, use a safe default
_max_seqs = 1
if request.n is not None and request.n > _max_seqs: if request.n is not None and request.n > _max_seqs:
return self.create_error_response( return self.create_error_response(
f"n={request.n} exceeds max_num_seqs={_max_seqs}. " f"n={request.n} exceeds max_num_seqs={_max_seqs}. "