fix(CRITICAL): CoreXGDN interface mismatch + engine death protection
Three fixes for the three bugs in latest docker log:
1. corex_gdn.py REWRITTEN — interface now matches qwen3_5.py:
OLD: CoreXGDN(num_heads, head_dim, layer_idx, chunk_size, eps)
NEW: CoreXGDN(num_v_heads, num_k_heads, head_k_dim, head_v_dim, conv_kernel_size, layer_idx)
OLD forward: (q, k, v, gate, beta, conv_state, temporal_state, attn_metadata)
NEW forward: (hidden_states, attn_metadata, conv_state, temporal_state,
in_proj_qkv, in_proj_z, in_proj_b, in_proj_a,
conv1d_weight, A_log, dt_bias, norm, out_proj)
Fixes: 'CoreXGDN.__init__() got unexpected keyword argument num_v_heads'
2. serving_chat.py — engine death protection for multimodal:
When model has no multimodal_config, return 400 instead of passing image data
to engine (which causes permanent AsyncEngineDeadError).
Fixes: 'ValueError: You set image=0 but found 1 items'
3. patch_ops.sh — ALWAYS deploy our modules (base image has bugs):
- qwen3_5.py: ALWAYS deploy (base has NaN)
- corex_gdn/moe/fa2.py: ALWAYS deploy (base interface mismatch)
- corex_fa2.py was MISSING from base → now deployed
This commit is contained in:
@@ -312,6 +312,14 @@ class OpenAIServingChat(OpenAIServing):
|
||||
engine_inputs = TokensPrompt(
|
||||
prompt_token_ids=prompt_inputs["prompt_token_ids"])
|
||||
if mm_data is not None:
|
||||
# Protect engine from death: if model doesn't support multimodal,
|
||||
# return 400 instead of crashing the entire engine.
|
||||
# ValueError "image=0 but found 1" kills the async engine permanently.
|
||||
mm_config = getattr(self.model_config, 'multimodal_config', None)
|
||||
if mm_config is None:
|
||||
logger.warning("Image data in request but model has no multimodal_config — rejecting to protect engine")
|
||||
return self.create_error_response(
|
||||
"This model does not support multimodal (image) inputs.")
|
||||
engine_inputs["multi_modal_data"] = mm_data
|
||||
|
||||
is_tracing_enabled = (await
|
||||
|
||||
Reference in New Issue
Block a user