Three fixes for the three bugs in latest docker log:
1. corex_gdn.py REWRITTEN — interface now matches qwen3_5.py:
OLD: CoreXGDN(num_heads, head_dim, layer_idx, chunk_size, eps)
NEW: CoreXGDN(num_v_heads, num_k_heads, head_k_dim, head_v_dim, conv_kernel_size, layer_idx)
OLD forward: (q, k, v, gate, beta, conv_state, temporal_state, attn_metadata)
NEW forward: (hidden_states, attn_metadata, conv_state, temporal_state,
in_proj_qkv, in_proj_z, in_proj_b, in_proj_a,
conv1d_weight, A_log, dt_bias, norm, out_proj)
Fixes: 'CoreXGDN.__init__() got unexpected keyword argument num_v_heads'
2. serving_chat.py — engine death protection for multimodal:
When model has no multimodal_config, return 400 instead of passing image data
to engine (which causes permanent AsyncEngineDeadError).
Fixes: 'ValueError: You set image=0 but found 1 items'
3. patch_ops.sh — ALWAYS deploy our modules (base image has bugs):
- qwen3_5.py: ALWAYS deploy (base has NaN)
- corex_gdn/moe/fa2.py: ALWAYS deploy (base interface mismatch)
- corex_fa2.py was MISSING from base → now deployed
Reverted the NO-FALLBACK rewrite of corex_gdn.py and qwen3_5.py.
Policy: do NOT rewrite modules that already exist in base image or
upstream_ref. If an interface doesn't match, fix the interface call
site — don't rewrite the entire module in pure PyTorch.
Base image has corex_gdn.py, corex_moe.py, corex_fa2.py with C++
backends. The right approach is to match their __init__ signatures,
not replace them with slower Python reimplementations.
Root cause from latest docker build log:
ValueError: You set image=0 in --limit-mm-per-prompt, but found 1 items
→ Engine background task crashes → AsyncEngineDeadError → all subsequent 503
Fixes:
1. computility-run.yaml: add --limit-mm-per-prompt image=1
Prevents multimodal ValueError from killing the engine process.
2. patch_ops.sh: DON'T overwrite base image's corex_gdn.py/corex_moe.py
Comp 168 log proves base image's corex modules work with libcorex_gdn.so.
Our overwrite broke CoreXGDN.__init__ (unexpected kwarg 'num_v_heads').
Only deploy ours if base has NO corex modules at all.
Also deploy corex_fa2.py if base lacks it.
3. qwen3_5.py: try multiple CoreXGDN init signatures
Base image CoreXGDN may accept different kwargs than ours.
Try kwargs form first, fall back to positional.
4. corex_gdn.py: accept both calling conventions in __init__
Future-proof for when we DO need to deploy ours.
5. Copied upstream_ref headers: ilu_layer_fused_moe.h, ilu_layer_attention.h
Last 2 missing ILU files from xllm. All 14/14 now present.
