fix(CRITICAL): engine death on image request + stop overwriting base corex modules

Root cause from latest docker build log:
  ValueError: You set image=0 in --limit-mm-per-prompt, but found 1 items
  → Engine background task crashes → AsyncEngineDeadError → all subsequent 503

Fixes:
1. computility-run.yaml: add --limit-mm-per-prompt image=1
   Prevents multimodal ValueError from killing the engine process.

2. patch_ops.sh: DON'T overwrite base image's corex_gdn.py/corex_moe.py
   Comp 168 log proves base image's corex modules work with libcorex_gdn.so.
   Our overwrite broke CoreXGDN.__init__ (unexpected kwarg 'num_v_heads').
   Only deploy ours if base has NO corex modules at all.
   Also deploy corex_fa2.py if base lacks it.

3. qwen3_5.py: try multiple CoreXGDN init signatures
   Base image CoreXGDN may accept different kwargs than ours.
   Try kwargs form first, fall back to positional.

4. corex_gdn.py: accept both calling conventions in __init__
   Future-proof for when we DO need to deploy ours.

5. Copied upstream_ref headers: ilu_layer_fused_moe.h, ilu_layer_attention.h
   Last 2 missing ILU files from xllm. All 14/14 now present.
This commit is contained in:
Claude
2026-08-10 09:12:05 +00:00
parent ff3562b941
commit f87689a4ef
7 changed files with 435 additions and 24 deletions

View File

@@ -463,6 +463,7 @@ class GatedDeltaNet(nn.Module):
self._use_corex_gdn = False
if _corex_gdn_available and _corex_gdn_module is not None:
try:
# Try base image's CoreXGDN signature first (may differ from ours)
self._corex_gdn_obj = _corex_gdn_module.CoreXGDN(
num_v_heads=self.num_v_heads // tp_size,
num_k_heads=self.num_k_heads // tp_size,
@@ -472,11 +473,25 @@ class GatedDeltaNet(nn.Module):
layer_idx=layer_idx,
)
self._use_corex_gdn = True
logger.info("GatedDeltaNet layer %d: CoreX fused GDN enabled", layer_idx)
except TypeError:
# Fallback: simpler signature
try:
self._corex_gdn_obj = _corex_gdn_module.CoreXGDN(
self.num_v_heads // tp_size,
self.head_k_dim,
layer_idx=layer_idx,
)
self._use_corex_gdn = True
except Exception as e2:
logger.warning(
"GatedDeltaNet layer %d: CoreX GDN init failed (%s), PyTorch",
layer_idx, e2)
except Exception as e:
logger.warning(
"GatedDeltaNet layer %d: CoreX GDN init failed (%s), using PyTorch",
"GatedDeltaNet layer %d: CoreX GDN init failed (%s), PyTorch",
layer_idx, e)
if self._use_corex_gdn and layer_idx == 0:
logger.info("GatedDeltaNet: CoreX fused GDN enabled")
def _conv1d_weight_loader(self, param: torch.Tensor,
loaded_weight: torch.Tensor) -> None: