fix(CRITICAL): engine death on image request + stop overwriting base corex modules
Root cause from latest docker build log: ValueError: You set image=0 in --limit-mm-per-prompt, but found 1 items → Engine background task crashes → AsyncEngineDeadError → all subsequent 503 Fixes: 1. computility-run.yaml: add --limit-mm-per-prompt image=1 Prevents multimodal ValueError from killing the engine process. 2. patch_ops.sh: DON'T overwrite base image's corex_gdn.py/corex_moe.py Comp 168 log proves base image's corex modules work with libcorex_gdn.so. Our overwrite broke CoreXGDN.__init__ (unexpected kwarg 'num_v_heads'). Only deploy ours if base has NO corex modules at all. Also deploy corex_fa2.py if base lacks it. 3. qwen3_5.py: try multiple CoreXGDN init signatures Base image CoreXGDN may accept different kwargs than ours. Try kwargs form first, fall back to positional. 4. corex_gdn.py: accept both calling conventions in __init__ Future-proof for when we DO need to deploy ours. 5. Copied upstream_ref headers: ilu_layer_fused_moe.h, ilu_layer_attention.h Last 2 missing ILU files from xllm. All 14/14 now present.
This commit is contained in:
@@ -463,6 +463,7 @@ class GatedDeltaNet(nn.Module):
|
||||
self._use_corex_gdn = False
|
||||
if _corex_gdn_available and _corex_gdn_module is not None:
|
||||
try:
|
||||
# Try base image's CoreXGDN signature first (may differ from ours)
|
||||
self._corex_gdn_obj = _corex_gdn_module.CoreXGDN(
|
||||
num_v_heads=self.num_v_heads // tp_size,
|
||||
num_k_heads=self.num_k_heads // tp_size,
|
||||
@@ -472,11 +473,25 @@ class GatedDeltaNet(nn.Module):
|
||||
layer_idx=layer_idx,
|
||||
)
|
||||
self._use_corex_gdn = True
|
||||
logger.info("GatedDeltaNet layer %d: CoreX fused GDN enabled", layer_idx)
|
||||
except TypeError:
|
||||
# Fallback: simpler signature
|
||||
try:
|
||||
self._corex_gdn_obj = _corex_gdn_module.CoreXGDN(
|
||||
self.num_v_heads // tp_size,
|
||||
self.head_k_dim,
|
||||
layer_idx=layer_idx,
|
||||
)
|
||||
self._use_corex_gdn = True
|
||||
except Exception as e2:
|
||||
logger.warning(
|
||||
"GatedDeltaNet layer %d: CoreX GDN init failed (%s), PyTorch",
|
||||
layer_idx, e2)
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"GatedDeltaNet layer %d: CoreX GDN init failed (%s), using PyTorch",
|
||||
"GatedDeltaNet layer %d: CoreX GDN init failed (%s), PyTorch",
|
||||
layer_idx, e)
|
||||
if self._use_corex_gdn and layer_idx == 0:
|
||||
logger.info("GatedDeltaNet: CoreX fused GDN enabled")
|
||||
|
||||
def _conv1d_weight_loader(self, param: torch.Tensor,
|
||||
loaded_weight: torch.Tensor) -> None:
|
||||
|
||||
Reference in New Issue
Block a user