dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

fix(CRITICAL): engine death on image request + stop overwriting base corex modules

Browse Source 
Root cause from latest docker build log:
  ValueError: You set image=0 in --limit-mm-per-prompt, but found 1 items
  → Engine background task crashes → AsyncEngineDeadError → all subsequent 503

Fixes:
1. computility-run.yaml: add --limit-mm-per-prompt image=1
   Prevents multimodal ValueError from killing the engine process.

2. patch_ops.sh: DON'T overwrite base image's corex_gdn.py/corex_moe.py
   Comp 168 log proves base image's corex modules work with libcorex_gdn.so.
   Our overwrite broke CoreXGDN.__init__ (unexpected kwarg 'num_v_heads').
   Only deploy ours if base has NO corex modules at all.
   Also deploy corex_fa2.py if base lacks it.

3. qwen3_5.py: try multiple CoreXGDN init signatures
   Base image CoreXGDN may accept different kwargs than ours.
   Try kwargs form first, fall back to positional.

4. corex_gdn.py: accept both calling conventions in __init__
   Future-proof for when we DO need to deploy ours.

5. Copied upstream_ref headers: ilu_layer_fused_moe.h, ilu_layer_attention.h
   Last 2 missing ILU files from xllm. All 14/14 now present.
This commit is contained in:
Claude
2026-08-10 09:12:05 +00:00
parent ff3562b941
commit f87689a4ef
7 changed files with 435 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

31
qwen3_6_scripts/patch_ops.sh
View File

@@ -180,17 +180,34 @@ if [ -n "$VLLM2" ]; then
cp ./chat_utils.py "$VLLM2/entrypoints/chat_utils.py" 2>/dev/null || true
fi
# Deploy corex_gdn.py + corex_moe.py → vllm model_executor/models/
# These provide the fused GDN prefill kernel and MoE pipeline that competitor 168 had
if [ -f "/workspace/ex_engine/python/corex_gdn.py" ]; then
# corex_gdn.py + corex_moe.py: DO NOT overwrite base image originals!
# Comp 168 log proves: base image's corex_gdn.py loads libcorex_gdn.so and works.
# Our overwrite breaks the interface (CoreXGDN.__init__ signature mismatch).
# Only deploy ours if base has NO corex modules at all.
if [ ! -f "$VLLM/model_executor/models/corex_gdn.py" ]; then
cp "/workspace/ex_engine/python/corex_gdn.py" "$VLLM/model_executor/models/corex_gdn.py" 2>/dev/null || true
echo "[patch_ops] corex_gdn.py deployed (base had none)"
fi
if [ ! -f "$VLLM/model_executor/models/corex_moe.py" ]; then
cp "/workspace/ex_engine/python/corex_moe.py" "$VLLM/model_executor/models/corex_moe.py" 2>/dev/null || true
echo "[patch_ops] Deployed: corex_gdn.py + corex_moe.py → $VLLM/model_executor/models/"
if [ -n "$VLLM2" ]; then
cp "/workspace/ex_engine/python/corex_gdn.py" "$VLLM2/model_executor/models/corex_gdn.py" 2>/dev/null || true
cp "/workspace/ex_engine/python/corex_moe.py" "$VLLM2/model_executor/models/corex_moe.py" 2>/dev/null || true
echo "[patch_ops] corex_moe.py deployed (base had none)"
fi
# corex_fa2.py: deploy if base doesn't have it
# Comp 168 log: corex_fa2.py provides FA2 packed/paged/chunked dispatch
if [ ! -f "$VLLM/model_executor/models/corex_fa2.py" ]; then
if [ -f "/workspace/ex_engine/python/corex_fa2.py" ]; then
cp "/workspace/ex_engine/python/corex_fa2.py" "$VLLM/model_executor/models/corex_fa2.py" 2>/dev/null || true
echo "[patch_ops] corex_fa2.py deployed (base had none)"
fi
fi
echo "[patch_ops] CoreX modules: preserved base originals where they exist"
if [ -n "$VLLM2" ]; then
for _CM in corex_gdn.py corex_moe.py corex_fa2.py; do
if [ -f "$VLLM/model_executor/models/$_CM" ] && [ ! -f "$VLLM2/model_executor/models/$_CM" ]; then
cp "$VLLM/model_executor/models/$_CM" "$VLLM2/model_executor/models/$_CM" 2>/dev/null || true
fi
done
fi
# Deploy EX Engine Python module + C++ bridge into vllm importable path
EX_ENGINE_SRC="/workspace/ex_engine"