fix: resolve all 8 deployment pipeline breaks
Breaks found and fixed: 1. Dockerfile: COPY 5 individual files → COPY entire ex_engine/ 2. patch_ops.sh EX_ENGINE_DIR: /workspace/ex_engine not found → added fallback 3. patch_ops.sh deploy: ix_ops.py to ex_engine/ (flat) → ex_engine/python/ (correct package) 4. ix_startup_patch.py: import from vllm.ex_engine.patch_vllm_ops → vllm.ex_engine.python.patch_vllm_ops 5. ix_moe_bridge.so: only deployed to ex_engine/ → also copy to model_executor/models/ and vllm root 6. ex_engine/__init__.py: missing re-exports → add imports so 'from vllm.ex_engine import x' works 7. gemm_grouped.so: compiled but never imported → add import + flag + prefill GEMM path in qwen3_5.py 8. build_moe_bridge.sh Python heredoc: SCRIPT_DIR not exported + wrong nested path → export + search both layouts Also added: - CUTLASS batched GEMM compile step (corex_batched_gemm.so for decode) - Full ex_engine/python/*.py deployment (was deploying only 2 of 19 files) - EX_ENGINE_INFRA_AUDIT.md documenting all findings
This commit is contained in:
@@ -75,14 +75,28 @@ echo "[moe_bridge] ixformer .so count: ${#IX_SO_FILES[@]}"
|
||||
# --- Build via torch.utils.cpp_extension ---
|
||||
mkdir -p "${SCRIPT_DIR}/prebuilt"
|
||||
|
||||
export SCRIPT_DIR VLLM_ROOT
|
||||
python3 << 'PYEOF'
|
||||
import os, sys, glob, shutil
|
||||
|
||||
script_dir = os.environ.get("SCRIPT_DIR", ".")
|
||||
vllm_root = os.environ.get("VLLM_ROOT", "")
|
||||
|
||||
moe_cu = os.path.join(script_dir, "ex_engine", "csrc", "moe_ops_impl.cu")
|
||||
bridge_cpp = os.path.join(script_dir, "ex_engine", "csrc", "ix_full_bridge_v2.cpp")
|
||||
# Find source files — try direct csrc/ first, then ex_engine/csrc/
|
||||
moe_cu = ""
|
||||
bridge_cpp = ""
|
||||
for base in [script_dir, os.path.join(script_dir, "ex_engine")]:
|
||||
candidate_cu = os.path.join(base, "csrc", "moe_ops_impl.cu")
|
||||
candidate_cpp = os.path.join(base, "csrc", "ix_full_bridge_v2.cpp")
|
||||
if os.path.isfile(candidate_cu):
|
||||
moe_cu = candidate_cu
|
||||
if os.path.isfile(candidate_cpp):
|
||||
bridge_cpp = candidate_cpp
|
||||
if not moe_cu or not bridge_cpp:
|
||||
print(f"[moe_bridge] ERROR: sources not found under {script_dir}")
|
||||
sys.exit(1)
|
||||
print(f"[moe_bridge] MOE_CU: {moe_cu}")
|
||||
print(f"[moe_bridge] BRIDGE_CPP: {bridge_cpp}")
|
||||
|
||||
# Collect linker flags
|
||||
extra_ldflags = []
|
||||
|
||||
Reference in New Issue
Block a user