project_6

dylanyunlong/project_6

Fork 0

Commit Graph

Author	SHA1	Message	Date
EX Engine	e04a3bace9	fix: fail-fast on ix_bridge failure + probe script for real machine 1. ix_bridge.py: RuntimeError instead of silent PyTorch fallback If JIT compile fails, crash immediately with diagnostic message. 0 score with no error log is worse than a visible crash. 2. qwen3_5.py: explicit WARNING log on import failure (not silent) Shows exact error so we can diagnose from docker log. 3. probe_ixformer_symbols.py: definitive test for real machine - Finds all ixformer .so files - nm/objdump for topk_softmax C++ symbol - Checks Python bindings - Attempts JIT compile + link (the real test) - Prints PASS/FAIL with next-step instructions Run on real machine: python3 probe_ixformer_symbols.py	2026-08-10 03:04:50 +00:00
EX Engine	d21b2505bb	fix: wire MoE topk via ixformer C++ bridge + disable broken flash_qla GDN Two call chain breaks fixed: 1. MoE routing (2304 calls/token): BEFORE: torch.softmax + torch.topk (3 Python GPU ops, no ixformer) AFTER: ix_bridge.py → ix_moe_bridge.cpp → ixformer::infer::topk_softmax() Source: upstream_ref/xllm/xllm/core/kernels/ilu/fused_moe.cpp line 46 The C++ API exists in base image SDK (ixformer.h declares it), only the Python binding (ixformer.functions) was missing. 2. GDN prefill (4 layers, 99.98% NaN): BEFORE: flash_qla SM70 kernel → abs mean=inf → nan_to_num → zeros AFTER: skip flash_qla, use _pytorch_forward directly Source: upstream_ref/xllm qwen3_gated_delta_net_base.cpp uses identical PyTorch chunked logic (no flash_qla). Sub168 (working build) never deployed flash_qla either. Files: - ex_engine/csrc/ix_moe_bridge.cpp: torch C++ extension calling ixformer C++ API - ex_engine/python/ix_bridge.py: JIT-compile loader with PyTorch fallback - qwen3_5.py: import ix_bridge for MoE, disable flash_qla for GDN - patch_ops.sh: deploy ix_bridge .cpp + .py into vllm model dir	2026-08-10 03:00:35 +00:00

Author

SHA1

Message

Date

EX Engine

e04a3bace9

fix: fail-fast on ix_bridge failure + probe script for real machine

1. ix_bridge.py: RuntimeError instead of silent PyTorch fallback
   If JIT compile fails, crash immediately with diagnostic message.
   0 score with no error log is worse than a visible crash.

2. qwen3_5.py: explicit WARNING log on import failure (not silent)
   Shows exact error so we can diagnose from docker log.

3. probe_ixformer_symbols.py: definitive test for real machine
   - Finds all ixformer .so files
   - nm/objdump for topk_softmax C++ symbol
   - Checks Python bindings
   - Attempts JIT compile + link (the real test)
   - Prints PASS/FAIL with next-step instructions

Run on real machine: python3 probe_ixformer_symbols.py

2026-08-10 03:04:50 +00:00

EX Engine

d21b2505bb

fix: wire MoE topk via ixformer C++ bridge + disable broken flash_qla GDN

Two call chain breaks fixed:

1. MoE routing (2304 calls/token):
   BEFORE: torch.softmax + torch.topk (3 Python GPU ops, no ixformer)
   AFTER:  ix_bridge.py → ix_moe_bridge.cpp → ixformer::infer::topk_softmax()
   Source: upstream_ref/xllm/xllm/core/kernels/ilu/fused_moe.cpp line 46
   The C++ API exists in base image SDK (ixformer.h declares it),
   only the Python binding (ixformer.functions) was missing.

2. GDN prefill (4 layers, 99.98% NaN):
   BEFORE: flash_qla SM70 kernel → abs mean=inf → nan_to_num → zeros
   AFTER:  skip flash_qla, use _pytorch_forward directly
   Source: upstream_ref/xllm qwen3_gated_delta_net_base.cpp uses
   identical PyTorch chunked logic (no flash_qla).
   Sub168 (working build) never deployed flash_qla either.

Files:
- ex_engine/csrc/ix_moe_bridge.cpp: torch C++ extension calling ixformer C++ API
- ex_engine/python/ix_bridge.py: JIT-compile loader with PyTorch fallback
- qwen3_5.py: import ix_bridge for MoE, disable flash_qla for GDN
- patch_ops.sh: deploy ix_bridge .cpp + .py into vllm model dir

2026-08-10 03:00:35 +00:00

2 Commits