EX Engine
f955dd127e
feat(EX): ix_full_bridge — all 14 ixformer::infer functions bridged
...
Upstream source: xllm/core/kernels/ilu/ixformer.h (Apache 2.0)
Wrapper patterns: xllm/core/kernels/ilu/{attention,norm,rope,activation,fused_moe,group_gemm}.cpp
Complete bridge (ix_full_bridge.cpp, 331 lines):
MoE: topk_softmax, gen_idx, expand, group_gemm, silu_mul, combine, fused_forward
Attention: paged_attention (decode), flash_attn_prefill (prefill)
Norm: rms_norm, fused_add_rms_norm
RoPE: rotary_embedding
Cache: reshape_and_cache
Linear: ixformer_linear
ix_bridge.py: tries ix_full_bridge first, falls back to ix_moe_bridge
patch_ops.sh: deploys both .cpp files to all JIT search paths
Copied ixformer.h + utils.h headers for reference
2026-08-10 04:01:42 +00:00
EX Engine
e04a3bace9
fix: fail-fast on ix_bridge failure + probe script for real machine
...
1. ix_bridge.py: RuntimeError instead of silent PyTorch fallback
If JIT compile fails, crash immediately with diagnostic message.
0 score with no error log is worse than a visible crash.
2. qwen3_5.py: explicit WARNING log on import failure (not silent)
Shows exact error so we can diagnose from docker log.
3. probe_ixformer_symbols.py: definitive test for real machine
- Finds all ixformer .so files
- nm/objdump for topk_softmax C++ symbol
- Checks Python bindings
- Attempts JIT compile + link (the real test)
- Prints PASS/FAIL with next-step instructions
Run on real machine: python3 probe_ixformer_symbols.py
2026-08-10 03:04:50 +00:00
EX Engine
d21b2505bb
fix: wire MoE topk via ixformer C++ bridge + disable broken flash_qla GDN
...
Two call chain breaks fixed:
1. MoE routing (2304 calls/token):
BEFORE: torch.softmax + torch.topk (3 Python GPU ops, no ixformer)
AFTER: ix_bridge.py → ix_moe_bridge.cpp → ixformer::infer::topk_softmax()
Source: upstream_ref/xllm/xllm/core/kernels/ilu/fused_moe.cpp line 46
The C++ API exists in base image SDK (ixformer.h declares it),
only the Python binding (ixformer.functions) was missing.
2. GDN prefill (4 layers, 99.98% NaN):
BEFORE: flash_qla SM70 kernel → abs mean=inf → nan_to_num → zeros
AFTER: skip flash_qla, use _pytorch_forward directly
Source: upstream_ref/xllm qwen3_gated_delta_net_base.cpp uses
identical PyTorch chunked logic (no flash_qla).
Sub168 (working build) never deployed flash_qla either.
Files:
- ex_engine/csrc/ix_moe_bridge.cpp: torch C++ extension calling ixformer C++ API
- ex_engine/python/ix_bridge.py: JIT-compile loader with PyTorch fallback
- qwen3_5.py: import ix_bridge for MoE, disable flash_qla for GDN
- patch_ops.sh: deploy ix_bridge .cpp + .py into vllm model dir
2026-08-10 03:00:35 +00:00
