project6-dev
d1c5e992aa
feat(SO): ix_moe_bridge.cpp — dlopen bridge for 12 ixformer::infer functions
...
THE CORE .so: ix_moe_bridge.cpp compiles to ix_moe_bridge.so which:
- Links against base image's libixformer.so at load time
- Exposes 12 functions to Python via pybind11:
MoE pipeline (7 steps):
topk_softmax() → ixformer::infer::topk_softmax
moe_gen_idx() → ixformer::infer::moe_compute_token_index_api
moe_expand_input() → ixformer::infer::moe_expand_input
moe_group_gemm() → ixformer::infer::moe_w16a16_group_gemm
silu_and_mul() → ixformer::infer::silu_and_mul
moe_combine_result()→ ixformer::infer::moe_output_reduce_sum
Inference ops (5 functions):
paged_attention() → ixformer::infer::xllm_paged_attention
rms_norm() → ixformer::infer::rms_norm
linear() → ixformer::infer::ixformer_linear
reshape_and_cache() → ixformer::infer::xllm_reshape_and_cache
rotary_embedding() → ixformer::infer::xllm_rotary_embedding
Build chain:
Dockerfile → build.sh → precompile_ix_bridge.py
→ torch.utils.cpp_extension.load(ix_moe_bridge.cpp, -lixformer)
→ ix_moe_bridge.cpython-310.so
Load chain:
Python: from ex_engine.python.ix_bridge import topk_softmax
→ ix_bridge.py loads ix_moe_bridge.so
→ dlopen links to libixformer.so
→ CUDA kernel on BI-V100
Interface source: upstream_ref/xllm_latest/core/kernels/ilu/ixformer.h
2026-08-11 02:37:03 +00:00
project6-dev
a54dbda3bb
fix(bridge): link against libixformer.so for silu_and_mul symbol
...
- ix_bridge.py: auto-discover ixformer .so files, pass as extra_ldflags
- ix_moe_bridge.cpp: fix mangled header from bad sed, add #include <optional>
- verify_single_gpu.py: also pass extra_ldflags during JIT compile
The undefined symbol _ZN8ixformer5infer12silu_and_mulERN2at6TensorES3_
lives in libixformer.so — need to explicitly link it.
2026-08-10 06:29:11 +00:00
project6-dev
ac3c8e28eb
fix(bridge): c10::nullopt → typed std::optional{} for CoreX torch compat
...
CoreX torch's c10::nullopt cannot implicitly convert to const std::optional<T>&.
Solution: use static typed empty optionals (kNoneTensor, kNoneBool).
Also unified all c10::optional forward decls to std::optional.
Applied same fix to ix_moe_bridge.cpp.
2026-08-10 06:25:15 +00:00
EX Engine
d00daa62f6
feat(MoE): full ixformer pipeline — topk → gen_idx → expand → group_gemm → silu → combine
...
Port complete MoE pipeline from upstream xllm/layers/ilu/fused_moe.cpp.
All 6 ixformer::infer functions now exposed via ix_moe_bridge.cpp:
1. topk_softmax — fused routing (was: 3 PyTorch ops)
2. moe_compute_token_index_api — build permutation maps
3. moe_expand_input — gather tokens by expert
4. moe_w16a16_group_gemm — batched expert GEMM (was: Python for-loop)
5. silu_and_mul — fused activation
6. moe_output_reduce_sum — weighted scatter-add
qwen3_5.py dispatch order:
1. Try ix_fused_moe_forward (full C++ pipeline, 7 kernel launches)
2. Try ix_topk_softmax only + PyTorch GEMM
3. Pure PyTorch fallback (torch.softmax + torch.topk + for-loop)
ix_bridge.py exposes both individual ops and fused_moe_forward().
No upstream code copied — only forward-declarations of ixformer C++ API
that the base image SDK already contains.
2026-08-10 03:06:14 +00:00
EX Engine
d21b2505bb
fix: wire MoE topk via ixformer C++ bridge + disable broken flash_qla GDN
...
Two call chain breaks fixed:
1. MoE routing (2304 calls/token):
BEFORE: torch.softmax + torch.topk (3 Python GPU ops, no ixformer)
AFTER: ix_bridge.py → ix_moe_bridge.cpp → ixformer::infer::topk_softmax()
Source: upstream_ref/xllm/xllm/core/kernels/ilu/fused_moe.cpp line 46
The C++ API exists in base image SDK (ixformer.h declares it),
only the Python binding (ixformer.functions) was missing.
2. GDN prefill (4 layers, 99.98% NaN):
BEFORE: flash_qla SM70 kernel → abs mean=inf → nan_to_num → zeros
AFTER: skip flash_qla, use _pytorch_forward directly
Source: upstream_ref/xllm qwen3_gated_delta_net_base.cpp uses
identical PyTorch chunked logic (no flash_qla).
Sub168 (working build) never deployed flash_qla either.
Files:
- ex_engine/csrc/ix_moe_bridge.cpp: torch C++ extension calling ixformer C++ API
- ex_engine/python/ix_bridge.py: JIT-compile loader with PyTorch fallback
- qwen3_5.py: import ix_bridge for MoE, disable flash_qla for GDN
- patch_ops.sh: deploy ix_bridge .cpp + .py into vllm model dir
2026-08-10 03:00:35 +00:00
