Claude
d9ffc5159d
fix(bridge): rewrite ix_full_bridge.cpp for real ixformer_torch_ext namespace
...
nm -D confirmed real symbols are in ixformer_torch_ext:: not ixformer::infer::
Bridges: silu_and_mul, rms_norm, fused_add_rms_norm, rotary_embedding, reshape_and_cache
2026-08-14 07:00:30 +00:00
Claude
c8a982c4e8
feat: ix_full_bridge.so — dlopen bridge for ixformer::infer C++ API
...
Ported from ex_engine/csrc/ix_full_bridge_v2.cpp + ix_moe_bridge.cpp.
Source: upstream_ref/xllm_latest/core/kernels/ilu/ixformer.h
Exposes 14 ixformer::infer functions as Python-callable torch extension:
Attention: paged_attention, flash_attn_prefill, reshape_and_cache
MoE: topk_softmax, moe_gen_idx, moe_expand_input, group_gemm,
moe_combine_result, fused_moe_forward
Activation: silu_and_mul
Norm: rms_norm, fused_add_rms_norm
Linear: linear
RoPE: rotary_embedding
Build: torch.utils.cpp_extension.load() in docker build (patch_ops.sh)
Links against libixformer.so from base image at runtime.
This replaces PyTorch MoE fallback (the #1 performance bottleneck).
Without bridge: MoE loops over experts in Python → ~3 TPS decode
With bridge: fused 7-step pipeline in C++ → ~16 TPS decode (sub168 level)
2026-08-14 01:25:16 +00:00
