ixformer probe results:
✗ moe_w16a16_group_gemm NOT in ixformer .so
✗ CUTLASS grouped GEMM needs cuda/std (variadic function error on corex)
✓ ixformer_linear EXISTS (fused matmul)
✓ torch.mm works (uses corex cublas)
Solution: moe_batched_gemm.cu
- C++ loop over experts (eliminates Python overhead)
- torch::mm for GEMM (corex cublas, not F.linear Python)
- Fused silu_and_mul CUDA kernel (not PyTorch ops)
- Weighted scatter-add in C++
- Skips empty experts (no wasted compute)
Integration in qwen3_5.py:
_USE_XLLM_MOE_GEMM dispatches to moe_experts_forward()
Falls back to Python for-loop if not available
Build: bash qwen3_6_scripts/build_xllm_kernels.sh
06d7713db6