dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
367 Commits 1 Branch 0 Tags
3e7fc565ffedaf962c2ffd415014b59f70d279d5
Commit Graph

5 Commits

Author SHA1 Message Date
project6-dev
a54dbda3bb fix(bridge): link against libixformer.so for silu_and_mul symbol 
- ix_bridge.py: auto-discover ixformer .so files, pass as extra_ldflags
- ix_moe_bridge.cpp: fix mangled header from bad sed, add #include <optional>
- verify_single_gpu.py: also pass extra_ldflags during JIT compile

The undefined symbol _ZN8ixformer5infer12silu_and_mulERN2at6TensorES3_
lives in libixformer.so — need to explicitly link it.
 2026-08-10 06:29:11 +00:00
EX Engine
f955dd127e feat(EX): ix_full_bridge — all 14 ixformer::infer functions bridged 
Upstream source: xllm/core/kernels/ilu/ixformer.h (Apache 2.0)
Wrapper patterns: xllm/core/kernels/ilu/{attention,norm,rope,activation,fused_moe,group_gemm}.cpp

Complete bridge (ix_full_bridge.cpp, 331 lines):
  MoE:       topk_softmax, gen_idx, expand, group_gemm, silu_mul, combine, fused_forward
  Attention: paged_attention (decode), flash_attn_prefill (prefill)
  Norm:      rms_norm, fused_add_rms_norm
  RoPE:      rotary_embedding
  Cache:     reshape_and_cache
  Linear:    ixformer_linear

ix_bridge.py: tries ix_full_bridge first, falls back to ix_moe_bridge
patch_ops.sh: deploys both .cpp files to all JIT search paths
Copied ixformer.h + utils.h headers for reference
 2026-08-10 04:01:42 +00:00
EX Engine
388f6b2d1a feat(MoE): wire full ix_fused_moe_forward as Tier 0 dispatch 
ix_bridge.py: expose all 6 ixformer::infer functions + fused_moe_forward()
qwen3_5.py: 4-tier MoE dispatch (fused C++ → CUB topk → ix topk → PyTorch)
patch_ops.sh: deploy ix_moe_bridge.cpp to 4 search paths for JIT
 2026-08-10 03:38:46 +00:00
EX Engine
e04a3bace9 fix: fail-fast on ix_bridge failure + probe script for real machine 
1. ix_bridge.py: RuntimeError instead of silent PyTorch fallback
   If JIT compile fails, crash immediately with diagnostic message.
   0 score with no error log is worse than a visible crash.

2. qwen3_5.py: explicit WARNING log on import failure (not silent)
   Shows exact error so we can diagnose from docker log.

3. probe_ixformer_symbols.py: definitive test for real machine
   - Finds all ixformer .so files
   - nm/objdump for topk_softmax C++ symbol
   - Checks Python bindings
   - Attempts JIT compile + link (the real test)
   - Prints PASS/FAIL with next-step instructions

Run on real machine: python3 probe_ixformer_symbols.py
 2026-08-10 03:04:50 +00:00
EX Engine
d21b2505bb fix: wire MoE topk via ixformer C++ bridge + disable broken flash_qla GDN 
Two call chain breaks fixed:

1. MoE routing (2304 calls/token):
   BEFORE: torch.softmax + torch.topk (3 Python GPU ops, no ixformer)
   AFTER:  ix_bridge.py → ix_moe_bridge.cpp → ixformer::infer::topk_softmax()
   Source: upstream_ref/xllm/xllm/core/kernels/ilu/fused_moe.cpp line 46
   The C++ API exists in base image SDK (ixformer.h declares it),
   only the Python binding (ixformer.functions) was missing.

2. GDN prefill (4 layers, 99.98% NaN):
   BEFORE: flash_qla SM70 kernel → abs mean=inf → nan_to_num → zeros
   AFTER:  skip flash_qla, use _pytorch_forward directly
   Source: upstream_ref/xllm qwen3_gated_delta_net_base.cpp uses
   identical PyTorch chunked logic (no flash_qla).
   Sub168 (working build) never deployed flash_qla either.

Files:
- ex_engine/csrc/ix_moe_bridge.cpp: torch C++ extension calling ixformer C++ API
- ex_engine/python/ix_bridge.py: JIT-compile loader with PyTorch fallback
- qwen3_5.py: import ix_bridge for MoE, disable flash_qla for GDN
- patch_ops.sh: deploy ix_bridge .cpp + .py into vllm model dir
 2026-08-10 03:00:35 +00:00