project_6

dylanyunlong/project_6

Fork 0

Commit Graph

Author	SHA1	Message	Date
project6-dev	0478628f17	fix(PROVEN): _moe_C compiles and runs on real BI-V100 hardware Tested on real machine (cc-b2042074, BI-V100, IX-ML 3.2.3): _moe_C.topk_softmax() → SUCCESS, correct output Two fixes proven on hardware: 1. cuda_compat.h: WARP_SIZE=64 (BI-V100 warp is 64, not 32) 2. topk_softmax_kernels.cu: cub/block/block_reduce.cuh instead of cub/cub.cuh (cub.cuh pulls radix_sort which has WARP_SIZE conflict) Key finding: ixformer SDK on this base image does NOT have topk_softmax. The ixformer::infer namespace from xllm's ixformer.h is for newer SDK. We MUST compile our own _moe_C kernel — which now works. Build flags (clang 16, ivcore10): CUDA: -O3 -cl-fast-relaxed-math (NOT --use_fast_math) C++: -O2 -std=c++17 Dockerfile simplified: 3 steps (was 6) _custom_ops.py: _moe_C as Priority 0, in-place vllm API	2026-08-11 01:50:48 +00:00
Claude	70c898ac8b	fix: ex_engine.python subpackage + flash_qla_sm70 deploy + vllm v0.5.5 MoE kernels 真机验证发现的问题: 1. qwen3_5.py 做 'from ex_engine.python.ix_bridge' 但包结构是 ex_engine.ix_bridge → 创建 python/ 子目录 + symlinks 2. flash_qla_sm70 只部署到 /workspace 没有到 vllm models 目录 → 显式 cp -r 到 VLLM/model_executor/models/ 3. 从 vllm v0.5.5 搬 MoE CUDA kernels (torch::Tensor API): - topk_softmax_kernels.cu (506行, CUB BlockReduce) - moe_align_block_size_kernels.cu (134行) - moe_pybind.cpp (pybind11 入口) 真机验证结果: ✓ ix_bridge import OK, available=True ✓ topk_softmax (64 experts, top8) OK — CUDA kernel 命中 ✓ ix_full_bridge silu_and_mul OK ✓ qwen3_5.py import OK ✓ ex_engine build 2/2 factors ✓ moe_topk_softmax_v3.so 编译成功 ✓ flash_qla_sm70_gdn_strided.so 编译成功 ✗ 单卡 32GB OOM (正常, 竞赛 4卡 tp=4)	2026-08-10 10:21:48 +00:00

Author

SHA1

Message

Date

project6-dev

0478628f17

fix(PROVEN): _moe_C compiles and runs on real BI-V100 hardware

Tested on real machine (cc-b2042074, BI-V100, IX-ML 3.2.3):
  _moe_C.topk_softmax() → SUCCESS, correct output

Two fixes proven on hardware:
1. cuda_compat.h: WARP_SIZE=64 (BI-V100 warp is 64, not 32)
2. topk_softmax_kernels.cu: cub/block/block_reduce.cuh instead of cub/cub.cuh
   (cub.cuh pulls radix_sort which has WARP_SIZE conflict)

Key finding: ixformer SDK on this base image does NOT have topk_softmax.
The ixformer::infer namespace from xllm's ixformer.h is for newer SDK.
We MUST compile our own _moe_C kernel — which now works.

Build flags (clang 16, ivcore10):
  CUDA: -O3 -cl-fast-relaxed-math (NOT --use_fast_math)
  C++:  -O2 -std=c++17

Dockerfile simplified: 3 steps (was 6)
_custom_ops.py: _moe_C as Priority 0, in-place vllm API

2026-08-11 01:50:48 +00:00

Claude

70c898ac8b

fix: ex_engine.python subpackage + flash_qla_sm70 deploy + vllm v0.5.5 MoE kernels

真机验证发现的问题:
1. qwen3_5.py 做 'from ex_engine.python.ix_bridge' 但包结构是 ex_engine.ix_bridge
   → 创建 python/ 子目录 + symlinks
2. flash_qla_sm70 只部署到 /workspace 没有到 vllm models 目录
   → 显式 cp -r 到 VLLM/model_executor/models/
3. 从 vllm v0.5.5 搬 MoE CUDA kernels (torch::Tensor API):
   - topk_softmax_kernels.cu (506行, CUB BlockReduce)
   - moe_align_block_size_kernels.cu (134行)
   - moe_pybind.cpp (pybind11 入口)

真机验证结果:
  ✓ ix_bridge import OK, available=True
  ✓ topk_softmax (64 experts, top8) OK — CUDA kernel 命中
  ✓ ix_full_bridge silu_and_mul OK
  ✓ qwen3_5.py import OK
  ✓ ex_engine build 2/2 factors
  ✓ moe_topk_softmax_v3.so 编译成功
  ✓ flash_qla_sm70_gdn_strided.so 编译成功
  ✗ 单卡 32GB OOM (正常, 竞赛 4卡 tp=4)

2026-08-10 10:21:48 +00:00

2 Commits