project_6

Files

dylan e18ece8f3a feat: port NaiveBatchedExperts from ds_vllm — view transpose + cublas transB

Source: upstream_ref/ds_vllm/vllm/model_executor/layers/fused_moe/experts/fused_batched_moe.py
        upstream_ref/ds_vllm/vllm/model_executor/layers/fused_moe/activation.py

New files (ported from ds_vllm, adapted for BI-V100):
  ex_engine/moe/__init__.py
  ex_engine/moe/activation.py
    - MoEActivation enum + apply_moe_activation
    - torch.ops._C.silu_and_mul replaced with F.silu(gate)*up fallback
  ex_engine/moe/naive_batched_experts.py
    - naive_batched_moe_forward()
    - Decode: per-expert loop, w13[eid].transpose(0,1) is VIEW (zero copy)
    - @ operator → cublas passes transB=CUBLAS_OP_T internally
    - Prefill: group tokens by expert, batch @ per expert

Modified:
  qwen3_6_scripts/qwen3_5.py
    - Import naive_batched_moe_forward
    - Tier 0.5: after ix_fused_moe, before corex point-optimized loop
    - Uses existing topk routing (xllm/corex/pytorch)

Key difference from previous approach:
  - NO physical transpose (was 22ms overhead)
  - NO weight gather into contiguous buffer
  - View transpose is O(0), cublas handles transB

2026-08-15 13:05:45 +00:00

csrc

fix: use F.silu in test script for old corex torch

2026-08-15 05:22:31 +00:00

fla_kernels

feat: import CUDA kernels from xllm/CCCL/FLA upstream repos

2026-08-14 07:48:52 +00:00

include

fix(CRITICAL): engine death on image request + stop overwriting base corex modules

2026-08-10 09:12:05 +00:00

moe

feat: port NaiveBatchedExperts from ds_vllm — view transpose + cublas transB