dylan 6f1904aa8c perf: MoE decode — pre-transposed bmm replaces F.linear (6.9ms vs 8.0ms, 14%) ...

Probe data (probe_moe_fused_breakdown.sh on BI-V100):
  F.linear loop 8 experts:     8.060 ms
  bmm pre-transposed full MoE: 6.918 ms  ← 14% faster
  transpose+contiguous runtime: 22.219 ms ← why CUTLASS was 27ms

Changes:
  - Lazy-cache w13_t (E,H,2I) and w2_t (E,I,H) on first decode call
  - FC1: torch.bmm(x_expand, w13_t_sel) replaces F.linear(x, w13_sel.reshape)
  - FC2: torch.bmm(act, w2_t_sel) replaces torch.bmm(w2_sel, act^T)
  - Zero runtime transpose cost after first call

2026-08-15 12:52:55 +00:00

..

cccl_preload

fix: restore libcccl_allocator.so + minimal .dockerignore + clean pycache

2026-08-14 03:51:43 +00:00

prebuilt/corex-3.2.3-ivcore10

prebuilt: 5 xllm CUDA kernel .so for BI-V100 (ivcore10)

2026-08-14 11:34:23 +00:00

qwen3_5

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

qwen3_5_moe

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

reasoning

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

vendor_overrides/vllm

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

wheels

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

api_server.py

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

bi100_env.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

bi100_profile.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

block_major_kv_cache.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

build_cccl_moe_sort_scatter.sh

fix: import torch.utils.cpp_extension explicitly in build script

2026-08-13 11:37:10 +00:00

build_corex_attn_head_rms_norm.sh

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

build_corex_batched_gemm.sh

fix(build): cutlass.h lives under tensorflow/include on this image

2026-08-15 12:03:12 +00:00

build_corex_block_major_kv_transfer.sh

fix(P0): extra=allow + topk_softmax fallback + deploy_local.sh + SO chain verify

2026-08-14 06:56:00 +00:00

build_corex_fused_paged_prefill_split4.sh

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

build_corex_gdn_beta_decay.sh

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

build_corex_gdn_causal_conv.sh

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

build_corex_gdn_chunk_recurrent.sh

feat: C++ GDN chunk+recurrent from xllm upstream + verification script

2026-08-13 04:01:59 +00:00

build_corex_gdn_gated_norm.sh

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

build_corex_gdn_packed_decode.sh

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

build_corex_gdn_qk_map.sh

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

build_corex_moe_direct_routed.sh

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

build_corex_moe_exact_reduce.sh

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

build_corex_moe_index_combine.sh

feat: moe_compute_index + moe_combine_result CUDA kernels from xllm upstream

2026-08-13 03:48:12 +00:00

build_corex_moe_topk_softmax.sh

fix: topk_softmax .so + fp32 router + enforce_eager + comp168 params

2026-08-12 11:14:01 +00:00

build_corex_moe_weight_gather.sh

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

build_corex_paged_kv_gather.sh

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

build_ix_attn_bridge.sh

feat: ix_moe_bridge + ix_attn_bridge — dlopen bridges for full ixformer::infer API

2026-08-14 07:32:31 +00:00

build_ix_bridge.sh

fix(bridge): rewrite ix_full_bridge.cpp for real ixformer_torch_ext namespace

2026-08-14 07:00:30 +00:00

build_ix_moe_bridge.sh

feat: ix_moe_bridge + ix_attn_bridge — dlopen bridges for full ixformer::infer API

2026-08-14 07:32:31 +00:00

build_xllm_kernels.sh

Revert "feat: batched MoE expert GEMM — replaces Python for-loop"

2026-08-14 11:47:37 +00:00

cat_cutlass_cu10_part2.sh

test: cat tensorop GEMM example + arch.h + cutlass.h from corex-samples

2026-08-15 05:41:03 +00:00

cat_cutlass_cu10.sh

fix: cat_cutlass_cu10.sh writes to cat_files/ directory instead of stdout

2026-08-15 05:31:33 +00:00

cccl_moe_sort_scatter_pybind.cpp

fix(CCCL): split compilation to isolate CCCL headers from torch/corex

2026-08-13 11:35:43 +00:00

cccl_moe_sort_scatter.cu

fix(CCCL): split compilation to isolate CCCL headers from torch/corex

2026-08-13 11:35:43 +00:00

chat_utils.py

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

cli_args.py

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

corex_attn_head_rms_norm.cu

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

corex_block_major_kv_transfer.cu

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

corex_fused_paged_prefill_split4.cu

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

corex_gdn_beta_decay.cu

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

corex_gdn_causal_conv.cu

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

corex_gdn_chunk_recurrent.cu

fix(pybind): c10::optional<Tensor>() default — c10::nullopt not registered in pybind

2026-08-14 02:05:30 +00:00

corex_gdn_gated_norm.cu

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

corex_gdn_packed_decode.cu

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

corex_gdn_qk_map.cu

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

corex_moe_direct_routed.cu

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

corex_moe_exact_reduce.cu

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

corex_moe_index_combine.cu

feat: moe_compute_index + moe_combine_result CUDA kernels from xllm upstream

2026-08-13 03:48:12 +00:00

corex_moe_topk_softmax.cu

fix: topk_softmax .so + fp32 router + enforce_eager + comp168 params

2026-08-12 11:14:01 +00:00

corex_moe_weight_gather.cu

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

corex_paged_kv_gather.cu

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

corex_query_tiled_paged_prefill.cu

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

deploy_local.sh

fix(P0): extra=allow + topk_softmax fallback + deploy_local.sh + SO chain verify

2026-08-14 06:56:00 +00:00

device_utils.cuh

fix: topk_softmax .so + fp32 router + enforce_eager + comp168 params

2026-08-12 11:14:01 +00:00

gdn_prefix.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

install_prebuilt_corex.sh

build: all 16 .so prebuilt, no JIT compile in docker

2026-08-14 07:14:46 +00:00

ix_attn_bridge.cpp

feat: ix_moe_bridge + ix_attn_bridge — dlopen bridges for full ixformer::infer API

2026-08-14 07:32:31 +00:00

ix_full_bridge.cpp

fix(bridge): rewrite ix_full_bridge.cpp for real ixformer_torch_ext namespace

2026-08-14 07:00:30 +00:00

ix_fused_moe.py

feat: ix_moe_bridge + ix_attn_bridge — dlopen bridges for full ixformer::infer API

2026-08-14 07:32:31 +00:00

ix_moe_bridge.cpp

feat: ix_moe_bridge + ix_attn_bridge — dlopen bridges for full ixformer::infer API

2026-08-14 07:32:31 +00:00

mamba_cache.py

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

moe_topk_sigmoid_kernels.cuh

fix: topk_softmax .so + fp32 router + enforce_eager + comp168 params

2026-08-12 11:14:01 +00:00

moe_topk_softmax_kernels.cuh

fix: topk_softmax .so + fp32 router + enforce_eager + comp168 params

2026-08-12 11:14:01 +00:00

paged_attn.py

perf: native ixformer decode — v1 ≤32K, v2 >32K (no Python fallback)

2026-08-13 07:09:50 +00:00

patch_block_major_cache_engine.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

patch_block_major_worker_capacity.py

revert: restore a3c45d3b yaml + Q-tiling + remove all OOM hacks

2026-08-14 02:28:48 +00:00

patch_block_manager_cache_trace.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

patch_corex_swap_blocks.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

patch_executor_startup_debug.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

patch_model_runner.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

patch_ops.sh

feat: ix_ops integration layer — wire ix_full_bridge.so into vllm hot path

2026-08-15 06:15:17 +00:00

patch_transformers_qwen3_5.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

patch_utils.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

patch_vllm_qwen3_5.py

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

patch_vllm_tool_parser.py

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

patch_worker_cache_transfer_order.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

patch_worker_profile_override.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

patch_worker_startup_profile_guard.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

patch_xformers_profile.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

patch_xformers_sdpa_batch_kernel.py

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

patch_xformers_sdpa_batch.py

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

patch_xformers_sdpa_seq_kernel.py

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

patch_xformers_sdpa_seq.py

fix: max-num-seqs=1 + pure Q-tiling — match a3c45d3b exactly

2026-08-14 05:40:05 +00:00

probe_corex_cub.sh

test: fix CUB compat tests + probe corex built-in CUB

2026-08-14 10:11:10 +00:00

probe_cutlass_arch.sh

test: probe CUTLASS + __CUDA_ARCH__ on BI-V100 for grouped GEMM feasibility

2026-08-14 11:59:51 +00:00

probe_cutlass_cu10.sh

test: probe Cu10 CUTLASS fork — find mma_cu10.h, tensor op files, batched_gemm example

2026-08-15 05:27:43 +00:00

probe_gemm_api.sh

test: probe cuinfer + cublasLt + cublas batched GEMM APIs for MoE expert dispatch

2026-08-14 12:03:52 +00:00

probe_ixformer_symbols.sh

test: probe ixformer .so symbols for MoE grouped GEMM

2026-08-14 11:40:13 +00:00

probe_ixinfer.sh

test: cat ixinfer.h + functional batched GEMM test on BI-V100

2026-08-14 12:07:00 +00:00

probe_k10_configs.sh

test: sweep 6 kernel 10 configs + cublas baseline — find best params for warp64

2026-08-15 05:08:11 +00:00

probe_k10_perf.sh

test: probe kernel 10 perf with CUDA events — isolate bottleneck

2026-08-14 17:23:41 +00:00

probe_moe_fused_breakdown.sh

test: MoE breakdown — F.linear vs torch.mm vs torch.bmm vs bmm pre-transposed

2026-08-15 12:46:37 +00:00

probe_torch_matmul_backend.sh

test: probe torch.matmul backend + ixformer.matmul/linear + Python loop overhead

2026-08-15 05:17:13 +00:00

probe_warp64.sh

test: probe warp_size=64 behavior + kernel 10 warp tiling with WARPSIZE=64 on BI-V100

2026-08-14 17:02:55 +00:00

protocol.py

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

qwen3_5.py

perf: MoE decode — pre-transposed bmm replaces F.linear (6.9ms vs 8.0ms, 14%)

2026-08-15 12:52:55 +00:00

qwen3coder_tool_parser.py

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

scheduler.py

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

sequence.py

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

serving_chat.py

fix(build): strip \r\n from all .py files — CRLF breaks patch_ops.sh text matching

2026-08-14 01:06:49 +00:00

serving_tokenization.py

feat: 替换为 project_7 验证通过的 wudixzy stack

2026-08-12 03:31:05 +00:00

test_cub_compat_v2.py

test: fix CUB compat tests + probe corex built-in CUB

2026-08-14 10:11:10 +00:00

test_cub_compat.py

test: CUB compatibility on BI-V100 — warp shuffle + BlockReduce

2026-08-14 08:21:48 +00:00

test_xllm_cuda_kernels.py

test: xllm CUDA kernel verification suite for BI-V100

2026-08-14 08:01:23 +00:00

verify_ast_chain.py

feat: xllm MoE CUDA kernels — fused_topk + compute_index + combine

2026-08-14 11:23:49 +00:00

verify_dlopen_chain.py

fix(P0): extra=allow + topk_softmax fallback + deploy_local.sh + SO chain verify

2026-08-14 06:56:00 +00:00