dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
641 Commits 1 Branch 0 Tags
f006ab1a01d85ffdf7bacb042dd02bba33eb60e2
Commit Graph

641 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Claude
f006ab1a01 test: cat tensorop GEMM example + arch.h + cutlass.h from corex-samples 2026-08-15 05:41:03 +00:00
root
b922d694dc data: Cu10 CUTLASS headers from corex-samples 2026-08-15 05:32:10 +00:00
Claude
1d36754efc fix: cat_cutlass_cu10.sh writes to cat_files/ directory instead of stdout 2026-08-15 05:31:33 +00:00
Claude
4abb4df215 test: cat Cu10 CUTLASS files — mma_cu10.h, iluvatar_mma.hpp, batched_gemm.cu, default_mma_core_cu10.h 2026-08-15 05:30:15 +00:00
Claude
6b9086c3a9 test: probe Cu10 CUTLASS fork — find mma_cu10.h, tensor op files, batched_gemm example 2026-08-15 05:27:43 +00:00
Claude
a465dd1d75 fix: use F.silu in test script for old corex torch 2026-08-15 05:22:31 +00:00
Claude
a12d070d82 fix: replace torch::silu with x*sigmoid(x) for old corex torch 2026-08-15 05:22:25 +00:00
Claude
c840c9159f feat: moe_tcu_dispatch.cpp — C++ MoE expert loop via torch::mm (TCU kernel) 
torch profiler confirmed: torch.mm launches Gemm_tcu_bi_kernel::gemm_h_h_tcu_25
which is BI-V100 TCU (Tensor Compute Unit) hardware-accelerated GEMM.
0.58ms per call vs our custom kernel 7.7ms — TCU is 13x faster.

Python for-loop overhead measured: 0.892 ms/expert = 7.1 ms for 8 experts.
This C++ dispatch eliminates that overhead while using the same TCU kernel.

Three entry points:
- moe_decode: full MoE forward (FC1 + SiLU*mul + FC2) for decode
- moe_prefill: group-by-expert MoE forward for prefill
- moe_expert_gemm_tcu: raw GEMM loop for benchmarking
 2026-08-15 05:20:05 +00:00
Claude
9514092980 test: probe torch.matmul backend + ixformer.matmul/linear + Python loop overhead 2026-08-15 05:17:13 +00:00
Claude
395b3e4042 test: clean rebuild + debug output for kernel 10 correctness 2026-08-15 05:14:18 +00:00
Claude
a8ca42b59c perf: hgemm_warptiling Config B — beats cublas on MoE-sized GEMM (0.7x) 
probe_k10_configs.sh results on BI-V100:
  256x4096 @ 4096x11008:
    cublas:   10.554 ms
    Config B:  7.649 ms (0.7x cublas — FASTER)
    Config A:  2308 ms  (old broken config)

Config B: BM128 BN128 BK16 WM64 WN64 WNITER2 TM8 TN4 NT128
Root cause of Config A slowness: WN=128 WNITER=4 caused
excessive register pressure and smem bank conflicts.
 2026-08-15 05:11:32 +00:00
Claude
21417319bc test: sweep 6 kernel 10 configs + cublas baseline — find best params for warp64 2026-08-15 05:08:11 +00:00
Claude
27bb8d28df test: probe kernel 10 perf with CUDA events — isolate bottleneck 2026-08-14 17:23:41 +00:00
Claude
2b12fe687e feat: hgemm_warptiling.cu — siboehm kernel 10 ported to WARPSIZE=64 FP16 
1:1 from upstream_ref/sgemm_cuda/10_kernel_warptiling.cuh.
3 changes: WARPSIZE 32→64, float→__half, FP32 accumulator.

Launch config (confirmed by probe_warp64.sh):
  NUM_THREADS=128, 2 warps of 64
  BM=128 BN=128 BK=16 WM=64 WN=128 WNITER=4 TM=4 TN=4
  WMITER=2, WSUBM=32, WSUBN=32, threads_per_warp=64 ✓
 2026-08-14 17:05:59 +00:00
Claude
11b8a98eea test: probe warp_size=64 behavior + kernel 10 warp tiling with WARPSIZE=64 on BI-V100 2026-08-14 17:02:55 +00:00
Claude
1af7e7cf48 fix: use c10::cuda::getCurrentCUDAStream().stream() for corex torch 2026-08-14 16:49:47 +00:00
Claude
3bee73207e fix: add cuda_runtime.h to hgemm_bind.cpp for cudaStream_t 2026-08-14 16:33:29 +00:00
Claude
09e5261ba6 refactor: hgemm_blocktiling.cu — strict 1:1 from siboehm kernel 6 
Only 3 changes from upstream_ref/sgemm_cuda/6_kernel_vectorize.cuh:
1. float → __half for A/B/C data and shared memory
2. float4 vectorized load → 4 scalar half loads (float4 needs 16-byte align)
3. threadResults accumulator stays float (FP32 accumulation)

Everything else identical: same shared mem layout, same indexing,
same A-transpose-while-loading, same thread tile computation.
No WARPSIZE. No cooperative_groups. No cuda::barrier.
 2026-08-14 16:24:22 +00:00
Claude
ab42fc1fd7 feat: hgemm_blocktiling.cu — FP16 GEMM kernel for MoE expert dispatch on BI-V100 
Adapted from siboehm/SGEMM_CUDA kernel 6 (vectorize + A transpose)
and wangzyon/NVIDIA_SGEMM_PRACTICE kernel 6 (mysgemm_v6).

Key design decisions:
- FP16 data with FP32 accumulation (avoid precision loss)
- No WARPSIZE dependency (safe for BI-V100 warp_size=64)
- Boundary checks for non-aligned M/N/K (MoE expert token counts vary)
- BM=128 BN=128 BK=8 TM=8 TN=8 (256 threads, fits BI-V100 128KB smem)
- A transpose in shared memory for coalesced reads

Two entry points:
1. hgemm(A, B) — standalone FP16 GEMM
2. moe_expert_gemm(input, weights, expert_counts) — MoE prefill path
   loops over experts with variable token counts

For decode (M=1), use cublasHgemmStridedBatched (confirmed working).

Upstream refs: upstream_ref/sgemm_cuda/6_kernel_vectorize.cuh
              upstream_ref/nvidia_sgemm_practice/kernel_6.cuh
 2026-08-14 16:22:00 +00:00
Claude
9ca33cf4d5 upstream: add GEMM kernel references from 4 repos for BI-V100 porting 
Sources (all CUDA 10.2 compatible, no CUTLASS/Triton dependency):
- leimao/CUDA-GEMM-Optimization: v00-v07, fp16 WMMA variant, double buffered
- siboehm/SGEMM_CUDA: kernel 1-12, warp tiling + double buffering
- wangzyon/NVIDIA_SGEMM_PRACTICE: kernel 1-7
- edtallison/sgemm-cuda: kernel 1-12 (reimplementation with notes)

Key porting issue: ALL kernels hardcode WARPSIZE=32.
BI-V100 has warp_size=64. Need to:
1. Replace all 32U / WARPSIZE constants with 64
2. Adjust warp subtile decomposition (WMITER, WNITER, WSUBM, WSUBN)
3. Adjust shared memory bank conflict avoidance (may have different bank count)
4. Test __shfl_down_sync with mask=0xFFFFFFFFFFFFFFFF (64-bit)
 2026-08-14 15:11:57 +00:00
Claude
29ecc2e602 feat: moe_expert_gemm.cpp — C++ loop over experts via ixformer_linear (replaces Python for-loop) 
Key difference from the reverted batched approach:
- Does NOT use torch::mm in a C++ loop (that was the reverted commit)
- Uses ixformer_torch_ext::ixformer_linear — the base image's optimized GEMM
- Same kernel the competitor (sub 168) uses via corex_moe.py
- Eliminates Python interpreter + dispatcher overhead per expert
- Links against _ixformer_torch.cpython-310.so (already in base image)

Decode: 1 Python call → 8 C++ ixformer_linear (vs 8 Python F.linear)
Prefill: 1 Python call → 64 C++ ixformer_linear (vs 64 Python F.linear)
 2026-08-14 12:09:50 +00:00
Claude
0ace44e293 test: cat ixinfer.h + functional batched GEMM test on BI-V100 2026-08-14 12:07:00 +00:00
Claude
bfc4de2cf3 test: probe cuinfer + cublasLt + cublas batched GEMM APIs for MoE expert dispatch 2026-08-14 12:03:52 +00:00
Claude
d6958070cb test: probe CUTLASS + __CUDA_ARCH__ on BI-V100 for grouped GEMM feasibility 2026-08-14 11:59:51 +00:00
claude
50a249e0a3 Revert "feat: batched MoE expert GEMM — replaces Python for-loop" 
This reverts commit 06d7713db6.
 2026-08-14 11:47:37 +00:00
claude
06d7713db6 feat: batched MoE expert GEMM — replaces Python for-loop 
ixformer probe results:
  ✗ moe_w16a16_group_gemm NOT in ixformer .so
  ✗ CUTLASS grouped GEMM needs cuda/std (variadic function error on corex)
  ✓ ixformer_linear EXISTS (fused matmul)
  ✓ torch.mm works (uses corex cublas)

Solution: moe_batched_gemm.cu
  - C++ loop over experts (eliminates Python overhead)
  - torch::mm for GEMM (corex cublas, not F.linear Python)
  - Fused silu_and_mul CUDA kernel (not PyTorch ops)
  - Weighted scatter-add in C++
  - Skips empty experts (no wasted compute)

Integration in qwen3_5.py:
  _USE_XLLM_MOE_GEMM dispatches to moe_experts_forward()
  Falls back to Python for-loop if not available

Build: bash qwen3_6_scripts/build_xllm_kernels.sh
 2026-08-14 11:43:46 +00:00
claude
93353a1414 test: probe ixformer .so symbols for MoE grouped GEMM 
CUTLASS grouped GEMM (example 24) requires SM80 Tensor Core + cuda/std headers.
Cannot compile on corex (same issue as CCCL 3.6 variadic functions).

Alternative path: ix_moe_bridge.so calls ixformer::infer::moe_w16a16_group_gemm
which is BI-V100 optimized grouped GEMM already in the base image.

This probe script checks if the MoE functions exist in ixformer .so
before attempting to build ix_moe_bridge.so.

Run: bash qwen3_6_scripts/probe_ixformer_symbols.sh
 2026-08-14 11:40:13 +00:00
claude
865c18f852 feat: integrate xllm_moe into qwen3_5.py MoE hot path 
xllm_moe.so provides 3 fused CUDA kernels compiled for ivcore10:
  - moe_fused_topk: CUB topk + softmax (replaces corex_moe_topk_softmax)
  - moe_compute_index: histogram + prefix_sum + place (replaces corex_moe_index_combine)
  - moe_combine_result: reorder + weighted sum (available but not yet wired to output)

Dispatch priority in _pure_pytorch_experts():
  Tier 0: xllm_moe (if available)
  Tier 1: corex_moe_* individual .so
  Tier 2: PyTorch fallback

Integration points:
  1. Topk routing: xllm_moe.moe_fused_topk → corex_moe_topk_softmax → torch.topk
  2. Index computation: xllm_moe.moe_compute_index → corex_moe_index_combine → torch.argsort
  3. Expert loop: still Python F.linear (next target: batch GEMM)

patch_ops.sh already deploys all prebuilt/*.so including xllm_moe.so
 2026-08-14 11:37:21 +00:00
root
e147c283e3 prebuilt: 5 xllm CUDA kernel .so for BI-V100 (ivcore10) 
All 12/12 AST call chain tests passed:
  xllm_norm.so       (1.4MB) rms_norm max_err=0.000000, fused_add_rms_norm err=0.001953
  xllm_activation.so (1.3MB) silu_and_mul err=0.001953, gelu_and_mul err=0.001953
  xllm_rope.so       (1.2MB) rotary_embedding q_diff=1360 k_diff=1404
  xllm_cache.so      (1.3MB) reshape_paged_cache err=0.000000
  xllm_moe.so        (?.?MB) fused_topk weight_sum_err<0.01, compute_index total=64, combine err<0.1

vs ixformer: rms_norm err=0.000000, silu_and_mul err=0.000000

Compiled with corex clang/16 --cuda-gpu-arch=ivcore10
Using corex CUB (/usr/local/corex/include/cub/), NOT CCCL 3.6
 2026-08-14 11:34:23 +00:00
claude
31d3ee99bb fix: MoE kernel include paths — device_utils.cuh + arch_condition.h 
Fixed xllm internal paths to our headers/ directory:
  kernels/cuda/device_utils.cuh → device_utils.cuh
  core/kernels/cuda/device_utils.cuh → device_utils.cuh
  core/kernels/cuda/arch_condition.h → arch_condition.h (copied)
 2026-08-14 11:31:54 +00:00
claude
df6a0f5d47 fix: remove cuda/functional from MoE topk kernels (not available on corex) 2026-08-14 11:29:19 +00:00
claude
a50adefdfc feat: xllm MoE CUDA kernels — fused_topk + compute_index + combine 
3 MoE kernel files adapted for corex:
  moe_fused_topk.cu: LOG(FATAL)→TORCH_CHECK, +torch/extension.h
  moe_compute_index.cu: CHECK_LE→TORCH_CHECK, uses cub::BlockScan (corex CUB)
  moe_combine.cu: fixed duplicate include, +torch/extension.h

New pybind binding: xllm_moe_bind.cpp
  → moe_fused_topk(gating, topk, renormalize, bias, scoring_func)
  → moe_compute_index(expert_id, num_experts)
  → moe_combine_result(gemm2, weights, N, topk)

AST verification added for all 3 functions
 2026-08-14 11:23:49 +00:00
claude
49cd7def89 fix: slot_ids dtype long→int32 in cache test 2026-08-14 11:20:07 +00:00
claude
1f51feee05 fix: cos_sin_cache dtype float→half in rope test 2026-08-14 11:18:02 +00:00
claude
7fc0c1defa test: AST call chain verification for all 4 xllm kernel .so 
verify_ast_chain.py tests every exported function:
  xllm_norm.so:       rms_norm (vs PyTorch), fused_add_rms_norm (vs PyTorch)
  xllm_activation.so: silu_and_mul (vs F.silu), gelu_and_mul (vs F.gelu)
  xllm_rope.so:       rotary_embedding (verify rotation applied)
  xllm_cache.so:      reshape_paged_cache (verify slot write)
  vs_ixformer:        compare our .so output vs base image ixformer

All 4 .so compiled successfully:
  xllm_norm.so       1.4MB  [rms_norm, fused_add_rms_norm]
  xllm_activation.so 1.3MB  [silu_and_mul, gelu_and_mul, act_and_mul]
  xllm_rope.so       1.2MB  [rotary_embedding]
  xllm_cache.so      1.3MB  [block_copy, reshape_paged_cache]
 2026-08-14 11:14:58 +00:00
claude
3d816cd18d fix: add ceil_div + DEVICE_INLINE to device_utils.cuh 
ceil_div<T> was in xllm utils.h (removed for glog).
DEVICE_INLINE macro also moved to shared header.
 2026-08-14 11:11:15 +00:00
claude
302aa9608a fix: block_copy.cu — DEVICE_INLINE, CHECK_EQ→TORCH_CHECK, cstdint 
Remaining glog dependencies removed:
  - DEVICE_INLINE macro defined inline
  - CHECK_EQ(a,b) → TORCH_CHECK(a == b)
  - CHECK_GT(a,b) → TORCH_CHECK(a > b)
  - #include <cstdint> for int32_t
 2026-08-14 11:06:07 +00:00
claude
900ae0b1ef fix: block_copy.cu remove utils.h (glog), CHECK→TORCH_CHECK 
3/4 kernels now compile:
  ✓ xllm_norm.so      (rms_norm, fused_add_rms_norm)
  ✓ xllm_activation.so (silu_and_mul, gelu_and_mul, act_and_mul)
  ✓ xllm_rope.so       (rotary_embedding)
  → xllm_cache.so      block_copy.cu had utils.h→glog — fixed
 2026-08-14 11:02:57 +00:00
claude
a206fc1d43 fix: activation.cu torch/extension.h + LOG(FATAL)→TORCH_CHECK, reshape_paged_cache.cu torch header 
xllm_norm.so: ✓ COMPILED AND LOADED (rms_norm, fused_add_rms_norm)

activation.cu fixes:
  - Add #include <torch/extension.h> (torch::Tensor not visible from torch/cuda.h alone)
  - Replace LOG(FATAL) with TORCH_CHECK (no glog)

reshape_paged_cache.cu:
  - Add #include <torch/extension.h>
 2026-08-14 10:52:54 +00:00
claude
093bfb380f feat: pybind11 bindings for xllm CUDA kernels 
norm.cu compiled successfully on BI-V100 (only warning: fp8 __host__ attr).
Failed at import because no PYBIND11_MODULE — now fixed.

New bindings/ directory with 4 binding files:
  xllm_norm_bind.cpp      → rms_norm, fused_add_rms_norm
  xllm_activation_bind.cpp → silu_and_mul, gelu_and_mul, act_and_mul
  xllm_rope_bind.cpp       → rotary_embedding
  xllm_cache_bind.cpp      → reshape_paged_cache, block_copy

Build script updated: each .so = kernel .cu + binding .cpp
 2026-08-14 10:50:13 +00:00
claude
415fff85f1 fix: add DISPATCH_FLOATING_TYPES macro to device_utils.cuh 
DISPATCH_FLOATING_TYPES was defined in xllm/core/kernels/cuda/utils.h
which was pulled in via cuda_ops_api.h → utils.h.
Since cuda_ops_api.h was removed (glog dependency), the macro was missing.

Now defined in device_utils.cuh with include guard, available to all kernel files:
  norm.cu, activation.cu, rope.cu, block_copy.cu, reshape_paged_cache.cu
 2026-08-14 10:45:37 +00:00
claude
0359103b9b fix: remove glog/cuda_ops_api.h dependency from all xllm CUDA kernels 
cuda_ops_api.h includes glog/logging.h and ATen/DynamicLibrary.h
which are not available in corex standalone compilation.

All kernel .cu files only need device_utils.cuh (provides namespace,
XLLM_KERNEL_ATTR macro, CUB includes, type helpers).

Fixed files:
  norm.cu, activation.cu, rope.cu, block_copy.cu, reshape_paged_cache.cu
  moe/moe_combine.cu, moe/moe_compute_index.cu, moe/moe_fused_topk.cu
 2026-08-14 10:30:10 +00:00
claude
51cb90b9ab fix: adapt xllm norm.cu for corex CUB (CUDA 10.2) 
Key change: replace CCCL 3.6 types with corex CUB equivalents
  - cuda::std::plus<> → cub::Sum
  - cuda::maximum<>  → cub::Max
  - Remove #include <cuda/std/functional>

Test results from real machine (3/4 passed):
  ✓ __shfl_down_sync works on ivcore10
  ✓ manual SMEM+shuffle block reduce works
  ✓ corex CUB cub::BlockReduce<float,256> compiles and runs correctly (32640)
  ✗ CCCL 3.6 variadic function issue — corex clang rejects device variadic

Confirmed: use /usr/local/corex/include/cub/ for all kernel code
           cccl_upstream is reference only, NOT compilable on corex

Build script: bash qwen3_6_scripts/build_xllm_kernels.sh
 2026-08-14 10:19:51 +00:00
claude
089b9ff4e2 test: fix CUB compat tests + probe corex built-in CUB 
v1 failures analyzed:
  1. __shfl_down_sync: load_inline bug (not hardware issue) — fixed with file-based compile
  2. CCCL 3.6: 'CUDA < 12 not supported' + variadic functions in device code
     → corex reports CUDA 10.2, corex clang rejects variadic in device code

v2 tests:
  [1/4] __shfl_down_sync — does ivcore10 support warp shuffle?
  [2/4] manual block reduce — SMEM + shuffle without CUB
  [3/4] cub::BlockReduce using COREX's own CUB (/usr/local/corex/include/cub/)
  [4/4] cub::BlockReduce using CCCL 3.6 with CCCL_IGNORE_DEPRECATED_CUDA_BELOW_12

probe_corex_cub.sh — discovers corex CUB version, compiles standalone test

Run on real machine:
  bash qwen3_6_scripts/probe_corex_cub.sh
  python3 qwen3_6_scripts/test_cub_compat_v2.py
 2026-08-14 10:11:10 +00:00
claude
793743f5c0 test: CUB compatibility on BI-V100 — warp shuffle + BlockReduce 
Three tests:
  1. __shfl_down_sync — warp shuffle PTX instruction
  2. Manual block reduce (SMEM + shuffle) — handwritten
  3. cub::BlockReduce<float, 256> — actual CCCL header

If test 1+2 pass but 3 fails → CUB headers need corex adaptation
If test 1 fails → ivcore10 doesn't support warp shuffle → need different reduction strategy

Run on real machine: python3 qwen3_6_scripts/test_cub_compat.py
 2026-08-14 08:21:48 +00:00
claude
ec140f3605 docs: complete porting assessment — 5 repos, 2660 build targets, ivcore10 compatibility 
cat'd every source:
  - CCCL c/parallel/src/reduce.cu: NVRTC JIT (cannot port directly)
  - CCCL c/parallel.v2/src/reduce.cu: hostjit/libnvcc (cannot port directly)
  - CUB block/warp headers: pure header-only (CAN compile with corex)
  - CUTLASS SM70 SIMT GEMM: CAN port (example 24 grouped_gemm)
  - FlashAttention: SM80 Tensor Core (cannot port), but layer_norm is SIMT
  - FLA: pure Triton (needs corex Triton verification)
  - xllm ILU: all already ported

Key finding: CCCL c/parallel depends on nvrtc JIT — must bypass with AOT compilation
using CUB header-only API + corex clang --cuda-gpu-arch=ivcore10
 2026-08-14 08:17:25 +00:00
claude
3a2cfc87c9 test: xllm CUDA kernel verification suite for BI-V100 
test_xllm_cuda_kernels.py — 7 test groups:
  1. activation.cu: silu_and_mul via ixf_F, compare vs torch.nn.functional.silu
  2. norm.cu: rms_norm + fused_add_rms_norm via ixf_F, compare vs PyTorch
  3. rope.cu: rotary_embedding via ixf_F, verify rotation applied
  4. moe_topk_softmax: corex .so, verify shapes + weights sum to 1
  5. ix_moe_bridge: full 7-step fused MoE pipeline (topk→expand→gemm→act→gemm→combine)
  6. ix_attn_bridge: load test (prefill_attention, decode_attention, linear)
  7. ix_full_bridge: silu_and_mul + rms_norm through bridge .so

Revert: undo unnecessary cccl_upstream sync (already up to date)

Run on real machine: python3 qwen3_6_scripts/test_xllm_cuda_kernels.py
 2026-08-14 08:01:23 +00:00
claude
8d75652949 feat: import CUDA kernels from xllm/CCCL/FLA upstream repos 
Sources cloned and tree'd (no --depth):
  - jd-opensource/xllm: ILU kernels, CUDA kernels, MoE kernels
  - NVIDIA/cccl: CUB tuning/dispatch headers (block-level primitives)
  - fla-org/flash-linear-attention: Triton GDN kernels
  - NVIDIA/cutlass: grouped GEMM reference (read, not copied)
  - Dao-AILab/flash-attention: attention kernel reference (SM80+, read only)

New CUDA kernels (from xllm, SM-agnostic, portable to BI-V100):
  ex_engine/xllm_kernels/cuda/activation.cu    (188 lines) — silu_and_mul, gelu
  ex_engine/xllm_kernels/cuda/norm.cu          (600 lines) — rms_norm, fused_add_rms_norm
  ex_engine/xllm_kernels/cuda/rope.cu          (258 lines) — rotary_embedding
  ex_engine/xllm_kernels/cuda/block_copy.cu    (209 lines) — copy_blocks, swap_blocks
  ex_engine/xllm_kernels/cuda/reshape_paged_cache.cu (101 lines) — KV cache ops
  ex_engine/xllm_kernels/cuda/headers/         (5 headers for compilation)

ILU bridge kernel sources (from xllm, verified SAME as upstream):
  ex_engine/xllm_kernels/ilu/    (10 files, 925 lines total)
  — activation.cpp, attention.cpp, fused_moe.cpp, group_gemm.cpp,
    matmul.cpp, norm.cpp, rope.cpp, ilu_ops_api.h, ixformer.h, utils.h

FLA Triton GDN kernels (for GatedDeltaNet without SM90+ FlashQLA):
  ex_engine/fla_kernels/gated_delta_rule/  (7 files, 2370 lines)
  — chunk_fwd.py (428), chunk.py (487), wy_fast.py (409),
    fused_recurrent.py (392), naive.py (161), gate.py (380)

CCCL sync (12 tuning + 14 dispatch headers updated from NVIDIA/cccl):
  cccl_upstream/cub/cub/device/dispatch/tuning/ — 12 changed files synced
  cccl_upstream/cub/cub/device/dispatch/ — 14 changed dispatch files synced

Compilation targets for real machine (ivcore10):
  1. CUDA kernels: --cuda-gpu-arch=ivcore10 via corex clang/16
  2. ILU bridges: torch.utils.cpp_extension linking ixformer .so
  3. FLA kernels: Triton JIT (if Triton works on BI-V100)
 2026-08-14 07:48:52 +00:00
claude
051b02d3cd feat: ix_moe_bridge + ix_attn_bridge — dlopen bridges for full ixformer::infer API 
Bridge architecture (from xllm/core/kernels/ilu/ixformer.h):

ix_moe_bridge.so (MoE 7-step fused pipeline):
  - topk_softmax → moe_compute_token_index_api → moe_expand_input
  - moe_w16a16_group_gemm (x2) → silu_and_mul → moe_output_reduce_sum
  - fused_moe_forward(): replaces entire Python expert loop
  - Fix: group_gemm format NT→TN (match xllm trans_b=true)

ix_attn_bridge.so (attention + linear):
  - ixinfer_flash_attn_unpad_with_block_tables (fused prefill)
  - xllm_paged_attention (fused paged decode)
  - ixformer_linear (matmul + activation)
  - residual_rms_norm (fused residual + norm)

Integration:
  - ix_fused_moe.py: Python loader (prebuilt .so → JIT → unavailable)
  - qwen3_5.py: Tier 0 dispatch in _pure_pytorch_experts()
  - patch_ops.sh: deploys ix_fused_moe.py + all prebuilt/*.so

Source: jd-opensource/xllm (fresh clone, all ILU kernels verified SAME)
Sync: upstream_ref/xllm_latest/models/llm/qwen3_next_hybrid_base.h (+32 lines)

Build on real machine:
  bash qwen3_6_scripts/build_ix_moe_bridge.sh
  bash qwen3_6_scripts/build_ix_attn_bridge.sh
 2026-08-14 07:32:31 +00:00
Claude
5e9b7c292a build: all 16 .so prebuilt, no JIT compile in docker 
- install_prebuilt_corex.sh: 14→16 artifacts, allow ix_full_bridge.so name
- patch_ops.sh: remove all on-site compilation, pure prebuilt install
 2026-08-14 07:14:46 +00:00