dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
604 Commits 1 Branch 0 Tags
900ae0b1efc19e46cd313eb3ba7eb694fa596f6b
Commit Graph

8 Commits

Author SHA1 Message Date
claude
900ae0b1ef fix: block_copy.cu remove utils.h (glog), CHECK→TORCH_CHECK 
3/4 kernels now compile:
  ✓ xllm_norm.so      (rms_norm, fused_add_rms_norm)
  ✓ xllm_activation.so (silu_and_mul, gelu_and_mul, act_and_mul)
  ✓ xllm_rope.so       (rotary_embedding)
  → xllm_cache.so      block_copy.cu had utils.h→glog — fixed
 2026-08-14 11:02:57 +00:00
claude
a206fc1d43 fix: activation.cu torch/extension.h + LOG(FATAL)→TORCH_CHECK, reshape_paged_cache.cu torch header 
xllm_norm.so: ✓ COMPILED AND LOADED (rms_norm, fused_add_rms_norm)

activation.cu fixes:
  - Add #include <torch/extension.h> (torch::Tensor not visible from torch/cuda.h alone)
  - Replace LOG(FATAL) with TORCH_CHECK (no glog)

reshape_paged_cache.cu:
  - Add #include <torch/extension.h>
 2026-08-14 10:52:54 +00:00
claude
093bfb380f feat: pybind11 bindings for xllm CUDA kernels 
norm.cu compiled successfully on BI-V100 (only warning: fp8 __host__ attr).
Failed at import because no PYBIND11_MODULE — now fixed.

New bindings/ directory with 4 binding files:
  xllm_norm_bind.cpp      → rms_norm, fused_add_rms_norm
  xllm_activation_bind.cpp → silu_and_mul, gelu_and_mul, act_and_mul
  xllm_rope_bind.cpp       → rotary_embedding
  xllm_cache_bind.cpp      → reshape_paged_cache, block_copy

Build script updated: each .so = kernel .cu + binding .cpp
 2026-08-14 10:50:13 +00:00
claude
415fff85f1 fix: add DISPATCH_FLOATING_TYPES macro to device_utils.cuh 
DISPATCH_FLOATING_TYPES was defined in xllm/core/kernels/cuda/utils.h
which was pulled in via cuda_ops_api.h → utils.h.
Since cuda_ops_api.h was removed (glog dependency), the macro was missing.

Now defined in device_utils.cuh with include guard, available to all kernel files:
  norm.cu, activation.cu, rope.cu, block_copy.cu, reshape_paged_cache.cu
 2026-08-14 10:45:37 +00:00
claude
0359103b9b fix: remove glog/cuda_ops_api.h dependency from all xllm CUDA kernels 
cuda_ops_api.h includes glog/logging.h and ATen/DynamicLibrary.h
which are not available in corex standalone compilation.

All kernel .cu files only need device_utils.cuh (provides namespace,
XLLM_KERNEL_ATTR macro, CUB includes, type helpers).

Fixed files:
  norm.cu, activation.cu, rope.cu, block_copy.cu, reshape_paged_cache.cu
  moe/moe_combine.cu, moe/moe_compute_index.cu, moe/moe_fused_topk.cu
 2026-08-14 10:30:10 +00:00
claude
51cb90b9ab fix: adapt xllm norm.cu for corex CUB (CUDA 10.2) 
Key change: replace CCCL 3.6 types with corex CUB equivalents
  - cuda::std::plus<> → cub::Sum
  - cuda::maximum<>  → cub::Max
  - Remove #include <cuda/std/functional>

Test results from real machine (3/4 passed):
  ✓ __shfl_down_sync works on ivcore10
  ✓ manual SMEM+shuffle block reduce works
  ✓ corex CUB cub::BlockReduce<float,256> compiles and runs correctly (32640)
  ✗ CCCL 3.6 variadic function issue — corex clang rejects device variadic

Confirmed: use /usr/local/corex/include/cub/ for all kernel code
           cccl_upstream is reference only, NOT compilable on corex

Build script: bash qwen3_6_scripts/build_xllm_kernels.sh
 2026-08-14 10:19:51 +00:00
claude
8d75652949 feat: import CUDA kernels from xllm/CCCL/FLA upstream repos 
Sources cloned and tree'd (no --depth):
  - jd-opensource/xllm: ILU kernels, CUDA kernels, MoE kernels
  - NVIDIA/cccl: CUB tuning/dispatch headers (block-level primitives)
  - fla-org/flash-linear-attention: Triton GDN kernels
  - NVIDIA/cutlass: grouped GEMM reference (read, not copied)
  - Dao-AILab/flash-attention: attention kernel reference (SM80+, read only)

New CUDA kernels (from xllm, SM-agnostic, portable to BI-V100):
  ex_engine/xllm_kernels/cuda/activation.cu    (188 lines) — silu_and_mul, gelu
  ex_engine/xllm_kernels/cuda/norm.cu          (600 lines) — rms_norm, fused_add_rms_norm
  ex_engine/xllm_kernels/cuda/rope.cu          (258 lines) — rotary_embedding
  ex_engine/xllm_kernels/cuda/block_copy.cu    (209 lines) — copy_blocks, swap_blocks
  ex_engine/xllm_kernels/cuda/reshape_paged_cache.cu (101 lines) — KV cache ops
  ex_engine/xllm_kernels/cuda/headers/         (5 headers for compilation)

ILU bridge kernel sources (from xllm, verified SAME as upstream):
  ex_engine/xllm_kernels/ilu/    (10 files, 925 lines total)
  — activation.cpp, attention.cpp, fused_moe.cpp, group_gemm.cpp,
    matmul.cpp, norm.cpp, rope.cpp, ilu_ops_api.h, ixformer.h, utils.h

FLA Triton GDN kernels (for GatedDeltaNet without SM90+ FlashQLA):
  ex_engine/fla_kernels/gated_delta_rule/  (7 files, 2370 lines)
  — chunk_fwd.py (428), chunk.py (487), wy_fast.py (409),
    fused_recurrent.py (392), naive.py (161), gate.py (380)

CCCL sync (12 tuning + 14 dispatch headers updated from NVIDIA/cccl):
  cccl_upstream/cub/cub/device/dispatch/tuning/ — 12 changed files synced
  cccl_upstream/cub/cub/device/dispatch/ — 14 changed dispatch files synced

Compilation targets for real machine (ivcore10):
  1. CUDA kernels: --cuda-gpu-arch=ivcore10 via corex clang/16
  2. ILU bridges: torch.utils.cpp_extension linking ixformer .so
  3. FLA kernels: Triton JIT (if Triton works on BI-V100)
 2026-08-14 07:48:52 +00:00
project6-dev
0b0c47fddd fix(critical): fold max_completion_tokens + max_num_seqs=2 + max_model_len=80000 + xllm_latest layer import 
Sub 655 root causes (confirmed from log analysis):
1. protocol.py: max_completion_tokens never folded into max_tokens
   → 162/881 replay requests rejected 400 (extra_forbidden)
2. max_num_seqs=1 → t2_n_2 test fails (needs n=2)
3. max_model_len=131072 → OOM crash at 62% replay, opencompass all 0

Fixes:
- protocol.py: model_validator fold_max_completion_tokens
- yaml: max_num_seqs=2, max_model_len=80000, PYTORCH_CUDA_ALLOC_CONF
- topk_softmax stays =0 (corex CUB BlockReduce incompatible on BI-V100)

xllm_latest import to ex_engine/:
- npu_torch layers: GDN(1164L), Qwen3.5 GDN, attention, fused_moe
- cuda/moe kernels: topk_softmax_kernels.cuh, moe_combine, moe_compute_index
- npu kernels: causal_conv1d, recurrent_gated_delta_rule
- model headers: qwen3_5.h, qwen3_next.h
 2026-08-13 03:19:39 +00:00