Claude
|
52e2ef31a8
|
feat: xllm_ops NO-FALLBACK kernel loader + 6 missing .so build targets + hot-path patcher
Build infrastructure:
- build_xllm_kernels.sh: add 5 missing build targets (norm, rope, activation, cache, moe)
Previously only built xllm_fused_qknorm_rope.so, now builds all 6 .so files
Kernel loader (xllm_ops.py):
- NO-FALLBACK architecture matching xllm/core/kernels/ilu/ dispatch chain
- Loads: xllm_norm.so, xllm_rope.so, xllm_activation.so, xllm_cache.so,
xllm_moe.so, ix_full_bridge.so, xllm_fused_qknorm_rope.so
- check_all(strict=True) verifies ALL .so at startup
Hot-path patcher (patch_vllm_hot_path.py):
- Monkey-patches vllm._custom_ops to route through xllm .so
- Critical fix: topk_softmax patch prevents comp 168 cascade
- Patches: topk_softmax, rms_norm, silu_and_mul, rotary_embedding, reshape_and_cache
Source mapping: xllm/core/kernels/ilu/*.cpp -> our xllm_*.so files
|
2026-08-15 14:13:13 +00:00 |
|
Claude
|
36676f2d1b
|
data: complete SGEMM upstream from 3 repos (siboehm+wangzyon+edtallison) + xllm fused_qknorm_rope + xattention kernels
SGEMM repos (upstream_ref/sgemm_cuda/, 41 files):
siboehm/SGEMM_CUDA: kernel 1-12, runner, CMake, cuBLAS benchmark
wangzyon/NVIDIA_SGEMM_PRACTICE: kernel 1-7 (Chinese comments), utils
edtallison/sgemm-cuda: kernel 01-09 (learning notes), Makefile
xllm kernels (ex_engine/xllm_kernels/cuda/):
fused_qknorm_rope.cu + bind — saves 128 kernel launches/fwd
xattention/ — 6 files from upstream xllm
headers: corex_compat_utils.h, topk_last_dim.cuh
ilu/CMakeLists.txt
SO_BUILD_MANIFEST.md — complete .so inventory and call chain analysis
|
2026-08-15 07:00:09 +00:00 |
|