perf: native ixformer decode (v1 ≤32K, v2 >32K) + flash_attn_varlen prefill
Replaces all Python PyTorch fallback attention with native ixformer kernels: Decode path: - ≤32K: paged_attention_v1 (5D KV layout, x=8) — verified on real BI-V100 - >32K: paged_attention_v2 (5D→4D permute) — verified 65K+ on real BI-V100 - Removes _forward_decode_pytorch Python fallback entirely Prefill path (profiling): - _run_sdpa_fallback now uses ixformer.flash_attn_varlen_func - head_dim=256 verified correct (diff<0.004) and 1.7x faster than PyTorch - Falls back to Q-tiling pure-math if ixformer unavailable Also includes: MoE kernel integration, GDN C++ kernels, diagnostic scripts, xllm upstream layer/kernel references, .dockerignore cleanup. All changes verified on real BI-V100 hardware (single card).
This commit is contained in:
28
ex_engine/csrc/ilu_CMakeLists.txt
Normal file
28
ex_engine/csrc/ilu_CMakeLists.txt
Normal file
@@ -0,0 +1,28 @@
|
||||
include(cc_library)
|
||||
set(CMAKE_CUDA_ARCHITECTURES ivcore11)
|
||||
file(GLOB_RECURSE ILU_HEADER_FILES
|
||||
"${CMAKE_CURRENT_LIST_DIR}/*.h"
|
||||
)
|
||||
|
||||
file(GLOB_RECURSE ILU_SOURCE_FILES
|
||||
"${CMAKE_CURRENT_LIST_DIR}/*.cpp"
|
||||
"${CMAKE_CURRENT_LIST_DIR}/*.cu"
|
||||
)
|
||||
|
||||
find_package(Python3 REQUIRED COMPONENTS Interpreter Development)
|
||||
|
||||
cc_library(
|
||||
NAME
|
||||
ilu_kernels
|
||||
HDRS
|
||||
${ILU_HEADER_FILES}
|
||||
SRCS
|
||||
${ILU_SOURCE_FILES}
|
||||
DEPS
|
||||
torch
|
||||
:util
|
||||
ixformer_kernels
|
||||
ixformer
|
||||
${Python3_LIBRARIES}
|
||||
cuinfer
|
||||
)
|
||||
Reference in New Issue
Block a user