Logo
Explore Help
Register Sign In
dylanyunlong/project_6
1
0
Fork 0
You've already forked project_6
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
f28223c9daf544eed16d88c36b6aa66078d5e1ff
project_6/ex_engine/csrc/ilu_CMakeLists.txt

29 lines
504 B
Plaintext
Raw Normal View History

perf: native ixformer decode (v1 ≤32K, v2 >32K) + flash_attn_varlen prefill Replaces all Python PyTorch fallback attention with native ixformer kernels: Decode path: - ≤32K: paged_attention_v1 (5D KV layout, x=8) — verified on real BI-V100 - >32K: paged_attention_v2 (5D→4D permute) — verified 65K+ on real BI-V100 - Removes _forward_decode_pytorch Python fallback entirely Prefill path (profiling): - _run_sdpa_fallback now uses ixformer.flash_attn_varlen_func - head_dim=256 verified correct (diff<0.004) and 1.7x faster than PyTorch - Falls back to Q-tiling pure-math if ixformer unavailable Also includes: MoE kernel integration, GDN C++ kernels, diagnostic scripts, xllm upstream layer/kernel references, .dockerignore cleanup. All changes verified on real BI-V100 hardware (single card).
2026-08-13 07:04:21 +00:00
include(cc_library)
set(CMAKE_CUDA_ARCHITECTURES ivcore11)
file(GLOB_RECURSE ILU_HEADER_FILES
"${CMAKE_CURRENT_LIST_DIR}/*.h"
)
file(GLOB_RECURSE ILU_SOURCE_FILES
"${CMAKE_CURRENT_LIST_DIR}/*.cpp"
"${CMAKE_CURRENT_LIST_DIR}/*.cu"
)
find_package(Python3 REQUIRED COMPONENTS Interpreter Development)
cc_library(
NAME
ilu_kernels
HDRS
${ILU_HEADER_FILES}
SRCS
${ILU_SOURCE_FILES}
DEPS
torch
:util
ixformer_kernels
ixformer
${Python3_LIBRARIES}
cuinfer
)
Reference in New Issue Copy Permalink
Powered by Gitea Version: 1.24.3 Page: 406ms Template: 2ms
English
Bahasa Indonesia Deutsch English Español Français Gaeilge Italiano Latviešu Magyar nyelv Nederlands Polski Português de Portugal Português do Brasil Suomi Svenska Türkçe Čeština Ελληνικά Български Русский Українська فارسی മലയാളം 日本語 简体中文 繁體中文(台灣) 繁體中文(香港) 한국어
Licenses API