f28223c9da
Replaces all Python PyTorch fallback attention with native ixformer kernels: Decode path: - ≤32K: paged_attention_v1 (5D KV layout, x=8) — verified on real BI-V100 - >32K: paged_attention_v2 (5D→4D permute) — verified 65K+ on real BI-V100 - Removes _forward_decode_pytorch Python fallback entirely Prefill path (profiling): - _run_sdpa_fallback now uses ixformer.flash_attn_varlen_func - head_dim=256 verified correct (diff<0.004) and 1.7x faster than PyTorch - Falls back to Q-tiling pure-math if ixformer unavailable Also includes: MoE kernel integration, GDN C++ kernels, diagnostic scripts, xllm upstream layer/kernel references, .dockerignore cleanup. All changes verified on real BI-V100 hardware (single card).
15 lines
165 B
Plaintext
Executable File
15 lines
165 B
Plaintext
Executable File
include(cc_library)
|
|
|
|
cc_library(
|
|
NAME
|
|
ilu_layers
|
|
HDRS
|
|
attention.h
|
|
fused_moe.h
|
|
SRCS
|
|
attention.cpp
|
|
fused_moe.cpp
|
|
DEPS
|
|
:common_layers
|
|
)
|