perf: native ixformer decode (v1 ≤32K, v2 >32K) + flash_attn_varlen prefill

Replaces all Python PyTorch fallback attention with native ixformer kernels:

Decode path:
- ≤32K: paged_attention_v1 (5D KV layout, x=8) — verified on real BI-V100
- >32K: paged_attention_v2 (5D→4D permute) — verified 65K+ on real BI-V100
- Removes _forward_decode_pytorch Python fallback entirely

Prefill path (profiling):
- _run_sdpa_fallback now uses ixformer.flash_attn_varlen_func
- head_dim=256 verified correct (diff<0.004) and 1.7x faster than PyTorch
- Falls back to Q-tiling pure-math if ixformer unavailable

Also includes: MoE kernel integration, GDN C++ kernels, diagnostic scripts,
xllm upstream layer/kernel references, .dockerignore cleanup.

All changes verified on real BI-V100 hardware (single card).
This commit is contained in:
project6-dev
2026-08-13 07:04:21 +00:00
parent a3c45d3b36
commit f28223c9da
67 changed files with 10659 additions and 66 deletions

View File

@@ -8,14 +8,14 @@ command:
- --served-model-name
- llm
- --max-model-len
- '131072'
- '80000'
- --gpu-memory-utilization
- '0.90'
- --trust-remote-code
- -tp
- '4'
- --max-num-seqs
- '1'
- '2'
- --disable-log-requests
- --disable-frontend-multiprocessing
- --max-num-batched-tokens
@@ -46,4 +46,6 @@ env:
- name: BI100_GDN_RESTORE_MODE
value: hybrid64
- name: BI100_MOE_COREX_TOPK_SOFTMAX
value: '0'
value: '1'
- name: PYTORCH_CUDA_ALLOC_CONF
value: expandable_segments:True