feat(SM70): wire up FlashQLA GDN kernel dispatch in prefill path

GDN forward dispatch chain:
1. CoreX fused kernel (if packaged) → fastest
2. FlashQLA SM70 CUDA kernel (prefill only) → verified on BI-V100
3. Pure PyTorch with NaN clamp → fallback

FlashQLA SM70 verified on real BI-V100:
- Compiled with clang++ --cuda-gpu-arch=ivcore10
- gdn_forward returns correct shapes, zero NaN
- 4 kernels: prefill, varlen prefill, decode global, decode ddtree

Also: apt ninja-build instead of pip ninja (pip version has no binary)
This commit is contained in:
Claude
2026-08-10 01:42:16 +00:00
parent 47958c4ed2
commit c077736968
2 changed files with 108 additions and 2 deletions

View File

@@ -57,8 +57,11 @@ for P in /usr/local/lib/python3.10/site-packages/transformers/models \
done
if [ -n "$TMODELS" ]; then
# Base engine requires transformers 4.55.3 for Qwen3_5Config support
pip install transformers==4.55.3 ninja -i https://pypi.tuna.tsinghua.edu.cn/simple --timeout 30 2>&1 || \
pip install transformers==4.55.3 -i https://pypi.tuna.tsinghua.edu.cn/simple --timeout 30 2>&1 || \
echo "[patch_ops] WARNING: pip install failed (may already be correct versions)"
# ninja-build required for torch.utils.cpp_extension CUDA compilation
apt-get update -qq && apt-get install -y -qq ninja-build 2>&1 || \
echo "[patch_ops] WARNING: ninja-build install failed — CUDA kernel will not compile"
cp -r ./qwen3_5 "$TMODELS/" 2>/dev/null && echo "[patch_ops] qwen3_5 config copied" || true
cp -r ./qwen3_5_moe "$TMODELS/" 2>/dev/null && echo "[patch_ops] qwen3_5_moe config copied" || true
python3 ./patch_transformers_qwen3_5.py 2>&1 || echo "[patch_ops] WARNING: transformers patch failed (non-fatal)"