feat(SM70): wire up FlashQLA GDN kernel dispatch in prefill path
GDN forward dispatch chain: 1. CoreX fused kernel (if packaged) → fastest 2. FlashQLA SM70 CUDA kernel (prefill only) → verified on BI-V100 3. Pure PyTorch with NaN clamp → fallback FlashQLA SM70 verified on real BI-V100: - Compiled with clang++ --cuda-gpu-arch=ivcore10 - gdn_forward returns correct shapes, zero NaN - 4 kernels: prefill, varlen prefill, decode global, decode ddtree Also: apt ninja-build instead of pip ninja (pip version has no binary)
This commit is contained in:
@@ -57,8 +57,11 @@ for P in /usr/local/lib/python3.10/site-packages/transformers/models \
|
||||
done
|
||||
if [ -n "$TMODELS" ]; then
|
||||
# Base engine requires transformers 4.55.3 for Qwen3_5Config support
|
||||
pip install transformers==4.55.3 ninja -i https://pypi.tuna.tsinghua.edu.cn/simple --timeout 30 2>&1 || \
|
||||
pip install transformers==4.55.3 -i https://pypi.tuna.tsinghua.edu.cn/simple --timeout 30 2>&1 || \
|
||||
echo "[patch_ops] WARNING: pip install failed (may already be correct versions)"
|
||||
# ninja-build required for torch.utils.cpp_extension CUDA compilation
|
||||
apt-get update -qq && apt-get install -y -qq ninja-build 2>&1 || \
|
||||
echo "[patch_ops] WARNING: ninja-build install failed — CUDA kernel will not compile"
|
||||
cp -r ./qwen3_5 "$TMODELS/" 2>/dev/null && echo "[patch_ops] qwen3_5 config copied" || true
|
||||
cp -r ./qwen3_5_moe "$TMODELS/" 2>/dev/null && echo "[patch_ops] qwen3_5_moe config copied" || true
|
||||
python3 ./patch_transformers_qwen3_5.py 2>&1 || echo "[patch_ops] WARNING: transformers patch failed (non-fatal)"
|
||||
|
||||
Reference in New Issue
Block a user