dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

Revert "fix(CRITICAL): remove xformers patches — Sub168 proves base ixformer attention works at 11.9 TPS, our patches reduced to 2.6 TPS"

Browse Source 
This reverts commit a8b16da5da.
This commit is contained in:
Claude
2026-08-11 02:33:45 +00:00
parent 87a19d2d00
commit 0eab333fb0
1 changed files with 5 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
qwen3_6_scripts/patch_ops.sh
View File

@@ -164,13 +164,12 @@ if [ -d "$_SITE" ]; then
fi fi
# =========================================================== # ===========================================================
# 5. XFormers patches — DISABLED # 5. XFormers patches — head_dim=256 bypass for BI-V100
# Sub 168 (base image) achieved output_tps=11.9 WITHOUT any xformers patches. # Comp 168 also had xformers patches (base uses xformers for attention)
# Sub 520 applied these patches → output_tps=2.6 (4.6x slower!)
# The patches replace ixformer flash attention with pure PyTorch O(L²) matmul.
# Base image ixformer attention works correctly — proven by Sub 168.
# =========================================================== # ===========================================================
echo "[patch_ops] xformers patches SKIPPED — base ixformer attention works (Sub 168 proof)" python3 ./patch_xformers_sdpa_seq.py 2>&1 || true
python3 ./patch_xformers_sdpa_batch.py 2>&1 || true
echo "[patch_ops] xformers patches applied"
# =========================================================== # ===========================================================
# 6. model_runner patch (prefix_cache_hit fix) # 6. model_runner patch (prefix_cache_hit fix)