From a8b16da5dae794b8e1bdfb1f0aee4da7d25c5397 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 11 Aug 2026 02:31:29 +0000 Subject: [PATCH] =?UTF-8?q?fix(CRITICAL):=20remove=20xformers=20patches=20?= =?UTF-8?q?=E2=80=94=20Sub168=20proves=20base=20ixformer=20attention=20wor?= =?UTF-8?q?ks=20at=2011.9=20TPS,=20our=20patches=20reduced=20to=202.6=20TP?= =?UTF-8?q?S?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Root cause of Sub 520 output_tps=2.6 (vs Sub 168 output_tps=11.9): - patch_xformers_sdpa_seq.py replaces ixformer flash attention with pure PyTorch O(L^2) matmul+softmax serial implementation - 32 full attention layers x every token = 4.6x slower Sub 168 (base image) proof: - output_tps_avg=11.9, output_tps_p50=13.0, output_tps_p90=18.1 - XFormers backend used WITHOUT any patches - ixformer flash_attn works correctly on BI-V100 This commit: skip xformers patches in patch_ops.sh Expected: output_tps should recover to ~11.9 (Sub 168 level) --- qwen3_6_scripts/patch_ops.sh | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/qwen3_6_scripts/patch_ops.sh b/qwen3_6_scripts/patch_ops.sh index a7fc5714..19ba24e7 100755 --- a/qwen3_6_scripts/patch_ops.sh +++ b/qwen3_6_scripts/patch_ops.sh @@ -164,12 +164,13 @@ if [ -d "$_SITE" ]; then fi # =========================================================== -# 5. XFormers patches — head_dim=256 bypass for BI-V100 -# Comp 168 also had xformers patches (base uses xformers for attention) +# 5. XFormers patches — DISABLED +# Sub 168 (base image) achieved output_tps=11.9 WITHOUT any xformers patches. +# Sub 520 applied these patches → output_tps=2.6 (4.6x slower!) +# The patches replace ixformer flash attention with pure PyTorch O(L²) matmul. +# Base image ixformer attention works correctly — proven by Sub 168. # =========================================================== -python3 ./patch_xformers_sdpa_seq.py 2>&1 || true -python3 ./patch_xformers_sdpa_batch.py 2>&1 || true -echo "[patch_ops] xformers patches applied" +echo "[patch_ops] xformers patches SKIPPED — base ixformer attention works (Sub 168 proof)" # =========================================================== # 6. model_runner patch (prefix_cache_hit fix)