From 0caabf285bc6307d4b7286ec888a49412043d0a1 Mon Sep 17 00:00:00 2001 From: muh-bot Date: Fri, 7 Aug 2026 02:46:46 +0000 Subject: [PATCH] =?UTF-8?q?fix(critical):=20v2=20kernel=20softmax=20normal?= =?UTF-8?q?ization=20was=20commented=20out=20=E2=80=94=20outputs=20were=20?= =?UTF-8?q?wrong?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit qwen3_6_scripts/prefix_prefill.py line 435: BEFORE: # acc /= l_i[:, None] (commented out = BUG) AFTER: acc = acc / l_i[:, None] (restored) Impact: _fwd_kernel_flash_attn_v2 was producing unnormalized attention output — every prefill with context length > BLOCK_M would have had incorrect softmax weights, causing wrong generation quality. This directly affects the effect test (偏差 ≤ ±4% benchmark). Root cause: the v1 kernel (_fwd_kernel) does online normalization (p_scale = beta/l_i_new), but v2 uses acc_scale = alpha only and defers normalization to the end. Someone commented out the final division, breaking v2. NOTE: The file that actually gets deployed is qwen3_6_scripts/, NOT vllm/. Previous commits edited vllm/ which has no effect on the built Docker image. --- qwen3_6_scripts/prefix_prefill.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/qwen3_6_scripts/prefix_prefill.py b/qwen3_6_scripts/prefix_prefill.py index cab0e139..6e78be83 100644 --- a/qwen3_6_scripts/prefix_prefill.py +++ b/qwen3_6_scripts/prefix_prefill.py @@ -432,7 +432,12 @@ if triton.__version__ >= "2.1.0": l_i = l_i_new m_i = m_i_new - # acc /= l_i[:, None] + # BUG FIX: v2 kernel accumulates unnormalized softmax weights. + # Without this final division, output = sum(softmax_unnorm * V) + # instead of the correct sum(softmax_normalized * V). + # v1 kernel does online normalization inside the loop (p_scale/acc_scale). + # v2 defers normalization — it MUST happen here. + acc = acc / l_i[:, None] # initialize pointers to output off_o = ( (cur_batch_in_all_start_index + offs_m[:, None]) * stride_obs +