Claude 2316199c97 [FIX] V2 shape mismatch bug — v_padded used num_heads for kv_h tensor
Bug: After GQA broadcast optimization, v_perm was [kv_h, seq_len, d]
in the GQA path, but unconditional v_padded allocation used num_heads:
  v_padded = torch.zeros((num_heads, padded_len, head_size))
  v_padded[:, :seq_len, :] = v_perm  # [24, padded, d] vs [4, seq, d] → CRASH

Fix: v_padded/v_parts allocation is now inside the non-GQA else branch.
GQA branch uses its own v_padded_kv with correct [kv_h, padded, d] shape.

This was a real runtime bug — V2 would have crashed on first call
for any GQA model (Qwen3.6, Llama, etc.).
2026-07-31 03:52:23 +00:00
2026-07-30 17:03:23 +08:00

project_6

Description
No description provided
Readme 427 MiB
Languages
C++ 41.8%
Cuda 31.6%
Python 22.2%
C 2.1%
CMake 1.1%
Other 1.1%