project_6 db0df78580 [gen_patch] fix 3 critical bugs: reduce struct selection, topk bits_per_pass injection, transform/batch_memcpy extraction
Bug 1: reduce struct selection — gen_patch selected bi100_plus_accum1_o4 (int8
path, items=32) instead of bi100_plus_float32_o4 (fp32 score accumulator,
items=24). vllm paged_attention always uses fp32 for score accumulation, so
the wrong struct was injecting items=32 into the 83%-weight hot path.

Fix: preference-ordered struct selection — float32 > accum2 > first non-default.
Now correctly selects bi100_plus_float32_o4 → NUM_ITEMS_PER_THREAD=24.

Bug 2: topk bits_per_pass not injected — gen_patch only extracted threads=512
from topk inline policy_selector, missing calc_bits_per_pass(key_size).
For Qwen3.6 float32 logits (key_size=4), bits_per_pass=11 (not 8).

Fix: topk-specific extraction that parses calc_bits_per_pass and returns
bits_per_pass=11. Now generates RADIX_BITS=11 patch for sampling_kernels.cu.

Bug 3: transform/batch_memcpy extraction failed — these headers use
policy struct naming, not bi100_* naming, so extract_bi100_structs was empty.

Fix: algorithm-specific fallback extraction for transform (reads
bi100_bytes_in_flight constexpr) and batch_memcpy (reads threads from
policy_selector return).

Validation: gen_patch 7 patches (was 6), test_smem_safety 191/191 safe
2026-08-05 03:15:12 +00:00
2026-07-30 17:03:23 +08:00

project_6

Description
No description provided
Readme 427 MiB
Languages
C++ 41.8%
Cuda 31.6%
Python 22.2%
C 2.1%
CMake 1.1%
Other 1.1%