dylanyunlon 8002900af0 [ENGINE] CCCL kernel_scan.cuh dual-algorithm dispatch: v1 (online norm) vs v2 (deferred norm) ...

Source: cccl_upstream/cub/cub/device/dispatch/kernels/kernel_scan.cuh
Target: vllm/attention/ops/prefix_prefill.py

CCCL kernel_scan.cuh implements compile-time algorithm selection:
  - lookback: AgentScan with delay_constructor_t (safe default)
  - lookahead: warpspeed pipeline stages (SM90+, deferred reduction)

Applied to prefix_prefill Triton kernels:
  - _fwd_kernel (v1) = lookback: online softmax norm per block
  - _fwd_kernel_flash_attn_v2 = lookahead: deferred normalization
    Saves (ctx_len / BLOCK_N) divisions per query row.

Before: v2 kernel NEVER called — dead code since initial commit.
After: v2 dispatched for standard Qwen3.6 path (no alibi, no sliding
window, power-of-2 head_dim, no FP8).

BI-V100: v2 saves 64 fdiv/row at ctx_len=4096, BLOCK_N=64.

2026-08-07 06:37:03 +00:00

..

__pycache__

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

adapter_commons

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

assets

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

attention

[ENGINE] CCCL kernel_scan.cuh dual-algorithm dispatch: v1 (online norm) vs v2 (deferred norm)

2026-08-07 06:37:03 +00:00

compilation

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

core

[BASE] block_table.py: CCCL dispatch_select_if alias_temporaries batch allocation

2026-08-06 04:22:02 +00:00

distributed

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

engine

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

entrypoints

[CRITICAL/base] Register qwen3_coder tool parser as hermes alias

2026-08-06 06:10:50 +00:00

executor

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

inputs

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

logging

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

lora

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

model_executor

refactor(moe): apply CCCL GridEvenShare + dispatch_batch_memcpy to BLOCK_SIZE_M

2026-08-07 03:22:57 +00:00

multimodal

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

platforms

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

plugins

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

prompt_adapter

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

spec_decode

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

transformers_utils

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

triton_utils

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

usage

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

worker

[BASE] cache_engine.py: CCCL temporary_storage layout two-phase KV cache allocation

2026-08-06 04:22:53 +00:00

__init__.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

_core_ext.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

_custom_ops.py

[v2] PARTITION_SIZE 512→1024 + fix import path

2026-08-05 03:32:23 +00:00

_ipex_ops.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

beam_search.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

block.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

config.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

connections.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

envs.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

forward_context.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

logger.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

outputs.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

pooling_params.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

py.typed

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

sampling_params.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

scalar_type.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

scripts.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

sequence.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

tracing.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

utils.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

version.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00