dylanyunlon ca3697f4b0 [ENGINE] CCCL SmemResource pattern: pre-allocate staging buffers in V2 paged attention ...

Eliminates per-decode-step torch.full/torch.zeros GPU allocations that cause
OOM after thousands of generation steps (case_truncation max_tokens=8192).

Three allocation sites replaced with staging buffer .fill_()/.zero_() reuse:
  - scores_padded: torch.full([H, padded_len], -inf) → _staging_scores slice
  - v_padded_kv: torch.zeros([kv_h, padded_len, d]) → _staging_v_kv slice
  - v_padded: torch.zeros([H, padded_len, d]) → _staging_v slice

Pattern from CCCL cub/detail/warpspeed/resource/smem_resource.cuh:
  SmemResource pre-allocates stageCount buffers, nextStage() cycles through them.
  PyTorch translation: allocate once at function entry, slice per step.

2026-08-07 04:37:44 +00:00

..

qwen3_5

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

qwen3_5_moe

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

reasoning

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

_custom_ops.py

[CRITICAL/deploy] fix 3 deployment gaps found from docker crash log

2026-08-06 07:01:14 +00:00

api_server.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

arg_utils.py

[ARCH] Eliminate AST patch scripts — full file replacements only

2026-08-05 08:24:43 +00:00

chat_utils.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

cli_args.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

logits_processor.py

[ARCH] Eliminate AST patch scripts — full file replacements only

2026-08-05 08:24:43 +00:00

mamba_cache.py

[fix/correctness] mamba_cache: safe swap via clone, not in-place fancy indexing

2026-08-06 06:33:26 +00:00

model_runner.py

[ENGINE] model_runner.py: CCCL CachingDeviceAllocator pattern — reduce CUDA graph capture from 1028→19 sizes

2026-08-07 01:59:18 +00:00

paged_attention_v2_pytorch.py

[ENGINE] CCCL SmemResource pattern: pre-allocate staging buffers in V2 paged attention

2026-08-07 04:37:44 +00:00

paged_attn.py

[critical/deploy] sync root paged_attn.py + prefix_prefill.py → qwen3_6_scripts/

2026-08-06 06:10:56 +00:00

patch_ops.sh

[CRITICAL/deploy] fix 3 deployment gaps found from docker crash log

2026-08-06 07:01:14 +00:00

patch_transformers_qwen3_5.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

prefix_prefill.py

fix(critical): v2 kernel softmax normalization was commented out — outputs were wrong

2026-08-07 02:46:46 +00:00

protocol.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

qwen3_5.py

[FIX] qwen3_5.py: replace solve_triangular with manual forward substitution

2026-08-06 03:02:29 +00:00

qwen3coder_tool_parser.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

registry.py

[ARCH] Eliminate 2 more patch scripts — registry.py + tool_parsers __init__.py

2026-08-05 08:36:52 +00:00

sampler.py

[BASE] qwen3_6_scripts/sampler.py: CCCL topk unsorted output optimization

2026-08-06 02:55:51 +00:00

scheduler.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

sequence.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

serving_chat.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

tool_parsers_init.py

[ARCH] Eliminate 2 more patch scripts — registry.py + tool_parsers __init__.py

2026-08-05 08:36:52 +00:00

verify_functional.py

feat(verify): expand functional test suite from 21 to 51 test cases

2026-08-07 02:01:23 +00:00

xformers.py

feat(xformers): replace Q-only tiling with Q+KV tiling + online softmax

2026-08-07 01:54:52 +00:00