Files
project_6/vllm/model_executor
dylanyunlon bf1cccb750 refactor(moe): apply CCCL GridEvenShare + dispatch_batch_memcpy to BLOCK_SIZE_M
CCCL source input: dispatch_batch_memcpy.cuh, agent_reduce.cuh, grid_even_share.cuh

dispatch_batch_memcpy.cuh two-level dispatch pattern:
  - Small buffers (warp-level): one CTA copies multiple small buffers
  - Large buffers (block-level): multiple CTAs collaborate on one buffer
  Applied: decode (M=1, numel=8) uses BLOCK_SIZE_M=16 (warp-level),
  prefill (M=4096, numel=32768) uses BLOCK_SIZE_M=256 (block-level).

GridEvenShare formula from grid_even_share.cuh:
  max_blocks = sm_count * subscription_factor = 16 * 5 = 80
  optimal_block_m = ceil(numel / max_blocks)
  Thresholds now derived from 80 * {16, 64, 128} instead of ad-hoc.

agent_reduce.cuh ConsumeFullTile pattern validates the existing
_moe_intermediate_cache buffer reuse (matches CCCL alias_temporaries
pre-allocation across kernel invocations).
2026-08-07 03:22:57 +00:00
..