c1936a55cb
block_histogram.cuh entire design:
Two algorithms for counting observations per bin:
1. BLOCK_HISTO_SORT: sort → detect discontinuities → run lengths = bin counts
Consistent throughput regardless of distribution.
2. BLOCK_HISTO_ATOMIC: atomicAdd per bin.
Fast for uniform, slow for skewed (atomic contention).
Template param selects algorithm at compile time.
Translation: We already do HISTO_SORT (argsort by expert_id → segment detect).
Added: compute seg_sizes histogram (seg_ends - seg_starts) which enables:
- Understanding expert load balance (skewed = some experts get 100 tokens,
others get 1 → HISTO_ATOMIC contention equivalent: Python loop overhead
for 1-token F.linear calls dominates)
- Future: batch 1-token segments into padded GEMM (HISTO_SORT guaranteed
consistent throughput, matches batch-friendly GEMM patterns)
+ dispatch_copy_mdspan contiguous-check in same commit area.
CCCL source: cub/cub/block/block_histogram.cuh (full 412-line file)
Maps to: qwen3_6_scripts/qwen3_5.py (_pure_pytorch_experts)