2e2a479c08
dispatch_copy_mdspan.cuh entire design: 1. Check is_exhaustive() + have_same_strides() (layout compatibility) 2. Fast path: if contiguous, use DeviceTransform (1D memcpy-like kernel) 3. Slow path: if non-contiguous, use DeviceFor::for_each_in_extents Translation to MoE segment loop: After sorting tokens by expert_id, tokens routed to the same expert often have consecutive original indices. When they do, hidden_states slice is zero-copy (view) vs fancy indexing (allocates new tensor). Check: tok_ids_seg[-1] == tok_ids_seg[0] + n - 1 (contiguous range) Fast: hidden_states[first:first+n] (zero-copy slice) Slow: hidden_states[tok_ids_seg] (gather with copy) CCCL source: cub/cub/device/dispatch/dispatch_copy_mdspan.cuh Maps to: qwen3_6_scripts/qwen3_5.py (_pure_pytorch_experts)