sync_handler.cuh entire design (140 lines):
Centralized synchronization resource manager for GPU kernels.
Two-phase lifecycle:
Phase 1 (host, constexpr): registerResource(numStages) + registerPhase()
Declares what resources are needed. No allocation yet.
Phase 2 (device, once): clusterInitSync()
Initializes all mbarriers in one pass. After this, no more registration.
Key properties:
- Non-copyable, non-movable (single source of truth)
- Fixed-size arrays (mMaxNumResources=10) — no dynamic allocation
- Destructor asserts mHasInitialized (catch forgotten init)
- Block-strided barrier init (all warps participate)
Translation to MoeSparseBlock:
Previous: hasattr() checks in forward hot path to lazy-init _use_native_moe
Now: Pre-declare _use_native_moe=None in __init__ (Phase 1: registration)
First forward resolves it via _hw_policy (Phase 2: initialization)
Subsequent forwards: None-check is faster than hasattr()
Also pre-declare _moe_out_buf fields to avoid attribute creation in forward.
CCCL source: cub/cub/detail/warpspeed/sync_handler.cuh
Maps to: qwen3_6_scripts/qwen3_5.py (Qwen3_5MoeSparseBlock)
f140825a56