perf(qwen3_5): CCCL block_reduce_warp_reductions → reduce DeltaNet loop iterations
CCCL design: when sequential path (Python forward substitution) dominates, reduce per-unit work by halving chunk_size from 32→16. 15 loop iterations beats 31, even with 2× more chunks. Also: add weight-skip warning logs (CCCL ScatterDirect pattern: never silently discard data). Docker logs will now show exactly which weights are skipped during load_weights, explaining the 1.12GB gap vs Sub168. CCCL sources this round: - block_reduce_warp_reductions.cuh: sequential vs parallel path selection - warp_exchange_smem.cuh: INSERT_PADDING for memory alignment - agent_reduce_by_key.cuh: TempStorage union + ScatterDirect
This commit is contained in:
@@ -118,14 +118,14 @@ class _HardwarePolicy:
|
||||
# solve_triangular path: one cuBLAS call per chunk, larger = fewer calls
|
||||
# Python loop path: C iterations per chunk, smaller = fewer iterations
|
||||
if self.solve_triangular_available:
|
||||
# Like CCCL max_items_per_thread=32 with threads=128 → tile=4096:
|
||||
# larger chunk = amortize kernel launch overhead
|
||||
# cuBLAS trsm: larger chunk = amortize kernel launch overhead
|
||||
self.deltanet_chunk_size = 64
|
||||
else:
|
||||
# Like CCCL reducing items for MUFU-heavy small-elem ops:
|
||||
# smaller chunk = fewer Python loop iterations (C iterations)
|
||||
# 32 iterations vs 64 = 2× fewer kernel launches in the loop
|
||||
self.deltanet_chunk_size = 32
|
||||
# Python forward substitution fallback: each chunk costs C iterations.
|
||||
# CCCL block_reduce_warp_reductions: when sequential path dominates,
|
||||
# reduce per-unit work (fewer iterations) even at cost of more units
|
||||
# (more chunks). 16 iterations × more chunks beats 32 iterations × fewer.
|
||||
self.deltanet_chunk_size = 16
|
||||
|
||||
# Prefill sub-chunk: controls peak memory per DeltaNet forward call.
|
||||
# CCCL target = cc_to_min_bytes_in_flight(cc): BI-V100 ≈ lower tier.
|
||||
@@ -1561,6 +1561,8 @@ class Qwen3_5ForCausalLM(nn.Module, HasInnerState, SupportsLoRA):
|
||||
if name.endswith(".bias") and name not in params_dict:
|
||||
continue
|
||||
if name not in params_dict:
|
||||
logger.warning("Skipped weight %s (not in params_dict, "
|
||||
"shape=%s)", name, loaded_weight.shape)
|
||||
continue
|
||||
param = params_dict[name]
|
||||
weight_loader = getattr(param, "weight_loader",
|
||||
@@ -1706,6 +1708,8 @@ class Qwen3_5MoeForCausalLM(Qwen3_5ForCausalLM):
|
||||
break
|
||||
else:
|
||||
if name not in params_dict:
|
||||
logger.warning("MoE: Skipped weight %s (not in params_dict, "
|
||||
"shape=%s)", name, loaded_weight.shape)
|
||||
continue
|
||||
param = params_dict[name]
|
||||
weight_loader = getattr(param, "weight_loader", default_weight_loader)
|
||||
|
||||
Reference in New Issue
Block a user