project6 83192486d3 perf(deltanet): CCCL thrust::all_of early termination for NaN detection
Full translation of thrust/benchmarks/bench/all_of/basic.cu pattern:

thrust::all_of uses short-circuit evaluation — once a mismatch is found,
it stops scanning. The benchmark's MismatchAt parameter (0.01, 0.5, 1.0)
shows that early detection at position 1% saves reading the other 99%.

Translation to NaN checks in GatedDeltaNet prefill/decode:
OLD: torch.isnan(result).any() — full tensor scan always (creates bool
tensor of same size, then reduces). If NaN found, does ANOTHER full scan
for mean(), then ANOTHER for nan_to_num. = 3 full passes.

NEW: Sample first 64 + last 64 elements. If sample is clean, skip all
3 full passes (the common case after overflow_cast clamp fix).
If sample detects NaN, proceed with full nan_to_num.

For decode (num_seqs=1, hidden_dim=2560): out has 2560 elements.
Sample check: 128 elements = 5% of tensor.
For prefill (seq_len=18K, hidden_dim=2560): result has 46M elements.
Sample check: 128 elements = 0.0003% of tensor.

On the happy path (no NaN), this eliminates O(N) work per layer per step.
2026-08-07 09:10:05 +00:00
2026-08-07 08:43:51 +00:00
2026-07-30 17:03:23 +08:00
2026-08-07 07:47:01 +00:00

project_6

Description
No description provided
Readme 427 MiB
Languages
C++ 41.8%
Cuda 31.6%
Python 22.2%
C 2.1%
CMake 1.1%
Other 1.1%