dylanyunlon 025059d78e fix(critical): raise decode threshold to prevent service crash
CCCL GridEvenShare principle: each work unit must complete within
bounded time. Python fallback decode was O(seq_len) per step —
at seq_len > 32K, each decode step took seconds, causing HTTP
timeout and service crash during case_truncation (max_tokens=8192).

Raised _PYTORCH_DECODE_THRESHOLD from 32768 to 999999 to force
all decode through ixformer native paged_attention_v1 kernel,
which is O(1) per decode step regardless of sequence length.

Competition submission Job 101 crashed at case_truncation phase
with RemoteDisconnected. Job 66 (competitor) passed this phase
using native kernel at all lengths. Root cause confirmed:
Python fallback too slow for production use.

Also derived from CCCL grid_even_share.cuh DispatchInit:
  big_share_items = normal_share_items + tile_items (at most +1 tile)
  Never let any block take unbounded work.
2026-08-07 04:39:19 +00:00
2026-07-30 17:03:23 +08:00

project_6

Description
No description provided
Readme 428 MiB
Languages
C++ 41.8%
Cuda 31.6%
Python 22.2%
C 2.1%
CMake 1.1%
Other 1.1%