68be2ff856e3ea4e7130796eefe9725749afdf48
CCCL source: cub/device/dispatch/dispatch_radix_sort.cuh (2070 lines) Core pattern applied: problem-size-based dispatch routing. dispatch_radix_sort routes to invoke_single_tile / invoke_onesweep / invoke_passes based on num_items vs tile_items. Same principle applied to request dispatch: 1. protocol.py: when max_tokens <= 128, disable thinking (small-tile path). Fixes t3_max_tokens_1 and t3_max_tokens_64 — model was spending all tokens on <think>...</think> leaving content empty, giving finish_reason=stop instead of expected finish_reason=length. 2. serving_chat.py: pre-clamp request.max_tokens to available context space BEFORE passing to engine. Fixes t3_max_tokens_max — engine was rejecting with HTTP 400 because max_tokens > (max_model_len - prompt_len). 3. serving_chat.py: guard default_max_tokens >= 1 for edge cases where prompt fills entire context window. Sub168 failed exactly these 3 tests plus d06_cache_hit (engine-level). These fixes target 3 of the 4 remaining failures.
project_6
Description
Languages
C++
41.8%
Cuda
31.6%
Python
22.2%
C
2.1%
CMake
1.1%
Other
1.1%