dylanyunlon 1c9ac93fee [ENGINE+TEST] 2 changes from CCCL random source reading ...

1. model_runner.py: CCCL CachingDeviceAllocator (example_device_radix_sort.cu)
   → CUDA graph capture 1028→19 sizes, saves ~50GB memory + 50s startup

2. verify_functional.py: TC-22→TC-30 from CCCL dispatch_segmented_reduce.cuh
   - TC-22/23: Unicode fidelity (Chinese/Japanese exact repeat)
   - TC-24: n=2 multiple choices (segmented output)
   - TC-25/26: Error handling (empty body, missing role)
   - TC-27/28: Sampling boundary (top_k=1, temperature=2.0)
   - TC-29/30: Endpoint health (/v1/models, /health)
   Total: 21→30 test cases (target: 50+ for competition)

CCCL sources read this round:
  cub/examples/device/example_device_radix_sort.cu → DoubleBuffer + CachingDeviceAllocator
  cudax/test/multi_gpu/concepts/has_gather_v.cu → TP gather pattern
  cub/cub/device/dispatch/dispatch_segmented_reduce.cuh → 3-tier policy (large/medium/small)

2026-08-07 02:01:06 +00:00

..

qwen3_5

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

qwen3_5_moe

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

reasoning

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

_custom_ops.py

[CRITICAL/deploy] fix 3 deployment gaps found from docker crash log

2026-08-06 07:01:14 +00:00

api_server.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

arg_utils.py

[ARCH] Eliminate AST patch scripts — full file replacements only

2026-08-05 08:24:43 +00:00

chat_utils.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

cli_args.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

logits_processor.py

[ARCH] Eliminate AST patch scripts — full file replacements only

2026-08-05 08:24:43 +00:00

mamba_cache.py

[fix/correctness] mamba_cache: safe swap via clone, not in-place fancy indexing

2026-08-06 06:33:26 +00:00

model_runner.py

[ENGINE] model_runner.py: CCCL CachingDeviceAllocator pattern — reduce CUDA graph capture from 1028→19 sizes

2026-08-07 01:59:18 +00:00

paged_attention_v2_pytorch.py

[CRITICAL/deploy] fix 3 deployment gaps found from docker crash log

2026-08-06 07:01:14 +00:00

paged_attn.py

[critical/deploy] sync root paged_attn.py + prefix_prefill.py → qwen3_6_scripts/

2026-08-06 06:10:56 +00:00

patch_ops.sh

[CRITICAL/deploy] fix 3 deployment gaps found from docker crash log

2026-08-06 07:01:14 +00:00

patch_transformers_qwen3_5.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

prefix_prefill.py

[critical/deploy] sync root paged_attn.py + prefix_prefill.py → qwen3_6_scripts/

2026-08-06 06:10:56 +00:00

protocol.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

qwen3_5.py

[FIX] qwen3_5.py: replace solve_triangular with manual forward substitution

2026-08-06 03:02:29 +00:00

qwen3coder_tool_parser.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

registry.py

[ARCH] Eliminate 2 more patch scripts — registry.py + tool_parsers __init__.py

2026-08-05 08:36:52 +00:00

sampler.py

[BASE] qwen3_6_scripts/sampler.py: CCCL topk unsorted output optimization

2026-08-06 02:55:51 +00:00

scheduler.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

sequence.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

serving_chat.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

tool_parsers_init.py

[ARCH] Eliminate 2 more patch scripts — registry.py + tool_parsers __init__.py

2026-08-05 08:36:52 +00:00

verify_functional.py

[ENGINE+TEST] 2 changes from CCCL random source reading

2026-08-07 02:01:06 +00:00

xformers.py

feat(xformers): replace Q-only tiling with Q+KV tiling + online softmax

2026-08-07 01:54:52 +00:00