Claude 5b8d7c6b76 arch(critical): deploy ALL customized files to container — qwen3_5.py was NEVER running ...

ROOT CAUSE FOUND: patch_ops.sh only deployed serving-layer files
(tool_parser, reasoning, protocol, serving_chat) but NEVER deployed:

  - qwen3_5.py (1712 lines of NaN-safe DeltaNet + CCCL patterns)
  - _custom_ops.py (MoE kernel fallback for BI-V100)
  - model_runner.py (has_inner_state for DeltaNet MambaCacheManager)
  - sampler.py, sequence.py, scheduler.py, arg_utils.py
  - xformers.py, paged_attn.py, prefix_prefill.py
  - logits_processor.py, mamba_cache.py

The container was running the BASE IMAGE's original qwen3_5.py which has:
  - NO NaN clamping (g.clamp, cumsum.clamp, state.clamp)
  - NO overflow_cast protection (CCCL pattern)
  - NO forward substitution fallback (cuSOLVER unavailable on BI-V100)
  - NO batched GEMM MoE decode (3 launches vs 16)
  - NO sorted-segment MoE prefill (CCCL histogram pattern)
  - NO GDN prefix-cache state save/restore

This explains why Docker logs showed 99.98% NaN in EVERY DeltaNet layer
despite our qwen3_5.py having comprehensive numerical guards.

Also fixes:
  - serving_chat.py: n>1 returns 400 instead of clamping (prevents OOM cascade)
  - serving_chat.py: improved d07 content fallback (multi-layer extraction)

2026-08-08 05:38:32 +00:00

..

qwen3_5

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

qwen3_5_moe

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

reasoning

fix(critical): 3 fixes from sub508 diagnosis — n>1 crash guard + thinking format + content fallback

2026-08-07 07:55:04 +00:00

_custom_ops.py

fix(critical): CCCL policy_selector degradation for MoE — PyTorch fallback for topk_softmax

2026-08-07 08:56:50 +00:00

api_server.py

fix(critical): CCCL-inspired graceful degradation — prevent OOM cascade

2026-08-07 09:55:06 +00:00

arg_utils.py

[ARCH] Eliminate AST patch scripts — full file replacements only

2026-08-05 08:24:43 +00:00

chat_utils.py

fix(critical): 3 bugs causing 0.0 score — OOM crash + thinking param + multimodal

2026-08-07 07:05:40 +00:00

cli_args.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

logits_processor.py

[ARCH] Eliminate AST patch scripts — full file replacements only

2026-08-05 08:24:43 +00:00

mamba_cache.py

[fix/correctness] mamba_cache: safe swap via clone, not in-place fancy indexing

2026-08-06 06:33:26 +00:00

model_runner.py

fix(engine): CCCL overflow_cast + checked_allocator patterns for NaN/OOM

2026-08-07 08:56:36 +00:00

paged_attention_v2_pytorch.py

[ENGINE] CCCL SmemResource pattern: pre-allocate staging buffers in V2 paged attention

2026-08-07 04:37:44 +00:00

paged_attn.py

perf: increase prefix attention tile budget 96MB→256MB

2026-08-07 04:44:18 +00:00

patch_numerical_stability.py

fix(cccl): robust fallback — inject module-level safety if regex misses

2026-08-07 09:58:35 +00:00

patch_ops.sh

arch(critical): deploy ALL customized files to container — qwen3_5.py was NEVER running

2026-08-08 05:38:32 +00:00

patch_transformers_qwen3_5.py

fix(critical): remove pip install transformers — was breaking corex kernel loading

2026-08-07 10:02:09 +00:00

prefix_prefill.py

fix(critical): v2 kernel softmax normalization was commented out — outputs were wrong

2026-08-07 02:46:46 +00:00

protocol.py

fix(critical): CCCL-inspired graceful degradation — prevent OOM cascade

2026-08-07 09:55:06 +00:00

qwen3_5.py

fix(deltanet): systematic NaN elimination via CCCL overflow_cast pattern

2026-08-07 09:26:11 +00:00

qwen3coder_tool_parser.py

fix(critical): disable thinking for tool_call requests — fixes d03_tool_call FAIL

2026-08-07 07:45:28 +00:00

registry.py

[ARCH] Eliminate 2 more patch scripts — registry.py + tool_parsers __init__.py

2026-08-05 08:36:52 +00:00

sampler.py

[BASE] qwen3_6_scripts/sampler.py: CCCL topk unsorted output optimization

2026-08-06 02:55:51 +00:00

scheduler.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

sequence.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

serving_chat.py

arch(critical): deploy ALL customized files to container — qwen3_5.py was NEVER running

2026-08-08 05:38:32 +00:00

tool_parsers_init.py

[ARCH] Eliminate 2 more patch scripts — registry.py + tool_parsers __init__.py

2026-08-05 08:36:52 +00:00

verify_functional.py

fix(verify): remove duplicate TC-22~30 test definitions — CCCL test_then.cu audit

2026-08-07 06:36:12 +00:00

xformers.py

feat(xformers): replace Q-only tiling with Q+KV tiling + online softmax

2026-08-07 01:54:52 +00:00