Based on a3c45d3b (last known working docker build):
- yaml: max-num-seqs=2 (fixes t2_n_2), TOPK_SOFTMAX=1 (use prebuilt .so)
- yaml: keep max-model-len=131072, gpu-mem=0.90 (prevents OOM)
- yaml: NO LD_PRELOAD (libcccl not built during docker build)
- xformers: revert to Q-tiling only (flash_attn caused OOM at profiling)
- .dockerignore: exclude all non-essential files from context
- remove libcccl_allocator.so from git tracking
What stays from recent work:
- 14 prebuilt .so (including corex_gdn_chunk_recurrent)
- qwen3_5.py with .float() fix and chunk_recurrent support
- All vendor_overrides and CCCL preload source (for future use)
commit 4c365b8c added 1106 CCCL device-level headers (294K lines) to
qwen3_6_scripts/cccl_preload/include/. These are NOT used by the allocator
preload (which only needs cub/util_allocator.cuh + 288 transitive deps)
and cannot compile on corex CUDA 10.2 anyway.
The extra headers doubled docker context from 15MB to 31MB, likely
causing platform build timeout or size limit failure.
Restoring to the original 288-header set that is proven to compile and run.
Two problems from real BI-V100 build:
1. 'CUDA versions below 12 are not supported'
→ Add CCCL_IGNORE_DEPRECATED_CUDA_BELOW_12 (official suppress macro)
2. corex thrust/complex.h conflicts with CCCL thrust headers
→ Split into two compilation units:
- cccl_moe_sort_scatter.cu: CCCL headers only, C API, no torch
- cccl_moe_sort_scatter_pybind.cpp: torch headers only, no CCCL
Same pattern as proven cccl_allocator_preload.cu
3. Variadic device functions rejected by corex clang:
→ is_referenceable.h: __test(...) → __test(long)
→ invoke.h: __any(...) → template __any(_T)
→ conjunction.h: __and_helper(...) → __and_helper(long)
SFINAE still works: int overload wins, long is fallback.
CUB CachingDeviceAllocator::DeviceAllocate calls cudaMalloc internally
on cache miss. Without a guard, our intercepted cudaMalloc recurses
into DeviceAllocate → cudaMalloc → DeviceAllocate → segfault.
thread_local g_in_allocator flag detects reentrant calls and forwards
them directly to the real cudaMalloc/cudaFree via dlsym(RTLD_NEXT).
CUB_DISABLE_NAMESPACE_MAGIC requires CUB_WRAPPED_NAMESPACE.
CUB_WRAPPED_NAMESPACE=cccl_preload wraps cub into cccl_preload::cub.
Source must use cccl_preload::cub::CachingDeviceAllocator.
CUB_WRAPPED_NAMESPACE=cccl_preload wraps cub into cccl_preload::cub
but cccl_allocator_preload.cu uses bare cub:: — compilation fails.
_CCCL_COMPILER_GCC=1 conflicts with CCCL auto-detection (redefined warning).
Drop both. CUB_DISABLE_NAMESPACE_MAGIC alone is sufficient.
