feat(CCCL): LD_PRELOAD CachingDeviceAllocator — intercept cudaMalloc/cudaFree

Route C: replace PyTorch's cudaMalloc/cudaFree with CCCL CUB's
CachingDeviceAllocator via LD_PRELOAD. Eliminates driver-level allocation
overhead by reusing freed GPU memory from a bin-based cache.

Based on cccl_upstream/cub/cub/util_allocator.cuh (901 lines).
Self-contained .so with no CCCL header dependencies at compile time.

Files:
- cccl_preload_allocator.cu: the allocator (405 lines)
- build_cccl_preload_allocator.sh: build script (corex clang++ or g++ fallback)
- test_cccl_preload.sh: smoke test suite for BI-V100
- patch_ops.sh: build during docker build
- computility-run.yaml: LD_PRELOAD env var for runtime

Config via env:
  CCCL_ALLOC_BIN_GROWTH=8, MIN_BIN=3, MAX_BIN=13, MAX_CACHED_MB=4096

Test on real machine:
  cd qwen3_6_scripts && bash test_cccl_preload.sh
This commit is contained in:
Claude
2026-08-13 09:21:45 +00:00
parent a1ae6e366f
commit 327c2c9044
5 changed files with 639 additions and 0 deletions

View File

@@ -246,6 +246,10 @@ if source != installed:
raise SystemExit("runtime api_server overlay identity mismatch")
PY
build_stage "compiling CCCL CachingDeviceAllocator LD_PRELOAD module"
bash ./build_cccl_preload_allocator.sh /workspace/qwen3_6_scripts || \
echo "[WARN] CCCL preload allocator build failed — will use default allocator"
build_stage "compiling CoreX CUDA extensions (moe_index_combine + gdn_chunk_recurrent)"
if [[ -x /usr/local/corex-3.2.3/bin/clang++ ]]; then
bash ./build_corex_moe_index_combine.sh "${VLLM_ROOT}" || \