feat(CCCL): LD_PRELOAD CachingDeviceAllocator — intercept cudaMalloc/cudaFree
Route C: replace PyTorch's cudaMalloc/cudaFree with CCCL CUB's CachingDeviceAllocator via LD_PRELOAD. Eliminates driver-level allocation overhead by reusing freed GPU memory from a bin-based cache. Based on cccl_upstream/cub/cub/util_allocator.cuh (901 lines). Self-contained .so with no CCCL header dependencies at compile time. Files: - cccl_preload_allocator.cu: the allocator (405 lines) - build_cccl_preload_allocator.sh: build script (corex clang++ or g++ fallback) - test_cccl_preload.sh: smoke test suite for BI-V100 - patch_ops.sh: build during docker build - computility-run.yaml: LD_PRELOAD env var for runtime Config via env: CCCL_ALLOC_BIN_GROWTH=8, MIN_BIN=3, MAX_BIN=13, MAX_CACHED_MB=4096 Test on real machine: cd qwen3_6_scripts && bash test_cccl_preload.sh
This commit is contained in:
@@ -246,6 +246,10 @@ if source != installed:
|
||||
raise SystemExit("runtime api_server overlay identity mismatch")
|
||||
PY
|
||||
|
||||
build_stage "compiling CCCL CachingDeviceAllocator LD_PRELOAD module"
|
||||
bash ./build_cccl_preload_allocator.sh /workspace/qwen3_6_scripts || \
|
||||
echo "[WARN] CCCL preload allocator build failed — will use default allocator"
|
||||
|
||||
build_stage "compiling CoreX CUDA extensions (moe_index_combine + gdn_chunk_recurrent)"
|
||||
if [[ -x /usr/local/corex-3.2.3/bin/clang++ ]]; then
|
||||
bash ./build_corex_moe_index_combine.sh "${VLLM_ROOT}" || \
|
||||
|
||||
Reference in New Issue
Block a user