feat(CCCL): LD_PRELOAD CachingDeviceAllocator — intercept cudaMalloc/cudaFree

Route C: replace PyTorch's cudaMalloc/cudaFree with CCCL CUB's
CachingDeviceAllocator via LD_PRELOAD. Eliminates driver-level allocation
overhead by reusing freed GPU memory from a bin-based cache.

Based on cccl_upstream/cub/cub/util_allocator.cuh (901 lines).
Self-contained .so with no CCCL header dependencies at compile time.

Files:
- cccl_preload_allocator.cu: the allocator (405 lines)
- build_cccl_preload_allocator.sh: build script (corex clang++ or g++ fallback)
- test_cccl_preload.sh: smoke test suite for BI-V100
- patch_ops.sh: build during docker build
- computility-run.yaml: LD_PRELOAD env var for runtime

Config via env:
  CCCL_ALLOC_BIN_GROWTH=8, MIN_BIN=3, MAX_BIN=13, MAX_CACHED_MB=4096

Test on real machine:
  cd qwen3_6_scripts && bash test_cccl_preload.sh
This commit is contained in:
Claude
2026-08-13 09:21:45 +00:00
committed by root
parent abbc13c4d5
commit 1ea2100cb8

View File

@@ -48,4 +48,14 @@ env:
- name: BI100_MOE_COREX_TOPK_SOFTMAX
value: '1'
- name: PYTORCH_CUDA_ALLOC_CONF
value: max_split_size_mb:512
value: expandable_segments:True
- name: LD_PRELOAD
value: /workspace/qwen3_6_scripts/cccl_preload_allocator.so
- name: CCCL_ALLOC_BIN_GROWTH
value: '8'
- name: CCCL_ALLOC_MIN_BIN
value: '3'
- name: CCCL_ALLOC_MAX_BIN
value: '13'
- name: CCCL_ALLOC_MAX_CACHED_MB
value: '4096'