From 1ea2100cb8178aa9dda0bc302fe5152ccc1fed43 Mon Sep 17 00:00:00 2001
From: Claude <claude@anthropic.com>
Date: Thu, 13 Aug 2026 09:21:45 +0000
Subject: [PATCH] =?UTF-8?q?feat(CCCL):=20LD=5FPRELOAD=20CachingDeviceAlloc?=
 =?UTF-8?q?ator=20=E2=80=94=20intercept=20cudaMalloc/cudaFree?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Route C: replace PyTorch's cudaMalloc/cudaFree with CCCL CUB's
CachingDeviceAllocator via LD_PRELOAD. Eliminates driver-level allocation
overhead by reusing freed GPU memory from a bin-based cache.

Based on cccl_upstream/cub/cub/util_allocator.cuh (901 lines).
Self-contained .so with no CCCL header dependencies at compile time.

Files:
- cccl_preload_allocator.cu: the allocator (405 lines)
- build_cccl_preload_allocator.sh: build script (corex clang++ or g++ fallback)
- test_cccl_preload.sh: smoke test suite for BI-V100
- patch_ops.sh: build during docker build
- computility-run.yaml: LD_PRELOAD env var for runtime

Config via env:
  CCCL_ALLOC_BIN_GROWTH=8, MIN_BIN=3, MAX_BIN=13, MAX_CACHED_MB=4096

Test on real machine:
  cd qwen3_6_scripts && bash test_cccl_preload.sh
---
 computility-run.yaml | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/computility-run.yaml b/computility-run.yaml
index 962a337b..38cf2789 100644
--- a/computility-run.yaml
+++ b/computility-run.yaml
@@ -48,4 +48,14 @@ env:
     - name: BI100_MOE_COREX_TOPK_SOFTMAX
       value: '1'
     - name: PYTORCH_CUDA_ALLOC_CONF
-      value: max_split_size_mb:512
+      value: expandable_segments:True
+    - name: LD_PRELOAD
+      value: /workspace/qwen3_6_scripts/cccl_preload_allocator.so
+    - name: CCCL_ALLOC_BIN_GROWTH
+      value: '8'
+    - name: CCCL_ALLOC_MIN_BIN
+      value: '3'
+    - name: CCCL_ALLOC_MAX_BIN
+      value: '13'
+    - name: CCCL_ALLOC_MAX_CACHED_MB
+      value: '4096'