From cf7824313fe2218a8de7d77e87975c574f425b33 Mon Sep 17 00:00:00 2001
From: project6-dev <dev@project6.ai>
Date: Thu, 13 Aug 2026 13:24:36 +0000
Subject: [PATCH] fix(OOM): add --num-gpu-blocks-override=4000 to skip
 profiling
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Profiling OOM: flash_attn_varlen_func allocates large temp buffers during
profile_run with 8192 dummy tokens. patch_worker_profile_override.py already
has skip logic when num_gpu_blocks_override is set.

4000 blocks × 16 tokens/block = 64K token KV capacity.
With max-model-len=80000 and prefix caching, this is sufficient.
Sub 168 reference: ran 262K context on 0.95 util without override because
base image profiling doesn't use flash_attn_varlen (uses native xformers).
---
 computility-run.yaml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/computility-run.yaml b/computility-run.yaml
index 0d4bb511..9c15aac6 100644
--- a/computility-run.yaml
+++ b/computility-run.yaml
@@ -32,6 +32,8 @@ command:
     - --enforce-eager
     - --dtype
     - half
+    - --num-gpu-blocks-override
+    - '4000'
 env:
     - name: VLLM_ENGINE_ITERATION_TIMEOUT_S
       value: 3600