fix(OOM): add --num-gpu-blocks-override=4000 to skip profiling

Profiling OOM: flash_attn_varlen_func allocates large temp buffers during
profile_run with 8192 dummy tokens. patch_worker_profile_override.py already
has skip logic when num_gpu_blocks_override is set.

4000 blocks × 16 tokens/block = 64K token KV capacity.
With max-model-len=80000 and prefix caching, this is sufficient.
Sub 168 reference: ran 262K context on 0.95 util without override because
base image profiling doesn't use flash_attn_varlen (uses native xformers).
This commit is contained in:
project6-dev
2026-08-13 13:24:36 +00:00
parent 8d2f30f065
commit cf7824313f

View File

@@ -32,6 +32,8 @@ command:
- --enforce-eager
- --dtype
- half
- --num-gpu-blocks-override
- '4000'
env:
- name: VLLM_ENGINE_ITERATION_TIMEOUT_S
value: 3600