fix(OOM): cap GPU blocks at 5000 via BI100_MAX_GPU_BLOCKS env var

Profiling zeros-out attention → vllm overestimates free memory → 7942 blocks
→ first real request OOMs. Cap at 5000 (80K tokens / 16 block_size).

patch_block_major_worker_capacity.py reads BI100_MAX_GPU_BLOCKS from env,
caps num_gpu_blocks after reserve_block_major_gpu_blocks.
This commit is contained in:
project6-dev
2026-08-14 00:12:39 +00:00
parent c6aa1b9c62
commit 456380eed0
2 changed files with 10 additions and 0 deletions

View File

@@ -49,3 +49,5 @@ env:
value: '1'
- name: PYTORCH_CUDA_ALLOC_CONF
value: max_split_size_mb:512
- name: BI100_MAX_GPU_BLOCKS
value: '5000'