dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

fix: num-gpu-blocks-override 4000→5500 — 4000 blocks can't fit a 70K prompt

Browse Source 
5500 blocks × 16 tokens = 88K token capacity.
Covers max-model-len=80000 + output generation headroom.
This commit is contained in:
project6-dev
2026-08-13 13:25:23 +00:00
parent cf7824313f
commit 71644e1530
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
computility-run.yaml
View File

@@ -33,7 +33,7 @@ command:
- --dtype - --dtype
- half - half
- --num-gpu-blocks-override - --num-gpu-blocks-override
- '4000' - '5500'
env: env:
- name: VLLM_ENGINE_ITERATION_TIMEOUT_S - name: VLLM_ENGINE_ITERATION_TIMEOUT_S
value: 3600 value: 3600