dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

fix: lower gpu-memory-utilization to 0.85 — prevent profiling OOM

Browse Source 
CCCL CachingDeviceAllocator reserves some memory for its bin cache.
With 0.90 utilization + max-model-len=80000, profiling stage OOMs.
0.85 leaves ~1.6GB headroom per GPU for profiling + allocator cache.
This commit is contained in:
Claude
2026-08-13 12:38:42 +00:00
parent 6092edebde
commit 451bdc8204
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
computility-run.yaml
View File

@@ -10,7 +10,7 @@ command:
- --max-model-len - --max-model-len
- '80000' - '80000'
- --gpu-memory-utilization - --gpu-memory-utilization
- '0.90' - '0.85'
- --trust-remote-code - --trust-remote-code
- -tp - -tp
- '4' - '4'