fix: lower gpu-memory-utilization to 0.85 — prevent profiling OOM

CCCL CachingDeviceAllocator reserves some memory for its bin cache.
With 0.90 utilization + max-model-len=80000, profiling stage OOMs.
0.85 leaves ~1.6GB headroom per GPU for profiling + allocator cache.
This commit is contained in:
Claude
2026-08-13 12:38:42 +00:00
parent 6092edebde
commit 451bdc8204

View File

@@ -10,7 +10,7 @@ command:
- --max-model-len
- '80000'
- --gpu-memory-utilization
- '0.90'
- '0.85'
- --trust-remote-code
- -tp
- '4'