fix(OOM): return zeros during profiling — skip both flash_attn AND Q-tiling
flash_attn_varlen OOMs at 4096 tokens, Q-tiling also OOMs (K tensor too large). During profiling (BI100_IN_STARTUP_PROFILE=1), return zeros immediately. Profiling only measures memory footprint, not output correctness. Restore: chunked_prefill=on, max_num_batched_tokens=4096.
This commit is contained in:
@@ -19,7 +19,7 @@ command:
|
||||
- --disable-log-requests
|
||||
- --disable-frontend-multiprocessing
|
||||
- --max-num-batched-tokens
|
||||
- '256'
|
||||
- '4096'
|
||||
- --enable-chunked-prefill
|
||||
- --max-seq-len-to-capture
|
||||
- '32768'
|
||||
|
||||
Reference in New Issue
Block a user