fix: revert max-num-seqs to 1 — segfault during profile_run with n=2
Profile run with 2 sequences triggers segfault in torch.distributed.gather during determine_num_available_blocks. t2_n_2 is worth 1 point, not worth the crash risk. max_completion_tokens fix is the priority.
This commit is contained in:
@@ -15,7 +15,7 @@ command:
|
||||
- -tp
|
||||
- '4'
|
||||
- --max-num-seqs
|
||||
- '2'
|
||||
- '1'
|
||||
- --disable-log-requests
|
||||
- --disable-frontend-multiprocessing
|
||||
- --max-num-batched-tokens
|
||||
|
||||
Reference in New Issue
Block a user