[Bugfix] Fix tokenizer_manager not getting 400 when req is too long (#3678)
Co-authored-by: voidxb <unkown>
This commit is contained in:
@@ -683,6 +683,8 @@ class Scheduler:
|
||||
self.server_args.allow_auto_truncate,
|
||||
)
|
||||
if error_msg:
|
||||
req.origin_input_ids = [0]
|
||||
req.sampling_params.max_new_tokens = 0
|
||||
self.waiting_queue.append(req)
|
||||
return
|
||||
|
||||
|
||||
Reference in New Issue
Block a user