[Bugfix] Fix tokenizer_manager not getting 400 when req is too long (#3678)

Co-authored-by: voidxb <unkown>
This commit is contained in:
Chang Su
2025-02-27 22:59:43 -08:00
committed by GitHub
parent 90bc26a813
commit eec3f6d1eb
2 changed files with 22 additions and 4 deletions

View File

@@ -683,6 +683,8 @@ class Scheduler:
self.server_args.allow_auto_truncate,
)
if error_msg:
req.origin_input_ids = [0]
req.sampling_params.max_new_tokens = 0
self.waiting_queue.append(req)
return