[Improvements] Merge health check route (#8444)
Signed-off-by: ybyang <ybyang7@iflytek.com> Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com> Co-authored-by: Kan Wu <wukanustc@gmail.com>
This commit is contained in:
@@ -1781,6 +1781,9 @@ class Scheduler(
|
||||
elif batch.forward_mode.is_dummy_first():
|
||||
self.set_next_batch_sampling_info_done(batch)
|
||||
|
||||
self.maybe_send_health_check_signal()
|
||||
|
||||
def maybe_send_health_check_signal(self):
|
||||
if self.return_health_check_ct:
|
||||
# Return some signal for the health check.
|
||||
# This is used to prevent the health check signal being blocked by long context prefill.
|
||||
|
||||
Reference in New Issue
Block a user