[BugFix][P/D] Modify the recalculation logic to prevent waiting requests from filling up the D node KVCache (#3641)
### What this PR does / why we need it?
Modify the recalculation logic to prevent waiting requests from filling
up the D node KVCache
- vLLM version: v0.11.0rc3
- vLLM main:
17c540a993
Signed-off-by: underfituu <hzhucong@163.com>
This commit is contained in:
@@ -362,7 +362,7 @@ class RecomputeScheduler(SchedulerInterface):
|
||||
skipped_waiting_requests = create_request_queue(self.policy)
|
||||
|
||||
# Next, schedule the WAITING requests.
|
||||
if not preempted_reqs:
|
||||
if not preempted_reqs and not recomputed_reqs:
|
||||
while self.waiting and token_budget > 0:
|
||||
if len(self.running) == self.max_num_running_reqs:
|
||||
break
|
||||
|
||||
Reference in New Issue
Block a user