[P/D] force with_prefill true after allreduce in kv producer (#3768)
### What this PR does / why we need it?
force with_prefill true after allreduce in kv producer
- vLLM version: v0.11.0rc3
- vLLM main:
c9461e05a4
---------
Signed-off-by: liziyu <liziyu16@huawei.com>
This commit is contained in:
@@ -1151,7 +1151,7 @@ class MooncakeLayerwiseConnectorWorker:
|
||||
connector_metadata: MooncakeLayerwiseConnectorMetadata,
|
||||
**kwargs) -> None:
|
||||
"""MooncakeLayerwiseConnector does not save explicitly."""
|
||||
if self.kv_role == 'kv_producer':
|
||||
if self.kv_role == 'kv_producer' and connector_metadata.request.keys():
|
||||
if self.pd_head_ratio != 1:
|
||||
if self.current_layer != 0:
|
||||
self.completion_event.wait()
|
||||
|
||||
Reference in New Issue
Block a user