[main][Bugix] fix kv pcp+pooling+pd separation bug (#6153)
### What this PR does / why we need it?
Rectify the problem that the pcp and pd separation and kv pooling
scenario.
In the pooling scenario, multi_nodes_meta_mapping is empty. As a result,
an error is reported when the remote_host information is obtained
through the get_remote_port_send_num method.
### Does this PR introduce _any_ user-facing change?
No
- vLLM version: v0.13.0
- vLLM main:
d68209402d
Signed-off-by: weiguihua2 <weiguihua2@huawei.com>
This commit is contained in:
@@ -1472,8 +1472,11 @@ class MooncakeConnectorWorker:
|
||||
) -> dict[int, RemotePortInfo]:
|
||||
remote_port_send_num: dict[int, RemotePortInfo] = {}
|
||||
for port in range(self._prefill_tp_size * meta.remote_pcp_size):
|
||||
remote_host = str(meta.remote_multi_nodes_meta_mapping[str(
|
||||
port)]['host'])
|
||||
remote_host_info = meta.remote_multi_nodes_meta_mapping.get(str(port), None)
|
||||
if remote_host_info is None:
|
||||
remote_host = meta.remote_host
|
||||
else:
|
||||
remote_host = remote_host_info['host']
|
||||
remote_port_send_num[meta.remote_port + port] = {
|
||||
'num': 0,
|
||||
'host': remote_host
|
||||
|
||||
Reference in New Issue
Block a user