[main][Bugix] fix kv pcp+pooling+pd separation bug (#6153)
### What this PR does / why we need it? Rectify the problem that the pcp and pd separation and kv pooling scenario. In the pooling scenario, multi_nodes_meta_mapping is empty. As a result, an error is reported when the remote_host information is obtained through the get_remote_port_send_num method. ### Does this PR introduce _any_ user-facing change? No - vLLM version: v0.13.0 - vLLM main: https://github.com/vllm-project/vllm/commit/d68209402ddab3f54a09bc1f4de9a9495a283b60 Signed-off-by: weiguihua2 <weiguihua2@huawei.com>
This commit is contained in:
1 parent
ff63626874
commit
4173255c0c
1 file changed
+5
-2
@@ -1472,8 +1472,11 @@ class MooncakeConnectorWorker:
|
||||
) -> dict[int, RemotePortInfo]:
|
||||
remote_port_send_num: dict[int, RemotePortInfo] = {}
|
||||
for port in range(self._prefill_tp_size * meta.remote_pcp_size):
|
||||
remote_host = str(meta.remote_multi_nodes_meta_mapping[str(
|
||||
port)]['host'])
|
||||
remote_host_info = meta.remote_multi_nodes_meta_mapping.get(str(port), None)
|
||||
if remote_host_info is None:
|
||||
remote_host = meta.remote_host
|
||||
else:
|
||||
remote_host = remote_host_info['host']
|
||||
remote_port_send_num[meta.remote_port + port] = {
|
||||
'num': 0,
|
||||
'host': remote_host
|
||||
|
||||
Reference in new issue
Block a user