c08364f761
### What this PR does / why we need it?
When using Mooncake on Ascend NPU, AscendDirectTransport randomly
allocates ports within range `[20000, 20000 + npu_per_node × 1000)`.
Reference:
[ascend_direct_transport.cpp#L554](https://github.com/kvcache-ai/Mooncake/blob/v0.3.7.post2/mooncake-transfer-engine/src/transport/ascend_transport/ascend_direct_transport/ascend_direct_transport.cpp#L475)
If `kv_port` overlaps with this range, users may encounter intermittent
startup failures:
```bash
zmq.error.ZMQError: Address already in use (addr='tcp://x.x.x.x:30012')
RuntimeError: KV Cache sending/receiving thread failed to start.
```
This pr fix intermittent kv_port conflict with AscendDirectTransport in
`Qwen3-235B-W8A8-EPLB.yaml`, and add Added `kv_port Configuration Guide`
section in `pd_disaggregation_mooncake_multi_node.md`.
test
Results(tests/e2e/nightly/multi_node/config/Qwen3-235B-W8A8-EPLB.yaml):
https://github.com/vllm-project/vllm-ascend/actions/runs/21540138907/job/62073265259
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
- vLLM version: v0.14.1
- vLLM main:
dc917cceb8
Signed-off-by: Meihan-chen <jcccx.cmh@gmail.com>
92 lines
2.7 KiB
YAML
92 lines
2.7 KiB
YAML
test_name: "test Qwen3-235B-A22B-W8A8 disaggregated_prefill"
|
|
model: "vllm-ascend/Qwen3-235B-A22B-W8A8"
|
|
num_nodes: 2
|
|
npu_per_node: 16
|
|
env_common:
|
|
HCCL_OP_EXPANSION_MODE: AIV
|
|
VLLM_USE_MODELSCOPE: true
|
|
TASK_QUEUE_ENABLE: 1
|
|
OMP_PROC_BIND: false
|
|
OMP_NUM_THREADS: 1
|
|
HCCL_BUFFSIZE: 1024
|
|
SERVER_PORT: 8080
|
|
DYNAMIC_EPLB: true
|
|
disaggregated_prefill:
|
|
enabled: true
|
|
prefiller_host_index: [0]
|
|
decoder_host_index: [1]
|
|
|
|
deployment:
|
|
-
|
|
server_cmd: >
|
|
vllm serve "vllm-ascend/Qwen3-235B-A22B-W8A8"
|
|
--host 0.0.0.0
|
|
--port $SERVER_PORT
|
|
--data-parallel-size 2
|
|
--data-parallel-size-local 2
|
|
--tensor-parallel-size 8
|
|
--seed 1024
|
|
--enable-expert-parallel
|
|
--max-num-seqs 16
|
|
--max-model-len 8192
|
|
--max-num-batched-tokens 8192
|
|
--quantization ascend
|
|
--trust-remote-code
|
|
--no-enable-prefix-caching
|
|
--gpu-memory-utilization 0.9
|
|
--kv-transfer-config
|
|
'{"kv_connector": "MooncakeConnectorV1",
|
|
"kv_role": "kv_producer",
|
|
"kv_port": "36000",
|
|
"engine_id": "0",
|
|
"kv_connector_extra_config": {
|
|
"prefill": {
|
|
"dp_size": 2,
|
|
"tp_size": 8
|
|
},
|
|
"decode": {
|
|
"dp_size": 2,
|
|
"tp_size": 8
|
|
}
|
|
}
|
|
}'
|
|
--additional-config
|
|
'{"eplb_config": {"dynamic_eplb":true,"expert_heat_collection_interval":2048,"algorithm_execution_interval":200}}'
|
|
|
|
-
|
|
server_cmd: >
|
|
vllm serve "vllm-ascend/Qwen3-235B-A22B-W8A8"
|
|
--host 0.0.0.0
|
|
--port $SERVER_PORT
|
|
--data-parallel-size 2
|
|
--data-parallel-size-local 2
|
|
--tensor-parallel-size 8
|
|
--seed 1024
|
|
--quantization ascend
|
|
--max-num-seqs 16
|
|
--max-model-len 8192
|
|
--max-num-batched-tokens 8192
|
|
--enable-expert-parallel
|
|
--trust-remote-code
|
|
--no-enable-prefix-caching
|
|
--gpu-memory-utilization 0.9
|
|
--kv-transfer-config
|
|
'{"kv_connector": "MooncakeConnectorV1",
|
|
"kv_role": "kv_consumer",
|
|
"kv_port": "36100",
|
|
"engine_id": "1",
|
|
"kv_connector_extra_config": {
|
|
"prefill": {
|
|
"dp_size": 2,
|
|
"tp_size": 8
|
|
},
|
|
"decode": {
|
|
"dp_size": 2,
|
|
"tp_size": 8
|
|
}
|
|
}
|
|
}'
|
|
--additional-config
|
|
'{"eplb_config": {"dynamic_eplb":true,"expert_heat_collection_interval":2048,"algorithm_execution_interval":200}}'
|
|
benchmarks:
|