[Bugfix] fix pcp + eplb error (#5561)
### What this PR does / why we need it?
Fix the bug in the PCP overlay feature
1、Fix the bug related to PCP and EPLB overlap by including PCP size in
the word_size calculation.
2、In the PCP pooling scenario, a prompt has been added for setting the
cp_kv_cache_interleave_size.
- vLLM version: v0.13.0
- vLLM main:
7157596103
Signed-off-by: weiguihua2 <weiguihua2@huawei.com>
This commit is contained in:
@@ -329,7 +329,7 @@ class NPUPlatform(Platform):
|
||||
raise AssertionError(
|
||||
f"cp_kv_cache_interleave_size({parallel_config.cp_kv_cache_interleave_size}) "
|
||||
f"and block_size({cache_config.block_size}) "
|
||||
"needs to be equal if use cp or dcp > 1 in P/D disaggregate scenario."
|
||||
"needs to be equal if use pcp or dcp > 1 in P/D disaggregate and kv pool scenario."
|
||||
)
|
||||
|
||||
if is_vl_model(vllm_config):
|
||||
|
||||
Reference in New Issue
Block a user