[bugfix] fix the complex and potentially problematic generate_kv_idx. (#5957)
### What this PR does / why we need it?
In long-sequence scenarios, the chunked-prefill component may encounter
dimension misalignment issues, which previously occurred during
precision testing on the code_generate_lite dataset. This PR removes
redundant computations and instead derives the value using existing
results and straightforward calculations.
- vLLM version: v0.13.0
- vLLM main:
2c24bc6996
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
This commit is contained in:
@@ -555,9 +555,6 @@ class NPUModelRunner(GPUModelRunner):
|
||||
self.num_spec_tokens)
|
||||
|
||||
if self.pcp_size > 1:
|
||||
if not self.vllm_config.model_config.use_mla:
|
||||
self.pcp_manager.generate_kv_idx(scheduler_output,
|
||||
self.input_batch)
|
||||
num_scheduled_tokens[:
|
||||
num_reqs], position_pcp = self.pcp_manager.update_tokens_for_pcp(
|
||||
num_scheduled_tokens[:num_reqs],
|
||||
|
||||
Reference in New Issue
Block a user