EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

[Bugfix] Fix kvpool precision synchronization (#4574)

Browse Source 
### What this PR does / why we need it?
Fix kvpool precision synchronization
Issue https://github.com/vllm-project/vllm-ascend/issues/4412


- vLLM version: v0.11.2

---------

Signed-off-by: LCAIZJ <leichao139636@163.com>
This commit is contained in:
Chao Lei
2025-11-30 09:39:07 +08:00
committed by GitHub
parent 2b3bfe432e
commit ff7061317f
2 changed files with 1 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
vllm_ascend/worker/model_runner_v1.py
View File

@@ -2339,7 +2339,6 @@ class NPUModelRunner(LoRAModelRunnerMixin):
attn_metadata, self.with_prefill, maybe_padded_num_tokens,
input_ids, positions, intermediate_tensors, inputs_embeds)
self.maybe_wait_for_kv_save()
finished_sending, finished_recving = self.get_finished_kv_transfer(
scheduler_output)
@@ -2603,7 +2602,7 @@ class NPUModelRunner(LoRAModelRunnerMixin):
# ngram and other speculative decoding methods use the sampled
# tokens on the CPU, so they are run after bookkeeping.
propose_draft_token_ids(valid_sampled_token_ids)
self.maybe_wait_for_kv_save()
if has_kv_transfer_group():
get_kv_transfer_group().clear_connector_metadata()