Files
xc-llm-ascend/vllm_ascend
Cao Yi aef9d4249d [Perf] Avoid CPU sync in mrope_positions copy by using full tensor copy (#7014)
### What this PR does / why we need it?

The index-select operation `mrope_positions.gpu[:,
:total_num_scheduled_tokens].copy_(...)` triggers a CPU-NPU
synchronization, which blocks subsequent operator dispatch and causes
bubbles visible in Profiling.

This PR changes to full tensor copy
(`mrope_positions.gpu.copy_(mrope_positions.cpu)`) to eliminate the sync
point. The trade-off is a negligible increase in memory usage since
`mrope_positions.cpu` is a small tensor.

**Result:** ~2-3% TPOT improvement with the profiling bubbles
eliminated.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Verified via Profiling that the CPU sync bubble is eliminated and TPOT
is reduced by 2-3%.
- vLLM version: v0.16.0
- vLLM main:
15d76f74e2

Signed-off-by: SlightwindSec <slightwindsec@gmail.com>
Co-authored-by: wanghuanjun2113 <wanghuanjun2113@gmail.com>
2026-03-09 14:46:37 +08:00
..