[Feature] adapt to uva buffer and main2main (#6657)
### What this PR does / why we need it?
vllm model runner v2 use uva buffer to prepare input data, but npu
doesn't support uva yet, this pr implement a uvawrapper class to mimic
gpu's uva backend. what's more, this pr make some modifications to adapt
to the newer main branch.
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
- vLLM main:
13397841ab
---------
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
This commit is contained in:
@@ -263,3 +263,12 @@
|
||||
# https://gitcode.com/Ascend/torchair/pull/2575
|
||||
# Future Plan:
|
||||
# Remove this patch when the PTA version used by vllm-ascend has been upgraded.
|
||||
# ** 14. File: worker/patch_v2_uva.py**
|
||||
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
# 1. `vllm.v1.worker.gpu.states.UvaBuffer`
|
||||
# Why:
|
||||
# ASCEND NPUs do not support UVA yet, so we need to wrap it in vLLM.
|
||||
# How:
|
||||
# make UvaBuffer a dummy class, mimic the interface of vllm UvaBuffer.
|
||||
# Future Plan:
|
||||
# Remove this patch when NPU support UVA.
|
||||
|
||||
Reference in New Issue
Block a user