EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

[Communicator] Add monkey patch (#30)

Browse Source 
Some PR for plugin support is not merged by vllm yet. This PR add monkey
patch to vllm-ascend to make vllm-ascend work with vllm directly.

This patch code should be removed once the related function is supported
by vllm originally.

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
This commit is contained in:
wangxiyuan
2025-02-11 19:15:35 +08:00
committed by GitHub
parent eb189aac81
commit f762ee89cc
4 changed files with 142 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
vllm_ascend/platform.py
View File

@@ -88,8 +88,9 @@ class NPUPlatform(Platform):
@classmethod
def check_and_update_config(cls, vllm_config: VllmConfig) -> None:
# Register ops when setup.
# Register ops and patch when setup.
from vllm_ascend import ops # noqa: F401
from vllm_ascend import patch # noqa: F401
parallel_config = vllm_config.parallel_config
if parallel_config.worker_cls == "auto":