EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

feat: support torchair graph mode in v1 engine (#789)

Browse Source 
### What this PR does / why we need it?
support torchair graph mode with v1 engine

---------

Signed-off-by: boying <897013703@qq.com>
This commit is contained in:
NeverRaR
2025-05-12 19:14:07 +08:00
committed by GitHub
parent 4a2505f81f
commit efabd722eb
5 changed files with 585 additions and 82 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
vllm_ascend/platform.py
View File

@@ -153,9 +153,9 @@ class NPUPlatform(Platform):
"enable_graph_mode is not supported because the version of torch is too low, forcing close enable_graph_mode"
)
vllm_config.additional_config["enable_graph_mode"] = False
if enable_graph_mode and envs.VLLM_USE_V1:
if enable_graph_mode and envs.VLLM_USE_V1 and envs.VLLM_MLA_DISABLE:
logger.warning(
"NPU graph mode is still experimental and not supported for V1 currently, "
"NPU graph mode is still experimental and not supported for V1 without mla currently, "
"it has been disabled automatically.")
vllm_config.additional_config["enable_graph_mode"] = False