EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Fix the device error when using ray as vllm-acend backend (#884)

Browse Source 
1. Remove RAY_EXPERIMENTAL_NOSET_ASCEND_RT_VISIBLE_DEVICES
2. Add lazy init for vllm_ascend_C

Signed-off-by: zhuo97 <1103045176@qq.com>
This commit is contained in:
zhuo97
2025-06-16 21:03:16 +08:00
committed by GitHub
parent 69b817ed65
commit f5404dc650
7 changed files with 40 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
vllm_ascend/ops/rotary_embedding.py
View File

@@ -22,11 +22,12 @@ import torch
from vllm.model_executor.layers.rotary_embedding import (
DeepseekScalingRotaryEmbedding, RotaryEmbedding)
from vllm_ascend.platform import CUSTOM_OP_ENABLED
from vllm_ascend.utils import enable_custom_op
def custom_rotary_embedding_enabled(query, neox_style, head_size):
return query.dtype == torch.float16 and neox_style and head_size % 32 == 0 and CUSTOM_OP_ENABLED
return query.dtype == torch.float16 and neox_style and head_size % 32 == 0 and enable_custom_op(
)
def rope_forward_oot(