EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

chip type judgement code optimization (#4485)

Browse Source 
### What this PR does / why we need it?
| | cpu envir | npu envir |
|---|---|---|
| set `SOC_VERSION` | check if `SOC_VERSION` is in dict `soc_to_device`,
if not, raise an error that can not support current chip type. | print a
warning log when `SOC_VERSION` is not equal to chip type from `npu-smi`,
same as left for others. |
| not set `SOC_VERSION` | raise an error that `SOC_VERSION` is necessary
when compiling in a cpu envir. | use chip type from `npu-smi` to compile
vllm-ascend. |

### Does this PR introduce _any_ user-facing change?

Now we must set env `SOC_VERSION` when compiling in cpu envir. 

### How was this patch tested?


- vLLM version: v0.11.2
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.2

Signed-off-by: zzzzwwjj <1183291235@qq.com>
This commit is contained in:
zzzzwwjj
2025-11-27 17:18:49 +08:00
committed by GitHub
parent 84d7f5a10d
commit 1fd56b1106
17 changed files with 42 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.github/Dockerfile.buildwheel vendored
View File

@@ -18,10 +18,12 @@ ARG PY_VERSION=3.11
FROM quay.io/ascend/manylinux:8.3.rc1-910b-manylinux_2_28-py${PY_VERSION}
ARG COMPILE_CUSTOM_KERNELS=1
ARG SOC_VERSION
# Define environments
ENV DEBIAN_FRONTEND=noninteractive
ENV COMPILE_CUSTOM_KERNELS=${COMPILE_CUSTOM_KERNELS}
ENV SOC_VERSION=$SOC_VERSION
RUN yum update -y && \
yum install -y python3-pip git vim wget net-tools gcc gcc-c++ make cmake numactl-devel && \
rm -rf /var/cache/yum

1
.github/workflows/image_310p_openeuler.yml vendored
View File

@@ -132,4 +132,5 @@ jobs:
file: Dockerfile.310p.openEuler
build-args: |
PIP_INDEX_URL=https://pypi.org/simple
SOC_VERSION=ascend310p1
provenance: false

1
.github/workflows/image_310p_ubuntu.yml vendored
View File

@@ -128,4 +128,5 @@ jobs:
tags: ${{ steps.meta.outputs.tags }}
build-args: |
PIP_INDEX_URL=https://pypi.org/simple
SOC_VERSION=ascend310p1
provenance: false

1
.github/workflows/image_a3_openeuler.yml vendored
View File

@@ -131,5 +131,6 @@ jobs:
file: Dockerfile.a3.openEuler
build-args: |
PIP_INDEX_URL=https://pypi.org/simple
SOC_VERSION=ascend910_9391
provenance: false

1
.github/workflows/image_a3_ubuntu.yml vendored
View File

@@ -127,5 +127,6 @@ jobs:
tags: ${{ steps.meta.outputs.tags }}
build-args: |
PIP_INDEX_URL=https://pypi.org/simple
SOC_VERSION=ascend910_9391
provenance: false

1
.github/workflows/image_openeuler.yml vendored
View File

@@ -131,4 +131,5 @@ jobs:
file: Dockerfile.openEuler
build-args: |
PIP_INDEX_URL=https://pypi.org/simple
SOC_VERSION=ascend910b1
provenance: false

1
.github/workflows/image_ubuntu.yml vendored
View File

@@ -128,4 +128,5 @@ jobs:
tags: ${{ steps.meta.outputs.tags }}
build-args: |
PIP_INDEX_URL=https://pypi.org/simple
SOC_VERSION=ascend910b1
provenance: false

2
.github/workflows/release_code.yml vendored
View File

@@ -59,6 +59,8 @@ jobs:
python3 -m pip install twine setuptools_scm
- name: Generate tar.gz
env:
SOC_VERSION: ascend910b1
run: |
python3 setup.py sdist
ls dist

1
.github/workflows/release_whl.yml vendored
View File

@@ -69,6 +69,7 @@ jobs:
ls
docker build -f ./.github/Dockerfile.buildwheel \
--build-arg PY_VERSION=${{ matrix.python-version }} \
--build-arg SOC_VERSION=ascend910b1 \
-t wheel:v1 .
docker run --rm \
-u $(id -u):$(id -g) \

1
.github/workflows/vllm_ascend_test_pr_light.yaml vendored
View File

@@ -81,6 +81,7 @@ jobs:
env:
VLLM_LOGGING_LEVEL: ERROR
VLLM_USE_MODELSCOPE: True
SOC_VERSION: ascend910b1
strategy:
matrix:
vllm_version: [v0.11.2]

2
Dockerfile
View File

@@ -20,10 +20,12 @@ FROM quay.io/ascend/cann:8.3.rc1-910b-ubuntu22.04-py3.11
ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"
ARG COMPILE_CUSTOM_KERNELS=1
ARG MOONCAKE_TAG="v0.3.7.post2"
ARG SOC_VERSION
# Define environments
ENV DEBIAN_FRONTEND=noninteractive
ENV COMPILE_CUSTOM_KERNELS=${COMPILE_CUSTOM_KERNELS}
ENV SOC_VERSION=$SOC_VERSION
WORKDIR /workspace

2
Dockerfile.310p
View File

@@ -19,10 +19,12 @@ FROM quay.io/ascend/cann:8.3.rc1-310p-ubuntu22.04-py3.11
ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"
ARG COMPILE_CUSTOM_KERNELS=1
ARG SOC_VERSION
# Define environments
ENV DEBIAN_FRONTEND=noninteractive
ENV COMPILE_CUSTOM_KERNELS=${COMPILE_CUSTOM_KERNELS}
ENV SOC_VERSION=$SOC_VERSION
RUN apt-get update -y && \
apt-get install -y python3-pip git vim wget net-tools gcc g++ cmake libnuma-dev && \

2
Dockerfile.310p.openEuler
View File

@@ -19,8 +19,10 @@ FROM quay.io/ascend/cann:8.3.rc1-310p-openeuler24.03-py3.11
ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"
ARG COMPILE_CUSTOM_KERNELS=1
ARG SOC_VERSION
ENV COMPILE_CUSTOM_KERNELS=${COMPILE_CUSTOM_KERNELS}
ENV SOC_VERSION=$SOC_VERSION
RUN yum update -y && \
yum install -y python3-pip git vim wget net-tools gcc gcc-c++ make cmake numactl-devel && \

2
Dockerfile.a3
View File

@@ -20,11 +20,13 @@ FROM quay.io/ascend/cann:8.3.rc1-a3-ubuntu22.04-py3.11
ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"
ARG COMPILE_CUSTOM_KERNELS=1
ARG MOONCAKE_TAG=v0.3.7.post2
ARG SOC_VERSION
COPY . /vllm-workspace/vllm-ascend/
# Define environments
ENV DEBIAN_FRONTEND=noninteractive
ENV COMPILE_CUSTOM_KERNELS=${COMPILE_CUSTOM_KERNELS}
ENV SOC_VERSION=$SOC_VERSION
RUN pip config set global.index-url ${PIP_INDEX_URL}

2
Dockerfile.a3.openEuler
View File

@@ -20,8 +20,10 @@ FROM quay.io/ascend/cann:8.3.rc1-a3-openeuler24.03-py3.11
ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"
ARG COMPILE_CUSTOM_KERNELS=1
ARG MOONCAKE_TAG="v0.3.7.post2"
ARG SOC_VERSION
ENV COMPILE_CUSTOM_KERNELS=${COMPILE_CUSTOM_KERNELS}
ENV SOC_VERSION=$SOC_VERSION
RUN pip config set global.index-url ${PIP_INDEX_URL}

2
Dockerfile.openEuler
View File

@@ -20,8 +20,10 @@ FROM quay.io/ascend/cann:8.3.rc1-910b-openeuler24.03-py3.11
ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"
ARG COMPILE_CUSTOM_KERNELS=1
ARG MOONCAKE_TAG="v0.3.7.post2"
ARG SOC_VERSION
ENV COMPILE_CUSTOM_KERNELS=${COMPILE_CUSTOM_KERNELS}
ENV SOC_VERSION=$SOC_VERSION
RUN pip config set global.index-url ${PIP_INDEX_URL}

27
setup.py
View File

@@ -73,7 +73,7 @@ def get_value_from_lines(lines: List[str], key: str) -> str:
return ""
def get_chip_info() -> str:
def get_chip_type() -> str:
try:
npu_info_lines = subprocess.check_output(
['npu-smi', 'info', '-l']).decode().strip().split('\n')
@@ -106,19 +106,27 @@ def get_chip_info() -> str:
except subprocess.CalledProcessError as e:
raise RuntimeError(f"Get chip info failed: {e}")
except FileNotFoundError:
# cpu envir, release code case, return `ascend910b1` by default
return "ascend910b1"
logging.warning(
"npu-smi command not found, if this is an npu envir, please check if npu driver is installed correctly."
)
return ""
envs = load_module_from_path("envs",
os.path.join(ROOT_DIR, "vllm_ascend", "envs.py"))
soc_version = get_chip_info()
soc_version = get_chip_type()
if not envs.SOC_VERSION:
if not soc_version:
raise RuntimeError(
"Could not determine chip type automatically via 'npu-smi'. "
"This can happen in a CPU-only environment. "
"Please set the 'SOC_VERSION' environment variable to specify the target chip."
)
envs.SOC_VERSION = soc_version
else:
if envs.SOC_VERSION != soc_version:
if soc_version and envs.SOC_VERSION != soc_version:
logging.warning(
f"env SOC_VERSION: {envs.SOC_VERSION} is not equal to soc_version from npu-smi: {soc_version}"
)
@@ -126,10 +134,6 @@ else:
def gen_build_info():
soc_version = envs.SOC_VERSION
if "310" in soc_version and not envs.COMPILE_CUSTOM_KERNELS:
raise ValueError(
"SOC version 310 only supports custom kernels. Please set COMPILE_CUSTOM_KERNELS=1 to enable custom kernels."
)
# TODO(zzzzwwjj): Add A5 case
soc_to_device = {
@@ -158,6 +162,11 @@ def gen_build_info():
assert soc_version in soc_to_device, f"Undefined soc_version: {soc_version}. Please file an issue to vllm-ascend."
device_type = soc_to_device[soc_version]
if device_type == "_310P" and not envs.COMPILE_CUSTOM_KERNELS:
raise ValueError(
"device type 310P only supports custom kernels. Please set COMPILE_CUSTOM_KERNELS=1 to enable custom kernels."
)
package_dir = os.path.join(ROOT_DIR, "vllm_ascend", "_build_info.py")
with open(package_dir, "w+") as f:
f.write('# Auto-generated file\n')