EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

Upgrade CANN version to 8.1.rc1 (#747)

Browse Source 
### What this PR does / why we need it?

Make CANN version bump separately from
https://github.com/vllm-project/vllm-ascend/pull/708

- Upgrade CANN version to 8.1.rc1
- Add prefix to speed up download
`m.daocloud.io/quay.io/ascend/cann:8.1.rc1-910b-ubuntu22.04-py3.10`
- Address tail sapce for Dockerfile.openEuler
- Add note for `/workspace` and `/vllm-workspace` as followup of
https://github.com/vllm-project/vllm-ascend/pull/741

### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?

CI passed

Co-authored-by: MengqingCao <cmq0113@163.com>

Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
Co-authored-by: MengqingCao <cmq0113@163.com>
This commit is contained in:
Yikun Jiang
2025-05-06 05:44:18 +08:00
committed by GitHub
parent d7e1110c8e
commit 79538b5d73
8 changed files with 23 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
.github/workflows/vllm_ascend_test.yaml vendored
View File

@@ -61,7 +61,8 @@ jobs:
name: vLLM Ascend test name: vLLM Ascend test
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
container: container:
image: quay.io/ascend/cann:8.0.0-910b-ubuntu22.04-py3.10 # TODO(yikun): Remove m.daocloud.io prefix when infra proxy ready
image: m.daocloud.io/quay.io/ascend/cann:8.1.rc1-910b-ubuntu22.04-py3.10
env: env:
HF_ENDPOINT: https://hf-mirror.com HF_ENDPOINT: https://hf-mirror.com
HF_TOKEN: ${{ secrets.HF_TOKEN }} HF_TOKEN: ${{ secrets.HF_TOKEN }}

2
Dockerfile
View File

@@ -15,7 +15,7 @@
# This file is a part of the vllm-ascend project. # This file is a part of the vllm-ascend project.
# #
FROM quay.io/ascend/cann:8.0.0-910b-ubuntu22.04-py3.10 FROM quay.io/ascend/cann:8.1.rc1-910b-ubuntu22.04-py3.10
ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple" ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"
ARG COMPILE_CUSTOM_KERNELS=1 ARG COMPILE_CUSTOM_KERNELS=1

6
Dockerfile.openEuler
View File

@@ -15,7 +15,7 @@
# This file is a part of the vllm-ascend project. # This file is a part of the vllm-ascend project.
# #
FROM quay.io/ascend/cann:8.0.0-910b-openeuler22.03-py3.10 FROM quay.io/ascend/cann:8.1.rc1-910b-openeuler22.03-py3.10
ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple" ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"
ARG COMPILE_CUSTOM_KERNELS=1 ARG COMPILE_CUSTOM_KERNELS=1
@@ -30,7 +30,7 @@ RUN pip config set global.index-url ${PIP_INDEX_URL}
WORKDIR /workspace WORKDIR /workspace
COPY . /vllm-workspace/vllm-ascend/ COPY . /vllm-workspace/vllm-ascend/
# Install vLLM # Install vLLM
ARG VLLM_REPO=https://github.com/vllm-project/vllm.git ARG VLLM_REPO=https://github.com/vllm-project/vllm.git
@@ -53,4 +53,4 @@ RUN source /usr/local/Ascend/ascend-toolkit/set_env.sh && \
RUN python3 -m pip install modelscope ray && \ RUN python3 -m pip install modelscope ray && \
python3 -m pip cache purge python3 -m pip cache purge
CMD ["/bin/bash"] CMD ["/bin/bash"]

2
README.md
View File

@@ -37,7 +37,7 @@ By using vLLM Ascend plugin, popular open-source models, including Transformer-l
- OS: Linux - OS: Linux
- Software: - Software:
* Python >= 3.9, < 3.12 * Python >= 3.9, < 3.12
* CANN >= 8.0.0 * CANN >= 8.1.rc1
* PyTorch >= 2.5.1, torch-npu >= 2.5.1 * PyTorch >= 2.5.1, torch-npu >= 2.5.1
* vLLM (the same version as vllm-ascend) * vLLM (the same version as vllm-ascend)

4
README.zh.md
View File

@@ -37,8 +37,8 @@ vLLM 昇腾插件 (`vllm-ascend`) 是一个由社区维护的让vLLM在Ascend NP
- 硬件：Atlas 800I A2 Inference系列、Atlas A2 Training系列 - 硬件：Atlas 800I A2 Inference系列、Atlas A2 Training系列
- 操作系统：Linux - 操作系统：Linux
- 软件： - 软件：
* Python >= 3.9 * Python >= 3.9, < 3.12
* CANN >= 8.0.RC2 * CANN >= 8.1.RC1
* PyTorch >= 2.5.1, torch-npu >= 2.5.1 * PyTorch >= 2.5.1, torch-npu >= 2.5.1
* vLLM (与vllm-ascend版本一致) * vLLM (与vllm-ascend版本一致)

2
docs/source/conf.py
View File

@@ -73,7 +73,7 @@ myst_substitutions = {
'pip_vllm_ascend_version': "0.8.4rc2", 'pip_vllm_ascend_version': "0.8.4rc2",
'pip_vllm_version': "0.8.4", 'pip_vllm_version': "0.8.4",
# CANN image tag # CANN image tag
'cann_image_tag': "8.0.0-910b-ubuntu22.04-py3.10", 'cann_image_tag': "8.1.rc1-910b-ubuntu22.04-py3.10",
} }
# Add any paths that contain templates here, relative to this directory. # Add any paths that contain templates here, relative to this directory.

25
docs/source/installation.md
View File

@@ -11,7 +11,7 @@ This document describes how to install vllm-ascend manually.
| Software | Supported version | Note | | Software | Supported version | Note |
|-----------|-------------------|----------------------------------------| |-----------|-------------------|----------------------------------------|
| CANN | >= 8.0.0 | Required for vllm-ascend and torch-npu | | CANN | >= 8.1.rc1 | Required for vllm-ascend and torch-npu |
| torch-npu | >= 2.5.1 | Required for vllm-ascend | | torch-npu | >= 2.5.1 | Required for vllm-ascend |
| torch | >= 2.5.1 | Required for torch-npu and vllm | | torch | >= 2.5.1 | Required for torch-npu and vllm |
@@ -69,10 +69,6 @@ docker run --rm \
:animate: fade-in-slide-down :animate: fade-in-slide-down
You can also install CANN manually: You can also install CANN manually:
```{note}
This guide takes aarch64 as an example. If you run on x86, you need to replace `aarch64` with `x86_64` for the package name shown below.
```
```bash ```bash
# Create a virtual environment # Create a virtual environment
python -m venv vllm-ascend-env python -m venv vllm-ascend-env
@@ -82,19 +78,19 @@ source vllm-ascend-env/bin/activate
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple attrs 'numpy<2.0.0' decorator sympy cffi pyyaml pathlib2 psutil protobuf scipy requests absl-py wheel typing_extensions pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple attrs 'numpy<2.0.0' decorator sympy cffi pyyaml pathlib2 psutil protobuf scipy requests absl-py wheel typing_extensions
# Download and install the CANN package. # Download and install the CANN package.
wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/CANN/CANN%208.0.0/Ascend-cann-toolkit_8.0.0_linux-aarch64.run wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/CANN/CANN%208.1.RC1/Ascend-cann-toolkit_8.1.RC1_linux-"$(uname -i)".run
chmod +x ./Ascend-cann-toolkit_8.0.0_linux-aarch64.run chmod +x ./Ascend-cann-toolkit_8.1.RC1_linux-"$(uname -i)".run
./Ascend-cann-toolkit_8.0.0_linux-aarch64.run --full ./Ascend-cann-toolkit_8.1.RC1_linux-"$(uname -i)".run --full
source /usr/local/Ascend/ascend-toolkit/set_env.sh source /usr/local/Ascend/ascend-toolkit/set_env.sh
wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/CANN/CANN%208.0.0/Ascend-cann-kernels-910b_8.0.0_linux-aarch64.run wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/CANN/CANN%208.1.RC1/Ascend-cann-kernels-910b_8.1.RC1_linux-"$(uname -i)".run
chmod +x ./Ascend-cann-kernels-910b_8.0.0_linux-aarch64.run chmod +x ./Ascend-cann-kernels-910b_8.1.RC1_linux-"$(uname -i)".run
./Ascend-cann-kernels-910b_8.0.0_linux-aarch64.run --install ./Ascend-cann-kernels-910b_8.1.RC1_linux-"$(uname -i)".run --install
wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/CANN/CANN%208.0.0/Ascend-cann-nnal_8.0.0_linux-aarch64.run wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/CANN/CANN%208.1.RC1/Ascend-cann-nnal_8.1.RC1_linux-"$(uname -i)".run
chmod +x ./Ascend-cann-nnal_8.0.0_linux-aarch64.run chmod +x ./Ascend-cann-nnal_8.1.RC1_linux-"$(uname -i)".run
./Ascend-cann-nnal_8.0.0_linux-aarch64.run --install ./Ascend-cann-nnal_8.1.RC1_linux-"$(uname -i)".run --install
source /usr/local/Ascend/nnal/atb/set_env.sh source /usr/local/Ascend/nnal/atb/set_env.sh
``` ```
@@ -223,6 +219,7 @@ docker run --rm \
-it $IMAGE bash -it $IMAGE bash
``` ```
The default workdir is `/workspace`, vLLM and vLLM Ascend code are placed in `/vllm-workspace` and installed in [development mode](https://setuptools.pypa.io/en/latest/userguide/development_mode.html)(`pip install -e`) to help developer immediately take place changes without requiring a new installation.
:::: ::::
::::: :::::

2
docs/source/quick_start.md
View File

@@ -62,6 +62,8 @@ docker run --rm \
:::: ::::
::::: :::::
The default workdir is `/workspace`, vLLM and vLLM Ascend code are placed in `/vllm-workspace` and installed in [development mode](https://setuptools.pypa.io/en/latest/userguide/development_mode.html)(`pip install -e`) to help developer immediately take place changes without requiring a new installation.
## Usage ## Usage
You can use Modelscope mirror to speed up download: You can use Modelscope mirror to speed up download: