diff --git a/.github/workflows/vllm_ascend_test.yaml b/.github/workflows/vllm_ascend_test.yaml index 5991d87..832ec65 100644 --- a/.github/workflows/vllm_ascend_test.yaml +++ b/.github/workflows/vllm_ascend_test.yaml @@ -119,7 +119,5 @@ jobs: pytest -sv tests - name: Run vllm-project/vllm test - env: - HF_TOKEN: ${{ secrets.HF_TOKEN }} run: | pytest -sv diff --git a/Dockerfile b/Dockerfile index 0854198..102e042 100644 --- a/Dockerfile +++ b/Dockerfile @@ -23,7 +23,7 @@ ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple" ENV DEBIAN_FRONTEND=noninteractive RUN apt-get update -y && \ - apt-get install -y python3-pip git vim && \ + apt-get install -y python3-pip git vim net-tools && \ rm -rf /var/cache/apt/* && \ rm -rf /var/lib/apt/lists/* @@ -33,15 +33,19 @@ COPY . /workspace/vllm-ascend/ RUN pip config set global.index-url ${PIP_INDEX_URL} -# Install vLLM main +# Install vLLM ARG VLLM_REPO=https://github.com/vllm-project/vllm.git -RUN git clone --depth 1 $VLLM_REPO /workspace/vllm +ARG VLLM_TAG=main +RUN git clone --depth 1 $VLLM_REPO --branch $VLLM_TAG /workspace/vllm RUN VLLM_TARGET_DEVICE="empty" python3 -m pip install /workspace/vllm/ -# Install vllm-ascend main +# Install vllm-ascend RUN python3 -m pip install /workspace/vllm-ascend/ --extra-index https://download.pytorch.org/whl/cpu/ -# Install modelscope (for fast download) and ray/net-tools (for multinode) -RUN python3 -m pip install modelscope ray net-tools +# Install torch-npu +RUN bash /workspace/vllm-ascend/pta_install.sh + +# Install modelscope (for fast download) and ray (for multinode) +RUN python3 -m pip install modelscope ray CMD ["/bin/bash"] diff --git a/README.md b/README.md index 3178ebf..16e7401 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,7 @@ By using vLLM Ascend plugin, popular open-source models, including Transformer-l - Software: * Python >= 3.9 * CANN >= 8.0.0 - * PyTorch >= 2.5.1, torch-npu >= 2.5.1.dev20250218 + * PyTorch >= 2.5.1, torch-npu >= 2.5.1.dev20250226 * vLLM (the same version as vllm-ascend) Find more about how to setup your environment step by step in [here](docs/source/installation.md). diff --git a/README.zh.md b/README.zh.md index 58903de..669f629 100644 --- a/README.zh.md +++ b/README.zh.md @@ -36,7 +36,7 @@ vLLM 昇腾插件 (`vllm-ascend`) 是一个让vLLM在Ascend NPU无缝运行的 - 软件: * Python >= 3.9 * CANN >= 8.0.RC2 - * PyTorch >= 2.5.1, torch-npu >= 2.5.1.dev20250218 + * PyTorch >= 2.5.1, torch-npu >= 2.5.1.dev20250226 * vLLM (与vllm-ascend版本一致) 在[此处](docs/source/installation.md),您可以了解如何逐步准备环境。 @@ -97,4 +97,4 @@ vllm-ascend有主干分支和开发分支。 ## 许可证 -Apache 许可证 2.0,如 [LICENSE](./LICENSE) 文件中所示。 \ No newline at end of file +Apache 许可证 2.0,如 [LICENSE](./LICENSE) 文件中所示。 diff --git a/docs/source/conf.py b/docs/source/conf.py index 40a1ad9..b1bb2bf 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -70,8 +70,8 @@ myst_substitutions = { 'vllm_ascend_version': 'main', # the newest release version of vllm-ascend and matched vLLM, used in pip install. # This value should be updated when cut down release. - 'pip_vllm_ascend_version': "v0.7.1rc1", - 'pip_vllm_version': "v0.7.1", + 'pip_vllm_ascend_version': "0.7.3rc1", + 'pip_vllm_version': "0.7.3", } # Add any paths that contain templates here, relative to this directory. diff --git a/docs/source/installation.md b/docs/source/installation.md index 3501b0c..78c54be 100644 --- a/docs/source/installation.md +++ b/docs/source/installation.md @@ -113,21 +113,37 @@ Once it's done, you can start to set up `vllm` and `vllm-ascend`. :selected: :sync: pip -You can install `vllm` and `vllm-ascend` from **pre-built wheel**: +You can install `vllm` and `vllm-ascend` from **pre-built wheel** (**Unreleased yet**, please build from source code): ```{code-block} bash :substitutions: -# Install vllm from source, since `pip install vllm` doesn't work on CPU currently. -# It'll be fixed in the next vllm release, e.g. v0.7.3. -git clone --branch |pip_vllm_version| https://github.com/vllm-project/vllm +# Install vllm-project/vllm from pypi +pip install vllm==|pip_vllm_version| +# Install vllm-project/vllm-ascend from pypi. +pip install vllm-ascend==|pip_vllm_ascend_version| --extra-index https://download.pytorch.org/whl/cpu/ +``` + +or build from **source code**: + +```{code-block} bash + :substitutions: + +# Install vLLM +git clone --depth 1 --branch |vllm_version| https://github.com/vllm-project/vllm cd vllm VLLM_TARGET_DEVICE=empty pip install . --extra-index https://download.pytorch.org/whl/cpu/ -# Install vllm-ascend from pypi. -pip install vllm-ascend==|pip_vllm_ascend_version| --extra-index https://download.pytorch.org/whl/cpu/ +# Install vLLM Ascend +git clone --depth 1 --branch |vllm_ascend_version| https://github.com/vllm-project/vllm-ascend.git +cd vllm-ascend +pip install -e . --extra-index https://download.pytorch.org/whl/cpu/ +``` +Current version depends on a unreleased `torch-npu`, you need to install manually: + +``` # Once the packages are installed, you need to install `torch-npu` manually, # because that vllm-ascend relies on an unreleased version of torch-npu. # This step will be removed in the next vllm-ascend release. @@ -140,25 +156,10 @@ pip install vllm-ascend==|pip_vllm_ascend_version| --extra-index https://downloa # mkdir pta cd pta -wget https://pytorch-package.obs.cn-north-4.myhuaweicloud.com/pta/Daily/v2.5.1/20250218.4/pytorch_v2.5.1_py310.tar.gz +wget https://pytorch-package.obs.cn-north-4.myhuaweicloud.com/pta/Daily/v2.5.1/20250226.4/pytorch_v2.5.1_py310.tar.gz tar -xvf pytorch_v2.5.1_py310.tar.gz pip install ./torch_npu-2.5.1.dev20250226-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl ``` - -or build from **source code**: - -```{code-block} bash - :substitutions: - -git clone --depth 1 --branch |vllm_version| https://github.com/vllm-project/vllm -cd vllm -VLLM_TARGET_DEVICE=empty pip install . --extra-index https://download.pytorch.org/whl/cpu/ - -git clone --depth 1 --branch |vllm_ascend_version| https://github.com/vllm-project/vllm-ascend.git -cd vllm-ascend -pip install -e . --extra-index https://download.pytorch.org/whl/cpu/ -``` - :::: ::::{tab-item} Using docker diff --git a/pta_install.sh b/pta_install.sh new file mode 100755 index 0000000..b37ba64 --- /dev/null +++ b/pta_install.sh @@ -0,0 +1,15 @@ +#!/bin/bash +mkdir pta +cd pta || exit +wget https://pytorch-package.obs.cn-north-4.myhuaweicloud.com/pta/Daily/v2.5.1/20250226.4/pytorch_v2.5.1_py310.tar.gz +tar -zxvf pytorch_v2.5.1_py310.tar.gz + +if [ "$(uname -i)" == "aarch64" ] +then + pip install ./torch_npu-2.5.1.dev20250226-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl +else + pip install ./torch_npu-2.5.1.dev20250226-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl +fi + +cd .. +rm -rf pta \ No newline at end of file