EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

[v0.11.0][Doc] Update doc (#3852)

Browse Source 
### What this PR does / why we need it?
Update doc


Signed-off-by: hfadzxy <starmoon_zhang@163.com>
This commit is contained in:
zhangxinyuehfad
2025-10-29 11:32:12 +08:00
committed by GitHub
parent 6188450269
commit 75de3fa172
49 changed files with 724 additions and 701 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
docs/source/installation.md
View File

@@ -16,13 +16,13 @@ This document describes how to install vllm-ascend manually.
| torch-npu | >= 2.7.1.dev20250724 | Required for vllm-ascend, No need to install manually, it will be auto installed in below steps |
| torch | >= 2.7.1 | Required for torch-npu and vllm |
You have 2 way to install:
There are two installation methods:
- **Using pip**: first prepare env manually or via CANN image, then install `vllm-ascend` using pip.
- **Using docker**: use the `vllm-ascend` pre-built docker image directly.
## Configure a new environment
Before installing, you need to make sure firmware/driver and CANN are installed correctly, refer to [link](https://ascend.github.io/docs/sources/ascend/quick_install.html) for more details.
Before installation, you need to make sure firmware/driver and CANN are installed correctly, refer to [Ascend Environment Setup Guide](https://ascend.github.io/docs/sources/ascend/quick_install.html) for more details.
### Configure hardware environment
@@ -71,11 +71,11 @@ docker run --rm \
You can also install CANN manually:
```bash
# Create a virtual environment
# Create a virtual environment.
python -m venv vllm-ascend-env
source vllm-ascend-env/bin/activate
# Install required python packages.
# Install required Python packages.
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple attrs 'numpy<2.0.0' decorator sympy cffi pyyaml pathlib2 psutil protobuf scipy requests absl-py wheel typing_extensions
# Download and install the CANN package.
@@ -102,11 +102,11 @@ source /usr/local/Ascend/nnal/atb/set_env.sh
::::{tab-item} Before using docker
:sync: docker
No more extra step if you are using `vllm-ascend` prebuilt docker image.
No more extra step if you are using `vllm-ascend` prebuilt Docker image.
::::
:::::
Once it's done, you can start to set up `vllm` and `vllm-ascend`.
Once it is done, you can start to set up `vllm` and `vllm-ascend`.
## Setup vllm and vllm-ascend
@@ -117,7 +117,7 @@ Once it's done, you can start to set up `vllm` and `vllm-ascend`.
:selected:
:sync: pip
First install system dependencies and config pip mirror:
First install system dependencies and configure pip mirror:
```bash
# Using apt-get with mirror
@@ -129,7 +129,7 @@ apt-get update -y && apt-get install -y gcc g++ cmake libnuma-dev wget git curl
pip config set global.index-url https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple
```
**[Optional]** Then config the extra-index of `pip` if you are working on a x86 machine or using torch-npu dev version:
**[Optional]** Then configure the extra-index of `pip` if you are working on an x86 machine or using torch-npu dev version:
```bash
# For torch-npu dev version or x86 machine
@@ -158,26 +158,26 @@ or build from **source code**:
```{code-block} bash
:substitutions:
# Install vLLM
# Install vLLM.
git clone --depth 1 --branch |vllm_version| https://github.com/vllm-project/vllm
cd vllm
VLLM_TARGET_DEVICE=empty pip install -v -e .
cd ..
# Install vLLM Ascend
# Install vLLM Ascend.
git clone --depth 1 --branch |vllm_ascend_version| https://github.com/vllm-project/vllm-ascend.git
cd vllm-ascend
pip install -v -e .
cd ..
```
vllm-ascend will build custom ops by default. If you don't want to build it, set `COMPILE_CUSTOM_KERNELS=0` environment to disable it.
vllm-ascend will build custom operators by default. If you don't want to build it, set `COMPILE_CUSTOM_KERNELS=0` environment to disable it.
:::
```{note}
If you are building from v0.7.3-dev and intend to use sleep mode feature, you should set `COMPILE_CUSTOM_KERNELS=1` manually.
To build custom ops, gcc/g++ higher than 8 and c++ 17 or higher is required. If you're using `pip install -e .` and encounter a torch-npu version conflict, please install with `pip install --no-build-isolation -e .` to build on system env.
If you encounter other problems during compiling, it is probably because unexpected compiler is being used, you may export `CXX_COMPILER` and `C_COMPILER` in env to specify your g++ and gcc locations before compiling.
To build custom operators, gcc/g++ higher than 8 and c++ 17 or higher is required. If you're using `pip install -e .` and encounter a torch-npu version conflict, please install with `pip install --no-build-isolation -e .` to build on system env.
If you encounter other problems during compiling, it is probably because unexpected compiler is being used, you may export `CXX_COMPILER` and `C_COMPILER` in environment to specify your g++ and gcc locations before compiling.
```
::::
@@ -220,7 +220,7 @@ docker run --rm \
-it $IMAGE bash
```
The default workdir is `/workspace`, vLLM and vLLM Ascend code are placed in `/vllm-workspace` and installed in [development mode](https://setuptools.pypa.io/en/latest/userguide/development_mode.html)(`pip install -e`) to help developer immediately take place changes without requiring a new installation.
The default workdir is `/workspace`, vLLM and vLLM Ascend code are placed in `/vllm-workspace` and installed in [development mode](https://setuptools.pypa.io/en/latest/userguide/development_mode.html) (`pip install -e`) to help developer immediately take place changes without requiring a new installation.
::::
:::::