EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

[CI/Build] Bump torch_npu to dev20250307.3 (#265)

Browse Source 
Update torch-npu version to fix torch npu exponential_ accuracy
With this update, the percision issue when setting `temperature > 0` is
fixed.

---------

Signed-off-by: Mengqing Cao <cmq0113@163.com>
This commit is contained in:
Mengqing Cao
2025-03-07 20:34:07 +08:00
committed by GitHub
parent faf8cd89cb
commit 91f7d8115d
6 changed files with 17 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
docs/source/installation.md
View File

@@ -12,7 +12,7 @@ This document describes how to install vllm-ascend manually.
| Software | Supported version | Note |
| ------------ | ----------------- | ---- |
| CANN | >= 8.0.0 | Required for vllm-ascend and torch-npu |
| torch-npu | >= 2.5.1.dev20250226 | Required for vllm-ascend |
| torch-npu | >= 2.5.1.dev20250307 | Required for vllm-ascend |
| torch | >= 2.5.1 | Required for torch-npu and vllm |
You have 2 way to install:
@@ -150,15 +150,15 @@ Current version depends on a unreleased `torch-npu`, you need to install manuall
#
# Here we take python 3.10 on aarch64 as an example. Feel free to install the correct version for your environment. See:
#
# https://pytorch-package.obs.cn-north-4.myhuaweicloud.com/pta/Daily/v2.5.1/20250226.4/pytorch_v2.5.1_py39.tar.gz
# https://pytorch-package.obs.cn-north-4.myhuaweicloud.com/pta/Daily/v2.5.1/20250226.4/pytorch_v2.5.1_py310.tar.gz
# https://pytorch-package.obs.cn-north-4.myhuaweicloud.com/pta/Daily/v2.5.1/20250226.4/pytorch_v2.5.1_py311.tar.gz
# https://pytorch-package.obs.cn-north-4.myhuaweicloud.com/pta/Daily/v2.5.1/20250307.3/pytorch_v2.5.1_py39.tar.gz
# https://pytorch-package.obs.cn-north-4.myhuaweicloud.com/pta/Daily/v2.5.1/20250307.3/pytorch_v2.5.1_py310.tar.gz
# https://pytorch-package.obs.cn-north-4.myhuaweicloud.com/pta/Daily/v2.5.1/20250307.3/pytorch_v2.5.1_py311.tar.gz
#
mkdir pta
cd pta
wget https://pytorch-package.obs.cn-north-4.myhuaweicloud.com/pta/Daily/v2.5.1/20250226.4/pytorch_v2.5.1_py310.tar.gz
wget https://pytorch-package.obs.cn-north-4.myhuaweicloud.com/pta/Daily/v2.5.1/20250307.3/pytorch_v2.5.1_py310.tar.gz
tar -xvf pytorch_v2.5.1_py310.tar.gz
pip install ./torch_npu-2.5.1.dev20250226-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
pip install ./torch_npu-2.5.1.dev20250307-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
```
::::

6
docs/source/tutorials.md
View File

@@ -212,7 +212,9 @@ Prompt: 'The future of AI is', Generated text: ' following you. As the technolog
Run docker container on each machine:
```shell
```{code-block} bash
:substitutions:
docker run \
--name vllm-ascend \
--device /dev/davinci0 \
@@ -233,7 +235,7 @@ docker run \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v /root/.cache:/root/.cache \
-p 8000:8000 \
-it quay.io/ascend/vllm-ascend:v0.7.1rc1 bash
-it quay.io/ascend/vllm-ascend:|vllm_ascend_version| bash
```
Choose one machine as head node, the other are worker nodes, then start ray on each machine: