EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

[v0.11.0][Doc] Update doc (#3852)

Browse Source 
### What this PR does / why we need it?
Update doc


Signed-off-by: hfadzxy <starmoon_zhang@163.com>
This commit is contained in:
zhangxinyuehfad
2025-10-29 11:32:12 +08:00
committed by GitHub
parent 6188450269
commit 75de3fa172
49 changed files with 724 additions and 701 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
docs/source/tutorials/multi_node_qwen3vl.md
View File

@@ -1,15 +1,15 @@
# Multi-Node-DP (Qwen3-VL-235B-A22B)
:::{note}
Qwen3 VL rely on the newest version of `transformers`(>4.56.2). Please install it from source until it's released.
Qwen3 VL relies on the newest version of `transformers` (>4.56.2). Please install it from source.
:::
## Verify Multi-Node Communication Environment
referring to [multi_node.md](https://vllm-ascend.readthedocs.io/en/latest/tutorials/multi_node.html#verification-process)
Refer to [multi_node.md](https://vllm-ascend.readthedocs.io/en/latest/tutorials/multi_node.html#verification-process).
## Run with docker
Assume you have an Atlas 800 A3(64G*16) nodes(or 2 * A2), and want to deploy the `Qwen3-VL-235B-A22B-Instruct` model across multi-node.
## Run with Docker
Assume you have Atlas 800 A3 (64G*16) nodes (or 2 * A2), and want to deploy the `Qwen3-VL-235B-A22B-Instruct` model across multiple nodes.
```{code-block} bash
:substitutions:
@@ -49,13 +49,13 @@ docker run --rm \
-it $IMAGE bash
```
Run the following scripts on two nodes respectively
Run the following scripts on two nodes respectively.
:::{note}
Before launch the inference server, ensure the following environment variables are set for multi node communication
Before launching the inference server, ensure the following environment variables are set for multi-node communication.
:::
node0
Node 0
```shell
#!/bin/sh
@@ -93,7 +93,7 @@ vllm serve Qwen/Qwen3-VL-235B-A22B-Instruct \
--gpu-memory-utilization 0.8 \
```
node1
Node 1
```shell
#!/bin/sh
@@ -136,7 +136,7 @@ vllm serve Qwen/Qwen3-VL-235B-A22B-Instruct \
--gpu-memory-utilization 0.8 \
```
If the service starts successfully, the following information will be displayed on node0:
If the service starts successfully, the following information will be displayed on node 0:
```shell
INFO: Started server process [44610]