[Doc] Update user doc index (#1581)
Add user doc index to make the user guide more clear
- vLLM version: v0.9.1
- vLLM main:
49e8c7ea25
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
This commit is contained in:
@@ -43,16 +43,10 @@ faqs
|
|||||||
:::{toctree}
|
:::{toctree}
|
||||||
:caption: User Guide
|
:caption: User Guide
|
||||||
:maxdepth: 1
|
:maxdepth: 1
|
||||||
user_guide/suppoted_features
|
user_guide/support_matrix/index
|
||||||
user_guide/supported_models
|
user_guide/configuration/index
|
||||||
user_guide/env_vars
|
user_guide/feature_guide/index
|
||||||
user_guide/additional_config
|
|
||||||
user_guide/sleep_mode
|
|
||||||
user_guide/graph_mode.md
|
|
||||||
user_guide/lora.md
|
|
||||||
user_guide/quantization.md
|
|
||||||
user_guide/release_notes
|
user_guide/release_notes
|
||||||
user_guide/structured_output
|
|
||||||
:::
|
:::
|
||||||
|
|
||||||
% How to contribute to the vLLM Ascend project
|
% How to contribute to the vLLM Ascend project
|
||||||
|
|||||||
@@ -54,7 +54,7 @@ hccn_tool -i 0 -ping -g address 10.20.0.20
|
|||||||
```
|
```
|
||||||
|
|
||||||
## Run with docker
|
## Run with docker
|
||||||
Assume you have two Altas 800 A2(64G*8) nodes, and want to deploy the `deepseek-v3-w8a8` quantitative model across multi-node.
|
Assume you have two Atlas 800 A2(64G*8) nodes, and want to deploy the `deepseek-v3-w8a8` quantitative model across multi-node.
|
||||||
|
|
||||||
```shell
|
```shell
|
||||||
# Define the image and container name
|
# Define the image and container name
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
vllm-ascend uses the following environment variables to configure the system:
|
vllm-ascend uses the following environment variables to configure the system:
|
||||||
|
|
||||||
:::{literalinclude} ../../../vllm_ascend/envs.py
|
:::{literalinclude} ../../../../vllm_ascend/envs.py
|
||||||
:language: python
|
:language: python
|
||||||
:start-after: begin-env-vars-definition
|
:start-after: begin-env-vars-definition
|
||||||
:end-before: end-env-vars-definition
|
:end-before: end-env-vars-definition
|
||||||
10
docs/source/user_guide/configuration/index.md
Normal file
10
docs/source/user_guide/configuration/index.md
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
# Configuration Guide
|
||||||
|
|
||||||
|
This section provides a detailed configuration guide of vLLM Ascend.
|
||||||
|
|
||||||
|
:::{toctree}
|
||||||
|
:caption: Configuration Guide
|
||||||
|
:maxdepth: 1
|
||||||
|
env_vars
|
||||||
|
additional_config
|
||||||
|
:::
|
||||||
@@ -59,7 +59,7 @@ online example:
|
|||||||
vllm serve Qwen/Qwen2-7B-Instruct --additional-config='{"torchair_graph_config": {"enabled": true},"ascend_scheduler_config": {"enabled": true,}}'
|
vllm serve Qwen/Qwen2-7B-Instruct --additional-config='{"torchair_graph_config": {"enabled": true},"ascend_scheduler_config": {"enabled": true,}}'
|
||||||
```
|
```
|
||||||
|
|
||||||
You can find more detail about additional config [here](./additional_config.md)
|
You can find more detail about additional config [here](../configuration/additional_config.md).
|
||||||
|
|
||||||
## Fallback to Eager Mode
|
## Fallback to Eager Mode
|
||||||
|
|
||||||
|
Before Width: | Height: | Size: 57 KiB After Width: | Height: | Size: 57 KiB |
13
docs/source/user_guide/feature_guide/index.md
Normal file
13
docs/source/user_guide/feature_guide/index.md
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
# Feature Guide
|
||||||
|
|
||||||
|
This section provides a detailed usage guide of vLLM Ascend features.
|
||||||
|
|
||||||
|
:::{toctree}
|
||||||
|
:caption: Feature Guide
|
||||||
|
:maxdepth: 1
|
||||||
|
graph_mode
|
||||||
|
quantization
|
||||||
|
sleep_mode
|
||||||
|
structured_output
|
||||||
|
lora
|
||||||
|
:::
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
# LoRA Adapters
|
# LoRA Adapters Guide
|
||||||
|
|
||||||
Like vLLM, vllm-ascend supports LoRA as well. The usage and more details can be found in [vLLM official document](https://docs.vllm.ai/en/latest/features/lora.html).
|
Like vLLM, vllm-ascend supports LoRA as well. The usage and more details can be found in [vLLM official document](https://docs.vllm.ai/en/latest/features/lora.html).
|
||||||
|
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
# Sleep Mode
|
# Sleep Mode Guide
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
@@ -1,13 +0,0 @@
|
|||||||
## {version}
|
|
||||||
### Highlights
|
|
||||||
- {feature}
|
|
||||||
### Bug fixes
|
|
||||||
- {bug}
|
|
||||||
### Other changes
|
|
||||||
- {change}
|
|
||||||
### Known issues
|
|
||||||
- {issue}
|
|
||||||
### Upgrade Notes
|
|
||||||
- {upgrade}
|
|
||||||
### Deprecation Notes
|
|
||||||
- {deprecation}
|
|
||||||
10
docs/source/user_guide/support_matrix/index.md
Normal file
10
docs/source/user_guide/support_matrix/index.md
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
# Features and models
|
||||||
|
|
||||||
|
This section provides a detailed supported matrix by vLLM Ascend.
|
||||||
|
|
||||||
|
:::{toctree}
|
||||||
|
:caption: Support Matrix
|
||||||
|
:maxdepth: 1
|
||||||
|
supported_models
|
||||||
|
supported_features
|
||||||
|
:::
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
# Supported Models
|
# Model Support
|
||||||
|
|
||||||
## Text-only Language Models
|
## Text-only Language Models
|
||||||
|
|
||||||
Reference in New Issue
Block a user