EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

[Docs] Improve bullets appearance and grammar (#4174)

Browse Source 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
This commit is contained in:
Michael Yao
2025-03-07 19:16:25 +08:00
committed by GitHub
parent b55a621ffb
commit c827c671f7
1 changed files with 36 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
docs/references/amd.md
View File

@@ -1,10 +1,8 @@
# SGLang on AMD # SGLang on AMD
## Introduction
This document describes how to set up an AMD-based environment for [SGLang](https://github.com/sgl-project/sglang). If you encounter issues or have questions, please [open an issue](https://github.com/sgl-project/sglang/issues) on the SGLang repository. This document describes how to set up an AMD-based environment for [SGLang](https://github.com/sgl-project/sglang). If you encounter issues or have questions, please [open an issue](https://github.com/sgl-project/sglang/issues) on the SGLang repository.
## System Configure ## System Configuration
When using AMD GPUs (such as MI300X), certain system-level optimizations help ensure stable performance. Here we take MI300X as an example. AMD provides official documentation for MI300X optimization and system tuning: When using AMD GPUs (such as MI300X), certain system-level optimizations help ensure stable performance. Here we take MI300X as an example. AMD provides official documentation for MI300X optimization and system tuning:
@@ -13,9 +11,9 @@ When using AMD GPUs (such as MI300X), certain system-level optimizations help en
- [AMD Instinct MI300X System Optimization](https://rocm.docs.amd.com/en/latest/how-to/system-optimization/mi300x.html) - [AMD Instinct MI300X System Optimization](https://rocm.docs.amd.com/en/latest/how-to/system-optimization/mi300x.html)
- [AMD Instinct MI300X Workload Optimization](https://rocm.docs.amd.com/en/latest/how-to/rocm-for-ai/inference-optimization/workload.html) - [AMD Instinct MI300X Workload Optimization](https://rocm.docs.amd.com/en/latest/how-to/rocm-for-ai/inference-optimization/workload.html)
**NOTE:** We strongly recommend reading theses docs entirely guide to fully utilize your system. **NOTE:** We strongly recommend reading these docs and guides entirely to fully utilize your system.
Below are a few key settings to confirm or enable: Below are a few key settings to confirm or enable for SGLang:
### Update GRUB Settings ### Update GRUB Settings
@@ -74,7 +72,6 @@ If you are using RDMA, please note that:
1. `--network host` and `--privileged` are required by RDMA. If you don't need RDMA, you can remove them. 1. `--network host` and `--privileged` are required by RDMA. If you don't need RDMA, you can remove them.
2. You may need to set `NCCL_IB_GID_INDEX` if you are using RoCE, for example: `export NCCL_IB_GID_INDEX=3`. 2. You may need to set `NCCL_IB_GID_INDEX` if you are using RoCE, for example: `export NCCL_IB_GID_INDEX=3`.
3. Launch the server. 3. Launch the server.
**NOTE:** Replace `<secret>` below with your [huggingface hub token](https://huggingface.co/docs/hub/en/security-tokens). **NOTE:** Replace `<secret>` below with your [huggingface hub token](https://huggingface.co/docs/hub/en/security-tokens).
@@ -108,7 +105,7 @@ With your AMD system properly configured and SGLang installed, you can now fully
### Running DeepSeek-V3 ### Running DeepSeek-V3
The only difference in running DeepSeek-V3 is when starting the server. Here's an example command: The only difference when running DeepSeek-V3 is in how you start the server. Here's an example command:
```bash ```bash
drun -p 30000:30000 \ drun -p 30000:30000 \
@@ -128,7 +125,7 @@ drun -p 30000:30000 \
### Running Llama3.1 ### Running Llama3.1
Running Llama3.1 is nearly identical. The only difference is in the model specified when starting the server, shown by the following example command: Running Llama3.1 is nearly identical to running DeepSeek-V3. The only difference is in the model specified when starting the server, shown by the following example command:
```bash ```bash
drun -p 30000:30000 \ drun -p 30000:30000 \
@@ -146,4 +143,4 @@ drun -p 30000:30000 \
### Warmup Step ### Warmup Step
When the server displays "The server is fired up and ready to roll!", it means the startup is successful. When the server displays `The server is fired up and ready to roll!`, it means the startup is successful.