[CPU] Add tutorial docs for SGL on CPU (#8000)
This commit is contained in:
@@ -52,6 +52,9 @@ cd ..
|
||||
pip install -e "python[all_hip]"
|
||||
```
|
||||
|
||||
Note: Please refer to [the CPU environment setup command list](../references/cpu.md#install-from-source)
|
||||
to set up the SGLang environment for running the models with CPU servers.
|
||||
|
||||
## Method 3: Using docker
|
||||
|
||||
The docker images are available on Docker Hub as [lmsysorg/sglang](https://hub.docker.com/r/lmsysorg/sglang/tags), built from [Dockerfile](https://github.com/sgl-project/sglang/tree/main/docker).
|
||||
@@ -87,6 +90,9 @@ drun -p 30000:30000 \
|
||||
drun v0.4.9.post3-rocm630 python3 -m sglang.bench_one_batch --batch-size 32 --input 1024 --output 128 --model amd/Meta-Llama-3.1-8B-Instruct-FP8-KV --tp 8 --quantization fp8
|
||||
```
|
||||
|
||||
Note: Please refer to [the CPU installation guide using Docker](../references/cpu.md#install-using-docker)
|
||||
to set up the SGLang environment for running the models with CPU servers.
|
||||
|
||||
## Method 4: Using docker compose
|
||||
|
||||
<details>
|
||||
|
||||
Reference in New Issue
Block a user