EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Update Install Method 2. From source (#2232)

Browse Source
This commit is contained in:
HAI
2024-11-27 22:46:55 -08:00
committed by GitHub
parent cd51758fad
commit b79fffdcb5
2 changed files with 12 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docker/Dockerfile.rocm
View File

@@ -1,5 +1,5 @@
# Usage (to build SGLang ROCm docker image): # Usage (to build SGLang ROCm docker image):
# docker build --build-arg SGL_BRANCH=v0.3.6.post2 -t testImage -f Dockerfile.rocm . # docker build --build-arg SGL_BRANCH=v0.3.6.post2 -t v0.3.6.post2-rocm620 -f Dockerfile.rocm .
# default base image # default base image
ARG BASE_IMAGE="rocm/vllm-dev:20241022" ARG BASE_IMAGE="rocm/vllm-dev:20241022"

11
docs/start/install.md
View File

@@ -28,6 +28,17 @@ pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/
Note: Please check the [FlashInfer installation doc](https://docs.flashinfer.ai/installation.html) to install the proper version according to your PyTorch and CUDA versions. Note: Please check the [FlashInfer installation doc](https://docs.flashinfer.ai/installation.html) to install the proper version according to your PyTorch and CUDA versions.
Note: To AMD ROCm system with Instinct/MI GPUs, do following instead:
```
# Use the last release branch
git clone -b v0.3.6.post2 https://github.com/sgl-project/sglang.git
cd sglang
pip install --upgrade pip
pip install -e "python[all_hip]"
```
## Method 3: Using docker ## Method 3: Using docker
The docker images are available on Docker Hub as [lmsysorg/sglang](https://hub.docker.com/r/lmsysorg/sglang/tags), built from [Dockerfile](https://github.com/sgl-project/sglang/tree/main/docker). The docker images are available on Docker Hub as [lmsysorg/sglang](https://hub.docker.com/r/lmsysorg/sglang/tags), built from [Dockerfile](https://github.com/sgl-project/sglang/tree/main/docker).
Replace `<secret>` below with your huggingface hub [token](https://huggingface.co/docs/hub/en/security-tokens). Replace `<secret>` below with your huggingface hub [token](https://huggingface.co/docs/hub/en/security-tokens).