EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Fix readme (#1236)

Browse Source
This commit is contained in:
Dr. Artificial曾小健
2024-08-28 14:51:41 +08:00
committed by GitHub
parent 3602692c7c
commit c8a9e79186
1 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -83,6 +83,7 @@ docker run --gpus all \
### Method 4: Using docker compose ### Method 4: Using docker compose
<details> <details>
<summary>More</summary>
> This method is recommended if you plan to serve it as a service. > This method is recommended if you plan to serve it as a service.
> A better approach is to use the [k8s-sglang-service.yaml](./docker/k8s-sglang-service.yaml). > A better approach is to use the [k8s-sglang-service.yaml](./docker/k8s-sglang-service.yaml).
@@ -94,6 +95,7 @@ docker run --gpus all \
### Method 5: Run on Kubernetes or Clouds with SkyPilot ### Method 5: Run on Kubernetes or Clouds with SkyPilot
<details> <details>
<summary>More</summary>
To deploy on Kubernetes or 12+ clouds, you can use [SkyPilot](https://github.com/skypilot-org/skypilot). To deploy on Kubernetes or 12+ clouds, you can use [SkyPilot](https://github.com/skypilot-org/skypilot).
@@ -262,6 +264,7 @@ Instructions for supporting a new model are [here](https://github.com/sgl-projec
#### Use Models From ModelScope #### Use Models From ModelScope
<details> <details>
<summary>More</summary>
To use a model from [ModelScope](https://www.modelscope.cn), set the environment variable SGLANG_USE_MODELSCOPE. To use a model from [ModelScope](https://www.modelscope.cn), set the environment variable SGLANG_USE_MODELSCOPE.
``` ```
@@ -276,6 +279,7 @@ SGLANG_USE_MODELSCOPE=true python -m sglang.launch_server --model-path qwen/Qwen
#### Run Llama 3.1 405B #### Run Llama 3.1 405B
<details> <details>
<summary>More</summary>
```bash ```bash
# Run 405B (fp8) on a single node # Run 405B (fp8) on a single node