EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Organize server_args (#277)

Browse Source
This commit is contained in:
Liangsheng Yin
2024-03-11 20:06:52 +08:00
committed by GitHub
parent faba293a0d
commit 1b35547927
12 changed files with 92 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/flashinfer.md
View File

@@ -16,10 +16,10 @@ please build it from source (the compilation takes a long time).
### Run a Server With Flashinfer Mode
Add `--model-mode flashinfer` argument to enable flashinfer when launching a server.
Add `--enable-flashinfer` argument to enable flashinfer when launching a server.
Example:
```bash
python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port 30000 --model-mode flashinfer
python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port 30000 --enable-flashinfer
```