EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

[Fix Doc.] Enable internal forwarding when starting the router (#4355)

Browse Source
This commit is contained in:
shizhediao
2025-03-12 15:53:26 -07:00
committed by GitHub
parent d40ee62b5d
commit 2c3656f276
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/router/router.md
View File

@@ -27,7 +27,7 @@ The router supports two working modes:
This will be a drop-in replacement for the existing `--dp-size` argument of SGLang Runtime. Under the hood, it uses multi-processes to launch multiple workers, wait for them to be ready, then connect the router to all workers. This will be a drop-in replacement for the existing `--dp-size` argument of SGLang Runtime. Under the hood, it uses multi-processes to launch multiple workers, wait for them to be ready, then connect the router to all workers.
```bash ```bash
python -m sglang_router.launch_server --model-path meta-llama/Meta-Llama-3.1-8B-Instruct --dp-size 4 python -m sglang_router.launch_server --model-path meta-llama/Meta-Llama-3.1-8B-Instruct --dp-size 4 --host 0.0.0.0
``` ```
After the server is ready, you can directly send requests to the router as the same way as sending requests to each single worker. After the server is ready, you can directly send requests to the router as the same way as sending requests to each single worker.