Sgl-router Prometheus metrics endpoint and usage track metrics (#6537)

This commit is contained in:
Chao Yang
2025-05-24 22:28:15 -07:00
committed by GitHub
parent 022012aae8
commit 1a39979993
9 changed files with 167 additions and 2 deletions

View File

@@ -81,6 +81,18 @@ router = Router(
Use the `--verbose` flag with the CLI for more detailed logs.
### Metrics
SGL Router exposes a Prometheus HTTP scrape endpoint for monitoring, which by default listens at 127.0.0.1:29000.
To change the endpoint to listen on all network interfaces and set the port to 9000, configure the following options when launching the router:
```
python -m sglang_router.launch_router \
--worker-urls http://localhost:8080 http://localhost:8081 \
--prometheus-host 0.0.0.0 \
--prometheus-port 9000
```
### Kubernetes Service Discovery
SGL Router supports automatic service discovery for worker nodes in Kubernetes environments. When enabled, the router will automatically: