Simo Lin
|
2e901e892f
|
[router] dedicated prefill HTTP client and request-path optimizations (#8923)
|
2025-08-07 17:31:45 -07:00 |
|
Tien Nguyen
|
fd05b56750
|
refactor(sgl-router): Replace once_cell with LazyLock in worker.rs and remove once_cell dependency from Cargo.toml (#8698)
|
2025-08-07 06:14:03 -07:00 |
|
Simo Lin
|
a69b637014
|
[router] fix req handling order, improve serialization, remove retry (#8888)
|
2025-08-06 23:24:39 -07:00 |
|
Simo Lin
|
8c7bb39dfb
|
[router] PD Router Simplification and Reorganization (#8838)
|
2025-08-05 21:20:38 -07:00 |
|
Simo Lin
|
5d62b56f7e
|
[router] complete router oai spec (#8828)
|
2025-08-05 18:30:19 -07:00 |
|
Simo Lin
|
354ac43555
|
[pd-router] Add Configurable Retry Logic for reduce backend pressure (#8744)
|
2025-08-04 20:42:07 -07:00 |
|
Simo Lin
|
2fa0462c39
|
[router] introduce dp worker abstraction (#8639)
|
2025-08-04 06:42:20 -07:00 |
|
Simo Lin
|
828a4fe944
|
[router] Implement HTTP Dependency Injection Pattern for Router System (#8714)
|
2025-08-02 19:16:47 -07:00 |
|
Simo Lin
|
6d4fd8826e
|
[router] minor code clean up and and refactoring (#8711)
|
2025-08-02 13:46:31 -07:00 |
|
Simo Lin
|
ab9b893e61
|
[bug] limit bootstrap room to to [0, 2^63 - 1] (#8684)
|
2025-08-01 14:41:01 -07:00 |
|
Simo Lin
|
66a398f49d
|
[router] migrate router from actix to axum (#8479)
|
2025-07-30 17:47:19 -07:00 |
|
Rui Chen
|
a730ce8162
|
[feature] [sgl-router] Add a dp-aware routing strategy (#6869)
|
2025-07-30 05:58:48 -07:00 |
|
Simo Lin
|
fe6a445d1e
|
[router] improve router logs and request id header (#8415)
|
2025-07-27 19:30:19 -07:00 |
|
Simo Lin
|
2ab97023e3
|
[router] add different policies for p node and d node (#8395)
|
2025-07-27 00:39:20 -07:00 |
|
Simo Lin
|
f6e07f2796
|
[router] fix pd model completion request (#8303)
|
2025-07-23 23:18:29 -07:00 |
|
Rui Chen
|
c33499a67b
|
fix: sgl-router remove dead code (#8257)
|
2025-07-22 08:41:23 -07:00 |
|
Simo Lin
|
5c8365a051
|
[router] add ut for pd router (#8208)
|
2025-07-20 23:12:52 -07:00 |
|
Simo Lin
|
1fc455e8b6
|
[router] add ut for pd request, metrics and config (#8184)
|
2025-07-20 10:53:42 -07:00 |
|
Simo Lin
|
41d33e4736
|
[router] add ut for worker and errors (#8170)
|
2025-07-19 14:38:33 -07:00 |
|
Charles Chen
|
f3d9736156
|
Fix suffix mismatch for the metrics. (#8168)
Signed-off-by: Charles Chen <chenliqian@chenliqian.cn>
|
2025-07-19 10:11:24 -07:00 |
|
Simo Lin
|
8fcc55cfa1
|
[router] router metrics cleanup (#8158)
|
2025-07-18 22:09:17 -07:00 |
|
Simo Lin
|
c8f31042a8
|
[router] Refactor router and policy traits with dependency injection (#7987)
Co-authored-by: Jin Pan <jpan236@wisc.edu>
Co-authored-by: Keru Yang <rukeyang@gmail.com>
Co-authored-by: Yingyi Huang <yingyihuang2000@outlook.com>
Co-authored-by: Philip Zhu <phlipzhux@gmail.com>
|
2025-07-18 14:24:24 -07:00 |
|
Simo Lin
|
f2d5c4920e
|
[router] add worker abstraction (#7960)
|
2025-07-11 20:17:48 -07:00 |
|
Simo Lin
|
4ed57807c2
|
[bugfix] add pd router policy validation (#7904)
|
2025-07-09 21:52:06 -07:00 |
|
Shuaiyi Zhang
|
7590f5224b
|
[router] Update metrics when request completes (#7899)
|
2025-07-09 08:18:09 -07:00 |
|
Shuaiyi Zhang
|
51ae40306a
|
[router] forward stream_options in request (#7860)
|
2025-07-08 08:03:38 -07:00 |
|
Simo Lin
|
ea3e7ffec7
|
[bugfix] Fix sgl-router get_server_info endpoint compatibility issue (#7813)
|
2025-07-06 19:52:57 -07:00 |
|
Zilin Zhu
|
82f021e22e
|
[router] add --log-level to sgl-router (#6512)
|
2025-07-02 19:33:04 -07:00 |
|
Simo Lin
|
e3f9b54819
|
[bugfix] fix runtime dropping panic in editable (#7628)
|
2025-06-29 15:38:28 -07:00 |
|
finetune
|
b3cff3651e
|
Fix sgl-router startup crash (#7619)
|
2025-06-29 14:41:34 -07:00 |
|
Simo Lin
|
c071198c1d
|
[router] add centralized configuration module for sgl-router (#7588)
|
2025-06-27 15:42:02 -07:00 |
|
Simo Lin
|
30f2a44a96
|
[misc] Add PD service discovery support in router (#7361)
|
2025-06-22 17:54:14 -07:00 |
|
Simo Lin
|
09ae5b20f3
|
Merge PDLB (Prefill-Decode Load Balancer) into SGLang Router (#7096)
|
2025-06-19 02:28:15 +08:00 |
|
Arthur Cheng
|
ff91474825
|
[Router] Fix k8s Service Discovery (#6766)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2025-06-02 16:57:23 -07:00 |
|
Chao Yang
|
1a39979993
|
Sgl-router Prometheus metrics endpoint and usage track metrics (#6537)
|
2025-05-24 22:28:15 -07:00 |
|
Zilin Zhu
|
669caa0a3f
|
[router] support http2 in router (#6487)
|
2025-05-21 01:42:45 -07:00 |
|
Zilin Zhu
|
e3bed74afb
|
[router] Add /list_workers endpoint to router (#6366)
|
2025-05-17 09:49:02 -07:00 |
|
applesaucethebun
|
2ce8793519
|
Add typo checker in pre-commit (#6179)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
|
2025-05-11 12:55:00 +08:00 |
|
Simo Lin
|
771669cbe0
|
[fix]: PyO3 macOS linking and consolidate on tracing for logging
|
2025-04-29 11:26:38 -07:00 |
|
Simo Lin
|
1468769bde
|
[Misc] add service discovery for sgl router
|
2025-04-29 10:21:19 -07:00 |
|
Simo Lin
|
f0365820e8
|
[Misc] add structure logging, write to file and log tracing for SGL Router
|
2025-04-27 16:54:10 -07:00 |
|
Yinghai Lu
|
f4d7ab7a63
|
[sgl-router] improvement to avoid hang (#4482)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
|
2025-03-17 10:37:50 -07:00 |
|
Byron Hsu
|
8cc300f536
|
Fix router test (#4483)
|
2025-03-16 22:49:47 -07:00 |
|
Michael Feil
|
37373ef2bb
|
sgl-router - issues on routing and project build. (#3870) (#3948)
|
2025-03-04 04:06:30 -08:00 |
|
Byron Hsu
|
9a0cc2e90e
|
[router] Forward all request headers from router to workers (#3070)
|
2025-01-23 20:30:31 -08:00 |
|
Byron Hsu
|
5de50653cd
|
[router] make error actionable (#3063)
|
2025-01-22 17:56:21 -08:00 |
|
Byron Hsu
|
3a8428ecaa
|
[router] Expose worker startup interval (#3019)
|
2025-01-20 14:36:54 -08:00 |
|
Byron Hsu
|
0311ce8e1c
|
[router] Expose worker startup secs & Return error instead of panic for router init (#3016)
|
2025-01-20 12:45:13 -08:00 |
|
Ata Fatahi
|
e3b3acfa6f
|
Rename rust folder to sgl-router (#2464)
Signed-off-by: Ata Fatahi <immrata@gmail.com>
|
2024-12-12 09:40:41 -08:00 |
|