Keyang Ru
|
ce67b2d586
|
[router]restructure protocol modules for better organization (#9321)
|
2025-08-19 01:07:58 +00:00 |
|
Chang Su
|
439df4548a
|
[router] Add spec for sglang scheduler (#9322)
|
2025-08-18 17:20:20 -07:00 |
|
Simo Lin
|
6e316588f8
|
[router] add reasoning parser base structure (#9310)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-08-18 09:26:09 -07:00 |
|
Simo Lin
|
24247b4168
|
[router] add tokenizer metrics (#9307)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-08-18 09:25:51 -07:00 |
|
Simo Lin
|
d08663eec1
|
[router] tokenizer factory, hf tokenizer, and stop sequence detector (#9293)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-08-17 22:38:38 -07:00 |
|
Simo Lin
|
ff0cf51c8e
|
[router] introducing tokenizer trait (#9287)
|
2025-08-17 16:30:01 -07:00 |
|
Simo Lin
|
b341b7dbce
|
[router] introduce prefill response draining for http compliance (#9281)
|
2025-08-17 14:23:04 -07:00 |
|
Jeff Nettleton
|
ce3ca9b02f
|
[router] add cargo clippy in CI and fix-up linting errors (#9242)
|
2025-08-17 11:03:56 -07:00 |
|
Simo Lin
|
bb10e3a1c3
|
[router] fix pd prefill http request complinace issue (#9237)
|
2025-08-16 22:36:45 -07:00 |
|
Simo Lin
|
da53e13cbb
|
[router] preserve original worker response header in router (#9236)
|
2025-08-15 11:01:47 -07:00 |
|
Jeff Nettleton
|
d7e38b2f6d
|
[router] clean up lint warnings with clippy execution (#9201)
|
2025-08-15 11:01:21 -07:00 |
|
Simo Lin
|
21b8846066
|
[router] allow more health check configuration (#9198)
|
2025-08-15 08:07:45 -07:00 |
|
Simo Lin
|
9d68bdb240
|
[router] Add Rust Binary Entrypoint for SGLang Router (#9089)
|
2025-08-11 21:37:36 -07:00 |
|
Simo Lin
|
38907fe639
|
refactor(pd-router): extract common patterns to reduce code duplication (#9081)
|
2025-08-11 13:32:31 -07:00 |
|
Simo Lin
|
6f81a710f7
|
[pd-router] add retry and circuit breakfor for pd router (#9051)
|
2025-08-11 05:53:26 -07:00 |
|
Simo Lin
|
067068f271
|
[router] regular router circuit breaker (#8997)
|
2025-08-10 21:19:30 -07:00 |
|
Simo Lin
|
473400e452
|
[router] upgrade kube version to latest (#9018)
|
2025-08-09 22:49:45 -07:00 |
|
Simo Lin
|
dd665f967f
|
[router] upgrade rand to latest version (#9017)
|
2025-08-09 22:49:30 -07:00 |
|
Simo Lin
|
7b7e56150e
|
[router] fix radix tree integration issues in PD router (#8982)
|
2025-08-08 14:47:51 -07:00 |
|
Simo Lin
|
1a8706c8b9
|
[router] reduce contention, fix double-count race (#8978)
|
2025-08-08 14:29:14 -07:00 |
|
Tony Lu
|
36bfddecb9
|
[router] add metrics for worker and policy (#8971)
Signed-off-by: Tony Lu <tonyluj@gmail.com>
|
2025-08-08 13:41:40 -07:00 |
|
Simo Lin
|
a59cbea92d
|
[router] harden retries + metrics; fix streaming load; header filtering (#8972)
|
2025-08-08 13:10:14 -07:00 |
|
Simo Lin
|
61a4680494
|
[router] router circuit breaker core (#8941)
|
2025-08-08 09:20:22 -07:00 |
|
Simo Lin
|
2e901e892f
|
[router] dedicated prefill HTTP client and request-path optimizations (#8923)
|
2025-08-07 17:31:45 -07:00 |
|
Tien Nguyen
|
fd05b56750
|
refactor(sgl-router): Replace once_cell with LazyLock in worker.rs and remove once_cell dependency from Cargo.toml (#8698)
|
2025-08-07 06:14:03 -07:00 |
|
Simo Lin
|
a69b637014
|
[router] fix req handling order, improve serialization, remove retry (#8888)
|
2025-08-06 23:24:39 -07:00 |
|
Simo Lin
|
8c7bb39dfb
|
[router] PD Router Simplification and Reorganization (#8838)
|
2025-08-05 21:20:38 -07:00 |
|
Simo Lin
|
5d62b56f7e
|
[router] complete router oai spec (#8828)
|
2025-08-05 18:30:19 -07:00 |
|
Simo Lin
|
354ac43555
|
[pd-router] Add Configurable Retry Logic for reduce backend pressure (#8744)
|
2025-08-04 20:42:07 -07:00 |
|
Simo Lin
|
2fa0462c39
|
[router] introduce dp worker abstraction (#8639)
|
2025-08-04 06:42:20 -07:00 |
|
Simo Lin
|
828a4fe944
|
[router] Implement HTTP Dependency Injection Pattern for Router System (#8714)
|
2025-08-02 19:16:47 -07:00 |
|
Simo Lin
|
6d4fd8826e
|
[router] minor code clean up and and refactoring (#8711)
|
2025-08-02 13:46:31 -07:00 |
|
Simo Lin
|
ab9b893e61
|
[bug] limit bootstrap room to to [0, 2^63 - 1] (#8684)
|
2025-08-01 14:41:01 -07:00 |
|
Simo Lin
|
66a398f49d
|
[router] migrate router from actix to axum (#8479)
|
2025-07-30 17:47:19 -07:00 |
|
Rui Chen
|
a730ce8162
|
[feature] [sgl-router] Add a dp-aware routing strategy (#6869)
|
2025-07-30 05:58:48 -07:00 |
|
Simo Lin
|
fe6a445d1e
|
[router] improve router logs and request id header (#8415)
|
2025-07-27 19:30:19 -07:00 |
|
Simo Lin
|
2ab97023e3
|
[router] add different policies for p node and d node (#8395)
|
2025-07-27 00:39:20 -07:00 |
|
Simo Lin
|
f6e07f2796
|
[router] fix pd model completion request (#8303)
|
2025-07-23 23:18:29 -07:00 |
|
Rui Chen
|
c33499a67b
|
fix: sgl-router remove dead code (#8257)
|
2025-07-22 08:41:23 -07:00 |
|
Simo Lin
|
5c8365a051
|
[router] add ut for pd router (#8208)
|
2025-07-20 23:12:52 -07:00 |
|
Simo Lin
|
1fc455e8b6
|
[router] add ut for pd request, metrics and config (#8184)
|
2025-07-20 10:53:42 -07:00 |
|
Simo Lin
|
41d33e4736
|
[router] add ut for worker and errors (#8170)
|
2025-07-19 14:38:33 -07:00 |
|
Charles Chen
|
f3d9736156
|
Fix suffix mismatch for the metrics. (#8168)
Signed-off-by: Charles Chen <chenliqian@chenliqian.cn>
|
2025-07-19 10:11:24 -07:00 |
|
Simo Lin
|
8fcc55cfa1
|
[router] router metrics cleanup (#8158)
|
2025-07-18 22:09:17 -07:00 |
|
Simo Lin
|
c8f31042a8
|
[router] Refactor router and policy traits with dependency injection (#7987)
Co-authored-by: Jin Pan <jpan236@wisc.edu>
Co-authored-by: Keru Yang <rukeyang@gmail.com>
Co-authored-by: Yingyi Huang <yingyihuang2000@outlook.com>
Co-authored-by: Philip Zhu <phlipzhux@gmail.com>
|
2025-07-18 14:24:24 -07:00 |
|
Simo Lin
|
f2d5c4920e
|
[router] add worker abstraction (#7960)
|
2025-07-11 20:17:48 -07:00 |
|
Simo Lin
|
4ed57807c2
|
[bugfix] add pd router policy validation (#7904)
|
2025-07-09 21:52:06 -07:00 |
|
Shuaiyi Zhang
|
7590f5224b
|
[router] Update metrics when request completes (#7899)
|
2025-07-09 08:18:09 -07:00 |
|
Shuaiyi Zhang
|
51ae40306a
|
[router] forward stream_options in request (#7860)
|
2025-07-08 08:03:38 -07:00 |
|
Simo Lin
|
ea3e7ffec7
|
[bugfix] Fix sgl-router get_server_info endpoint compatibility issue (#7813)
|
2025-07-06 19:52:57 -07:00 |
|