Simo Lin
|
5343058875
|
[router] grpc router bootstraps (#9759)
|
2025-08-28 12:07:06 -07:00 |
|
Simo Lin
|
07c9d8fba2
|
[router] add llama3.2 multi json streaming parser (#9735)
|
2025-08-28 05:57:13 -07:00 |
|
Simo Lin
|
e1f7cf57dc
|
[router] additional llama32 parser unit test and multi json support (#9732)
|
2025-08-27 20:34:11 -07:00 |
|
Simo Lin
|
2bb9d454b5
|
[router] additional pythonic parser unit test (#9730)
|
2025-08-27 19:55:59 -07:00 |
|
Keyang Ru
|
3f2d0cefcd
|
[router] Add MCP Tool Handler (#9615)
|
2025-08-27 19:12:39 -07:00 |
|
Simo Lin
|
07ee0ab750
|
[router] add gpt-oss and glm4 tool parser (#9703)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-08-27 11:26:00 -07:00 |
|
Simo Lin
|
5c06dcb75a
|
[router] add kimi-k2 tool parser (#9702)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-08-27 11:04:55 -07:00 |
|
Simo Lin
|
6f6beca49d
|
[router] add step3 tool parser (#9695)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-08-27 10:44:52 -07:00 |
|
Simo Lin
|
6e4e1c8cdc
|
[router] add deepseek tool parser (#9694)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-08-27 06:18:24 -07:00 |
|
Chang Su
|
90313fb09a
|
[router] add token bucket rate limiter (#9656)
|
2025-08-26 10:36:26 -07:00 |
|
Stefan He
|
cbc0e4d779
|
Fix lint for router (#9636)
|
2025-08-26 00:38:53 -07:00 |
|
Simo Lin
|
e2e378caba
|
[router] add ut for mistral, llama, pythonic, and streaming tool parser (#9632)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-08-25 22:02:15 -07:00 |
|
Keyang Ru
|
5ef545e678
|
[router] Move all protocols to spec.rs file (#9519)
|
2025-08-22 14:18:47 -07:00 |
|
Chang Su
|
53e2cd464f
|
[router] remove all tokenizer metrics for performance (#9474)
|
2025-08-21 18:35:24 -07:00 |
|
Simo Lin
|
78ae175866
|
[router] add tokenizer benchmark (#9427)
|
2025-08-21 11:09:39 -07:00 |
|
Chang Su
|
e65231022f
|
[router] add tokenizer integration test with real mini tokenizer (#9413)
|
2025-08-20 17:56:23 -07:00 |
|
Keyang Ru
|
3828db4309
|
[router] Add IGW (Inference Gateway) Feature Flag (#9371)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-08-20 17:38:57 -07:00 |
|
Keyang Ru
|
5ae5ecaa15
|
[router] Implement OpenAI Responses API specification (#9367)
|
2025-08-19 20:14:47 -07:00 |
|
Simo Lin
|
5fbad308cd
|
[router] add tokenizer chat template support (#9370)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-08-19 20:14:02 -07:00 |
|
Keyang Ru
|
ce67b2d586
|
[router]restructure protocol modules for better organization (#9321)
|
2025-08-19 01:07:58 +00:00 |
|
Jeff Nettleton
|
ce3ca9b02f
|
[router] add cargo clippy in CI and fix-up linting errors (#9242)
|
2025-08-17 11:03:56 -07:00 |
|
Simo Lin
|
21b8846066
|
[router] allow more health check configuration (#9198)
|
2025-08-15 08:07:45 -07:00 |
|
Simo Lin
|
9d68bdb240
|
[router] Add Rust Binary Entrypoint for SGLang Router (#9089)
|
2025-08-11 21:37:36 -07:00 |
|
Simo Lin
|
067068f271
|
[router] regular router circuit breaker (#8997)
|
2025-08-10 21:19:30 -07:00 |
|
Simo Lin
|
61a4680494
|
[router] router circuit breaker core (#8941)
|
2025-08-08 09:20:22 -07:00 |
|
Simo Lin
|
a69b637014
|
[router] fix req handling order, improve serialization, remove retry (#8888)
|
2025-08-06 23:24:39 -07:00 |
|
Simo Lin
|
8c7bb39dfb
|
[router] PD Router Simplification and Reorganization (#8838)
|
2025-08-05 21:20:38 -07:00 |
|
Simo Lin
|
5d62b56f7e
|
[router] complete router oai spec (#8828)
|
2025-08-05 18:30:19 -07:00 |
|
Simo Lin
|
354ac43555
|
[pd-router] Add Configurable Retry Logic for reduce backend pressure (#8744)
|
2025-08-04 20:42:07 -07:00 |
|
Simo Lin
|
828a4fe944
|
[router] Implement HTTP Dependency Injection Pattern for Router System (#8714)
|
2025-08-02 19:16:47 -07:00 |
|
Simo Lin
|
66a398f49d
|
[router] migrate router from actix to axum (#8479)
|
2025-07-30 17:47:19 -07:00 |
|
Rui Chen
|
a730ce8162
|
[feature] [sgl-router] Add a dp-aware routing strategy (#6869)
|
2025-07-30 05:58:48 -07:00 |
|
Simo Lin
|
fe6a445d1e
|
[router] improve router logs and request id header (#8415)
|
2025-07-27 19:30:19 -07:00 |
|
Simo Lin
|
2ab97023e3
|
[router] add different policies for p node and d node (#8395)
|
2025-07-27 00:39:20 -07:00 |
|
Simo Lin
|
39fe1e880d
|
[router] add request format unit test (#8300)
|
2025-07-24 14:30:37 -07:00 |
|
Simo Lin
|
33c4b4d04e
|
[router] add streaming unit test (#8299)
|
2025-07-24 14:30:27 -07:00 |
|
Simo Lin
|
2f86f3ad62
|
[router] add endpoint unit test (#8298)
|
2025-07-23 23:26:44 -07:00 |
|
Simo Lin
|
f6e07f2796
|
[router] fix pd model completion request (#8303)
|
2025-07-23 23:18:29 -07:00 |
|
Simo Lin
|
6f8f4aeea4
|
[router] add common ut infra to mock worker and app (#8295)
|
2025-07-23 10:07:51 -07:00 |
|
Simo Lin
|
5c8365a051
|
[router] add ut for pd router (#8208)
|
2025-07-20 23:12:52 -07:00 |
|
Simo Lin
|
c8f31042a8
|
[router] Refactor router and policy traits with dependency injection (#7987)
Co-authored-by: Jin Pan <jpan236@wisc.edu>
Co-authored-by: Keru Yang <rukeyang@gmail.com>
Co-authored-by: Yingyi Huang <yingyihuang2000@outlook.com>
Co-authored-by: Philip Zhu <phlipzhux@gmail.com>
|
2025-07-18 14:24:24 -07:00 |
|
Simo Lin
|
f2d5c4920e
|
[router] add worker abstraction (#7960)
|
2025-07-11 20:17:48 -07:00 |
|
Shuaiyi Zhang
|
51ae40306a
|
[router] forward stream_options in request (#7860)
|
2025-07-08 08:03:38 -07:00 |
|
Simo Lin
|
3abc30364d
|
[ci] add router benchmark script and CI (#7498)
|
2025-06-25 01:28:25 -07:00 |
|
Simo Lin
|
30f2a44a96
|
[misc] Add PD service discovery support in router (#7361)
|
2025-06-22 17:54:14 -07:00 |
|
Simo Lin
|
09ae5b20f3
|
Merge PDLB (Prefill-Decode Load Balancer) into SGLang Router (#7096)
|
2025-06-19 02:28:15 +08:00 |
|