Simo Lin
|
5dccf69713
|
[router] create worker removal step and clean up worker manager (#11921)
|
2025-10-22 13:26:06 -07:00 |
|
Chang Su
|
590bc4b7a7
|
[router][grpc] Fix background tasks stored with wrong id (#11945)
|
2025-10-21 18:38:51 -07:00 |
|
Chang Su
|
70f6309cd4
|
[router][grpc] Support v1/responses API (#11926)
|
2025-10-21 17:41:48 -07:00 |
|
Chang Su
|
e69094df64
|
[router][grpc] Remove continue_final_message in ChatTemplateParams and add minijinja-contrib (#11882)
|
2025-10-20 18:03:09 -07:00 |
|
ybyang
|
d513ee93ef
|
[2/2] [feature] support openai like classification api in router (#11670)
|
2025-10-18 19:31:08 -07:00 |
|
Simo Lin
|
a7ae61ed77
|
[router] Add Configurable L0 and L1 Tokenizer Caching (#11688)
|
2025-10-18 18:33:53 -07:00 |
|
Chang Su
|
ca240eefb4
|
[router][grpc] Support parallel queue puts in grpc_request_manager and remove mutex for grpc_client (#11798)
|
2025-10-17 20:49:43 -07:00 |
|
Chang Su
|
dc01313da1
|
[router] Add rustfmt and set group imports by default (#11732)
|
2025-10-16 17:33:29 -07:00 |
|
Simo Lin
|
64affab495
|
[router] fix p and d worker filtering and bootstrap port handling (#11729)
|
2025-10-16 14:19:39 -07:00 |
|
Keyang Ru
|
4c9bcb9d56
|
[Router] Refactor protocol definitions: split spec.rs into modular files (#11677)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-10-16 13:44:44 -07:00 |
|
Simo Lin
|
0b9915c132
|
[router] update generate spec to align with sgl io struct (#11591)
|
2025-10-14 02:51:33 -04:00 |
|
Chang Su
|
27ef1459e6
|
[router][protocols] Add Axum validate extractor and use it for /v1/chat/completions endpoint (#11588)
|
2025-10-13 22:51:15 -07:00 |
|
Chang Su
|
4b694e7d5a
|
[router][grpc] Add error handling to generate_tool_constraints (#11562)
|
2025-10-13 12:26:09 -07:00 |
|
Chang Su
|
7b59b0b8b0
|
[router][grpc] Further delegate non-stream processing to processing.rs (#11553)
|
2025-10-13 10:36:27 -07:00 |
|
Chang Su
|
92777135a0
|
[router][grpc] Consolidate parser checks for chat completions (#11439)
|
2025-10-10 20:44:29 -04:00 |
|
Simo Lin
|
c495833186
|
[router] leverage RAII to actively cancel request during client disconnect (#11399)
|
2025-10-10 20:43:38 -04:00 |
|
Chang Su
|
ab926dd697
|
[router][grpc] Fix streaming bugs: empty tool names, state pollution, and panics (#11373)
|
2025-10-09 06:53:23 -04:00 |
|
Chang Su
|
a1080b72a0
|
[router] Fix all unused_qualifications (#11341)
|
2025-10-08 13:55:27 -07:00 |
|
Chang Su
|
a65ca73911
|
[router][grpc] Cleanup debug logs in grpc_server and grpc_router (#11340)
|
2025-10-08 13:26:19 -07:00 |
|
Simo Lin
|
677aa0e25f
|
[router] improve reasoning parser lock and reduce req cloning (#11336)
|
2025-10-08 11:18:15 -07:00 |
|
Simo Lin
|
01c9ee1ab4
|
[router] refactor generate to use new pipeline arch (#11323)
|
2025-10-08 09:38:50 -07:00 |
|
Chang Su
|
edd86b8853
|
[router][grpc] Refactor chat handler in grpc/ to use centralized orchestrator (#11314)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2025-10-07 20:50:20 -07:00 |
|
Chang Su
|
420c99acfe
|
[router][grpc] Fix error message format in grpc chat handler (#11307)
|
2025-10-07 13:54:02 -07:00 |
|
Simo Lin
|
79d3495177
|
[router] add reasoning and tool parser argument in router (#11290)
|
2025-10-07 09:08:32 -04:00 |
|
Chang Su
|
b07c9c76c5
|
[router][grpc] Refine streaming processes (#11277)
|
2025-10-06 15:15:01 -07:00 |
|
Chang Su
|
466992b2d0
|
[router][tool call] Clean up redundant detect_format and has_tool_markers (#11270)
|
2025-10-06 14:04:02 -07:00 |
|
Simo Lin
|
d736e0b65e
|
[router] add grpc router pd mode for chat and generate (#11140)
|
2025-10-04 06:58:28 -07:00 |
|
Chang Su
|
963175d5c0
|
[router][grpc] Support streaming for v1/chat/completions (#11179)
|
2025-10-02 14:35:16 -07:00 |
|
Chang Su
|
b658be6f6a
|
[router][grpc] Support tool call parser in streaming (#11160)
|
2025-10-02 03:18:50 -07:00 |
|
Chang Su
|
8ce830a8b0
|
[router][bugfix] Fix input_logprobs handling with None value and logprob_start_len = -1 (#11113)
|
2025-09-30 16:09:40 -07:00 |
|
Chang Su
|
d1676cd483
|
[router][tool call] Full support for ToolChoice (#11085)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2025-09-29 22:36:03 -07:00 |
|
Simo Lin
|
33b3c0f85f
|
[router] grpc router generate endpoint support (#11070)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-09-29 22:07:53 -07:00 |
|
Chang Su
|
5937a56d47
|
[router][grpc] Add logprobs support to router (#11082)
|
2025-09-29 15:55:06 -07:00 |
|
Chang Su
|
f065e5bea5
|
[router] Use get_pooled in process_single_choice (#11079)
|
2025-09-29 15:48:00 -07:00 |
|
Chang Su
|
c1c8dd1dd0
|
[router][tool parser] Modify tool parser to return both normal text and tool calls (non-stream) (#10995)
|
2025-09-27 18:10:17 -04:00 |
|
Chang Su
|
37f3325b06
|
[router][grpc] Support E2E non-stream chat completions (#10980)
|
2025-09-26 22:02:06 -07:00 |
|
Chang Su
|
0c3db88978
|
[router][grpc] Add helpfer functions for decoder in router.rs and fix specs (#10971)
|
2025-09-26 20:10:45 -04:00 |
|
Simo Lin
|
aae7ead2d0
|
[router] remove old/oudated/useless comments across code base (#10968)
|
2025-09-26 10:48:50 -07:00 |
|
Simo Lin
|
be059b83d6
|
[router] grpc router regular mode import cleanup (#10963)
|
2025-09-26 04:06:59 -07:00 |
|
Simo Lin
|
5d4fe1ceee
|
[router] add move grpc worker management from router to worker manager (#10960)
|
2025-09-26 03:57:57 -07:00 |
|
Simo Lin
|
1b011e68dc
|
[router] move grpc client from router to worker and builder (#10958)
|
2025-09-26 03:13:47 -07:00 |
|
Simo Lin
|
1e57b9472d
|
[router] add grpc client get and set (#10955)
|
2025-09-26 03:07:05 -07:00 |
|
Chang Su
|
37158f2018
|
router: Support parallel sampling num > 1 in grpc_server and non-stream handling (#10929)
|
2025-09-25 20:03:35 -07:00 |
|
Chang Su
|
5e21d6aec0
|
refactor: Move grpc/client.rs to grpc_client/sglang_scheduler.rs (#10924)
|
2025-09-25 17:21:22 -04:00 |
|
Chang Su
|
916784746b
|
router: Fix constraint proto and build_constraint in grpc router (#10881)
|
2025-09-25 11:12:06 -04:00 |
|
Simo Lin
|
e738703547
|
[router] consolidate worker get loads (#10880)
|
2025-09-24 22:13:31 -04:00 |
|
Simo Lin
|
7a06ef984d
|
[router] consolidate health endpoints and flush cache (#10876)
|
2025-09-24 15:23:21 -07:00 |
|
Chang Su
|
4a87ba217f
|
router-grpc: Add tools processing and other paramters for apply_chat_template (#10877)
|
2025-09-24 15:23:06 -07:00 |
|
Chang Su
|
9209b209be
|
router-grpc: Support jinja chat template content format detection (#10832)
|
2025-09-24 11:45:01 -07:00 |
|
Chang Su
|
7ff93e613f
|
router(grpc): Implement route for chat_cmpl endpoint (#10761)
|
2025-09-23 11:26:33 -07:00 |
|