Yusong Gao
|
c35dcfdb30
|
[PD] fix: skip warmup request in disaggregation mode to prevent crash on timeout (#5292)
|
2025-04-11 23:03:07 +08:00 |
|
Jon Durbin
|
04eb6062e4
|
Include context length in /v1/models response. (#4809)
|
2025-03-27 20:23:18 -07:00 |
|
fzyzcjy
|
52029bd1e3
|
Fix warmup error when dp=1 (#4753)
|
2025-03-25 02:01:21 -07:00 |
|
fzyzcjy
|
eb934bdf3b
|
Fix test_expert_distribution failure (#4752)
|
2025-03-25 01:17:03 -07:00 |
|
yuhsaun-t
|
199bb01d00
|
Add endpoints to dump selected expert ids (#4435)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-03-24 21:34:19 -07:00 |
|
fzyzcjy
|
6b7038babd
|
Speedup warmup when DP > 1 (#4695)
|
2025-03-24 21:08:05 -07:00 |
|
Rin Intachuen
|
d1112d8548
|
Add endpoint for file support, purely to speed up processing of input_embeds. (#2797)
|
2025-03-16 18:30:37 -07:00 |
|
Ying Sheng
|
1b859295f4
|
[Eagle] Remove the greedy branch and some redundant code (#4363)
Co-authored-by: Sehoon Kim <sehoon@x.ai>
|
2025-03-16 02:48:55 -07:00 |
|
Wang Ran (汪然)
|
158430473e
|
Fix typos (#4368)
|
2025-03-15 21:27:58 -07:00 |
|
Wang Ran (汪然)
|
91b19949d7
|
typo: Update http_server.py (#4350)
|
2025-03-12 15:05:30 -07:00 |
|
Xihuai Wang
|
95575aa76a
|
Reasoning parser (#4000)
Co-authored-by: Lucas Pickup <lupickup@microsoft.com>
|
2025-03-03 21:16:36 -08:00 |
|
Lianmin Zheng
|
935cda944b
|
Misc clean up; Remove the support of jump forward (#4032)
|
2025-03-03 07:02:14 -08:00 |
|
Lianmin Zheng
|
ac2387279e
|
Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>
|
2025-03-03 00:12:04 -08:00 |
|
Chayenne
|
930da877c4
|
rename FunctionCallReqInput to ParseFunctionCallReq (#3976)
|
2025-02-28 18:46:25 -08:00 |
|
KCFindstr
|
bc20e93f2d
|
[feat] Add Vertex AI compatible prediction route for /generate (#3866)
|
2025-02-27 19:42:15 -08:00 |
|
Lianmin Zheng
|
f2388f6b95
|
Revert "Rename TokenizerManager to StdOrchestrator" (#3828)
|
2025-02-24 14:47:59 -08:00 |
|
fzyzcjy
|
45360b2fa9
|
Improve: Rename TokenizerManager to StdOrchestrator (#3116)
|
2025-02-23 00:30:58 -08:00 |
|
Andrew Smith
|
1df6eabd5d
|
feat: Add SageMaker support (#3740)
|
2025-02-21 19:31:09 +08:00 |
|
YAMY
|
b045841bae
|
Feature/function calling update (#2700)
Co-authored-by: Mingyuan Ma <mamingyuan2001@berkeley.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
|
2025-01-26 09:57:51 -08:00 |
|
Lianmin Zheng
|
03464890e0
|
Separate two entry points: Engine and HTTP server (#2996)
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
|
2025-01-19 22:09:24 -08:00 |
|