Praneth Paruchuri
|
cb736df854
|
Support for Phi-1.5 & Phi-2 models (#7862)
|
2025-07-13 18:43:40 -07:00 |
|
Lifu Huang
|
e2ed9d049a
|
Refactor dynamic LoRA update to fix incorrect handling of variant weight shapes (#7844)
|
2025-07-13 18:36:01 -07:00 |
|
Yineng Zhang
|
22bd857cb5
|
docs: update README (#7985)
|
2025-07-12 13:31:11 -07:00 |
|
Yineng Zhang
|
eb118d88c4
|
chore: bump v0.4.9.post2 (#7963)
|
2025-07-11 21:11:20 -07:00 |
|
ronnie_zheng
|
86044712c6
|
[feature] kv transfer support of ascend npu (#7795)
Co-authored-by: liupeng <liupeng374@huawei.com>
|
2025-07-11 00:07:51 -07:00 |
|
Atream
|
615553079d
|
Support Kimi K2 (#7940)
|
2025-07-11 00:02:21 -07:00 |
|
Binyao Jiang
|
2d54d4bb64
|
Feat: Support Phi-3.5-MoE in SGLang (#7907)
|
2025-07-09 23:51:33 -07:00 |
|
Yineng Zhang
|
066f4ec91f
|
chore: bump v0.4.9.post1 (#7882)
|
2025-07-09 00:28:17 -07:00 |
|
Yikai Zhang
|
0870232195
|
Update native_api doc to match the change in the get_model_info endpoint (#7660)
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
|
2025-07-08 21:05:58 -07:00 |
|
Shangming Cai
|
64c5907e12
|
[PD] Add guidance for prefill bootstrap timeout (#7846)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-07-08 21:00:34 -07:00 |
|
Xinyuan Tong
|
43e20c0647
|
Support Mimo-VL (#7579)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-07-08 14:00:25 -07:00 |
|
Yineng Zhang
|
ec5f9c6269
|
chore: bump v0.4.9 (#7802)
|
2025-07-05 17:40:29 -07:00 |
|
Yuchen Cheng
|
1e3e3add3d
|
fix(docs): fix the broken link in docs/references/production_metrics.md (#7741)
Signed-off-by: rudeigerc <rudeigerc@gmail.com>
|
2025-07-03 23:46:07 -07:00 |
|
Xinyuan Tong
|
43f93f632c
|
fix CI: update native api ipynb (#7754)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-07-03 15:25:00 -07:00 |
|
ronnie_zheng
|
1e0e549766
|
Ascend attention backend(PA&MLA) (#7722)
Co-authored-by: Maksim <makcum888e@mail.ru>
Co-authored-by: VDV1985 <vladdv85@mail.ru>
|
2025-07-03 09:23:19 -07:00 |
|
Yi Zhang
|
93b6785d78
|
add description for llama4 eagle3 (#7688)
|
2025-07-01 01:19:19 -07:00 |
|
ybyang
|
7349717e4b
|
[doc] update lws doc for pd (#7318)
|
2025-07-01 10:39:04 +08:00 |
|
Lianmin Zheng
|
22352d47a9
|
Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632)
Co-authored-by: Kan Wu <wukanustc@gmail.com>
|
2025-06-29 23:16:19 -07:00 |
|
tarinkk
|
eb6c2c1663
|
Hybrid kv cache for LLaMA4 (#6563)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: tarinkk <rt572@physics.rutger.edu>
Co-authored-by: tarinkk <rt572@rutgers.physics.edu>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
|
2025-06-27 18:58:55 -07:00 |
|
Yineng Zhang
|
69183f8808
|
chore: bump v0.4.8.post1 (#7559)
|
2025-06-26 02:21:12 -07:00 |
|
Shangming Cai
|
5c2142579a
|
[PD] Raise error for incompatible mooncake version and some minor fixes (#7527)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-25 18:55:24 -07:00 |
|
Yineng Zhang
|
7c3a12c000
|
chore: bump v0.4.8 (#7493)
|
2025-06-23 23:14:22 -07:00 |
|
Lianmin Zheng
|
30ceccc74a
|
Update hyperparameter_tuning.md (#7454)
|
2025-06-22 22:42:55 -07:00 |
|
Chang Su
|
72676cd6c0
|
feat(oai refactor): Replace openai_api with entrypoints/openai (#7351)
Co-authored-by: Jin Pan <jpan236@wisc.edu>
|
2025-06-21 13:21:06 -07:00 |
|
Jinn
|
ab74f8f09d
|
Remove batches api in docs & example (#7400)
|
2025-06-20 19:46:31 -07:00 |
|
woodx
|
97011abc8a
|
[Doc] add embedding rerank doc (#7364)
|
2025-06-19 21:53:54 -07:00 |
|
Yineng Zhang
|
fadf18fdd5
|
docs: update installation (#7366)
|
2025-06-19 12:00:19 -07:00 |
|
linzhuo
|
1de4db9bef
|
update invalid link in doc (#7297)
|
2025-06-18 01:37:36 -07:00 |
|
Yijie Zhu
|
a39d928782
|
support qwen2 running on ascend npu device (#7022)
Co-authored-by: 刁莹煜 <diaoyingyu1@hisilicon.com>
|
2025-06-17 11:24:10 -07:00 |
|
Yineng Zhang
|
f9dc9dd28b
|
chore: bump v0.4.7.post1 (#7248)
|
2025-06-16 15:20:29 -07:00 |
|
Lianmin Zheng
|
21615cc3fe
|
Minor style and doc fix (#7228)
|
2025-06-16 01:03:13 -07:00 |
|
Lifu Huang
|
98538822d5
|
Add Phi-4-mm to supported VLM supported model list. (#7178)
|
2025-06-13 23:17:40 -07:00 |
|
Povilas Kanapickas
|
bd7cfbd2f8
|
[Fix] Reduce busy polling when scheduler is idle (#6026)
|
2025-06-12 14:58:22 -07:00 |
|
Lianmin Zheng
|
dbdf76ca98
|
Clean up docs for server args and sampling parameters (generated by grok) (#7076)
|
2025-06-10 19:55:42 -07:00 |
|
Ximingwang-09
|
f2a75a66c4
|
update doc (#7046)
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
|
2025-06-11 10:02:01 +08:00 |
|
Lianmin Zheng
|
0f218731e3
|
Do not run frontend_reasoning.ipynb to reduce the CI load (#7073)
|
2025-06-10 17:15:31 -07:00 |
|
Yudi Xue
|
14c18d25df
|
Frontend language separate reasoning support (#6031)
|
2025-06-10 17:11:29 -07:00 |
|
Lianmin Zheng
|
90bd3e32d6
|
Improve perf tuning docs (#7071)
|
2025-06-10 16:55:04 -07:00 |
|
kyle-pena-kuzco
|
b56de8f943
|
Open AI API hidden states (#6716)
|
2025-06-10 14:37:29 -07:00 |
|
Lianmin Zheng
|
bb185b0e92
|
Update README.md (#7040)
|
2025-06-10 01:59:14 -07:00 |
|
Yineng Zhang
|
4f723edd3b
|
chore: bump v0.4.7 (#7038)
|
2025-06-10 01:56:20 -07:00 |
|
Yueyang Pan
|
98c00a2df1
|
Fix torch profiler bugs for bench_offline_throughput.py (#6557)
|
2025-06-09 20:33:41 +08:00 |
|
HAI
|
b819381fec
|
AITER backend extension and workload optimizations (#6838)
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Hubert Lu <Hubert.Lu@amd.com>
|
2025-06-05 23:00:18 -07:00 |
|
shangmingc
|
dd1012fcbe
|
[PD] Fix potential perf spike caused by tracker gc and optimize doc (#6764)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-05 10:56:02 -07:00 |
|
zyksir
|
8e3797be1c
|
support 1 shot allreduce in 1-node and 2-node using mscclpp (#6277)
|
2025-06-04 22:11:24 -07:00 |
|
Xinyuan Tong
|
cf9815ba69
|
[Refactor] Multimodal data processing for VLM (#6659)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-06-04 11:22:33 -07:00 |
|
Marc Sun
|
37f1547587
|
[FEAT] Add transformers backend support (#5929)
|
2025-06-03 21:05:29 -07:00 |
|
Lianmin Zheng
|
2d72fc47cf
|
Improve profiler and integrate profiler in bench_one_batch_server (#6787)
|
2025-05-31 15:53:55 -07:00 |
|
shangmingc
|
6cb00c6398
|
[PD] Optimize time out logic and add env var doc for mooncake (#6761)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-30 00:45:02 -07:00 |
|
Baizhou Zhang
|
791b3bfabb
|
[Feature] Support Flashinfer fp8 blockwise GEMM kernel on Blackwell (#6479)
|
2025-05-28 16:03:43 -07:00 |
|