Vishwanath Venkatesan
|
2cd2e27f80
|
SGLang HiCache NIXL Connector (#8488)
Signed-off-by: Vishwanath Venkatesan <vvenkatesan@nvidia.com>
Co-authored-by: Moein Khazraee <moein@nvidia.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-07-31 13:09:42 -07:00 |
|
Chang Su
|
743638bc03
|
misc: Remove debug print to logger.info (#8633)
|
2025-07-31 12:56:52 -07:00 |
|
Michael Yao
|
061c8959ff
|
Fix typos in py_test/test_launch_server.py (#6227)
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
|
2025-07-31 12:48:47 -07:00 |
|
Brayden Zhong
|
4acf690206
|
[Optimization][Perf] Disable the GC during CUDA graph capture to speed up by up to 3x (#8577)
|
2025-07-31 11:31:21 -07:00 |
|
Simo Lin
|
aee0ef52f5
|
[router] update router pypi version (#8628)
|
2025-07-31 11:24:12 -07:00 |
|
Simo Lin
|
ae807774f5
|
[ci] fix genai-bench execution cmd (#8629)
|
2025-07-31 10:40:54 -07:00 |
|
Ke Bao
|
8fbcfd0723
|
Update step3v default config (#8626)
|
2025-08-01 00:49:26 +08:00 |
|
Ke Bao
|
3c307dc057
|
Fix hf3fs_fuse import error (#8623)
|
2025-07-31 22:42:31 +08:00 |
|
Tao He
|
5d15fb8c9d
|
[bugifx] QWen-1M context support[2/3] using current cuda stream in the DCA's kernel for bugfix. (#8611)
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: sa-buc <linzhu.ht@w32d09270.cloud.sqa.na131>
|
2025-07-31 22:41:39 +08:00 |
|
Shangming Cai
|
016fd25127
|
[PD] Use batch transfer for rdma transport and add notes for mnnvl usage (#8595)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-07-31 21:29:34 +08:00 |
|
Yineng Zhang
|
023288645b
|
chore: bump v0.4.10 (#8608)
|
2025-07-31 20:50:17 +08:00 |
|
Cheng Wan
|
7a1f7fc504
|
[Feature] Hybrid EP and TP (#8590)
|
2025-07-31 02:53:25 -07:00 |
|
Chang Su
|
51c38163c1
|
model: support Step3V (#8583)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: nnnobody-code <nnnobody@foxmail.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Qiaolin-Yu <qy254@cornell.edu>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
|
2025-07-31 02:41:00 -07:00 |
|
yihong
|
09f1a247ce
|
fix: fork should not run pypi router (#8604)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
|
2025-07-31 02:37:13 -07:00 |
|
Cheng Wan
|
32fa1e9cc2
|
[4/N] MoE Refactor: Unified Triton Kernel for FusedMoE and EPMoE (#8515)
|
2025-07-31 02:34:02 -07:00 |
|
Liangsheng Yin
|
e7dc163f57
|
add SVG logo (#8603)
|
2025-07-31 15:56:26 +08:00 |
|
Cheng Wan
|
e179e0b797
|
update sgl-kernel for EP: python part (#8550)
|
2025-07-31 00:14:39 -07:00 |
|
huangtingwei
|
d904959233
|
Support l3 cache (mooncake store) for hiradix cache (#7211)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: AniZpZ <zhuangsen.zp@antgroup.com>
Co-authored-by: zuoyuan <zhangzuo21@mails.tsinghua.edu.cn>
Co-authored-by: @wangyueneng.wyn <wangyueneng.wyn@antgroup.com>
Co-authored-by: JinYan Su <jinyansu792@gmail.com>
|
2025-07-30 23:15:51 -07:00 |
|
huangtingwei
|
26c8a310bd
|
fix incorrect increase of hit count (#8533)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-07-31 06:02:42 +00:00 |
|
yi wang
|
5963e50503
|
[bugfix] Fix 2 minor bugs in the hicache storage layer (#8404)
|
2025-07-31 05:47:14 +00:00 |
|
Yineng Zhang
|
43118f5f2a
|
chore: bump sgl-kernel v0.2.8 (#8599)
|
2025-07-30 22:23:52 -07:00 |
|
Cheng Wan
|
a5f5ab4030
|
update sgl-kernel for EP: kernel part (#8514)
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
|
2025-07-30 22:19:55 -07:00 |
|
Binyao Jiang
|
59aab76f0a
|
Bug: Fix google gemma3n-mm audio input not working bug (#8365)
|
2025-07-30 21:23:09 -07:00 |
|
Charles Chen
|
659bfd1023
|
Add GKE's default CUDA runtime lib location to PATH and LD_LIBRARY_PATH. (#8544)
|
2025-07-30 20:28:07 -07:00 |
|
Lifu Huang
|
67e53b16f5
|
Bump transfomers to 4.54.1 to fix Gemma cache issue. (#8541)
|
2025-07-30 19:50:54 -07:00 |
|
Qi Yuhang
|
9b9e82539b
|
[Fix]Fix index oob in get_group_gemm_starts kernel. (#8564)
|
2025-07-30 19:49:35 -07:00 |
|
Simo Lin
|
66a398f49d
|
[router] migrate router from actix to axum (#8479)
|
2025-07-30 17:47:19 -07:00 |
|
pansicheng
|
299803343d
|
Add hf3fs support for hicache storage (based on #7704) (#7280)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-07-30 17:42:41 -07:00 |
|
Chang Su
|
a79a5d7012
|
Revert "Fix the input tools format and history tool_calls in OpenAI API (#6556)" (#8584)
|
2025-07-30 13:12:05 -07:00 |
|
Adarsh Shirawalmath
|
ec5f944271
|
[Model] Add support for Arcee Foundational Model (#8154)
|
2025-07-30 10:45:25 -07:00 |
|
Yuan Luo
|
3bdcdd134b
|
[Hot-Fix] moe_aligned_block_size CI failed in AMD (#8461)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
|
2025-07-31 00:28:32 +08:00 |
|
Rui Chen
|
a730ce8162
|
[feature] [sgl-router] Add a dp-aware routing strategy (#6869)
|
2025-07-30 05:58:48 -07:00 |
|
Shangming Cai
|
55ecdc0a8e
|
Update CODEOWNERS (#8562)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-07-30 01:05:57 -07:00 |
|
Elfie Guo
|
e3f08c77bc
|
Update cutlass_moe.py (#8545)
|
2025-07-29 23:46:34 -07:00 |
|
Simo Lin
|
a9fd80336d
|
[router] allow longer time out for router e2e (#8560)
|
2025-07-29 23:43:37 -07:00 |
|
hzh0425
|
2fbb754e1d
|
feature(pd-hicache): Prefill instances support reusing the RemoteStorage Cache via HiCache. (#8516)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-07-29 21:19:25 -07:00 |
|
hzh0425
|
a85ebf50b8
|
feat(hicache): support file backend reading directory config form env. (#8498)
|
2025-07-29 21:18:46 -07:00 |
|
Cheng Wan
|
9effeb5bdd
|
Support EPLB in FusedMoE (#8448)
|
2025-07-29 16:02:41 -07:00 |
|
Mick
|
1992ef9ba7
|
fix: temporarily disable cuda-ipc for mm data tensor (#8431)
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2025-07-29 22:42:03 +00:00 |
|
Stefan He
|
c0fd77e839
|
bring back kimi vl ci (#8537)
|
2025-07-29 13:14:18 -07:00 |
|
Lianmin Zheng
|
a4c3b121d8
|
Split the scheduler into multiple mixin classes to reduce the file size (#8483)
|
2025-07-29 12:46:50 -07:00 |
|
Ke Bao
|
5973675bc3
|
Fix moe align kernel test (#8531)
|
2025-07-29 11:03:02 -07:00 |
|
Elfie Guo
|
4d16c88b6e
|
Update cutlass_moe.py (#8535)
|
2025-07-29 10:49:41 -07:00 |
|
Xiaoyu Zhang
|
7a4309cc8a
|
[sgl-kernel performace] fix fp8 quant kernels dispatch __nv_fp8_e4m3 bug to improve performance 10%-20% (#8499)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
|
2025-07-29 23:31:54 +08:00 |
|
Lianmin Zheng
|
813670660c
|
Update README.md (#8528)
|
2025-07-29 04:09:19 -07:00 |
|
Lianmin Zheng
|
263c9236a0
|
Always trigger pr-test (#8527)
|
2025-07-29 04:05:19 -07:00 |
|
Yineng Zhang
|
6478831be9
|
chore: bump v0.4.9.post6 (#8517)
|
2025-07-29 02:30:07 -07:00 |
|
TimWang
|
2e1d2d7e66
|
Add PVC and update resource limits in k8s config (#8489)
|
2025-07-28 23:15:31 -07:00 |
|
Lifu Huang
|
fb16fbaf52
|
Fix incorrect KV cache allocation for MTP models. (#8482)
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
|
2025-07-28 22:54:50 -07:00 |
|
fzyzcjy
|
0ce84c822b
|
Support colocating requests (#7973)
|
2025-07-28 22:51:49 -07:00 |
|