IAN
|
11e27d0926
|
[PD]: Support Muti Prefill in one node (#5704)
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
|
2025-04-26 00:30:47 +08:00 |
|
shangmingc
|
50eda8398e
|
[PD] Add kvargs table and thread pool for kvcache sender of mooncake (#5738)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-04-25 18:15:01 +08:00 |
|
Liangsheng Yin
|
c55550cbf0
|
[PD] Better logs (#5715)
|
2025-04-25 17:25:45 +08:00 |
|
Brayden Zhong
|
43fb95c2fa
|
[Model] Support ArcticForCausalLM architecture (Snowflake/snowflake-arctic-instruct) (#5078)
Co-authored-by: vincent-4 <vincentzhongy+githubvincent4@gmail.com>
|
2025-04-25 15:24:09 +08:00 |
|
Ravi Theja
|
7d9679b74d
|
Add MMMU benchmark results (#4491)
Co-authored-by: Ravi Theja Desetty <ravitheja@Ravis-MacBook-Pro.local>
|
2025-04-25 15:23:53 +08:00 |
|
Michael Yao
|
b5be56944b
|
[Doc] Fix a link to Weilin Zhao (#5706)
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
|
2025-04-25 02:02:27 +08:00 |
|
Ravi Theja
|
d2b8d0b8d8
|
Add example to use sgl engine with fastapi (#5648)
Co-authored-by: Ravi Theja Desetty <ravitheja@Ravis-MacBook-Pro.local>
|
2025-04-24 23:57:05 +08:00 |
|
Baizhou Zhang
|
a14654dd68
|
Fix weight loading bug for Deepseek v3+nextn (#5684)
|
2025-04-24 21:29:56 +08:00 |
|
Yuhong Guo
|
5d93a950ee
|
[BugFix] Fix combination of MTP and --n-share-experts-fusionwith R1 (#5707)
|
2025-04-24 21:13:51 +08:00 |
|
Mick
|
c998d04b46
|
vlm: enable radix cache for qwen-vl models (#5349)
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-04-23 20:35:05 -07:00 |
|
Yineng Zhang
|
7d0edf3cae
|
chore: bump sgl-kernel 0.1.0 (#5688)
|
2025-04-23 14:23:59 -07:00 |
|
Yineng Zhang
|
ce4ecba477
|
fix: only compile ApplyTokenBitmaskInplace cu124+ (#5686)
|
2025-04-23 14:17:42 -07:00 |
|
Yineng Zhang
|
b1f6d89b5f
|
fix: update truss bench_serving (#5683)
|
2025-04-23 13:28:35 -07:00 |
|
Michael Yao
|
7c99103f4c
|
[Doc] Fix two 404 links caused by sglang typo (#5667)
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
|
2025-04-23 23:21:55 +08:00 |
|
Lianmin Zheng
|
de071366cd
|
tune the threshold of gemma-2-27b-it in test_nightly_gsm8k_eval.py (#5677)
|
2025-04-23 05:31:17 -07:00 |
|
shangmingc
|
e0673969b9
|
[PD] Add support for dp attention with mooncake (#5530)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-04-23 17:20:27 +08:00 |
|
Yineng Zhang
|
127ff8982e
|
fix torchvision::nms not exist (#5671)
|
2025-04-23 02:17:21 -07:00 |
|
Yineng Zhang
|
8777a1d24b
|
fix gemma3 unit test (#5670)
|
2025-04-23 02:14:01 -07:00 |
|
Cheng Wan
|
711efe7814
|
Integrating PD disaggregation with DP attention and DeepEP (#5435)
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
|
2025-04-23 01:46:01 -07:00 |
|
Yineng Zhang
|
fbb5f229d4
|
fix awq_dequantize import (#5669)
|
2025-04-23 01:36:26 -07:00 |
|
Yineng Zhang
|
15fabcc07f
|
fix sgl-kernel unit tests (#5666)
|
2025-04-23 01:18:30 -07:00 |
|
Elfie Guo
|
e62c49557d
|
[1/2] Add FP8 Blockscale MoE CUTLASS kernel for Blackwell (#5281)
|
2025-04-22 22:28:20 -07:00 |
|
fzyzcjy
|
71d1785f2d
|
Remove unnecessary torch.full in DeepSeek (#5601)
|
2025-04-22 21:24:29 -07:00 |
|
Baizhou Zhang
|
3f87f83116
|
Fuse q_a_proj and kv_a_proj (#5619)
|
2025-04-22 20:35:08 -07:00 |
|
Baizhou Zhang
|
ce5412b62e
|
Turn on DeepGemm By Default and Update Doc (#5628)
|
2025-04-22 16:10:08 -07:00 |
|
Yineng Zhang
|
7282ab741a
|
fix: update bench_speculative (#5649)
|
2025-04-22 16:08:15 -07:00 |
|
HAI
|
b0feda090c
|
Revert "Support aiter RMSNorm in AMD" (#5646)
|
2025-04-22 15:20:24 -07:00 |
|
Ke Bao
|
6b6e748775
|
Remove q concat in FA3 backend for DeepSeek decode (#5638)
|
2025-04-22 11:43:12 -07:00 |
|
JieXin Liang
|
917324862e
|
[fix] reduce dp capture bs (#5634)
Co-authored-by: alcanerian <alcanerian@gmail.com>
|
2025-04-22 11:08:45 -07:00 |
|
lukec
|
2ed96c7a8a
|
fix flashmla bug (#5272)
|
2025-04-22 10:36:23 -07:00 |
|
saltyfish66
|
2aa3f5e2d0
|
[feature] Add H20 fp8_w8a8 FusedMoE config for --n-share-experts-fusion=16 (#5641)
Co-authored-by: yuethe <yuethe@tencent.com>
|
2025-04-22 09:33:13 -07:00 |
|
lambert0312
|
76d17c7ecb
|
Fix shared experts fusion error without quantization (#5632)
|
2025-04-22 09:22:26 -07:00 |
|
Connector Switch
|
70d040f904
|
[NFC] Remove duplicate compressed-tensors (#5640)
|
2025-04-22 09:10:25 -07:00 |
|
JieXin Liang
|
4418f599a5
|
Fix FA3 DeepSeek prefill performance regression (#5624)
Co-authored-by: ispobock <ispobaoke@gmail.com>
|
2025-04-22 01:41:41 -07:00 |
|
Yineng Zhang
|
04f2abcb34
|
fix: gemma 3 not use softcap (#5622)
|
2025-04-22 01:16:08 -07:00 |
|
JieXin Liang
|
506be6b892
|
[fix] fix compile_deep_gemm missing kv_b_proj (#5620)
|
2025-04-22 00:06:36 -07:00 |
|
JieXin Liang
|
2343d8df7d
|
[fix] force use deepgemm in compile_deep_gemm (#5618)
|
2025-04-21 21:36:02 -07:00 |
|
Michael Yao
|
92bb64bc86
|
[Doc] Fix a 404 link to llama-405b (#5615)
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
|
2025-04-21 20:39:37 -07:00 |
|
Ke Bao
|
11b23ae97b
|
Remove extra copy in deepseek forward absorb (#5578)
Co-authored-by: saienduri <saimanas.enduri@amd.com>
|
2025-04-21 19:33:21 -07:00 |
|
Yineng Zhang
|
b9c87e781d
|
chore: bump v0.4.5.post3 (#5611)
|
2025-04-21 18:16:20 -07:00 |
|
michael-amd
|
968ef51562
|
Support aiter RMSNorm in AMD (#5510)
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
|
2025-04-21 17:40:39 -07:00 |
|
Lianmin Zheng
|
1343200299
|
Clean up mem settings (#5610)
|
2025-04-21 17:19:00 -07:00 |
|
JieXin Liang
|
c2942907d5
|
[feature] enable pre compile jit deep_gemm (#5580)
|
2025-04-21 16:52:53 -07:00 |
|
Liangsheng Yin
|
e69a219074
|
Enhance GPU memory settings (#5604)
|
2025-04-21 15:15:00 -07:00 |
|
Byron Hsu
|
bf98d2e377
|
[PD] Support prefill overlap + Ensure no race condition (#5609)
|
2025-04-21 12:12:56 -07:00 |
|
Byron Hsu
|
e65b9f21e3
|
[PD] Support decode overlap schedule (#5608)
|
2025-04-21 12:06:16 -07:00 |
|
Trevor Morris
|
4dce1cc608
|
[PD] Add NIXL transfer backend (#5477)
|
2025-04-22 01:36:12 +08:00 |
|
Byron Hsu
|
deded17f38
|
[PD] Fix edge case and simplify large page size + chunked prefill (#5589)
|
2025-04-21 10:27:02 -07:00 |
|
shangmingc
|
f29a718f63
|
[PD] Fix generate endpoint of min_lb for PD (#5598)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-04-21 21:39:18 +08:00 |
|
Yongtong Wu
|
3f57b00a59
|
Support PD bootstrap fields on /v1/chat/completions endpoint (#5488)
|
2025-04-21 01:10:58 -07:00 |
|