Stefan He
|
00fbd8a484
|
Fix typo of flash_cache (#7513)
|
2025-06-25 02:04:41 -07:00 |
|
valarLip
|
802815e40b
|
take aiter get_rope back (#7521)
|
2025-06-25 02:03:33 -07:00 |
|
valarLip
|
4c6675c4fc
|
enable aiter fp8 blockscale quant (#7520)
|
2025-06-25 02:02:31 -07:00 |
|
Hongbo Xu
|
e21aa1df67
|
[PD] Add different TP sizes support for no-MLA models (#6793)
Co-authored-by: shangmingc <csmthu@gmail.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-25 02:00:22 -07:00 |
|
zixuanzhang226
|
f3cbd24541
|
feat: send kvmetrics from sglang scheduler (#6721)
|
2025-06-25 01:57:49 -07:00 |
|
ll819214
|
506a2d5934
|
npu fused op (#7386)
Co-authored-by: Li Junwen <lijunwen13@hisilicon.com>
|
2025-06-25 01:54:20 -07:00 |
|
Shangming Cai
|
a07f8ae4b7
|
[CI] Upgrade mooncake to v0.3.4.post2 to fix potential slice failed bug (#7522)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-25 01:49:22 -07:00 |
|
Chunyuan WU
|
7eb47b0f3d
|
[CPU] [BF16] Call fused_experts_cpu, weight_packed_linear and bmm_cpu kernel in DeepSeek model (#6641)
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2025-06-25 01:43:33 -07:00 |
|
DangKai
|
bc2e5645c4
|
fix: force synchronization between TP workers when update_weights (#6626)
Co-authored-by: dangkai.dk <dangkai.dk@alibaba-inc.com>
|
2025-06-25 01:35:59 -07:00 |
|
Simo Lin
|
3abc30364d
|
[ci] add router benchmark script and CI (#7498)
|
2025-06-25 01:28:25 -07:00 |
|
linzhuo
|
afeed46530
|
clean duplicate code (#7512)
|
2025-06-25 01:22:20 -07:00 |
|
yilian49
|
587b4c6e92
|
EPLB support for MTP (#7510)
|
2025-06-25 01:16:56 -07:00 |
|
Shangming Cai
|
7b9a174a7a
|
[PD][Spec] Fix hidden state transfer for spec decode (#7516)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-25 00:42:07 -07:00 |
|
ybyang
|
03c039c48e
|
[OAI] patch origin request_id logic (#7508)
|
2025-06-24 20:09:38 -07:00 |
|
Ke Bao
|
57ab776910
|
Fuse sorted_token_ids padding to moe_align_block_size kernel (#7437)
|
2025-06-24 17:44:27 -07:00 |
|
Chang Su
|
112b496a6c
|
misc: Improvement to serving_chat.py and add more ut (#7489)
|
2025-06-24 17:19:51 -07:00 |
|
Chang Su
|
3562256bb2
|
fix: Add --model as an alias for --model-path in server_args (#7505)
|
2025-06-24 12:08:08 -07:00 |
|
Trevor Morris
|
5f527834a8
|
[PD] NIXL: Register kv args in advance and cleanup finished requests (#6717)
|
2025-06-24 11:26:09 -07:00 |
|
xianzhiT
|
9f1787fa60
|
Support multi-thread model weight loading (#7277)
|
2025-06-24 10:39:10 -07:00 |
|
Xiaoyu Zhang
|
8ecad0b16f
|
[benchmark] fbgemm benchmark support bandwidth report and support fbgemm_cutlass_gmm (#7422)
|
2025-06-24 09:44:55 -07:00 |
|
Ke Bao
|
7151194bb2
|
Remove cumsum_buffer initilization (#7439)
|
2025-06-24 02:22:16 -07:00 |
|
Francis
|
2ed68d7a6c
|
[PD Disaggregation] replace transfer with batch transfer for better performance (#7236)
|
2025-06-24 02:12:04 -07:00 |
|
valarLip
|
e984d5073b
|
enable aiter_biased_grouped_topk kernel (#7423)
|
2025-06-24 02:09:42 -07:00 |
|
Alex Sun
|
755f314785
|
[AMD] add aiter fused moe in DeepEP path (#7268)
|
2025-06-24 09:05:47 +00:00 |
|
Yineng Zhang
|
7c3a12c000
|
chore: bump v0.4.8 (#7493)
|
2025-06-23 23:14:22 -07:00 |
|
Yineng Zhang
|
e846d95ef6
|
chore: bump sgl-kernel v0.2.0 (#7490)
|
2025-06-23 22:29:50 -07:00 |
|
Charles Chen
|
15f3401343
|
Fix MTP with Deepseek R1 Fp4 (#7376)
|
2025-06-23 21:38:07 -07:00 |
|
Chang Su
|
d04163b3fa
|
Fix RequestValidationError response format (#7487)
|
2025-06-23 20:35:11 -07:00 |
|
huangtingwei
|
7732bbe458
|
bugfix: Prevent global mutation of conv.stop_str across requests (#7347)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-06-23 19:36:23 -07:00 |
|
u4lr451
|
ed0a0b692c
|
Perormance: Enable cuda graph for dp idle batch (#7269)
Co-authored-by: austindeng <austindeng@tencent.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: ch-wan <cwan39@gatech.edu>
|
2025-06-23 17:34:13 -07:00 |
|
Chang Su
|
fa42e41962
|
ci: Revert openai_server related tests in AMD suites (#7449)
|
2025-06-23 15:28:22 -07:00 |
|
Yuhong Guo
|
e5afb88b1c
|
Support weight loading without mmap (#7469)
|
2025-06-23 15:13:59 -07:00 |
|
Charles Chen
|
e5ddeb04d5
|
Quick fix for DeepGemm requant to also cover MTP. (#7378)
|
2025-06-23 12:08:54 -07:00 |
|
JieXin Liang
|
bdbb8d009a
|
[perf] slightly imporve DeepSeek-R1-FP4 TP8 (#7481)
|
2025-06-23 12:05:30 -07:00 |
|
Zhiqiang Xie
|
34c3f9b2d3
|
kvcache io kernels and test case (#7382)
|
2025-06-23 11:58:59 -07:00 |
|
Liangsheng Yin
|
76139bfba0
|
update mooncake in dockerfile (#7480)
|
2025-06-24 02:29:30 +08:00 |
|
Cheng Wan
|
f8d48fd311
|
Fix dtype for idle input in spec decoding (#7456)
|
2025-06-23 11:23:25 -07:00 |
|
Chang Su
|
34b6b8426f
|
feat(func_call): Add more check in BaseFormatDetector.parse_streaming_increment (#7479)
|
2025-06-23 11:15:47 -07:00 |
|
Liangsheng Yin
|
25549433e8
|
Fix prefill OOM due to wrong token calculation when page > 1 (#7397)
|
2025-06-24 02:12:29 +08:00 |
|
Shangming Cai
|
d6dddc19ff
|
[CI] Upgrade mooncake to 0.3.4.post1 to fix 8 gpu tests (#7472)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-24 02:10:50 +08:00 |
|
Lianmin Zheng
|
55e03b10c4
|
Fix a bug in BatchTokenIDOut & Misc style and dependency updates (#7457)
|
2025-06-23 06:20:39 -07:00 |
|
kk
|
8aa68ed5c4
|
Solve docker build failed in the virtual machine (#7290)
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
|
2025-06-23 09:10:30 +00:00 |
|
xutizhou
|
506c4928f5
|
feat: integrate deepgemm into EPMoE (#6821)
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: TianQiLin666666 <1834987979@qq.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-06-23 01:38:58 -07:00 |
|
Lianmin Zheng
|
30ceccc74a
|
Update hyperparameter_tuning.md (#7454)
|
2025-06-22 22:42:55 -07:00 |
|
Cheng Wan
|
ac5010e0ba
|
Fix CUDA Graph Check under Deepep with DP FFN (#7451)
|
2025-06-22 20:35:58 -07:00 |
|
Yi Zhang
|
3cee035e99
|
add fused moe config for qwen3 in triton3.3.1 (#7445)
|
2025-06-22 18:29:25 -07:00 |
|
Simo Lin
|
30f2a44a96
|
[misc] Add PD service discovery support in router (#7361)
|
2025-06-22 17:54:14 -07:00 |
|
kk
|
bd4f581896
|
Fix torch compile run (#7391)
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
|
2025-06-22 15:33:09 -07:00 |
|
Ke Bao
|
50f1b6d6b1
|
Remove copy after bmm (#7441)
|
2025-06-22 15:07:44 -07:00 |
|
Trevor Morris
|
5962e70d8d
|
FlashInfer NVFP4 MoE with EP & 2-stream shared expert (#7327)
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
Co-authored-by: alcanderian <alcanderian@gmail.com>
|
2025-06-22 13:38:47 -07:00 |
|