Lifu Huang
|
2373faa317
|
Fix flakiness in LoRA batch test. (#7552)
|
2025-06-27 19:51:43 -07:00 |
|
fzyzcjy
|
9efb2993da
|
Tiny add logs for expert location updater (#7308)
|
2025-06-27 19:12:33 -07:00 |
|
Chunyuan WU
|
a5317b2fd3
|
[CPU] add optimizations for INT8 and FP8 DeepSeek (#6769)
Co-authored-by: Zheng, Beilei <beilei.zheng@intel.com>
|
2025-06-27 19:04:29 -07:00 |
|
tarinkk
|
eb6c2c1663
|
Hybrid kv cache for LLaMA4 (#6563)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: tarinkk <rt572@physics.rutger.edu>
Co-authored-by: tarinkk <rt572@rutgers.physics.edu>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
|
2025-06-27 18:58:55 -07:00 |
|
Xinyuan Tong
|
357921aa51
|
Fix: Minicpm (#7612)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-06-27 17:32:29 -07:00 |
|
Simo Lin
|
c071198c1d
|
[router] add centralized configuration module for sgl-router (#7588)
|
2025-06-27 15:42:02 -07:00 |
|
Lifu Huang
|
d7374d7467
|
Fix broken CI TestVILAServer (#7610)
|
2025-06-27 15:01:03 -07:00 |
|
Lianmin Zheng
|
ce3a3e8783
|
Move multimodal processors into a separate folder (#7581)
|
2025-06-27 11:58:24 -07:00 |
|
Qiaolin Yu
|
41650b0d70
|
feat: support compatibility between MTP and two-batch-overlap (#7225)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-06-27 01:10:27 -07:00 |
|
Xinyuan Tong
|
1b95162008
|
Updates transformers and timm dependencies (#7577)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-06-27 00:30:17 -07:00 |
|
Keyang Ru
|
29bd4c8135
|
[CI] Add CI Testing for Prefill-Decode Disaggregation with Router (#7540)
|
2025-06-27 00:18:56 -07:00 |
|
Ata Fatahi
|
031f64aa1b
|
Add e2e test for multi instance multi stage memory release/resume occupuation (#7208)
Signed-off-by: Ata Fatahi <immrata@gmail.com>
|
2025-06-26 17:40:38 -07:00 |
|
fzyzcjy
|
3d7cdb2ebd
|
Fix MTP error when enabling two-batch overlap (#7569)
|
2025-06-26 15:40:54 -07:00 |
|
Xinyuan Tong
|
604efe07e1
|
Updates Gemma3n MLP layer to adapt latest transformers version (#7573)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-06-26 15:07:22 -07:00 |
|
Cheng Wan
|
1b8cf77b01
|
[Fix] incorrect assert in EPLB (#7575)
|
2025-06-26 14:59:20 -07:00 |
|
Trevor Morris
|
bb9b608c86
|
[PD][NIXL] Set is_sorted=False to fix NIXL_ERR_NOT_FOUND (#7330)
|
2025-06-26 10:39:39 -07:00 |
|
Yineng Zhang
|
69183f8808
|
chore: bump v0.4.8.post1 (#7559)
|
2025-06-26 02:21:12 -07:00 |
|
Xinyuan Tong
|
9b00990bea
|
chore: remove vlm unnecessary import (#7541)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
|
2025-06-26 01:38:15 -07:00 |
|
Mick
|
4d67025a1d
|
chore: improve ci bug reporting (#7542)
|
2025-06-26 01:32:44 -07:00 |
|
YanbingJiang
|
0e05fe8cf4
|
Update seed in CPU UTs to avoid flaky failure with single test (#7544)
|
2025-06-25 21:25:50 -07:00 |
|
Meng, Peng
|
2390a2bc8d
|
Add Tencent HunYuanMoEV1 model support (#7549)
|
2025-06-25 20:59:53 -07:00 |
|
Ruihang Lai
|
16d76b9f23
|
[CMake] Fix sgl-kernel CMakeLists for Blackwell (#7543)
|
2025-06-25 19:00:46 -07:00 |
|
Shangming Cai
|
5c2142579a
|
[PD] Raise error for incompatible mooncake version and some minor fixes (#7527)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-25 18:55:24 -07:00 |
|
Qiaolin Yu
|
b8df43ab9c
|
Fix gathered_buffer issues in tbo (#7531)
|
2025-06-25 14:42:21 -07:00 |
|
Yuhong Guo
|
a1c1ebe935
|
Fix FP8 KV Cache Support in FA3 Backend (#7148)
|
2025-06-25 02:14:40 -07:00 |
|
mlmz
|
fe2a0f962f
|
minor: 'role' must be system/assistant/tool, but case insensitive for now (#7499)
|
2025-06-25 02:11:03 -07:00 |
|
eigen
|
20beb3702b
|
feat: add return hidden_states at async generation (#7507)
|
2025-06-25 02:10:09 -07:00 |
|
Stefan He
|
00fbd8a484
|
Fix typo of flash_cache (#7513)
|
2025-06-25 02:04:41 -07:00 |
|
valarLip
|
802815e40b
|
take aiter get_rope back (#7521)
|
2025-06-25 02:03:33 -07:00 |
|
valarLip
|
4c6675c4fc
|
enable aiter fp8 blockscale quant (#7520)
|
2025-06-25 02:02:31 -07:00 |
|
Hongbo Xu
|
e21aa1df67
|
[PD] Add different TP sizes support for no-MLA models (#6793)
Co-authored-by: shangmingc <csmthu@gmail.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-25 02:00:22 -07:00 |
|
zixuanzhang226
|
f3cbd24541
|
feat: send kvmetrics from sglang scheduler (#6721)
|
2025-06-25 01:57:49 -07:00 |
|
ll819214
|
506a2d5934
|
npu fused op (#7386)
Co-authored-by: Li Junwen <lijunwen13@hisilicon.com>
|
2025-06-25 01:54:20 -07:00 |
|
Shangming Cai
|
a07f8ae4b7
|
[CI] Upgrade mooncake to v0.3.4.post2 to fix potential slice failed bug (#7522)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-25 01:49:22 -07:00 |
|
Chunyuan WU
|
7eb47b0f3d
|
[CPU] [BF16] Call fused_experts_cpu, weight_packed_linear and bmm_cpu kernel in DeepSeek model (#6641)
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2025-06-25 01:43:33 -07:00 |
|
DangKai
|
bc2e5645c4
|
fix: force synchronization between TP workers when update_weights (#6626)
Co-authored-by: dangkai.dk <dangkai.dk@alibaba-inc.com>
|
2025-06-25 01:35:59 -07:00 |
|
Simo Lin
|
3abc30364d
|
[ci] add router benchmark script and CI (#7498)
|
2025-06-25 01:28:25 -07:00 |
|
linzhuo
|
afeed46530
|
clean duplicate code (#7512)
|
2025-06-25 01:22:20 -07:00 |
|
yilian49
|
587b4c6e92
|
EPLB support for MTP (#7510)
|
2025-06-25 01:16:56 -07:00 |
|
Shangming Cai
|
7b9a174a7a
|
[PD][Spec] Fix hidden state transfer for spec decode (#7516)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-25 00:42:07 -07:00 |
|
ybyang
|
03c039c48e
|
[OAI] patch origin request_id logic (#7508)
|
2025-06-24 20:09:38 -07:00 |
|
Ke Bao
|
57ab776910
|
Fuse sorted_token_ids padding to moe_align_block_size kernel (#7437)
|
2025-06-24 17:44:27 -07:00 |
|
Chang Su
|
112b496a6c
|
misc: Improvement to serving_chat.py and add more ut (#7489)
|
2025-06-24 17:19:51 -07:00 |
|
Chang Su
|
3562256bb2
|
fix: Add --model as an alias for --model-path in server_args (#7505)
|
2025-06-24 12:08:08 -07:00 |
|
Trevor Morris
|
5f527834a8
|
[PD] NIXL: Register kv args in advance and cleanup finished requests (#6717)
|
2025-06-24 11:26:09 -07:00 |
|
xianzhiT
|
9f1787fa60
|
Support multi-thread model weight loading (#7277)
|
2025-06-24 10:39:10 -07:00 |
|
Xiaoyu Zhang
|
8ecad0b16f
|
[benchmark] fbgemm benchmark support bandwidth report and support fbgemm_cutlass_gmm (#7422)
|
2025-06-24 09:44:55 -07:00 |
|
Ke Bao
|
7151194bb2
|
Remove cumsum_buffer initilization (#7439)
|
2025-06-24 02:22:16 -07:00 |
|
Francis
|
2ed68d7a6c
|
[PD Disaggregation] replace transfer with batch transfer for better performance (#7236)
|
2025-06-24 02:12:04 -07:00 |
|
valarLip
|
e984d5073b
|
enable aiter_biased_grouped_topk kernel (#7423)
|
2025-06-24 02:09:42 -07:00 |
|