Simo Lin
|
c8f31042a8
|
[router] Refactor router and policy traits with dependency injection (#7987)
Co-authored-by: Jin Pan <jpan236@wisc.edu>
Co-authored-by: Keru Yang <rukeyang@gmail.com>
Co-authored-by: Yingyi Huang <yingyihuang2000@outlook.com>
Co-authored-by: Philip Zhu <phlipzhux@gmail.com>
|
2025-07-18 14:24:24 -07:00 |
|
Hongbo Xu
|
1f76fc8747
|
[3/n] chore: decouple AWQ implementation from vLLM dependency (#8113)
Co-authored-by: AniZpZ <zhuangsen.zp@antgroup.com>
|
2025-07-18 11:45:22 -07:00 |
|
Even Zhou
|
6737671c82
|
[Bugfix] Fix w8a8_int8 import error on NPU (#8147)
|
2025-07-18 11:34:55 -07:00 |
|
Enrique Shockwave
|
fd63b62eaa
|
fix compressed tensors WNA16 imports (#8142)
|
2025-07-18 11:34:14 -07:00 |
|
Peng Zhang
|
719b29f218
|
feat: enchance green context stream creation robust with backward compatibility (#8136)
|
2025-07-18 02:45:03 -07:00 |
|
Sai Enduri
|
d0510f08fe
|
Revert "Fix different device type adjustment in PP" (#8141)
|
2025-07-18 01:12:11 -07:00 |
|
Zhiqiang Xie
|
9d33fcfb8e
|
Hicache Storage Layer Prototype (#7704)
|
2025-07-18 15:20:19 +08:00 |
|
jianan-gu
|
7891bac16b
|
[Quantization][w8a8_int8] Fix weight loading issue for w8a8_int8 path with "ignore" layer list in quantization config (#7820)
|
2025-07-17 22:03:56 -07:00 |
|
jianan-gu
|
48c1fa7bb6
|
[CPU][Llama4] Fix Llama4 MoE inputs with "apply_router_weight_on_input" (#7889)
|
2025-07-17 21:43:25 -07:00 |
|
yilian49
|
8aa5ae6b04
|
load draft model fix (#7506)
|
2025-07-17 21:41:32 -07:00 |
|
Minglei Zhu
|
8a32355704
|
Feat: Support Granite 3.0 MoE in SGLang (#7959)
|
2025-07-17 20:56:03 -07:00 |
|
Qi Yuhang
|
6e92da8fca
|
[Fix][Ready]Fix register spilling in cutlass nvfp4 gemm kernel on Blackwell (#8127)
|
2025-07-17 20:49:36 -07:00 |
|
Mick
|
e1020dc588
|
refactor: simply MultimodalTokens logic (#7924)
|
2025-07-17 17:59:15 -07:00 |
|
Zhao Chen
|
3586b4cef2
|
feat: add production metric for retracted requests due to insufficient kvcache (#7030)
Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com>
|
2025-07-17 11:59:05 -07:00 |
|
Asher
|
4296021499
|
[Hunyuan]: Fix Dense Model Support (#8117)
Signed-off-by: Asher Zhang <asherszhang@tencent.com>
|
2025-07-17 10:00:11 -07:00 |
|
Ziqi Fan
|
01857fab61
|
fix: update HostKVCache init to report correct msg when available memory is not enough (#8102)
|
2025-07-17 21:24:34 +08:00 |
|
fzyzcjy
|
519ff5c8e6
|
Super tiny fix typo (#8046)
|
2025-07-17 21:15:51 +08:00 |
|
Yuan Luo
|
af1cc8fe2d
|
[kernel] opt moe align block kernel by block/warp scan algorithm (#7884)
|
2025-07-17 19:33:02 +08:00 |
|
Cheng Wan
|
49b8777460
|
Refactor: move all quantization-related code to srt/layer/quantization (#7989)
|
2025-07-17 00:47:07 -07:00 |
|
Cheng Wan
|
02404a1e35
|
[ci] recover 8-gpu deepep test (#8105)
|
2025-07-17 00:46:40 -07:00 |
|
hzh0425
|
5c08a36cbf
|
[Fix] ensure DeepGEMM is only enabled for FP8_W8A8 models (#8110)
|
2025-07-16 21:33:29 -07:00 |
|
Cheng Wan
|
9069884b51
|
[ci] disable memory imbalance check for draft worker (#8108)
|
2025-07-16 20:41:47 -07:00 |
|
Simo Lin
|
8a7a7770e5
|
[ci] limit cmake build nproc (#8100)
|
2025-07-16 18:09:28 -07:00 |
|
Yingchun Lai
|
795668dc73
|
feat: add tp_rank, pp_rank and dp_rank labels for scheduler metrics (#7597)
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
|
2025-07-16 17:55:59 -07:00 |
|
Mick
|
4395c87a9b
|
refactor: unify names of the feature field of MultimodalDataItem (#8075)
|
2025-07-16 17:52:38 -07:00 |
|
Peng Zhang
|
c28ad1990d
|
[1/n] chore: decouple quantization implementation from vLLM dependency (#7992)
|
2025-07-16 15:56:26 -07:00 |
|
Xiaoze Fan
|
570d33437b
|
[Feature] Layer-wise Prefill (#7634)
Signed-off-by: jason-fxz <jason341132@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-07-17 01:57:46 +08:00 |
|
Simo Lin
|
d9eb5efc71
|
[misc] update nvshmem and pin deepEP commit hash (#8098)
|
2025-07-16 08:54:55 -07:00 |
|
Peng Zhang
|
6dc4af4937
|
fix greenctx stream compability (#8090)
|
2025-07-16 07:08:46 -07:00 |
|
YanbingJiang
|
b188a89a5d
|
Fix CI xeon test with triton 3.3.1 (#8086)
|
2025-07-16 02:12:23 -07:00 |
|
Mick
|
497efe747d
|
Revert "feat: replace Decord with video_reader-rs" (#8077)
|
2025-07-15 20:04:56 -07:00 |
|
Qiaolin Yu
|
69f453e5a4
|
Use device_group for all_gather when disabling overlap scheduling (#8001)
|
2025-07-15 19:38:58 -07:00 |
|
Qiaolin Yu
|
3bc43c683e
|
Fix different device type adjustment in PP (#7760)
|
2025-07-15 19:37:14 -07:00 |
|
Xinyuan Tong
|
7498522f7d
|
update transformers to 4.53.2 (#8029)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-07-15 18:24:39 -07:00 |
|
strgrb
|
194841e329
|
remove kv_a.congigous in DeepseekV2AttentionMLA (#8058)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-07-15 18:20:41 -07:00 |
|
kozo
|
ebff5fcb06
|
feat: replace Decord with video_reader-rs (#5163)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-07-15 18:17:34 -07:00 |
|
Sai Enduri
|
f06bd210c0
|
Update amd docker image. (#8045)
Co-authored-by: Hubert Lu <55214931+hubertlu-tw@users.noreply.github.com>
|
2025-07-15 15:09:56 -07:00 |
|
yhang
|
14f1f1514b
|
H20 tune config for Kimi (#8047)
|
2025-07-15 13:48:31 -07:00 |
|
Albert
|
38216cf049
|
concurrently load weights of DeepseekV2ForCausalLM (#7943)
Signed-off-by: Tianyu Zhou <albert.zty@antgroup.com>
|
2025-07-15 13:41:19 -07:00 |
|
Yineng Zhang
|
4a8837950a
|
fix: resolve arm build issue (#8052)
|
2025-07-15 01:19:34 -07:00 |
|
jiawei
|
f1f1d1d40d
|
Fix the input tools format and history tool_calls in OpenAI API (#6556)
|
2025-07-15 00:58:55 -07:00 |
|
Xinyuan Tong
|
9120e83d03
|
fix: remove redundant rotary embedding cache recomputation in MiniCPM (#8022)
|
2025-07-15 00:12:45 -07:00 |
|
Xinyuan Tong
|
6e923dbd30
|
feat: update multimodal data handling in engine entrypoint (#8002)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-07-15 00:12:22 -07:00 |
|
Qi Yuhang
|
c268c11c71
|
[feat]Support fusion kernel for constructing quant input and scale factor for fp8_blockwise_scaled_grouped_mm (#8023)
|
2025-07-15 00:02:44 -07:00 |
|
Chang Su
|
e6d5988442
|
Update CODEOWNERS (#8044)
|
2025-07-15 14:38:15 +08:00 |
|
杨睿
|
9b560c3e1c
|
fix: modality length mismatch with image_data (#7887)
|
2025-07-15 14:27:54 +08:00 |
|
Sai Enduri
|
5dc5866e8e
|
Setup workflow for releasing mi300x and mi350x dockers. (#8035)
|
2025-07-14 21:51:43 -07:00 |
|
ehuaa
|
64e78bb31b
|
prevent server crash from potential invalid grammar (#7897)
|
2025-07-15 11:21:45 +08:00 |
|
hzh0425
|
7c39e8a198
|
Fix Bug 'get_cpu_copy not Implemented' in pd offloading mode (#7982)
|
2025-07-14 14:57:10 -07:00 |
|
Lifu Huang
|
d969504d9a
|
Fix flaky CI: test_vlm_models (#8006)
|
2025-07-14 14:56:41 -07:00 |
|