Xinyuan Tong
|
aab35bccb4
|
fix: draft model IMA by overide max_positional_embeddings (#10787)
Co-authored-by: Qiaolin Yu <qy254@cornell.edu>
|
2025-09-23 12:56:16 -07:00 |
|
Yineng Zhang
|
42aedb02af
|
[Auto Sync] Update protocol.py (20250923) (#10820)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cicirori <32845984+cicirori@users.noreply.github.com>
|
2025-09-23 12:49:56 -07:00 |
|
Yiakwy
|
984730b732
|
add tunning files for QWEN-3-NEXT (#10794)
|
2025-09-23 12:46:30 -07:00 |
|
Shangming Cai
|
23632d350c
|
Fix latest main ci (#10799)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2025-09-23 12:46:13 -07:00 |
|
Chang Su
|
08b8c0c3cd
|
[router] fix axum default body limit (#10818)
|
2025-09-23 12:44:17 -07:00 |
|
Lzhang-hub
|
d42975c641
|
Remove duplicate code in qwen2 model (#10540)
|
2025-09-24 02:40:51 +08:00 |
|
ZhengHSI
|
adc24a3a0c
|
fix ceval (#10504)
Co-authored-by: lukec <118525388+sleepcoo@users.noreply.github.com>
|
2025-09-24 02:35:25 +08:00 |
|
Chang Su
|
7ff93e613f
|
router(grpc): Implement route for chat_cmpl endpoint (#10761)
|
2025-09-23 11:26:33 -07:00 |
|
Simo Lin
|
b24b2e7ed7
|
[router] use dashmap for radix tree instead of hash for multi model (#10814)
|
2025-09-23 11:25:53 -07:00 |
|
sogalin
|
7135db5d31
|
[ROCm] Update aiter to v0.1.5.post3 (#10812)
|
2025-09-23 10:54:12 -07:00 |
|
Jimmy
|
4b5ef3002c
|
[fix][pd-disag]no need set next batch sampling info done in prefill (#10259)
|
2025-09-24 01:24:36 +08:00 |
|
Zheng Li
|
4f564b9e83
|
model: support qwen3-vl series (#10323)
Co-authored-by: ocss884 <ocss.lin@gmail.com>
Co-authored-by: cao1zhg <653506626@qq.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: 瑀澈 <yuche.lz@alibaba-inc.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-09-23 10:15:52 -07:00 |
|
Simo Lin
|
98c3b04ff2
|
[router] responses api POST and GET with local storage (#10581)
Co-authored-by: key4ng <rukeyang@gmail.com>
|
2025-09-23 09:12:02 -07:00 |
|
Simo Lin
|
ddab4fc7c7
|
[router] fix cache aware routing strategy and lock contention (#10773)
|
2025-09-23 08:53:49 -07:00 |
|
Shangming Cai
|
d21c35224d
|
Fix hicache mooncake backend CI (#10792)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2025-09-23 02:04:44 -07:00 |
|
Liangsheng Yin
|
4a762041d7
|
move environ into sglang.srt to avoid break SRT auto sync. (#10791)
|
2025-09-23 02:04:20 -07:00 |
|
Lianmin Zheng
|
ea338676b5
|
Clean up server args (#10770)
|
2025-09-23 00:22:32 -07:00 |
|
ishandhanani
|
b06db198ba
|
followup: clean up dockerfiles and release yamls (#10783)
|
2025-09-23 00:19:46 -07:00 |
|
Yineng Zhang
|
8c1ef0f914
|
chore: upgrade sgl-kernel 0.3.12 (#10782)
|
2025-09-23 00:18:54 -07:00 |
|
Liangsheng Yin
|
f5a2faf2b8
|
Introduce FutureMap (#10715)
|
2025-09-22 23:27:30 -07:00 |
|
ishandhanani
|
1c82d9db28
|
feat: unify dockerfiles (#10705)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-09-22 23:23:48 -07:00 |
|
Lifu Huang
|
9241f4fd20
|
Move cached kernel to srt.utils (#10776)
|
2025-09-22 23:00:36 -07:00 |
|
fzyzcjy
|
063c3791fe
|
Fix trtllm_mla slow concat kernel in MTP (#10777)
|
2025-09-22 22:47:49 -07:00 |
|
Liangsheng Yin
|
632b7d8cc9
|
Use simulate acc len from sglang.environ (#10771)
|
2025-09-23 12:59:50 +08:00 |
|
Chang Su
|
16adf3dcab
|
[router] fix logger type mismatch (#10774)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2025-09-22 21:02:28 -07:00 |
|
Simo Lin
|
c3a1d7759f
|
[router] remove pd router draining channel (#10767)
|
2025-09-22 20:49:33 -07:00 |
|
Simo Lin
|
89971c4c3c
|
[router] refactor router and worker management 4/n (#10756)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-09-22 18:35:10 -07:00 |
|
Lianmin Zheng
|
113f8f65a2
|
[Auto Sync] Update configurer.py (20250923) (#10765)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
|
2025-09-22 17:34:34 -07:00 |
|
ronnie_zheng
|
e22f3a5ec9
|
[Ascend]optimize Qwen3 on Ascend (#10574)
Co-authored-by: c30031083 <chenxu140@huawei.com>
|
2025-09-22 17:18:36 -07:00 |
|
ronnie_zheng
|
095093ee5a
|
[Ascend] optimize Qwen-vl on Ascend (#10556)
Co-authored-by: wangqihui01 <wangqh10@163.com>
|
2025-09-22 17:18:16 -07:00 |
|
Even Zhou
|
d27a6f7092
|
[Feature] Add MLAProcess for DeepSeek MLA on NPU (#10130)
|
2025-09-22 17:17:48 -07:00 |
|
Yineng Zhang
|
0753ef831e
|
[Auto Sync] Update flashattention_backend.py (20250922) (#10762)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Gordon Gustafson <ggustafson@together.ai>
|
2025-09-22 16:41:42 -07:00 |
|
ishandhanani
|
662393f27d
|
fix: kv events with tp > 1 (#10541)
|
2025-09-22 15:55:44 -07:00 |
|
pranavm-nvidia
|
b1bb8e7490
|
Enables TRT-LLM backend to be used for target_verify (#10281)
Co-authored-by: Pranav Marathe <pranavm@ipp1-3309.ipp1a1.colossus.nvidia.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
|
2025-09-22 15:54:00 -07:00 |
|
Lianmin Zheng
|
38c00ed7a1
|
Fix multimodal registry and code sync scripts (#10759)
Co-authored-by: cctry <shiyang@x.ai>
|
2025-09-22 15:36:01 -07:00 |
|
pansicheng
|
d4041a5eeb
|
refactor zero copy (#10300)
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
|
2025-09-22 15:17:31 -07:00 |
|
Vedant Jhaveri
|
2f555c4cee
|
[Generative Score API] Added test_scores_api.py to github CICD to run per commit (#10755)
Co-authored-by: Vedant Jhaveri <vjhaveri@linkedin.com>
Co-authored-by: Sundara Raman Ramachandran <sundar24295@gmail.com>
|
2025-09-22 14:41:57 -07:00 |
|
Yineng Zhang
|
e53df7c009
|
chore: bump sgl-kernel v0.3.12 (#10732)
|
2025-09-22 14:39:25 -07:00 |
|
Jue WANG
|
9c53dad809
|
Fix MTP MoE weight loading with NVFP4 target model. (#10758)
|
2025-09-22 14:21:57 -07:00 |
|
Simo Lin
|
7ca1bea63d
|
[router] update ci so only execute benchmarks when labels are added (#10757)
|
2025-09-22 13:23:07 -07:00 |
|
Simo Lin
|
97c3823931
|
[router] refactor router and worker management 3/n (#10727)
|
2025-09-22 12:17:50 -07:00 |
|
Chang Su
|
60dbbd086a
|
bugfix: Fix get_worker_urls_for_model in http/router.rs (#10754)
|
2025-09-22 14:10:31 -04:00 |
|
Baizhou Zhang
|
aa1c5cf5bd
|
Add warnings and remove dependency for deterministic inference (#10724)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-09-22 10:56:02 -07:00 |
|
Adarsh Shirawalmath
|
592caab66a
|
[Docs, minor] Fix LLM doc matrix (#10753)
|
2025-09-23 01:29:55 +08:00 |
|
Lifu Huang
|
2101d93b4f
|
Fix CI TestChunkedSGMV (#10737)
|
2025-09-22 16:09:58 +08:00 |
|
Shangming Cai
|
70e4b21853
|
Fix flaky logprobs test (#10728)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2025-09-22 00:46:26 -07:00 |
|
feng397
|
944f1ea0ec
|
fix capture_bs when speculative decoding enabled (#10730)
|
2025-09-22 00:43:12 -07:00 |
|
Yingchun Lai
|
9d7e82a0ab
|
EPLB: prefer to use physical experts in the same node (#9849)
|
2025-09-22 00:34:30 -07:00 |
|
Yang Yu
|
f05805515b
|
Convert FLASHINFER_WORKSPACE_SIZE to integer (#10731)
|
2025-09-22 00:26:42 -07:00 |
|
Lifu Huang
|
635ccda673
|
[4/4] Introduce CachedKernel to reduce CSGMV kernel launch overheads by 60% (#10709)
|
2025-09-21 22:26:42 -07:00 |
|