EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
5,463 Commits 8 Branches 0 Tags
0753ef831eebd8d261ad136718aaa4bb62a65e33
Commit Graph

5463 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yineng Zhang
0753ef831e [Auto Sync] Update flashattention_backend.py (20250922) (#10762) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Gordon Gustafson <ggustafson@together.ai>
 2025-09-22 16:41:42 -07:00
ishandhanani
662393f27d fix: kv events with tp > 1 (#10541) 2025-09-22 15:55:44 -07:00
pranavm-nvidia
b1bb8e7490 Enables TRT-LLM backend to be used for target_verify (#10281) 
Co-authored-by: Pranav Marathe <pranavm@ipp1-3309.ipp1a1.colossus.nvidia.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-09-22 15:54:00 -07:00
Lianmin Zheng
38c00ed7a1 Fix multimodal registry and code sync scripts (#10759) 
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-22 15:36:01 -07:00
pansicheng
d4041a5eeb refactor zero copy (#10300) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-09-22 15:17:31 -07:00
Vedant Jhaveri
2f555c4cee [Generative Score API] Added test_scores_api.py to github CICD to run per commit (#10755) 
Co-authored-by: Vedant Jhaveri <vjhaveri@linkedin.com>
Co-authored-by: Sundara Raman Ramachandran <sundar24295@gmail.com>
 2025-09-22 14:41:57 -07:00
Yineng Zhang
e53df7c009 chore: bump sgl-kernel v0.3.12 (#10732) 2025-09-22 14:39:25 -07:00
Jue WANG
9c53dad809 Fix MTP MoE weight loading with NVFP4 target model. (#10758) 2025-09-22 14:21:57 -07:00
Simo Lin
7ca1bea63d [router] update ci so only execute benchmarks when labels are added (#10757) 2025-09-22 13:23:07 -07:00
Simo Lin
97c3823931 [router] refactor router and worker management 3/n (#10727) 2025-09-22 12:17:50 -07:00
Chang Su
60dbbd086a bugfix: Fix get_worker_urls_for_model in http/router.rs (#10754) 2025-09-22 14:10:31 -04:00
Baizhou Zhang
aa1c5cf5bd Add warnings and remove dependency for deterministic inference (#10724) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-09-22 10:56:02 -07:00
Adarsh Shirawalmath
592caab66a [Docs, minor] Fix LLM doc matrix (#10753) 2025-09-23 01:29:55 +08:00
Lifu Huang
2101d93b4f Fix CI TestChunkedSGMV (#10737) 2025-09-22 16:09:58 +08:00
Shangming Cai
70e4b21853 Fix flaky logprobs test (#10728) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-22 00:46:26 -07:00
feng397
944f1ea0ec fix capture_bs when speculative decoding enabled (#10730) 2025-09-22 00:43:12 -07:00
Yingchun Lai
9d7e82a0ab EPLB: prefer to use physical experts in the same node (#9849) 2025-09-22 00:34:30 -07:00
Yang Yu
f05805515b Convert FLASHINFER_WORKSPACE_SIZE to integer (#10731) 2025-09-22 00:26:42 -07:00
Lifu Huang
635ccda673 [4/4] Introduce CachedKernel to reduce CSGMV kernel launch overheads by 60% (#10709) 2025-09-21 22:26:42 -07:00
ronnie_zheng
1c3dbad8fe [Ascend] codeowner updates for ascend related files (#10699) 2025-09-21 19:47:01 -07:00
Qiaolin Yu
e2ac7888b8 [2/2] Support deterministic inference for temperature > 0 (#10678) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
 2025-09-21 19:36:08 -07:00
Stefan He
86527a4799 [deterministic inference] Move batch invariant pkg to sglang (#10695) 2025-09-21 19:35:14 -07:00
Ethan (Yusheng) Su
134b4f7ec2 Support deterministic inference with triton backend (#10694) 2025-09-22 09:20:40 +08:00
Yineng Zhang
f67d1f45bc [Auto Sync] Update deepseek_v2.py (20250922) (#10717) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Michael Granado <mgranado@together.ai>
 2025-09-21 17:43:50 -07:00
Qi Yuhang
0f04a5f428 Optimize cutlass int8 gemm kernel for large M on SM89 Ada GPU (#10714) 2025-09-21 17:04:27 -07:00
Yineng Zhang
2f18602f13 fix: disable gpt-oss b200 ut (#10716) 2025-09-21 17:02:25 -07:00
Jimmy
56321e9fc2 [Router]fix: fix get_load missing api_key (#10385) 2025-09-21 15:28:38 -04:00
Xinyuan Tong
12d6cf18f0 Refactors radix cache for extra key support (#10317) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-09-22 02:16:16 +08:00
sglang-bot
fc3e542009 Update release-docs.yml (#10706) 2025-09-21 00:22:21 -07:00
Lifu Huang
08ecd0aa2a [3/4] Speed up CSGMV backend perf by 10% through dynamic chunking + kernel optimization (#10592) 2025-09-20 22:47:48 -07:00
fzyzcjy
720c1c8ca3 Super tiny fix extra logs (#10697) 2025-09-20 21:30:54 -07:00
Yineng Zhang
d403c143e3 feat: update server args (#10696) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-20 18:52:19 -07:00
Stefan He
cba0d8c309 [Feature] Support deterministic inference with FA3 backend (#10651) 2025-09-20 17:50:21 -07:00
Yineng Zhang
f1d7892318 [Auto Sync] Update modelopt_quant.py (20250920) (#10688) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-20 02:37:49 -07:00
Yineng Zhang
7c876de7f5 fix: remove awq_dequantize deps (#10686) 2025-09-20 01:47:01 -07:00
Yineng Zhang
ba94b82986 fix: update run_suite (#10685) 2025-09-20 01:22:06 -07:00
dmitrygx
2b7417bf6a fix(disagg): fix sending KV cache in case of MLA for NIXL backend (#10673) 2025-09-20 00:30:10 -07:00
Baizhou Zhang
f111649580 Replace os.environ in layernorm.py (#10684) 2025-09-20 00:20:33 -07:00
Binyao Jiang
bd7eb0205a [Performance] Qwen3-Next: optimize causal_conv1d_fn triton kernel - up to 9% faster (#10680) 2025-09-20 00:12:43 -07:00
Shangming Cai
74cd6e3902 chore: upgrade mooncake 0.3.6.post1 to fix gb200 dockerfile (#10681) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-20 00:12:26 -07:00
Yineng Zhang
b17e67df36 [Auto Sync] Update deepseek_v2.py (20250920) (#10683) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-19 23:43:31 -07:00
Baizhou Zhang
8ecef73f12 [1/2] Support deterministic inference with flashinfer attention backend (#10645) 
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
 2025-09-19 23:34:29 -07:00
Simo Lin
1d1ce62495 [router] refactor router and worker management 2.5/n (#10677) 2025-09-19 20:54:40 -07:00
Yineng Zhang
60e2a7cead [Auto Sync] Update model_runner.py (20250920) (#10679) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-19 18:26:54 -07:00
Jinyan Chen
d88ef4a388 limit sgl-kernel causal conv1d to cuda only (#10648) 
Co-authored-by: Jinyan Chen <jinyanc@nvidia.com>
 2025-09-19 16:59:37 -07:00
Yineng Zhang
6f993e8b9e chore: cleanup docker image (#10671) 2025-09-19 16:56:49 -07:00
Chang Su
03ce92e594 router-spec: Reorder ChatCompletionRequest and fix validation logic (#10675) 2025-09-19 16:41:21 -07:00
Simo Lin
00eb5eb721 [router] refactor router and worker management 2/n (#10666) 2025-09-19 12:37:57 -07:00
Yineng Zhang
dab4663b4e [Auto Sync] Update .clang-format (20250919) (#10670) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-19 12:31:44 -07:00
Yineng Zhang
610a6d6e86 fix: resolve sync issue (#10668) 2025-09-19 12:05:39 -07:00