EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
4,844 Commits 8 Branches 0 Tags
34e5e11f0ff6f2111241915d88f59fc44dfcf200
Commit Graph

3270 Commits

Author SHA1 Message Date
fzyzcjy
34e5e11f0f Tiny make device_loading_context more static (#9478) 2025-08-23 02:07:15 -07:00
fzyzcjy
2600fc0d47 Overlapped weight offload (#8034) 2025-08-23 02:06:46 -07:00
hlu1
ccd3fb946e [fix] Fix mxfp4 triton MoE tp bug (#9473) 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
 2025-08-23 01:48:40 -07:00
Chang Su
c9dd70fbde tool-call(dsv3): Improve deepseek-v3 chat template and tool_choice = required (#9525) 2025-08-23 01:46:56 -07:00
Yineng Zhang
6b2b8bf0e1 fix: blackwell dsv3 fp8 issue temporary solution (#9530) 2025-08-23 01:33:21 -07:00
fzyzcjy
0374304a2c Add enable_flashinfer_mxfp4_bf16_moe for higher precision and slower moe backend (#9004) 2025-08-23 15:38:40 +08:00
Chanh Nguyen
127d4b0d5e Support GC Freezing to improve latency & throughput (#9241) 
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
 2025-08-23 13:43:09 +08:00
Moein Khazraee
7e880286b5 Add support for extensions of interface and pre-registrations to NIXL HiCache (#9211) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-08-22 20:06:13 -07:00
sogalin
c4500233ff Add Qwen3-30B-A3B-Thinking-2507 support on AMD GPUs. (#9456) 2025-08-22 13:14:42 -07:00
Hubert Lu
f445a1d9a3 [AMD] Fix Llama 4 FP8 accuracy issues on MI300X (#7699) 2025-08-22 13:13:45 -07:00
datdo-msft
110a65989b [MTP] Force greedy sampling on AMD (#9127) 2025-08-22 11:14:43 -07:00
Wenxuan Tan
0f587e80d3 Use Tensor Core Decode when gqa group size >= 4 (#8624) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-22 23:25:15 +08:00
huangtingwei
6078d5fcc0 [HiCacheStorage] backup optimization for MLA model (#8865) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-08-22 18:03:51 +08:00
pansicheng
70cf4abccc 3fs zerocopy (#9109) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-08-22 17:56:38 +08:00
Xuchun Shang
cebf45994b [bugfix] Make --enable-hierarchical-cache and --disable-radix-cache mutually exclusive (#9452) 
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
 2025-08-22 17:49:52 +08:00
Qiaolin Yu
9c0c1e30b2 Disable torch.compile for get_last_loc_large_page_size_large_top_k (#9507) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2025-08-22 02:05:02 -07:00
Mick
a1f011d09a minor: determine mm attn backend based on platforms (#9303) 2025-08-22 01:08:41 -07:00
Qiaolin Yu
9ec314c6ac Support speculative decoding in the trtllm_mha attention backend (#9331) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2025-08-21 23:53:35 -07:00
Elfie Guo
243e745d07 Add trtllm_mla and cutlass_mla for ragged fmha for chunked prefill (#9480) 2025-08-21 23:01:36 -07:00
timmy-feng
61a0e600df torch.compile() mrope (#9487) 2025-08-21 23:01:08 -07:00
Yongfei Xu
9708d353b7 Support MHA with chunked prefix cache for flashinfer/flashmla backend, support page size > 1 for MHA chunked prefix (#8616) 
Co-authored-by: xuyongfei.xyf <xuyongfei.xyf@antgroup.com>
 2025-08-21 18:19:44 -07:00
Pavani Majety
3cc3d9b950 Add Support for Page Size greater than 1 for Flashinfer MLA Backend (#8593) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-08-21 18:15:06 -07:00
Xinyuan Tong
6c855db82c Revert "bugfix: Fix output_ids extraction in detokenizer_manager" (#9467) 2025-08-21 17:24:25 -07:00
Yineng Zhang
0f9318f7d0 feat: update auto_choose_speculative_params (#9470) 2025-08-21 17:12:12 -07:00
Yineng Zhang
849957bc76 fix: tmp revert gpt oss tp sharding on hopper (#9469) 2025-08-21 17:03:21 -07:00
Stefan He
cded039b57 [FA3] Init Spec Page Table only when Spec is enabled to save ~40MB (#9455) 2025-08-21 15:11:38 -07:00
zixuanzhang226
275f9df381 feat: add fused moe config for GLM-4.5-Air-FP8 on B200 (#9463) 2025-08-21 15:10:20 -07:00
Xinyuan Tong
e8449ab515 Add deepseek v3.1 thinking parser support and update docs (#9464) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-08-21 15:09:40 -07:00
Yineng Zhang
4746aaea41 fix: support fb fp8 (#9462) 2025-08-21 14:31:43 -07:00
gongwei-130
10d34f74e2 fix: should return a invalid request response when schema missing (#9461) 2025-08-21 14:06:50 -07:00
gongwei-130
9ba7253094 accomendate reasoning_effort set in chat_template_kwargs (#9458) 2025-08-21 13:22:03 -07:00
Hongbo Xu
9c8e4f69c3 [5/n]decouple quantization implementation from vLLM dependency (#9454) 2025-08-21 12:52:07 -07:00
hlu1
dae9a80f43 [fix] Fix mxfp4 weight loading bug with TP sharding in GPT-OSS (#9433) 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-08-21 03:50:51 -07:00
fzyzcjy
e85cb1ce9d Fix quant kernel test errors and benchmark wrong output speeds (#7604) 2025-08-21 03:48:41 -07:00
fzyzcjy
55d336cb08 Refactor weight offloading logic (#8521) 2025-08-21 03:48:13 -07:00
DiweiSun
029e0af31d ci: enhance xeon ci (#9395) 2025-08-21 03:35:17 -07:00
pranavm-nvidia
64574ef8c0 Enables speculative decoding for the trtllm_mla attention backend (#9238) 2025-08-21 01:18:21 -07:00
Kaixi Hou
18da2c96ec [NVIDIA] Fix trtllm fp4 moe backend when used in MTP (#9384) 2025-08-21 00:54:01 -07:00
Liangsheng Yin
9b5f0f64f5 Fix tiny misalign with previous truncation setting in tokenizer_manager (#9430) 2025-08-21 14:05:35 +08:00
VDV1985
2c4b4b786b [feature] Ascend NPU graph support (#9399) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
Co-authored-by: yezhifeng (D) <y00897525@china.huawei.com>
Co-authored-by: anon189Ty <Stari_Falcon@outlook.com>
Co-authored-by: Maksim <makcum888e@mail.ru>
Co-authored-by: ssshinigami <44640852+ssshinigami@users.noreply.github.com>
 2025-08-20 21:13:27 -07:00
Martin Vit
7cd2ee06d7 feat: Add Triton fallback option and SM120 MoE configs for FP8 models (#9251) 2025-08-20 19:33:15 -07:00
Liangsheng Yin
eb19ccadae [bug] fix errors related to context length in SD (#9388) 2025-08-21 10:32:34 +08:00
Shangming Cai
25ef53f05f [PD] Fix nvlink transport accuracy through transferring metadata with tcp (#9261) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-08-20 19:29:10 -07:00
Cao E
c674bf9c6b Fix biased_grouped_topk_cpu (#9420) 2025-08-20 19:18:48 -07:00
Qiaolin Yu
af1973b871 Fix max_seq_len_k in trtllm_mha attention backend (#9416) 2025-08-20 19:17:13 -07:00
strgrb
88fbc31b50 Support trtllm_allreduce_fusion in flashinfer for cuda<12.8 (#9339) 
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
 2025-08-20 16:54:30 -07:00
nathan
8f5b9910c1 Add support for Qwen3-seq-cls (#9357) 2025-08-20 16:51:56 -07:00
Xinyuan Tong
84719b527a fix: InternS1 don't recognize image, updates image token for InternVL processor (#9381) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-08-20 16:43:03 -07:00
jiapingW
e99729c9f3 Fixed the issue where eagle3 TPOT was not as good as without eagle3. (#9404) 2025-08-20 16:42:01 -07:00
Nicolas Castet
c10b8e6a0f Support DP attention with GPT-OSS (#9359) 2025-08-20 16:36:31 -07:00