EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Commit Graph

  • 988accbc1e Update docker file for supporting PD-Disaggregation on MI300x (#9494) kk 2025-08-22 14:48:40 +08:00
  • b6b2287e4b chore: bump sgl-kernel v0.3.6.post2 (#9475) Yineng Zhang 2025-08-21 23:02:08 -07:00
  • 243e745d07 Add trtllm_mla and cutlass_mla for ragged fmha for chunked prefill (#9480) Elfie Guo 2025-08-21 23:01:36 -07:00
  • 61a0e600df torch.compile() mrope (#9487) timmy-feng 2025-08-22 02:01:08 -04:00
  • 0f8cee8cd3 [router] fix router load guard tracking for streaming (#9491) Simo Lin 2025-08-21 22:48:29 -07:00
  • 816c4c8572 [router] add tool parser base structure and partial json parser (#9482) Chang Su 2025-08-21 22:08:56 -07:00
  • 13ec8d427e [Docs]Update reasoning parser doc & fix outdated link (#9492) Xinyuan Tong 2025-08-22 13:08:28 +08:00
  • 05bd789791 [docs]: fix reasoning context in docs (#9483) Chayenne 2025-08-21 20:04:12 -07:00
  • 5fd311d33e [code clean] add H20 cutlass groupGemm default config (#9333) kousakawang 2025-08-22 10:23:29 +08:00
  • 53e2cd464f [router] remove all tokenizer metrics for performance (#9474) Chang Su 2025-08-21 18:35:24 -07:00
  • 9708d353b7 Support MHA with chunked prefix cache for flashinfer/flashmla backend, support page size > 1 for MHA chunked prefix (#8616) Yongfei Xu 2025-08-22 09:19:44 +08:00
  • 704ced1b2e [AMD] Remove the deprecated C10_WARP_SIZE (#9356) Hubert Lu 2025-08-21 18:16:35 -07:00
  • 3cc3d9b950 Add Support for Page Size greater than 1 for Flashinfer MLA Backend (#8593) Pavani Majety 2025-08-21 18:15:06 -07:00
  • 0b3a5b1151 Update reasoning parser doc (#9468) Xinyuan Tong 2025-08-22 08:25:30 +08:00
  • 6c855db82c Revert "bugfix: Fix output_ids extraction in detokenizer_manager" (#9467) Xinyuan Tong 2025-08-22 08:24:25 +08:00
  • 0f9318f7d0 feat: update auto_choose_speculative_params (#9470) Yineng Zhang 2025-08-21 17:12:12 -07:00
  • 849957bc76 fix: tmp revert gpt oss tp sharding on hopper (#9469) Yineng Zhang 2025-08-21 17:03:21 -07:00
  • cded039b57 [FA3] Init Spec Page Table only when Spec is enabled to save ~40MB (#9455) Stefan He 2025-08-21 15:11:38 -07:00
  • 275f9df381 feat: add fused moe config for GLM-4.5-Air-FP8 on B200 (#9463) zixuanzhang226 2025-08-21 15:10:20 -07:00
  • e8449ab515 Add deepseek v3.1 thinking parser support and update docs (#9464) Xinyuan Tong 2025-08-22 06:09:40 +08:00
  • 4746aaea41 fix: support fb fp8 (#9462) Yineng Zhang 2025-08-21 14:31:43 -07:00
  • 10d34f74e2 fix: should return a invalid request response when schema missing (#9461) gongwei-130 2025-08-21 14:06:50 -07:00
  • 9ba7253094 accomendate reasoning_effort set in chat_template_kwargs (#9458) gongwei-130 2025-08-21 13:22:03 -07:00
  • 9c8e4f69c3 [5/n]decouple quantization implementation from vLLM dependency (#9454) Hongbo Xu 2025-08-22 03:52:07 +08:00
  • 78ae175866 [router] add tokenizer benchmark (#9427) Simo Lin 2025-08-21 11:09:39 -07:00
  • dae9a80f43 [fix] Fix mxfp4 weight loading bug with TP sharding in GPT-OSS (#9433) hlu1 2025-08-21 03:50:51 -07:00
  • e85cb1ce9d Fix quant kernel test errors and benchmark wrong output speeds (#7604) fzyzcjy 2025-08-21 18:48:41 +08:00
  • 55d336cb08 Refactor weight offloading logic (#8521) fzyzcjy 2025-08-21 18:48:13 +08:00
  • de4990a5b2 [Bug] Fix w4afp8 moe kernel (#9392) Yuhao Yao 2025-08-21 18:45:18 +08:00
  • 029e0af31d ci: enhance xeon ci (#9395) DiweiSun 2025-08-21 18:35:17 +08:00
  • 64574ef8c0 Enables speculative decoding for the trtllm_mla attention backend (#9238) pranavm-nvidia 2025-08-21 01:18:21 -07:00
  • 18da2c96ec [NVIDIA] Fix trtllm fp4 moe backend when used in MTP (#9384) Kaixi Hou 2025-08-21 00:54:01 -07:00
  • 9b5f0f64f5 Fix tiny misalign with previous truncation setting in tokenizer_manager (#9430) Liangsheng Yin 2025-08-21 14:05:35 +08:00
  • 70bb066ee4 Fix FP4 inference corruption issue in glm4.5-air model (#9346) Azure 2025-08-21 13:13:47 +08:00
  • 2c4b4b786b [feature] Ascend NPU graph support (#9399) VDV1985 2025-08-21 07:13:27 +03:00
  • 7cd2ee06d7 feat: Add Triton fallback option and SM120 MoE configs for FP8 models (#9251) Martin Vit 2025-08-21 04:33:15 +02:00
  • eb19ccadae [bug] fix errors related to context length in SD (#9388) Liangsheng Yin 2025-08-21 10:32:34 +08:00
  • 25ef53f05f [PD] Fix nvlink transport accuracy through transferring metadata with tcp (#9261) Shangming Cai 2025-08-21 10:29:10 +08:00
  • c674bf9c6b Fix biased_grouped_topk_cpu (#9420) Cao E 2025-08-21 10:18:48 +08:00
  • af1973b871 Fix max_seq_len_k in trtllm_mha attention backend (#9416) Qiaolin Yu 2025-08-20 19:17:13 -07:00
  • 5cfbb4c136 [router] add glm and step3 reasoning parser (#9415) Chang Su 2025-08-20 18:33:10 -07:00
  • e65231022f [router] add tokenizer integration test with real mini tokenizer (#9413) Chang Su 2025-08-20 17:56:23 -07:00
  • 3828db4309 [router] Add IGW (Inference Gateway) Feature Flag (#9371) Keyang Ru 2025-08-20 17:38:57 -07:00
  • 88fbc31b50 Support trtllm_allreduce_fusion in flashinfer for cuda<12.8 (#9339) strgrb 2025-08-21 07:54:30 +08:00
  • 8f5b9910c1 Add support for Qwen3-seq-cls (#9357) nathan 2025-08-21 01:51:56 +02:00
  • ef3004d90a misc: parse bench_serving result as markdown table (#9377) Mick 2025-08-21 07:44:20 +08:00
  • 84719b527a fix: InternS1 don't recognize image, updates image token for InternVL processor (#9381) Xinyuan Tong 2025-08-21 07:43:03 +08:00
  • e99729c9f3 Fixed the issue where eagle3 TPOT was not as good as without eagle3. (#9404) jiapingW 2025-08-21 07:42:01 +08:00
  • c10b8e6a0f Support DP attention with GPT-OSS (#9359) Nicolas Castet 2025-08-20 18:36:31 -05:00
  • d4bce29721 Fix incorrect logic in chat template handling. (#9336) Lifu Huang 2025-08-20 16:25:36 -07:00
  • b0980af89f Support pinning adapter via server args. (#9249) Lifu Huang 2025-08-20 16:25:01 -07:00
  • 24eaebeb4b Fix FlashInfer GPU <-> CPU sync (#9409) Nathan Wang 2025-08-20 18:26:12 -04:00
  • a91e90d9a3 [2/2] Fuse routed scaling factor into select_experts (#8690) Trevor Morris 2025-08-20 15:10:16 -07:00
  • f96413c444 Refactor allreduce add rmsnorm pattern (#9278) Xiaoyu Zhang 2025-08-20 17:03:08 +08:00
  • 08ebdf79d0 Fix the --allow-auto-truncate argument in tokenizer manager. (#9391) Liangsheng Yin 2025-08-20 16:56:47 +08:00
  • 42c8704560 Add PDL support for quant kernel and rope kernel (#9106) fzyzcjy 2025-08-20 16:56:29 +08:00
  • c9bf3877a0 Reduce overhead for fa by not calling heavy CUDA property check (#7375) Yichen Yan 2025-08-20 16:26:28 +08:00
  • de2dd73831 Revert "[feature] Rework Ascend NPU graph support" (#9385) Even Zhou 2025-08-20 15:35:10 +08:00
  • 1ec9769753 [Docs] Update contribution guide (#9383) Lianmin Zheng 2025-08-19 23:37:45 -07:00
  • d8ed60f254 [CI] Fix disaggregation failure tolerance CI (#9378) Shangming Cai 2025-08-20 14:31:08 +08:00
  • f1b0eda55c [readme] Add SGLang x AMD SF meetup information (#9380) Mingyi 2025-08-19 22:25:09 -07:00
  • f20b6a3f2b [minor] Sync style changes (#9376) Lianmin Zheng 2025-08-19 21:35:01 -07:00
  • 3680d6f88b [feature] Rework Ascend NPU graph support (#9350) Even Zhou 2025-08-20 11:32:27 +08:00
  • f515449582 Fix gpt-oss response api streaming issue (#9368) Keyang Ru 2025-08-19 20:19:42 -07:00
  • e0ce171d79 Fix triton backend eagle illegal memory access (#9344) Ke Bao 2025-08-20 11:16:26 +08:00
  • fe43e889f8 Fix mini lb timeout issue (#9369) fzyzcjy 2025-08-20 11:15:16 +08:00
  • 5ae5ecaa15 [router] Implement OpenAI Responses API specification (#9367) Keyang Ru 2025-08-19 20:14:47 -07:00
  • 5fbad308cd [router] add tokenizer chat template support (#9370) Simo Lin 2025-08-19 20:14:02 -07:00
  • 7638f5e44e [router] Implement gRPC SGLangSchedulerClient (#9364) Chang Su 2025-08-19 16:44:11 -07:00
  • b45f753cba [router] adds reasoning parser pooling and thread-safe (#9360) Simo Lin 2025-08-19 13:35:39 -07:00
  • c5057262fa [Router] Add validation module for API parameters (#9335) Keyang Ru 2025-08-19 13:25:53 -07:00
  • 46fe8b8cb2 [CI] Fix lint issues (#9361) Chang Su 2025-08-19 13:05:36 -07:00
  • 0b95a01a8f [router] add tiktokenizer and sequence in router (#9354) Simo Lin 2025-08-19 10:46:28 -07:00
  • a3b810ebdb fix: enable multi-GPU Triton fused MoE tuning (#6295) mpashkovskiy 2025-08-19 20:16:58 +03:00
  • 94959237bf [router] add dsr1, kimi, and qwen reasoning parser (#9353) Simo Lin 2025-08-19 10:15:24 -07:00
  • f4fafacc5d Revert "[feature] Ascend NPU graph support (#8027)" (#9348) Even Zhou 2025-08-20 01:11:23 +08:00
  • 01d47a27b6 [Bugfix] fix kv buffer register & dp attention & deepepmoe (#9327) chenxu140 2025-08-20 01:09:48 +08:00
  • ecc9f3e47a [Minor] Fix the style of sgl-kernel (#9332) Lianmin Zheng 2025-08-18 23:45:00 -07:00
  • 7e8187e004 docs: fix spec (#9326) Yineng Zhang 2025-08-18 19:35:46 -07:00
  • e483ab6d20 enable marlin fp8 blockwise (#8990) Enrique Shockwave 2025-08-19 02:53:15 +01:00
  • 720cd308ba Add CMakeLists.txt binary_dir (#7019) EduardDurech 2025-08-19 03:36:33 +02:00
  • ce67b2d586 [router]restructure protocol modules for better organization (#9321) Keyang Ru 2025-08-18 18:07:58 -07:00
  • 3c2c9f6c9e [Bug] Fix input arguments of flashinfer_trtllm_moe (#9317) Jiaqi Gu 2025-08-18 18:03:19 -07:00
  • a31ea44824 support for interns1-mini (#9299) zxy 2025-08-19 08:56:04 +08:00
  • 439df4548a [router] Add spec for sglang scheduler (#9322) Chang Su 2025-08-18 17:20:20 -07:00
  • 5626e20b2b Tiny fix CI (#9306) fzyzcjy 2025-08-19 07:54:36 +08:00
  • c6c379ab31 [AMD] Reorganize hip-related header files in sgl-kernel (#9320) Hubert Lu 2025-08-18 16:53:44 -07:00
  • c2fbf60f39 [GLM4.1V and GLM4.5V] Add vision transformer num_dummy_head support: max tp=4 -> max tp=8 (#9059) Binyao Jiang 2025-08-18 14:40:13 -07:00
  • 98b44e9e56 [PD] Propagate internal server errors from aborted requests to clients instead of blindly returning 200's (#8936) datdo-msft 2025-08-18 14:23:46 -07:00
  • 6805f6da40 upgrade xgrammar 0.1.23 and openai-harmony 0.0.4 (#9284) Swipe4057 2025-08-19 00:02:00 +03:00
  • ca533580f2 [Docs] Correct and clarify notes in Engine docstring (#9313) 江家瑋 2025-08-19 04:24:19 +08:00
  • 886454e8e7 [MISC] use dynamic choices for tool-call-parser argument (#9316) Keyang Ru 2025-08-18 13:02:10 -07:00
  • 0cf3fbeb18 should return invalide request for empty prompt (#9315) gongwei-130 2025-08-18 11:44:11 -07:00
  • 2256d62d36 Modelopt quant config adaptation (#8829) Zhiyu 2025-08-18 11:27:30 -07:00
  • 6cdcbcc674 [fix] fix enable_pdl for blackwell (#9011) JieXin Liang 2025-08-19 01:16:08 +08:00
  • c480a3f6ea Minor style fixes for sgl-kernel (#9289) Lianmin Zheng 2025-08-18 09:38:35 -07:00
  • 6e316588f8 [router] add reasoning parser base structure (#9310) Simo Lin 2025-08-18 09:26:09 -07:00
  • 24247b4168 [router] add tokenizer metrics (#9307) Simo Lin 2025-08-18 09:25:51 -07:00
  • 4c0bb411e5 Further fix memory pool leak error (#9298) fzyzcjy 2025-08-18 15:58:06 +08:00
  • 968e181826 Fix triton_fused_moe unit test and benchmark (#9276) Yuan Luo 2025-08-18 15:54:33 +08:00