EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Commit Graph

  • f9dc9dd28b chore: bump v0.4.7.post1 (#7248) Yineng Zhang 2025-06-16 15:20:29 -07:00
  • 62a7aa2efc Update CI flakes. (#7244) Sai Enduri 2025-06-16 15:19:32 -07:00
  • 5ca07eed90 [fix] fix DeepGEMM blackwell input quant & ut & fix style and log (#7247) JieXin Liang 2025-06-17 02:45:54 +08:00
  • e30ef368ab Feat/support rerank (#6058) woodx 2025-06-17 01:50:01 +08:00
  • 91a066ec6a Tiny remove comments about DeepEP on H20 (#7234) fzyzcjy 2025-06-17 00:13:57 +08:00
  • c494386728 minor fix (#7245) Liangsheng Yin 2025-06-16 23:30:26 +08:00
  • 53a525bf33 [Eagle] Fix kernel call after updating speculative sampling kernels (#7231) Lianmin Zheng 2025-06-16 07:25:59 -07:00
  • 7ddf8e83d2 [EAGLE] Fix draft kv cache layout for fa3 and topk > 1 (#7239) Lianmin Zheng 2025-06-16 05:47:51 -07:00
  • 8321f8e45e Release sgl-kernel 0.1.9 (#7232) Lianmin Zheng 2025-06-16 03:37:40 -07:00
  • cfceb83d05 Fix sampling for speculative decoding & simplify kernels (#7207) Lianmin Zheng 2025-06-16 03:28:30 -07:00
  • b1286a116a [EAGLE] Refactor code for page size > 1 & more simplifications (#7213) Lianmin Zheng 2025-06-16 03:04:29 -07:00
  • 21615cc3fe Minor style and doc fix (#7228) Lianmin Zheng 2025-06-16 01:03:13 -07:00
  • 0ae1e9a755 refine fused_moe benchmark (#7221) Xiaoyu Zhang 2025-06-16 12:21:32 +08:00
  • e07d064729 Support LoRA in MMMU benchmark script. (#7218) Lifu Huang 2025-06-15 21:17:57 -07:00
  • 3c2274fbee Implement gather before attn (#6378) Cheng Wan 2025-06-15 21:08:56 -07:00
  • d2679f5109 Fix ChunkCache object has no attribute 'disable' (#7217) Baizhou Zhang 2025-06-15 20:55:15 -07:00
  • 96be97bfff Minor PD style fix (#7215) Byron Hsu 2025-06-15 16:12:12 -07:00
  • 88f9c347b2 [PD] use int32 for kv indices & get num_reserved_decode_tokens from server_args (#7214) Byron Hsu 2025-06-15 11:51:03 -07:00
  • fff10809bf Revert "[EAGLE] Refactor code for page size > 1 & more simplifications" (#7210) Lianmin Zheng 2025-06-15 02:48:00 -07:00
  • 5f1ab32717 [EAGLE] Refactor code for page size > 1 & more simplifications (#7163) Lianmin Zheng 2025-06-14 23:16:23 -07:00
  • 7df7c679b6 feat: use zstd for docker (#7205) Yineng Zhang 2025-06-14 23:13:29 -07:00
  • 38af4f68a9 Fix grammar abort & Minor style fixes (#7204) Lianmin Zheng 2025-06-14 22:49:41 -07:00
  • a6305c7d50 Lianmin/simplify memory pool (#7202) Lianmin Zheng 2025-06-14 22:25:37 -07:00
  • a023856b12 Move host memory pools into a separate file (#7200) Lianmin Zheng 2025-06-14 21:31:42 -07:00
  • db0cc57e75 [PD] Support decode retract and update decode.py (#7196) Byron Hsu 2025-06-14 19:48:05 -07:00
  • 349bb2c92a Fix error when disabling new DeepGEMM (#7198) fzyzcjy 2025-06-15 10:24:54 +08:00
  • 0b8939bc2c Fix NCCL 2.27.3 not in docker image (#7195) fzyzcjy 2025-06-15 09:36:11 +08:00
  • ed89837cf4 chore: upgrade sgl-kernel v0.1.8.post2 (#7186) JieXin Liang 2025-06-15 09:26:18 +08:00
  • 55561e2553 [fix] fix determine_num_fused_shared_experts (#7180) JieXin Liang 2025-06-15 08:41:22 +08:00
  • 4473320380 chore: bump v0.1.8.post2 (#7189) Yineng Zhang 2025-06-14 17:01:48 -07:00
  • 0bd67ba2bd Fix a minor bug related to DeepGEMM upgrade (#7191) Zhijian Liu 2025-06-14 16:54:40 -07:00
  • 7d316991b2 [PD] Update prefill.py (#7190) Byron Hsu 2025-06-14 15:59:54 -07:00
  • ab1a4fa5cb [fix] fix cutlass_mla_backend with cuda_graph and add sm_scale for sgl-kernel cutlass_mla (#7184) JieXin Liang 2025-06-15 03:45:41 +08:00
  • ed54bf9d19 [fix] fix dsv3 weight loader tqdm and simplify shared experts fusion (#7181) JieXin Liang 2025-06-15 02:56:29 +08:00
  • b57d87c297 Fix shared experts fusion + weight requant (#7177) fzyzcjy 2025-06-14 17:35:18 +08:00
  • 98538822d5 Add Phi-4-mm to supported VLM supported model list. (#7178) Lifu Huang 2025-06-13 23:17:40 -07:00
  • f47a1b1d0f Increase timeout in test/srt/test_disaggregation.py (#7175) Lianmin Zheng 2025-06-13 23:12:14 -07:00
  • 93cec4335f Support new DeepGEMM (#7172) fzyzcjy 2025-06-14 14:00:17 +08:00
  • ba589b88fc Improve test cases for eagle infer (#7173) Lianmin Zheng 2025-06-13 22:25:13 -07:00
  • 50876abc47 Add test for refactored openai server (#7161) Jinn 2025-06-13 22:42:57 -05:00
  • b4c41f7276 Refactor DeepGEMM integration (#7150) fzyzcjy 2025-06-14 11:41:03 +08:00
  • 8b8f2e7463 Support new DeepGEMM input format in silu_and_mul_masked_post_quant_fwd (#7153) fzyzcjy 2025-06-14 11:40:24 +08:00
  • 0fc3d992bb Split the eagle test into two files (#7170) Lianmin Zheng 2025-06-13 20:14:26 -07:00
  • be2d985df8 Minor style change of triton backend (#7165) Lianmin Zheng 2025-06-13 16:01:23 -07:00
  • 5b1afa7814 Re-quantize DeepSeek model weights to support DeepGEMM new input format (#7156) fzyzcjy 2025-06-14 06:57:45 +08:00
  • c49c1d9226 Remove 200us slow concat kernel (part 2: srt) (#7020) fzyzcjy 2025-06-14 06:19:31 +08:00
  • 0f1dfa1efe Tiny add sanity checks for DeepGEMM inputs (#7157) fzyzcjy 2025-06-14 05:36:27 +08:00
  • e3ec6bf4b6 Minor speed up block_quant_dequant (#6814) fzyzcjy 2025-06-14 05:32:46 +08:00
  • b04df75acd Fix DeepEP error in some environments (#7154) fzyzcjy 2025-06-14 05:27:25 +08:00
  • bec3e48402 Support new DeepGEMM format in per token group quant (part 2: srt) (#7155) fzyzcjy 2025-06-14 05:25:40 +08:00
  • 8ab7d93c2e chore: bump v0.1.8.post1 (#7152) Yineng Zhang 2025-06-13 03:14:26 -07:00
  • 5c66c4424f Support new DeepGEMM format in per token group quant (#7146) fzyzcjy 2025-06-13 17:00:22 +08:00
  • aa46ed34d2 Remove 200us slow concat kernel (part 1: kernel) (#7145) fzyzcjy 2025-06-13 16:58:29 +08:00
  • 2f4ec752bc filter by num_hidden_layers (#7056) pansicheng 2025-06-13 15:53:09 +08:00
  • da47621ccc Minor speedup topk postprocessing (#7058) fzyzcjy 2025-06-13 15:50:18 +08:00
  • 22a6b9fc05 Remove unnecessary metadata_expand.max_seq_len_k operations in fa3 to… (#7140) Binyao Jiang 2025-06-12 23:25:52 -07:00
  • b02df20a8d Minor add utility to read expert distribution recorder output (#7134) fzyzcjy 2025-06-13 07:50:05 +08:00
  • bd7cfbd2f8 [Fix] Reduce busy polling when scheduler is idle (#6026) Povilas Kanapickas 2025-06-13 00:58:22 +03:00
  • 4b9971e401 Add gfx950 support for sgl-kernel. (#7092) sogalin 2025-06-13 02:07:48 +08:00
  • dcc79d325b Update CODEOWNERS (#7126) Lianmin Zheng 2025-06-12 02:12:49 -07:00
  • 7046e0fab7 feat: update blackwell setup (#7119) Yineng Zhang 2025-06-12 01:54:40 -07:00
  • 930746d93c Improve log status (#7115) Liangsheng Yin 2025-06-12 14:38:24 +08:00
  • 84727a5139 [sgl-kernel] Add cuda kernel for moe_ep_silu_and_mul (#6919) Yuan Luo 2025-06-12 11:43:08 +08:00
  • ef32677444 Fix positional argument (#7093) Quanfeng Li 2025-06-12 09:31:13 +08:00
  • 021f76e4f4 [Perf] Refactor LoRAManager to eliminate stream syncs and redundant computations (#6994) Lifu Huang 2025-06-11 16:18:57 -07:00
  • 777688b892 [feat]: Emit fixed-size KV blocks events (#6824) Faradawn Yang 2025-06-11 15:07:58 -05:00
  • 0ca594eda9 [FIX]remove redundant code in logits_processor.py (#7079) Neo 2025-06-12 02:49:30 +08:00
  • 31d6dee5c4 Support VILA models (#6106) Zijian 2025-06-12 02:47:25 +08:00
  • 02543b545c Fix misusing the "_is_cuda". (#7091) sogalin 2025-06-12 02:21:31 +08:00
  • 25a6a9aa22 Fix circular import in test_prefix_chunk_info.py (#7097) Baizhou Zhang 2025-06-11 10:57:45 -07:00
  • 83d87685c5 vlm: adapt internvl to VisionAttention (#6870) Mick 2025-06-11 16:16:04 +08:00
  • 2a5f0100e0 Fix GGuf and add back test_gguf.py (#7067) Baizhou Zhang 2025-06-10 21:07:20 -07:00
  • dbdf76ca98 Clean up docs for server args and sampling parameters (generated by grok) (#7076) Lianmin Zheng 2025-06-10 19:55:42 -07:00
  • f2a75a66c4 update doc (#7046) Ximingwang-09 2025-06-11 10:02:01 +08:00
  • 6b12d6a8d5 Simplify the heuristics for setting --mem-fraction-static (#7054) Lianmin Zheng 2025-06-10 19:01:39 -07:00
  • 0f218731e3 Do not run frontend_reasoning.ipynb to reduce the CI load (#7073) Lianmin Zheng 2025-06-10 17:15:31 -07:00
  • 14c18d25df Frontend language separate reasoning support (#6031) Yudi Xue 2025-06-10 17:11:29 -07:00
  • 90bd3e32d6 Improve perf tuning docs (#7071) Lianmin Zheng 2025-06-10 16:55:04 -07:00
  • ca9291181d [Feature] Add Logit Bias (#6579) Brayden Zhong 2025-06-10 18:39:25 -04:00
  • 344adb00ec fix arm sgl-kernel link issue (#7066) Yineng Zhang 2025-06-10 15:23:22 -07:00
  • b56de8f943 Open AI API hidden states (#6716) kyle-pena-kuzco 2025-06-10 17:37:29 -04:00
  • ce5ee3bdf0 chore: update dev docker (#7064) Yineng Zhang 2025-06-10 12:57:58 -07:00
  • a0e4d4eb53 Fix missing tool call id if tool call index >0 in streaming tool call output. (#7049) Xu Wenqing 2025-06-11 03:56:43 +08:00
  • 2f58445531 Revert "Add sanity checks when a test file is not added to CI (#6947)" (#7063) Yineng Zhang 2025-06-10 12:43:25 -07:00
  • fe55947acd Add sanity checks when a test file is not added to CI (#6947) fzyzcjy 2025-06-11 03:34:57 +08:00
  • 19995dd78e Tiny fix cutlass_mla_get_workspace_size stub incorrect signature (#7057) fzyzcjy 2025-06-11 03:27:57 +08:00
  • 3b014bc13d Fix test_lora.py CI (#7061) Baizhou Zhang 2025-06-10 12:24:46 -07:00
  • d7c3e8e93d [fix] libmlx5.so already in base image (#7060) Hank Han 2025-06-11 03:22:24 +08:00
  • 8ea7df6114 [WA] fix output data is nan in CI test "test_moe_eval_accuracy_large.py" (#7021) kk 2025-06-11 00:08:10 +08:00
  • 4a102a2b02 Minor style fix in cuda_graph_runner.py (#7053) Lianmin Zheng 2025-06-10 06:32:41 -07:00
  • 6406408a70 Clean up server_args.py (#7037) Lianmin Zheng 2025-06-10 05:34:29 -07:00
  • 019851d099 Fix eagle on AMD (#7051) Lianmin Zheng 2025-06-10 05:22:40 -07:00
  • 2dae104dca Minor cleanup of fa3 backend (#6999) Lianmin Zheng 2025-06-10 03:58:44 -07:00
  • cef6655b26 fix 24.12 docker (#7045) Yineng Zhang 2025-06-10 02:43:22 -07:00
  • 27196d4148 [Docker] Upgrading base image from 24.04 to 24.12 (#7043) Swipe4057 2025-06-10 12:23:05 +03:00
  • bb185b0e92 Update README.md (#7040) Lianmin Zheng 2025-06-10 01:59:14 -07:00
  • 4f723edd3b chore: bump v0.4.7 (#7038) Yineng Zhang 2025-06-10 01:56:20 -07:00
  • fcde67b016 CPU: map changes from developing branch in sgl-kernel (#6833) YanbingJiang 2025-06-10 16:08:15 +08:00
  • 81372f3bef Fix fused_moe triton configs (#7029) yudian0504 2025-06-10 14:23:03 +08:00
  • baa6624d7c [CI] Add CI workflow for sgl-router docker build (#7027) Arthur Cheng 2025-06-09 23:16:44 -07:00