Tongbao Zhang
|
3980ff1be6
|
rename benchmark_deepgemm_fp8_group_gemm.py (#4605)
|
2025-03-23 23:35:20 -07:00 |
|
Stefan He
|
5d7edc8e55
|
Support FA3 as Attention backend by using --attention-backend fa3 (#4680)
Co-authored-by: qsong <qsong@linkedin.com>
Co-authored-by: qingquansong <ustcsqq@gmail.com>
|
2025-03-23 23:28:11 -07:00 |
|
Alex Sun
|
af6535e7aa
|
[ROCm] Enable MTP (NextN) on AMD GPU (#4631)
|
2025-03-23 22:58:05 -07:00 |
|
c1lovez1
|
93cf7fc5cd
|
Unify variable naming: replace is_in_free_group with is_not_in_free_group (#4698)
|
2025-03-23 21:51:08 -07:00 |
|
Kyungmin Lee
|
2a206b22ed
|
Fix RotaryEmbedding when using Triton backend for EXAONE-3.5-2.4B (#4064)
|
2025-03-23 17:58:12 -07:00 |
|
Zhiqiang Xie
|
4d25305700
|
Move mem_state update into debug mode (#4525)
|
2025-03-23 00:52:27 -07:00 |
|
Mick
|
11577cedb7
|
refactor: bug fixes and refactor for vlm (#4661)
|
2025-03-22 22:48:49 -07:00 |
|
fzyzcjy
|
ca75741e86
|
Support async in DeepEP (#4610)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
|
2025-03-22 22:39:56 -07:00 |
|
fzyzcjy
|
c6d549e773
|
Multiple tiny code cleanups (#4608)
|
2025-03-22 22:39:11 -07:00 |
|
Yi Zhang
|
3c09548d1f
|
close gemma2 in test_verl_engine.py temporarily (#4685)
|
2025-03-22 16:36:46 -07:00 |
|
BroadbentJim
|
8796cebb2c
|
fix typo SGLang supports three grammar backends (#4679)
|
2025-03-22 14:33:48 -07:00 |
|
xutizhou
|
c2bd094d6e
|
Optimize Permute Kernel in DeepEP (#4643)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-03-22 14:30:34 -07:00 |
|
Adarsh Shirawalmath
|
f8f9244a61
|
[Bug Fix] Add partial rotary factor support for Phi-4 and upgrade to transformers v4.50.0 (#3984)
Co-authored-by: Chayenne <zhaochen20@outlook.com>
|
2025-03-22 14:27:39 -07:00 |
|
Zhiqiang Xie
|
ecbfe58bb0
|
Bug fix for metrics counter (#4660)
|
2025-03-22 13:39:21 -07:00 |
|
Ke Bao
|
8f163b1653
|
Add EAGLE mtbench benchmark script (#4676)
Co-authored-by: chromecast56 <jamesll@mit.edu>
|
2025-03-22 13:34:01 -07:00 |
|
Yineng Zhang
|
e7a8610d51
|
fix flaky ut (#4670)
|
2025-03-22 12:36:50 -07:00 |
|
Adarsh Shirawalmath
|
a2cc62a6db
|
[CI fix] test skipping modelopt on AMD (#4677)
|
2025-03-22 12:36:02 -07:00 |
|
Adarsh Shirawalmath
|
fb8886037c
|
[Docs] Update docs for gemma3 and VLM chat templates (#4674)
|
2025-03-22 08:02:19 -07:00 |
|
AniZpZ
|
321ab756bc
|
[1/3] fix dsv3 awq issue (#4556)
Co-authored-by: leoneo <1320612015@qq.com>
|
2025-03-22 01:07:17 -07:00 |
|
penguin_wwy
|
38f25e87fc
|
Correcting default configuration when benchmarking fused_moe (#4665)
|
2025-03-22 00:52:34 -07:00 |
|
Yun Dai
|
8cd4250401
|
[quantization] fix channelwise conversion with scalar weight scale (#4596)
|
2025-03-22 00:47:52 -07:00 |
|
Chunan Zeng
|
6a384d5c01
|
Speed up per token and per tensor quant by 15% (#4639)
|
2025-03-22 00:37:57 -07:00 |
|
Yineng Zhang
|
f69e0696bc
|
update news for README (#4664)
|
2025-03-21 23:25:59 -07:00 |
|
mlmz
|
f6ab4ca6bc
|
fix: fix ipython running error for Engine due to outlines nest_asyncio (#4582)
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
|
2025-03-21 19:11:15 -07:00 |
|
Byron Hsu
|
c7c7dbebbe
|
[PD] Release initial code (#4654)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Ying1123 <sqy1415@gmail.com>
Co-authored-by: merrymercy <lianminzheng@gmail.com>
Co-authored-by: makro
Co-authored-by: dhou-xai
|
2025-03-21 14:47:47 -07:00 |
|
Yuhong Guo
|
417fc72f6f
|
Align completion and chat_completion response to OpenAI API (#4637)
|
2025-03-20 22:59:04 -07:00 |
|
Michael Yao
|
c6ec70290f
|
[docs] Add links and fix grammars in deploy_on_k8s.md (#4641)
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
|
2025-03-20 22:55:23 -07:00 |
|
lukec
|
4c56e5dbee
|
Set deepgemm to the default value in the hopper architecture. (#4613)
|
2025-03-20 22:03:00 -07:00 |
|
Cheng Wan
|
7b5fc71972
|
fix SUPPORT_CUTLASS_BLOCK_FP8 flag (#4640)
|
2025-03-20 21:45:07 -07:00 |
|
Shu Wang
|
ad4e58bf67
|
Support fp8 gemm for blackwell (#4558)
|
2025-03-20 12:40:28 -07:00 |
|
Ke Bao
|
bfb03c6182
|
Update doc for MTP and DP attention (#4622)
|
2025-03-20 11:31:48 -07:00 |
|
Yuhong Guo
|
b36ab493b3
|
Enable setting sglang logger from Env Variable SGLANG_LOGGING_CONFIG_PATH (#4592)
Signed-off-by: Yuhong Guo <yuhong.gyh@antgroup.com>
|
2025-03-20 02:10:32 -07:00 |
|
JieXin Liang
|
9e93ef3f8e
|
[fix] fix illegal mem access and clean up triton attention backend (#4571)
|
2025-03-20 02:01:52 -07:00 |
|
Chuyue Sun
|
fad86a6863
|
Support n in OpenAI API completions (#3446)
Co-authored-by: Shan Yu <shanyu1@g.ucla.edu>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: chuyue sun <chuyue@lmsys.us-northcentral1-a.compute.internal>
|
2025-03-20 13:46:46 +08:00 |
|
strgrb
|
df7014a8d2
|
avoid cudaStreamSynchronize in DeepSeekV2AttentionMLA (#4577)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-03-19 10:02:26 -07:00 |
|
JieXin Liang
|
4942074174
|
[fix] fix initialization of _ENABLE_TORCH_INFERENCE_MODE (#4549)
|
2025-03-19 09:57:59 -07:00 |
|
Hongbo Xu
|
ba52fd1868
|
Add clang-format to pre-commit config (#4583)
|
2025-03-19 09:50:19 -07:00 |
|
lukec
|
b6944f97a6
|
Support FlashMLA backend cuda graph (#4514)
Co-authored-by: yinfan98 <1106310035@qq.com>
Co-authored-by: Hongbosherlock <hongbosherlock@gmail.com>
Co-authored-by: ispobock <ispobaoke@163.com>
|
2025-03-19 08:25:34 -07:00 |
|
Jinyan Chen
|
f44db16c8e
|
[Feature] Integrate DeepEP into SGLang (#4232)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
|
2025-03-19 08:16:31 -07:00 |
|
strgrb
|
f9c53cbb42
|
Create col-major and tma-aligned x_scale for deep_gemm.gemm_fp8_fp8_bf16_nt (#4515)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-03-19 00:02:43 -07:00 |
|
Baizhou Zhang
|
90532b7627
|
[Fix] Fix raw_bs bug when using flashinfer mla and eagle (#4557)
|
2025-03-18 21:26:53 -07:00 |
|
JieXin Liang
|
c0e9a36c5f
|
Optimize Triton decoding kernel for dynamic workload (#4553)
|
2025-03-18 21:25:38 -07:00 |
|
aoshen524
|
588865f0e0
|
[Feature] Support Tensor Parallelism and Weight Slicing for Lora (#4274)
Co-authored-by: ShenAo1111 <1377693092@qq.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-03-18 20:33:07 -07:00 |
|
Cheng Wan
|
3196999f63
|
Reduce computation and communication in DP attention (#4521)
|
2025-03-18 13:41:36 -07:00 |
|
James Liu
|
9e0186f352
|
[Feature] Support EAGLE 3 (#4247)
|
2025-03-18 07:35:23 -07:00 |
|
Wei Wu
|
8baf9a0c18
|
[Fix] Type annotation correction for UpdateWeightsFromTensorReqInput (#4532)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-03-18 00:52:47 -07:00 |
|
Yineng Zhang
|
c787298547
|
use sgl custom all reduce (#4441)
|
2025-03-18 00:46:41 -07:00 |
|
Ke Bao
|
45212ce18b
|
Add deepseek v2 torch compile pr test (#4538)
|
2025-03-18 00:29:24 -07:00 |
|
Yineng Zhang
|
c16b33ccac
|
cleanup deps 3/n (#4541)
|
2025-03-18 00:11:36 -07:00 |
|
Albert
|
2d0045125f
|
Fix the incorrect args in benchmark_and_profiling.md (#4542)
Signed-off-by: Tianyu Zhou <albert.zty@antgroup.com>
|
2025-03-18 00:07:06 -07:00 |
|