Liangsheng Yin
|
40d9b8acce
|
Improve overlap scheduling (#5788)
|
2025-04-28 11:19:16 +08:00 |
|
Simo Lin
|
f0365820e8
|
[Misc] add structure logging, write to file and log tracing for SGL Router
|
2025-04-27 16:54:10 -07:00 |
|
Huapeng Zhou
|
86317c09e9
|
[Docs] update grafana setup guide in production metrics (#5643)
Co-authored-by: NoahM <88418672+zhudianGG@users.noreply.github.com>
|
2025-04-27 15:36:33 -07:00 |
|
Lianmin Zheng
|
daed453e84
|
[CI] Improve github summary & enable fa3 for more models (#5796)
|
2025-04-27 15:29:46 -07:00 |
|
Lianmin Zheng
|
ded04b2e0a
|
Update nightly-test.yml (#5797)
|
2025-04-27 15:27:24 -07:00 |
|
Baizhou Zhang
|
84022c0e56
|
Release v0.4.6 (#5795)
|
2025-04-27 14:07:05 -07:00 |
|
Baizhou Zhang
|
f9fb33efc3
|
Add 8-GPU Test for Deepseek-V3 (#5691)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2025-04-27 12:46:12 -07:00 |
|
Lianmin Zheng
|
a38f6932cc
|
[CI] Fix test case (#5790)
|
2025-04-27 08:55:35 -07:00 |
|
Liangsheng Yin
|
beb65c7433
|
[PD]Reduce kv transfer threads (#5791)
|
2025-04-27 23:03:30 +08:00 |
|
Lianmin Zheng
|
621e96bf9b
|
[CI] Fix ci tests (#5769)
|
2025-04-27 07:18:10 -07:00 |
|
Lianmin Zheng
|
35ca04d2fa
|
[CI] fix port conflicts (#5789)
|
2025-04-27 05:17:44 -07:00 |
|
Lianmin Zheng
|
3c4e0ee64d
|
[CI] Tune threshold (#5787)
|
2025-04-27 04:10:22 -07:00 |
|
Lianmin Zheng
|
9c088829ee
|
Revert "Use device_id in dist init to reduce NCCL communicator warmup & creation overhead" (#5786)
|
2025-04-27 04:03:02 -07:00 |
|
Lianmin Zheng
|
005aad32ad
|
Revert "[fix] fix bench_one_batch_server" (#5785)
|
2025-04-27 03:48:33 -07:00 |
|
Lianmin Zheng
|
4d23ba08f5
|
Simplify FA3 tests (#5779)
|
2025-04-27 01:30:17 -07:00 |
|
Lianmin Zheng
|
6e313c1b8b
|
Revert "Revert "fix: import vllm_rotary_embedding error when head_size not in 64, 128, 256, 512"" (#5777)
|
2025-04-27 01:04:15 -07:00 |
|
Baizhou Zhang
|
a45a4b239d
|
Split local attention test from fa3 test (#5774)
|
2025-04-27 01:03:31 -07:00 |
|
Lianmin Zheng
|
981a2619d5
|
Fix eagle test case (#5776)
|
2025-04-27 01:00:54 -07:00 |
|
Lianmin Zheng
|
8ba313304d
|
Revert "fix: import vllm_rotary_embedding error when head_size not in 64, 128, 256, 512" (#5772)
|
2025-04-26 23:26:08 -07:00 |
|
zhanweidu
|
021020632a
|
add switch to disable open api doc (#3744)
Signed-off-by: congcongke <zhanweidu@163.com>
|
2025-04-26 23:18:47 -07:00 |
|
Kebe
|
7e944246c3
|
Add memory_saver check (#4986)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2025-04-26 20:20:05 -07:00 |
|
lambert0312
|
a086a11305
|
Use sgl-kernel sgl_per_token_group_quant_int8 (#4971)
|
2025-04-26 20:19:42 -07:00 |
|
Michał Moskal
|
bdbe5f816b
|
update llguidance to 0.7.11; adds StructTag (#4870)
|
2025-04-26 20:13:57 -07:00 |
|
aoshen524
|
9ad28f639e
|
fix(srt): check if sample_indices is not None before usage. (#5633)
|
2025-04-26 19:51:01 -07:00 |
|
yan97ao
|
d7b1ce65a5
|
Handle JSONDecodeError while processing request data (#5599)
|
2025-04-26 19:50:50 -07:00 |
|
JieXin Liang
|
f55933e1cc
|
[misc] more decode step log for batch_one_batch (#5565)
|
2025-04-26 19:50:28 -07:00 |
|
Stefan He
|
408ba02218
|
Add Llama 4 to FA3 test (#5509)
|
2025-04-26 19:49:31 -07:00 |
|
vzed
|
094891c01a
|
fix: Use is not None instead of != None for None checks. (#5687)
|
2025-04-26 19:26:57 -07:00 |
|
Frankey_8080
|
a21ef36352
|
support for the DeepSeek model by enabling streaming response parsing (#5592)
|
2025-04-26 18:59:31 -07:00 |
|
JieXin Liang
|
3c4dc38a9a
|
[fix] fix bench_one_batch_server (#5607)
|
2025-04-26 18:49:45 -07:00 |
|
DavidBao
|
d8fbc7c096
|
[feature] support for roberta embedding models (#5730)
|
2025-04-26 18:47:06 -07:00 |
|
saienduri
|
c5e1026f47
|
Update amd docker image to sglang:v0.4.5.post3-rocm630. (#5697)
|
2025-04-26 18:46:57 -07:00 |
|
Ke Bao
|
799c4bb502
|
Fuse MLA set kv cache kernel (#5748)
|
2025-04-26 18:42:22 -07:00 |
|
Mick
|
02723e1b0d
|
CI: rewrite test_vision_chunked_prefill to speedup (#5682)
|
2025-04-26 18:33:13 -07:00 |
|
vzed
|
df2cf583ce
|
we fix the non existent access of decrypted_config_file (#5685)
|
2025-04-26 18:32:37 -07:00 |
|
saltyfish66
|
133ded039a
|
perf: update H20 fused_moe_triton kernel config to get higher throughput during prefilling (#5716)
|
2025-04-26 18:15:07 -07:00 |
|
Yuhong Guo
|
f87a6ab359
|
Resolves the 404 Not Found error when running compile_deep_gemm.py in multi-node setups (#5720)
|
2025-04-26 18:13:13 -07:00 |
|
JieXin Liang
|
eebfdb9459
|
[fix] fix potential bumpy throughtput with deepgemm (#5722)
|
2025-04-26 18:12:48 -07:00 |
|
Wenxuan Tan
|
dfb322642f
|
Use device_id in dist init to reduce NCCL communicator warmup & creation overhead (#5728)
|
2025-04-26 18:11:09 -07:00 |
|
Kyungmin Lee
|
63c13a2c73
|
fix: import vllm_rotary_embedding error when head_size not in 64, 128, 256, 512 (#5733)
|
2025-04-26 18:10:23 -07:00 |
|
liwenju0
|
4d1e52abea
|
Add an assertion to enhance the robustness of the operator (#5736)
|
2025-04-26 18:09:12 -07:00 |
|
Lianmin Zheng
|
155890e4d1
|
[Minor] fix documentations (#5756)
|
2025-04-26 17:48:43 -07:00 |
|
Yi Zhang
|
1f963d7f64
|
Bugfix for minicpmo vision test (#5760)
|
2025-04-26 23:18:02 +08:00 |
|
ZXN
|
04d0123fd9
|
[Fix]: support deepseek-vl2-tiny model (#5552)
Co-authored-by: bppps <zouyu.zzx@alibaba-inc.com>
|
2025-04-26 17:52:53 +08:00 |
|
Mick
|
feda9b11b3
|
fix: fix one more bug from merging mm_inputs (#5718)
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: XinyuanTong <115166877+JustinTong0323@users.noreply.github.com>
|
2025-04-25 17:28:33 -07:00 |
|
Ke Bao
|
c3948ba67e
|
Reorder loop in shared expert weight loading (#5719)
|
2025-04-25 17:27:42 -07:00 |
|
Michael Yao
|
269c457e05
|
[Docs] Update runtime/engine/readme.md (#5737)
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
|
2025-04-25 16:39:29 -07:00 |
|
Xiaoyu Zhang
|
18ce468d56
|
update triton 3.2.0 h200 fused moe triton config and add warning about triton fused_moe_kernel performance degradation due to different Triton versions. (#5740)
|
2025-04-25 16:24:59 -07:00 |
|
Lianmin Zheng
|
21514ff5bd
|
Disable flaky eagle tests (#5753)
|
2025-04-25 15:54:39 -07:00 |
|
Lianmin Zheng
|
5641a09458
|
Revert "[Model] Support ArcticForCausalLM architecture (Snowflake/snowflake-arctic-instruct)" (#5754)
|
2025-04-25 15:50:28 -07:00 |
|