valarLip
|
e984d5073b
|
enable aiter_biased_grouped_topk kernel (#7423)
|
2025-06-24 02:09:42 -07:00 |
|
Alex Sun
|
755f314785
|
[AMD] add aiter fused moe in DeepEP path (#7268)
|
2025-06-24 09:05:47 +00:00 |
|
Yineng Zhang
|
7c3a12c000
|
chore: bump v0.4.8 (#7493)
|
2025-06-23 23:14:22 -07:00 |
|
Yineng Zhang
|
e846d95ef6
|
chore: bump sgl-kernel v0.2.0 (#7490)
|
2025-06-23 22:29:50 -07:00 |
|
Charles Chen
|
15f3401343
|
Fix MTP with Deepseek R1 Fp4 (#7376)
|
2025-06-23 21:38:07 -07:00 |
|
Chang Su
|
d04163b3fa
|
Fix RequestValidationError response format (#7487)
|
2025-06-23 20:35:11 -07:00 |
|
huangtingwei
|
7732bbe458
|
bugfix: Prevent global mutation of conv.stop_str across requests (#7347)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-06-23 19:36:23 -07:00 |
|
u4lr451
|
ed0a0b692c
|
Perormance: Enable cuda graph for dp idle batch (#7269)
Co-authored-by: austindeng <austindeng@tencent.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: ch-wan <cwan39@gatech.edu>
|
2025-06-23 17:34:13 -07:00 |
|
Chang Su
|
fa42e41962
|
ci: Revert openai_server related tests in AMD suites (#7449)
|
2025-06-23 15:28:22 -07:00 |
|
Yuhong Guo
|
e5afb88b1c
|
Support weight loading without mmap (#7469)
|
2025-06-23 15:13:59 -07:00 |
|
Charles Chen
|
e5ddeb04d5
|
Quick fix for DeepGemm requant to also cover MTP. (#7378)
|
2025-06-23 12:08:54 -07:00 |
|
JieXin Liang
|
bdbb8d009a
|
[perf] slightly imporve DeepSeek-R1-FP4 TP8 (#7481)
|
2025-06-23 12:05:30 -07:00 |
|
Zhiqiang Xie
|
34c3f9b2d3
|
kvcache io kernels and test case (#7382)
|
2025-06-23 11:58:59 -07:00 |
|
Liangsheng Yin
|
76139bfba0
|
update mooncake in dockerfile (#7480)
|
2025-06-24 02:29:30 +08:00 |
|
Cheng Wan
|
f8d48fd311
|
Fix dtype for idle input in spec decoding (#7456)
|
2025-06-23 11:23:25 -07:00 |
|
Chang Su
|
34b6b8426f
|
feat(func_call): Add more check in BaseFormatDetector.parse_streaming_increment (#7479)
|
2025-06-23 11:15:47 -07:00 |
|
Liangsheng Yin
|
25549433e8
|
Fix prefill OOM due to wrong token calculation when page > 1 (#7397)
|
2025-06-24 02:12:29 +08:00 |
|
Shangming Cai
|
d6dddc19ff
|
[CI] Upgrade mooncake to 0.3.4.post1 to fix 8 gpu tests (#7472)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-24 02:10:50 +08:00 |
|
Lianmin Zheng
|
55e03b10c4
|
Fix a bug in BatchTokenIDOut & Misc style and dependency updates (#7457)
|
2025-06-23 06:20:39 -07:00 |
|
kk
|
8aa68ed5c4
|
Solve docker build failed in the virtual machine (#7290)
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
|
2025-06-23 09:10:30 +00:00 |
|
xutizhou
|
506c4928f5
|
feat: integrate deepgemm into EPMoE (#6821)
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: TianQiLin666666 <1834987979@qq.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-06-23 01:38:58 -07:00 |
|
Lianmin Zheng
|
30ceccc74a
|
Update hyperparameter_tuning.md (#7454)
|
2025-06-22 22:42:55 -07:00 |
|
Cheng Wan
|
ac5010e0ba
|
Fix CUDA Graph Check under Deepep with DP FFN (#7451)
|
2025-06-22 20:35:58 -07:00 |
|
Yi Zhang
|
3cee035e99
|
add fused moe config for qwen3 in triton3.3.1 (#7445)
|
2025-06-22 18:29:25 -07:00 |
|
Simo Lin
|
30f2a44a96
|
[misc] Add PD service discovery support in router (#7361)
|
2025-06-22 17:54:14 -07:00 |
|
kk
|
bd4f581896
|
Fix torch compile run (#7391)
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
|
2025-06-22 15:33:09 -07:00 |
|
Ke Bao
|
50f1b6d6b1
|
Remove copy after bmm (#7441)
|
2025-06-22 15:07:44 -07:00 |
|
Trevor Morris
|
5962e70d8d
|
FlashInfer NVFP4 MoE with EP & 2-stream shared expert (#7327)
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
Co-authored-by: alcanderian <alcanderian@gmail.com>
|
2025-06-22 13:38:47 -07:00 |
|
fzyzcjy
|
edc21cc8ae
|
Tiny add logging for GC (#7406)
|
2025-06-22 12:40:02 +08:00 |
|
Liangsheng Yin
|
05c9bc8956
|
[minor] simplify the TokenToKVPoolAllocator (#7414)
|
2025-06-22 12:37:18 +08:00 |
|
Chang Su
|
b7a2df0a44
|
refactor(test): reorganize OpenAI test file structure (#7408)
|
2025-06-21 19:37:48 -07:00 |
|
Lifu Huang
|
1998ce4046
|
Refactor LoRAManager and LoRAMemoryPool state management logic for dynamic LoRA loading support (#7412)
|
2025-06-21 16:09:19 -07:00 |
|
Chang Su
|
72676cd6c0
|
feat(oai refactor): Replace openai_api with entrypoints/openai (#7351)
Co-authored-by: Jin Pan <jpan236@wisc.edu>
|
2025-06-21 13:21:06 -07:00 |
|
Atream
|
02bf31ef29
|
[fix] PD disaggregation when enable mtp and tp!=dp (#7420)
|
2025-06-21 12:03:11 -07:00 |
|
Liangsheng Yin
|
5ea5d22170
|
Fix CPU offloading for MLA memory pool (#7409)
|
2025-06-22 02:39:05 +08:00 |
|
Chang Su
|
fdfd5224bf
|
fix: Fix CI test_function_call_parser.py (#7425)
|
2025-06-22 00:25:08 +08:00 |
|
pansicheng
|
7f3ee861ae
|
fix overlap pagecount (#6984)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-06-21 15:34:45 +08:00 |
|
ehuaa
|
bec5891083
|
[BugFix]fix qwen25 invoke function call streaming responses with curly braces as the starting indicator (#7394)
|
2025-06-20 22:50:08 -07:00 |
|
woodx
|
9edf6608c9
|
[BugFix]: fix EmbeddingReqInput single input error (#7396)
|
2025-06-20 20:35:21 -07:00 |
|
Jinn
|
ab74f8f09d
|
Remove batches api in docs & example (#7400)
|
2025-06-20 19:46:31 -07:00 |
|
Keyang Ru
|
5e7fdc79fa
|
[OAI Server Refactor] [ChatCompletions & Completions] Support Return Hidden State (#7329)
Signed-off-by: keru <rukeyang@gmail.com>
|
2025-06-20 19:18:53 -07:00 |
|
Yineng Zhang
|
4d8d9b8efd
|
chore: upgrade mooncake-transfer-engine 0.3.4 (#7401)
|
2025-06-20 16:38:54 -07:00 |
|
Cheng Wan
|
5041df2d01
|
Fix 7285 Merge Conflicts (#7403)
|
2025-06-20 16:02:50 -07:00 |
|
Cheng Wan
|
256801e973
|
Update usage_processor.py (#7402)
|
2025-06-20 15:55:38 -07:00 |
|
Cheng Wan
|
73b13e69b4
|
Optimize DP attn scheduling for speculative decoding (#7285)
|
2025-06-20 15:06:41 -07:00 |
|
Cheng Wan
|
8609e637a9
|
Fix All-Gather under world size one (#7219)
|
2025-06-20 14:57:34 -07:00 |
|
yhyang201
|
dea2b84bc3
|
[OAI Server Refactor] [ChatCompletions & Completions] Implement UsageInfo Processor (#7360)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-06-20 14:51:21 -07:00 |
|
woodx
|
cfb2fb5afc
|
[OAI refactor] Add rerank and score serving (#7399)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-06-20 14:51:10 -07:00 |
|
Cheng Wan
|
22bfed7509
|
[DeepSeekNextN] fix: residual of head norm can be None (#7398)
|
2025-06-20 14:45:16 -07:00 |
|
Cheng Wan
|
e879d8b7a8
|
[Feature] Comprehensive Hybrid Parallelism Support (#6389)
|
2025-06-20 14:43:11 -07:00 |
|