Yi Zhang
|
3cee035e99
|
add fused moe config for qwen3 in triton3.3.1 (#7445)
|
2025-06-22 18:29:25 -07:00 |
|
Simo Lin
|
30f2a44a96
|
[misc] Add PD service discovery support in router (#7361)
|
2025-06-22 17:54:14 -07:00 |
|
kk
|
bd4f581896
|
Fix torch compile run (#7391)
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
|
2025-06-22 15:33:09 -07:00 |
|
Ke Bao
|
50f1b6d6b1
|
Remove copy after bmm (#7441)
|
2025-06-22 15:07:44 -07:00 |
|
Trevor Morris
|
5962e70d8d
|
FlashInfer NVFP4 MoE with EP & 2-stream shared expert (#7327)
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
Co-authored-by: alcanderian <alcanderian@gmail.com>
|
2025-06-22 13:38:47 -07:00 |
|
fzyzcjy
|
edc21cc8ae
|
Tiny add logging for GC (#7406)
|
2025-06-22 12:40:02 +08:00 |
|
Liangsheng Yin
|
05c9bc8956
|
[minor] simplify the TokenToKVPoolAllocator (#7414)
|
2025-06-22 12:37:18 +08:00 |
|
Chang Su
|
b7a2df0a44
|
refactor(test): reorganize OpenAI test file structure (#7408)
|
2025-06-21 19:37:48 -07:00 |
|
Lifu Huang
|
1998ce4046
|
Refactor LoRAManager and LoRAMemoryPool state management logic for dynamic LoRA loading support (#7412)
|
2025-06-21 16:09:19 -07:00 |
|
Chang Su
|
72676cd6c0
|
feat(oai refactor): Replace openai_api with entrypoints/openai (#7351)
Co-authored-by: Jin Pan <jpan236@wisc.edu>
|
2025-06-21 13:21:06 -07:00 |
|
Atream
|
02bf31ef29
|
[fix] PD disaggregation when enable mtp and tp!=dp (#7420)
|
2025-06-21 12:03:11 -07:00 |
|
Liangsheng Yin
|
5ea5d22170
|
Fix CPU offloading for MLA memory pool (#7409)
|
2025-06-22 02:39:05 +08:00 |
|
Chang Su
|
fdfd5224bf
|
fix: Fix CI test_function_call_parser.py (#7425)
|
2025-06-22 00:25:08 +08:00 |
|
pansicheng
|
7f3ee861ae
|
fix overlap pagecount (#6984)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-06-21 15:34:45 +08:00 |
|
ehuaa
|
bec5891083
|
[BugFix]fix qwen25 invoke function call streaming responses with curly braces as the starting indicator (#7394)
|
2025-06-20 22:50:08 -07:00 |
|
woodx
|
9edf6608c9
|
[BugFix]: fix EmbeddingReqInput single input error (#7396)
|
2025-06-20 20:35:21 -07:00 |
|
Jinn
|
ab74f8f09d
|
Remove batches api in docs & example (#7400)
|
2025-06-20 19:46:31 -07:00 |
|
Keyang Ru
|
5e7fdc79fa
|
[OAI Server Refactor] [ChatCompletions & Completions] Support Return Hidden State (#7329)
Signed-off-by: keru <rukeyang@gmail.com>
|
2025-06-20 19:18:53 -07:00 |
|
Yineng Zhang
|
4d8d9b8efd
|
chore: upgrade mooncake-transfer-engine 0.3.4 (#7401)
|
2025-06-20 16:38:54 -07:00 |
|
Cheng Wan
|
5041df2d01
|
Fix 7285 Merge Conflicts (#7403)
|
2025-06-20 16:02:50 -07:00 |
|
Cheng Wan
|
256801e973
|
Update usage_processor.py (#7402)
|
2025-06-20 15:55:38 -07:00 |
|
Cheng Wan
|
73b13e69b4
|
Optimize DP attn scheduling for speculative decoding (#7285)
|
2025-06-20 15:06:41 -07:00 |
|
Cheng Wan
|
8609e637a9
|
Fix All-Gather under world size one (#7219)
|
2025-06-20 14:57:34 -07:00 |
|
yhyang201
|
dea2b84bc3
|
[OAI Server Refactor] [ChatCompletions & Completions] Implement UsageInfo Processor (#7360)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-06-20 14:51:21 -07:00 |
|
woodx
|
cfb2fb5afc
|
[OAI refactor] Add rerank and score serving (#7399)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-06-20 14:51:10 -07:00 |
|
Cheng Wan
|
22bfed7509
|
[DeepSeekNextN] fix: residual of head norm can be None (#7398)
|
2025-06-20 14:45:16 -07:00 |
|
Cheng Wan
|
e879d8b7a8
|
[Feature] Comprehensive Hybrid Parallelism Support (#6389)
|
2025-06-20 14:43:11 -07:00 |
|
Xinyuan Tong
|
0998808009
|
Refine OpenAI serving entrypoint to remove batch requests (#7372)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Chang Su <csu272@usc.edu>
|
2025-06-20 14:33:43 -07:00 |
|
Wenbo Yang
|
794be55af2
|
Support THUDM/GLM-4-0414 (GLM-Z1) Glm4ForCausalLM architecture. (#5485)
|
2025-06-20 10:07:43 -07:00 |
|
Shangming Cai
|
187b85b7f3
|
[PD] Optimize custom mem pool usage and bump mooncake version (#7393)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-20 09:50:39 -07:00 |
|
strgrb
|
ceba0ce4f6
|
support return logprobs for pipeline (#7356)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-06-19 23:50:45 -07:00 |
|
Ata Fatahi
|
1ab6be1b26
|
Purge VerlEngine (#7326)
Signed-off-by: Ata Fatahi <immrata@gmail.com>
|
2025-06-19 23:47:21 -07:00 |
|
woodx
|
4df5fc2156
|
Feat/refactor embedding server (#7322)
|
2025-06-19 23:46:01 -07:00 |
|
Li Hui
|
a06912ad8b
|
Fix judgment condition for enabling Deepseek V3/R1 shared expert fusion optimization (#7371)
|
2025-06-19 21:58:00 -07:00 |
|
woodx
|
97011abc8a
|
[Doc] add embedding rerank doc (#7364)
|
2025-06-19 21:53:54 -07:00 |
|
Huang Long
|
1d6515ef2a
|
[Bugfix]Fix hang bug using dp attention with HiRadixCache (#7159)
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
|
2025-06-19 20:34:36 -07:00 |
|
Li Hui
|
dea8aa7ab8
|
Clean unused import for mimo mtp model (#7370)
|
2025-06-19 19:16:14 -07:00 |
|
ybyang
|
906dbc34f1
|
[Docker] optimize dockerfile remove deepep and blackwell merge it to… (#7343)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-06-19 17:42:40 -07:00 |
|
Yineng Zhang
|
fadf18fdd5
|
docs: update installation (#7366)
|
2025-06-19 12:00:19 -07:00 |
|
Shangming Cai
|
f88e70853e
|
[Bugfix][PD] Set conclude state before clear when failure happens (#7362)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-19 11:26:53 -07:00 |
|
Atream
|
4f838c09cd
|
[PD] Transfer hidden states for mtp when disaggregation (#7242)
|
2025-06-19 11:22:47 -07:00 |
|
Chang Su
|
d20a073bc3
|
Put _normalize_rid before other normalization in io_struct (#7363)
|
2025-06-19 11:21:25 -07:00 |
|
DarkSharpness
|
47367b768d
|
[Refactor] Clean up radix cache related API (#7303)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-06-20 00:58:48 +08:00 |
|
fzyzcjy
|
650127a173
|
Reintroduce tiny fix sampler error when prob is not contiguous (#7354)
|
2025-06-19 09:39:34 -07:00 |
|
Stefan He
|
3774f07825
|
Multi-Stage Awake: Support Resume and Pause KV Cache and Weights separately (#7099)
|
2025-06-19 00:56:37 -07:00 |
|
Chunyuan WU
|
9179ea1595
|
add seed in CPU UTs to avoid flaky failure (#7333)
|
2025-06-18 19:12:14 -07:00 |
|
Yineng Zhang
|
20a503c7d1
|
fix: resolve blackwell deepep image issue (#7331)
|
2025-06-18 17:04:02 -07:00 |
|
Jinn
|
ffd1a26e09
|
Add more refactored openai test & in CI (#7284)
|
2025-06-18 13:52:55 -07:00 |
|
Simo Lin
|
09ae5b20f3
|
Merge PDLB (Prefill-Decode Load Balancer) into SGLang Router (#7096)
|
2025-06-19 02:28:15 +08:00 |
|
ybyang
|
712bf9ec9b
|
[pd] optimize dockerfile for pd disaggregation (#7319)
Co-authored-by: zhyncs <me@zhyncs.com>
|
2025-06-18 11:26:41 -07:00 |
|