fzyzcjy
|
1b19df4b2a
|
Refactor communication logic of DeepSeek for extensibility and understandability (#6321)
|
2025-05-19 20:14:48 -07:00 |
|
fzyzcjy
|
f0653886a5
|
Expert distribution recording without overhead for EPLB (#4957)
|
2025-05-19 20:07:43 -07:00 |
|
Yineng Zhang
|
b146555749
|
Revert "Implement return_hidden_states for the OpenAI API (#6137)" (#6440)
|
2025-05-19 18:21:29 -07:00 |
|
Yi Zhang
|
b06215daed
|
[BUG] fix stop_profile crash (#6431)
|
2025-05-19 17:30:33 -07:00 |
|
Trevor Morris
|
7adf245ba2
|
[Metrics] Add KV events publishing (#6098)
|
2025-05-19 14:19:54 -07:00 |
|
lukec
|
844e2f227a
|
Fix nodeepgemm init (#6417)
|
2025-05-19 00:44:03 -07:00 |
|
kyle-pena-kuzco
|
4f39bcf7ab
|
Implement return_hidden_states for the OpenAI API (#6137)
|
2025-05-18 22:30:25 -07:00 |
|
fzyzcjy
|
31c9569bb8
|
Fix request id error (#6401)
|
2025-05-18 18:58:59 -07:00 |
|
Chang Su
|
1be6956d1b
|
[Bugfix] Fix field error in v1_embedding_request (#6400)
|
2025-05-18 15:58:29 -07:00 |
|
Mick
|
626ccb7d3f
|
vlm: tensor hash kernel (#5974)
|
2025-05-18 15:38:16 -07:00 |
|
fzyzcjy
|
72bfb0baf0
|
Refactor DeepSeek MoE layer to unify the two forward branches (#6325)
|
2025-05-18 15:34:36 -07:00 |
|
wangxiyu191
|
155214952b
|
refactor: Extract repeated member variables in KVCache subclasses to base class. (#6323)
|
2025-05-18 15:28:15 -07:00 |
|
Chang Su
|
ebe58d545d
|
[Misc] Implement RankZeroFilter for rank-specific logging in model_runner.py (#6333)
|
2025-05-18 15:27:13 -07:00 |
|
Chang Su
|
066cf44546
|
[OAI] Add rid tracing for v1/embeddings and fix rid type in Chat (#6397)
|
2025-05-18 13:05:38 -07:00 |
|
JieXin Liang
|
1f30c05d4a
|
[fix] fix fa3 forward_decode with spec_decode (#6395)
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
|
2025-05-18 12:50:15 -07:00 |
|
doujiang24
|
9d24c3ffb0
|
chore: tiny remove duplicated code (#6392)
Signed-off-by: doujiang24 <doujiang24@gmail.com>
|
2025-05-18 02:17:32 -07:00 |
|
Yury Sulsky
|
24161c5913
|
The Gemma template is missing a newline after the user role. (#6331)
Co-authored-by: Yury Sulsky <ysulsky@tesla.com>
|
2025-05-18 01:57:27 -07:00 |
|
libra
|
11553c1a37
|
Add pipeline parallelism for Qwen2 and Qwen3 Model (#6250)
|
2025-05-18 00:42:55 -07:00 |
|
Mick
|
01dd39bac1
|
refactor: minor refactors regarding multimodal processing (#6187)
|
2025-05-17 22:53:20 -07:00 |
|
Lianmin Zheng
|
b3f3d610fd
|
Do not use FA3 for mistral (#6379)
|
2025-05-17 19:47:34 -07:00 |
|
Yineng Zhang
|
f07c6a009b
|
chore: upgrade sgl-kernel v0.1.3 (#6377)
|
2025-05-17 19:47:05 -07:00 |
|
Lianmin Zheng
|
4bb816d444
|
Fix CI tests (#6362)
|
2025-05-17 19:16:45 -07:00 |
|
ybyang
|
c250939ecb
|
[Fix Chat API] add request id for chat/completion for tracing (#6364)
|
2025-05-17 18:58:22 -07:00 |
|
ishandhanani
|
b6909aa223
|
fix: allow launch_dummy_health_check_server to start inside of running asyncio loop (#6330)
|
2025-05-17 18:32:41 -07:00 |
|
fzyzcjy
|
f87283573e
|
Add expert distribution APIs for engine (#6290)
|
2025-05-17 18:31:51 -07:00 |
|
fzyzcjy
|
73187152a4
|
Reland tiny refactor DefaultModelLoader.Source (#6041)
|
2025-05-17 17:11:20 -07:00 |
|
fzyzcjy
|
4086566516
|
Fix expert distribution recorder and profiler command stuck forever (#6284)
|
2025-05-17 17:10:44 -07:00 |
|
fzyzcjy
|
fd08c04821
|
Support custom DeepEP tuning config (#6257)
|
2025-05-17 17:09:42 -07:00 |
|
fzyzcjy
|
26ebb849eb
|
Tiny refactor bench_serving to extract RequestFuncOutput.init_new (#6108)
|
2025-05-17 17:08:52 -07:00 |
|
fzyzcjy
|
02973cd9a4
|
Tiny refactor bench_serving to improve extensibility (#6134)
|
2025-05-17 17:07:58 -07:00 |
|
fzyzcjy
|
6d95a35abf
|
Support outputing details for bench_serving (#6107)
|
2025-05-17 17:06:52 -07:00 |
|
fzyzcjy
|
01d2838c0f
|
Fix stop_profile does not wait for finishing (#4741)
|
2025-05-17 17:06:15 -07:00 |
|
xutizhou
|
e3b8a72291
|
[fix] illegal memory in _fwd_kernel_ep_scatter_2 and _fwd_kernel_ep_gather (#6348)
|
2025-05-17 17:01:42 -07:00 |
|
Lifu Huang
|
3cf1473a09
|
Use monotonic clock for interval measurement (#6211)
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
|
2025-05-17 16:49:18 -07:00 |
|
fzyzcjy
|
2716830802
|
Speed up when having padding tokens in DeepEP (#6175)
|
2025-05-17 16:44:05 -07:00 |
|
Chang Su
|
205d5cb407
|
perf: Optimize local attention memory allocation in FlashAttentionBackend (#6356)
|
2025-05-17 01:45:46 -07:00 |
|
fzyzcjy
|
2df9d40aa6
|
Minor code cleanup refactor for DeepSeek models (#6324)
|
2025-05-16 19:06:03 -07:00 |
|
fzyzcjy
|
8dc191f237
|
Fix one wasted kernel in DeepSeek and minor refactor (#6316)
|
2025-05-16 19:05:33 -07:00 |
|
Kiv Chen
|
64825b8395
|
model(vlm): mistral 3.1 (#5099)
Co-authored-by: KivenChen <sleigh-queue-0y@icloud.com>
|
2025-05-16 18:36:18 -07:00 |
|
Lianmin Zheng
|
dcc0a45618
|
Fix amd ci (#6360)
|
2025-05-16 15:33:10 -07:00 |
|
Lianmin Zheng
|
c2b7ddca49
|
[Minor] cleanup unused imports (#6358)
|
2025-05-16 14:52:38 -07:00 |
|
Fr4nk1in
|
4bd2952a37
|
feat: add dp attention support for Qwen 2/3 MoE models, fixes #6088 (#6121)
Co-authored-by: King.Zevin <zevin@mail.ustc.edu.cn>
Co-authored-by: Yi Zhang <1109276519@qq.com>
|
2025-05-16 14:44:10 -07:00 |
|
Elfie Guo
|
6fc9357503
|
[2/2] Add python wrapper for CUTLASS FP8 Blockscale MoE Kernel. (#5694)
|
2025-05-16 13:14:07 -07:00 |
|
Baizhou Zhang
|
839fb31e5f
|
[Fix] Improve dependencies for Blackwell image (#6334)
|
2025-05-16 12:38:22 -07:00 |
|
Yury Sulsky
|
f19a9204cd
|
Support precomputed multimodal features for Qwen-VL and Gemma3 models. (#6136)
Co-authored-by: Yury Sulsky <ysulsky@tesla.com>
|
2025-05-16 12:26:15 -07:00 |
|
Lianmin Zheng
|
e07a6977e7
|
Minor improvements of TokenizerManager / health check (#6327)
|
2025-05-15 15:29:25 -07:00 |
|
Qiaolin Yu
|
cd8d4b9dfc
|
Fix lora bench (#6302)
|
2025-05-15 10:09:55 -07:00 |
|
fzyzcjy
|
f194e14fb7
|
Reduce MoE memory usage (#6147)
|
2025-05-15 09:38:28 -07:00 |
|
Yi Liu
|
cfc9f9ab8d
|
Fix gpu mem check on CPU (#6317)
Signed-off-by: yiliu30 <yi4.liu@intel.com>
|
2025-05-15 09:37:45 -07:00 |
|
JieXin Liang
|
9a405274e2
|
[misc] remove redundant platform codes (#6298)
|
2025-05-15 00:51:30 -07:00 |
|