Zhiqiang Xie
|
8af7048dcf
|
Query remaining memory dynamically for PrefillAdder (#2941)
|
2025-01-17 20:20:26 -08:00 |
|
bjmsong
|
d3024f4fc8
|
support e4m3 kvcache in qwen2 & add kv scaling facotr json (#2894)
Co-authored-by: bjmsong <bjmsong@126.com>
|
2025-01-18 11:43:22 +08:00 |
|
Zhiqiang Xie
|
13387e6b7a
|
Multi-turn benchmark for hierarchical caching (#2942)
|
2025-01-17 16:17:24 -08:00 |
|
Wen Sun
|
120c3634ef
|
Fix Llama-3.1-405B References Docs (#2944)
|
2025-01-17 14:46:38 -08:00 |
|
Yineng Zhang
|
78e5b22f29
|
feat: use get_rope for gemma2 (#2954)
|
2025-01-18 02:57:18 +08:00 |
|
Yineng Zhang
|
7a15e9ad36
|
cleanup models unused import 2/n (#2952)
|
2025-01-18 01:09:19 +08:00 |
|
Ke Bao
|
dc2ac0cbdb
|
Update pr template (#2951)
|
2025-01-18 00:44:16 +08:00 |
|
Ke Bao
|
d47c5101f1
|
Add ut for qwen model (#2947)
|
2025-01-18 00:03:54 +08:00 |
|
Yineng Zhang
|
033c715b46
|
cleanup models dependencies 1/n (#2948)
|
2025-01-17 23:46:48 +08:00 |
|
Yineng Zhang
|
d06c1ab587
|
update ci install dependency (#2949)
|
2025-01-17 23:42:23 +08:00 |
|
Yineng Zhang
|
c5644cace9
|
docs: add Cursor for adoption and sponsorship (#2950)
|
2025-01-17 23:41:57 +08:00 |
|
Ke Bao
|
53e6552fed
|
Fix qwen accuracy issue (#2945)
|
2025-01-17 22:35:26 +08:00 |
|
Yineng Zhang
|
5dc54f1a62
|
feat: remove vllm distributed (#2907)
Co-authored-by: Zhangyi <1109276519@qq.com>
|
2025-01-17 22:31:51 +08:00 |
|
Ke Bao
|
f3e9b4894b
|
Fix sgl-kernel ci (#2938)
|
2025-01-17 17:26:21 +08:00 |
|
Lianmin Zheng
|
6a7973add8
|
Update release-docs.yml (#2937)
|
2025-01-17 00:36:40 -08:00 |
|
Chunyuan WU
|
63051738a9
|
Enable CPU device on SGLang (#2806)
|
2025-01-16 21:22:53 -08:00 |
|
Chang Su
|
a8ccacc8b8
|
[Frontend] Fix request length check and add option to disallow auto truncation in scheduler (#2876)
|
2025-01-16 14:51:19 -08:00 |
|
Lianmin Zheng
|
0427416b59
|
Fix zmq binding (#2930)
Co-authored-by: Chunyuan WU <chunyuan.wu@intel.com>
|
2025-01-16 14:36:07 -08:00 |
|
Chayenne
|
bf3edc2c60
|
Docs: Update pull_request_template.md (#2928)
|
2025-01-16 13:04:11 -08:00 |
|
Xiaoyu Zhang
|
78e974b2a5
|
[kernel] MiniMax-Text-01 decode lightning_attn with triton (#2920)
|
2025-01-16 12:51:38 -08:00 |
|
Lianmin Zheng
|
bc6915e3b9
|
Improve type annotation and styles (#2926)
|
2025-01-16 12:51:11 -08:00 |
|
saienduri
|
a883f0790d
|
Update release-docker-amd.yml to run on amd docker runner. (#2927)
|
2025-01-16 12:42:29 -08:00 |
|
Lianmin Zheng
|
8b6ce52e92
|
Support multi-node DP attention (#2925)
Co-authored-by: dhou-xai <dhou@x.ai>
|
2025-01-16 11:15:00 -08:00 |
|
Ke Bao
|
58f3f2b840
|
Add CI for sgl-kernel (#2924)
|
2025-01-17 01:26:51 +08:00 |
|
Lianmin Zheng
|
93d690617e
|
Simplify the process launch code in server.py (#2923)
|
2025-01-16 07:52:17 -08:00 |
|
Yun Dai
|
e00e5385e0
|
add profiling to bench_one_batch script (#2821)
|
2025-01-16 07:24:24 -08:00 |
|
Rin Intachuen
|
a2f602b541
|
fixed lm_head.weight error for quantized qwen (#2910)
|
2025-01-16 06:51:43 -08:00 |
|
Lianmin Zheng
|
8f2c522aba
|
Improve benchmark scripts and error message printing (#2922)
|
2025-01-16 06:24:31 -08:00 |
|
Yineng Zhang
|
7596417732
|
minor: use bear for compilation database (#2919)
|
2025-01-16 18:39:11 +08:00 |
|
Yineng Zhang
|
2dc957d421
|
fix setup for sgl kernel (#2917)
|
2025-01-16 18:17:34 +08:00 |
|
Yineng Zhang
|
bf8d07a6f9
|
feat: patch linear base (#2915)
|
2025-01-16 18:00:03 +08:00 |
|
Xiaoyu Zhang
|
ab31793661
|
[kernel] MiniMax-Text-01 prefill lightning_attn with triton (#2911)
|
2025-01-16 14:18:29 +08:00 |
|
Yineng Zhang
|
b7f3fec13c
|
minor: rename bench for sgl kernel (#2909)
|
2025-01-16 05:55:43 +08:00 |
|
Yineng Zhang
|
58f42b1dd8
|
minor: update pr test (#2908)
|
2025-01-16 05:51:49 +08:00 |
|
yizhang2077
|
767c9dec03
|
adapt custom allreduce for tensorrt llm (#2511)
|
2025-01-16 04:57:35 +08:00 |
|
Yineng Zhang
|
a53454c55e
|
fix: sgl-kernel link cuda (#2906)
|
2025-01-16 04:53:23 +08:00 |
|
yizhang2077
|
6cb3974e77
|
optimize custom allreduce kernel (#2904)
|
2025-01-16 03:04:25 +08:00 |
|
Lianmin Zheng
|
f65c13b559
|
Remove normalized_prompt_logprobs from the engine to make code easier to maintain (#2902)
|
2025-01-15 04:54:14 -08:00 |
|
Cody Yu
|
b803b395b7
|
Disable graceful shutdown of tokenizer manager when not in the main thread (#2872)
|
2025-01-15 03:29:33 -08:00 |
|
Ke Bao
|
bfbda62c8b
|
Add ut for w8a8 int8 quantization (#2897)
|
2025-01-15 18:29:14 +08:00 |
|
Yineng Zhang
|
b3e99dfb22
|
chore: bump v0.4.1.post6 (#2899)
|
2025-01-15 16:23:42 +08:00 |
|
Xiaoyu Zhang
|
f005758f2b
|
introduce CUB in sgl-kernel (#2887)
|
2025-01-14 19:48:59 +08:00 |
|
Yineng Zhang
|
f5c6c66794
|
feat: support internlm 3 dense (#2888)
|
2025-01-14 19:23:26 +08:00 |
|
Ke Bao
|
cc0485bef2
|
Support w8a8 int8 quantization config (#2881)
|
2025-01-14 17:07:49 +08:00 |
|
kk
|
b8cd09f27a
|
update ROCm docker for layernorm kernel optimization (#2885)
Co-authored-by: wunhuang <wunhuang@amd.com>
|
2025-01-14 16:59:43 +08:00 |
|
Ke Bao
|
c19d84829c
|
Adjust flashinfer workspace size for Qwen2 models (#2879)
|
2025-01-14 13:34:22 +08:00 |
|
Yineng Zhang
|
80002562a8
|
docs: update README (#2878)
|
2025-01-14 12:48:17 +08:00 |
|
Lianmin Zheng
|
46d4431889
|
Add a new api configure_logging to allow dumping the requests (#2875)
|
2025-01-13 14:24:00 -08:00 |
|
fzyzcjy
|
923f518337
|
CUDA-graph-compatible releasing and resuming KV cache and model weight memory (#2630)
|
2025-01-13 11:38:51 -08:00 |
|
Xiaoyu Zhang
|
d08c77c434
|
Sampling penalties memory interface (#2870)
|
2025-01-13 23:09:00 +08:00 |
|