Ke Wen
|
cf2489762b
|
Add Tensor Parallel to torch_native_llama (#1876)
|
2024-11-15 21:26:00 -08:00 |
|
HAI
|
e5c6715003
|
Fix core (MI300X) with --enable-overlap (#2048)
|
2024-11-15 21:24:42 -08:00 |
|
Xiaoyu Zhang
|
023d0a73df
|
fix small typos in docs (#2047)
|
2024-11-15 11:09:10 -08:00 |
|
Lianmin Zheng
|
32c9a7ec11
|
Release v0.3.5.post2 (#2046)
|
2024-11-15 06:54:00 -08:00 |
|
Lianmin Zheng
|
b01df48cf2
|
[Fix] Adjust default chunked prefill size and cuda graph max bs according to GPU memory capacity (#2044)
|
2024-11-15 06:21:57 -08:00 |
|
Lianmin Zheng
|
c29b98e043
|
Fix json benchmark (#2043)
|
2024-11-15 05:33:43 -08:00 |
|
DarkSharpness
|
954f4e6bd6
|
benchmark json schema (#2030)
|
2024-11-15 05:06:19 -08:00 |
|
Lianmin Zheng
|
2558d6a675
|
Fix the default arguments of bench_offline_throughput.py & simplify detokenizer manager (#2042)
|
2024-11-15 05:02:44 -08:00 |
|
ws
|
29ebe3dff4
|
fix: align enable_overlap_scheduler naming between code and docs (#2038)
|
2024-11-15 03:39:10 -08:00 |
|
zolinthecow
|
f6dd648620
|
Offline LLM Engine Benchmark Throughput (#1968)
Co-authored-by: ByronHsu <byronhsu1230@gmail.com>
|
2024-11-14 21:59:33 -08:00 |
|
Lianmin Zheng
|
ea53c63bad
|
Expose no_stop_trim and skip_special_tokens in openai api (#2039)
|
2024-11-14 19:09:21 -08:00 |
|
Lianmin Zheng
|
a10d530943
|
Fix outlines version (#2036)
|
2024-11-14 12:52:40 -08:00 |
|
Lianmin Zheng
|
aae5434bdf
|
Fix unit tests (#2034)
|
2024-11-14 11:08:37 -08:00 |
|
Lianmin Zheng
|
c3eac1b010
|
Fix torch.compile for MoE (#2033)
|
2024-11-14 01:30:24 -08:00 |
|
HAI
|
b275ce0043
|
Github runner instructions for AMD (#2031)
|
2024-11-13 23:57:18 -08:00 |
|
Patrick Yi
|
13ce3e4b5d
|
Add download_dir ServerArgs property (#2027)
|
2024-11-13 23:26:56 -08:00 |
|
Tzu Gwo
|
df246e699d
|
chore: open lto and optimization in release profile (#2028)
|
2024-11-14 15:02:39 +08:00 |
|
chottolabs
|
fb9fb3518b
|
set content to empty string (#2026)
|
2024-11-14 01:06:02 +00:00 |
|
Lianmin Zheng
|
c722d9bdc3
|
Fix dependency and error message for xgrammar (#2024)
|
2024-11-13 14:04:25 -08:00 |
|
Lianmin Zheng
|
218ab3611d
|
Do not let invalid grammar crash the server (#2023)
|
2024-11-13 11:39:16 -08:00 |
|
Lianmin Zheng
|
f407fcf9ef
|
Release v0.3.5.post1 (#2022)
|
2024-11-13 10:27:12 -08:00 |
|
Lianmin Zheng
|
54479d6f30
|
Fix grammar backend for tensor parallelism (#2020)
|
2024-11-13 01:49:45 -08:00 |
|
Lianmin Zheng
|
ba069a24d3
|
Fix grammar backend (#2018)
|
2024-11-12 21:17:38 -08:00 |
|
DarkSharpness
|
125b1199c5
|
support parallel grammar preprocessing (#1996)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2024-11-12 08:45:28 -08:00 |
|
Xiaoyu Zhang
|
eff468dd5a
|
fix test_embedding_models prompt length too long's bug (#2015)
|
2024-11-12 23:21:16 +08:00 |
|
Xiaoyu Zhang
|
a1bd719031
|
fix a bug in v1_embeeding_request (#2014)
|
2024-11-12 16:49:45 +08:00 |
|
Lianmin Zheng
|
78c1d6445f
|
Fix finish reason (#2013)
|
2024-11-11 23:24:41 -08:00 |
|
Xiaoyu Zhang
|
027e65248f
|
support echo=true and logprobs in openai api when logprobs=1 in lm-evaluation-harness (#1998)
|
2024-11-11 23:21:20 -08:00 |
|
Ke Bao
|
b808a38365
|
Filter empty prompt in random bench serving (#2011)
|
2024-11-12 14:53:41 +08:00 |
|
Byron Hsu
|
602ebc661d
|
remove sglang folder in rust (#2010)
|
2024-11-11 20:45:52 -08:00 |
|
Lianmin Zheng
|
530ae1bdc8
|
Fix weight loading for tied word embedding when TP > 1 (#2009)
|
2024-11-11 17:52:42 -08:00 |
|
Lianmin Zheng
|
befc6beb86
|
Fix a typo in io_struct.py (#2008)
|
2024-11-11 16:34:10 -08:00 |
|
Lianmin Zheng
|
59a5ba9be0
|
[Minor] Remove unused imports (#2006)
|
2024-11-11 15:36:14 -08:00 |
|
Byron Hsu
|
86c37d010a
|
fix sglang_router not found (#2005)
|
2024-11-11 15:20:14 -08:00 |
|
RangiLyu
|
f18b9c7252
|
support internlm2-reward (#1994)
|
2024-11-11 15:09:58 -08:00 |
|
Byron Hsu
|
3e33574374
|
run rust test on ubuntu instead of 1-gpu-runner (#2003)
|
2024-11-11 14:46:08 -08:00 |
|
Byron Hsu
|
0d94f1dd03
|
Bump router to 0.0.3 (#2004)
|
2024-11-11 14:42:22 -08:00 |
|
Byron Hsu
|
e728258d34
|
release router from py38 to py312 (#2002)
|
2024-11-11 14:30:25 -08:00 |
|
Byron Hsu
|
239eafbd2e
|
Fix rust unit test and pypi token (#2001)
|
2024-11-11 14:18:21 -08:00 |
|
James Xu
|
9d427265fd
|
Add Engine::encode example (#2000)
|
2024-11-11 13:43:35 -08:00 |
|
Byron Hsu
|
00ffde206f
|
setup router python binding ci (#1999)
|
2024-11-11 12:19:32 -08:00 |
|
James Xu
|
ddeb9d42de
|
Add engine encode (#1995)
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
|
2024-11-11 11:48:17 -08:00 |
|
Yineng Zhang
|
aaf0a3156e
|
docs: add slides link in README (#1997)
|
2024-11-11 05:03:16 -08:00 |
|
Byron Hsu
|
f9633fa9b9
|
[rust] cache-aware DP - approx tree (#1934)
|
2024-11-10 21:57:32 -08:00 |
|
HAI
|
087ab83223
|
[Performance, Triton] Optimize over mask compute to tl.load in fused_moe_kernel (#1980)
|
2024-11-10 18:54:43 -08:00 |
|
Byron Hsu
|
8169c6f4ef
|
Add gen-shared-prefix dataset in bench_serving (#1990)
|
2024-11-11 08:39:56 +08:00 |
|
Lianmin Zheng
|
3d043319aa
|
[CI] Balance unit tests (#1988)
|
2024-11-10 11:45:01 -08:00 |
|
yizhang2077
|
a8aad9357d
|
qwen2vl fix bug for #1971 #1897 (#1984)
|
2024-11-10 08:10:45 -08:00 |
|
Yineng Zhang
|
47ffe7af81
|
docs: add shm size for docker run (#1986)
|
2024-11-10 22:14:48 +08:00 |
|
Yineng Zhang
|
b3523af8eb
|
fix: update pyzmq version (#1983)
|
2024-11-10 21:33:23 +08:00 |
|