Chayenne
|
5a5f18432f
|
Fix docs ci (#1888)
|
2024-11-02 11:57:22 -07:00 |
|
Lianmin Zheng
|
7b394e5f2b
|
Fix docs (#1889)
|
2024-11-02 11:46:00 -07:00 |
|
Chayenne
|
3b60558dd7
|
Native api (#1886)
Co-authored-by: Chayenne <zhaochenyang@g.ucla.edu>
|
2024-11-02 01:02:17 -07:00 |
|
Lianmin Zheng
|
5a9a4f41c6
|
Update index.rst (#1885)
|
2024-11-02 00:20:33 -07:00 |
|
Chayenne
|
72e979bfb5
|
add native api docs (#1883)
Co-authored-by: Chayenne <zhaochenyang@g.ucla.edu>
|
2024-11-02 00:17:30 -07:00 |
|
Ran Chen
|
146f613405
|
Fix incorrect context length for llama3.2-11b (#1873)
|
2024-11-02 00:04:50 -07:00 |
|
Lianmin Zheng
|
660ecb731f
|
Fix doc links (#1882)
|
2024-11-01 20:42:30 -07:00 |
|
Lianmin Zheng
|
2565cb0f40
|
Update docs and workflow (#1881)
|
2024-11-01 20:29:41 -07:00 |
|
Lianmin Zheng
|
066e8a4ef0
|
Update docs title (#1879)
|
2024-11-01 20:00:41 -07:00 |
|
Lianmin Zheng
|
2134f0898c
|
Fix links in the docs (#1878)
|
2024-11-01 18:25:55 -07:00 |
|
Lianmin Zheng
|
a54f278d44
|
Add a FAQ documentation (#1877)
|
2024-11-01 18:16:29 -07:00 |
|
Lianmin Zheng
|
d1b31b0684
|
Improve docs and fix the broken links (#1875)
|
2024-11-01 17:47:44 -07:00 |
|
jacky.cheng
|
d59a47828c
|
[3rdparty, document] Updated Documentation that covers performance tuning techniques for AMD Instinct GPUs. (#1871)
Co-authored-by: root <root@dell300x-pla-t10-23.pla.dcgpu>
|
2024-11-01 12:12:59 -07:00 |
|
Yineng Zhang
|
104bf2609b
|
minor: update nightly eval (#1867)
|
2024-11-01 21:38:29 +08:00 |
|
Chayenne
|
3bf3d011ed
|
Add vlm document (#1866)
Co-authored-by: Chayenne <zhaochenyang@g.ucla.edu>
|
2024-11-01 00:51:15 -07:00 |
|
Yineng Zhang
|
d86a2d6562
|
minor: add human eval (#1754)
|
2024-11-01 14:29:20 +08:00 |
|
Ke Bao
|
16eb33ffe2
|
Update vocab embedding deps and add TP switch (#1856)
|
2024-10-31 20:13:07 -07:00 |
|
Chayenne
|
61cf00e112
|
change file tree (#1859)
Co-authored-by: Chayenne <zhaochenyang@g.ucla.edu>
|
2024-10-31 20:10:16 -07:00 |
|
Liangsheng Yin
|
b9fd178f1b
|
Fix retraction + overlap (#1860)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2024-10-31 18:27:42 -07:00 |
|
HAI
|
d8e9d61f86
|
[Build, ROCm] Dockerfile.rocm for Instinct GPUs, with package updates (#1861)
|
2024-10-31 16:38:16 -07:00 |
|
Lianmin Zheng
|
a2e0424abf
|
Fix memory leak for chunked prefill 2 (#1858)
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
|
2024-10-31 14:51:51 -07:00 |
|
geeker-smallwhite
|
8ce202a493
|
delete unused character (#1855)
|
2024-10-31 19:33:55 +08:00 |
|
Lianmin Zheng
|
d913d52c9a
|
Fix warnings in doc build (#1852)
|
2024-10-30 22:28:00 -07:00 |
|
Lianmin Zheng
|
0ab7bcaf66
|
Simplify documentation in README.md (#1851)
|
2024-10-30 21:57:49 -07:00 |
|
Byron Hsu
|
438526a814
|
Refactor tokenizer manager (#1846)
|
2024-10-30 21:32:18 -07:00 |
|
Lianmin Zheng
|
f7102fbd2b
|
Fix mixed chunked prefill (#1850)
|
2024-10-30 21:20:41 -07:00 |
|
Byron Hsu
|
a7a0a6886b
|
Make decode log interval configurable (#1847)
|
2024-10-30 19:59:20 -07:00 |
|
HAI
|
2d4ce1b792
|
[Performance, Triton Kernel Args] _decode_grouped_softmax_reducev_fwd… (#1845)
|
2024-10-30 17:33:36 -07:00 |
|
Chayenne
|
4ba815b84e
|
Fix suggest edit (#1842)
Co-authored-by: Chayenne <zhaochenyang@g.ucla.edu>
|
2024-10-30 12:28:12 -07:00 |
|
HAI
|
5f65e2b830
|
[Performance, Hardware] MoE weights padding to AMD MI300x GPUs (#1836)
|
2024-10-30 12:17:32 -07:00 |
|
Ying Sheng
|
4e2af03cfa
|
[Production] Drain requests before exit when receive SIGTERM (#1838)
|
2024-10-30 10:22:56 -07:00 |
|
Lianmin Zheng
|
3184aa95a7
|
Update README.md (#1840)
|
2024-10-30 03:16:43 -07:00 |
|
Lianmin Zheng
|
b548801ddb
|
Update docs (#1839)
|
2024-10-30 02:49:08 -07:00 |
|
Chayenne
|
539df95d2c
|
Imporve openai api documents (#1827)
Co-authored-by: Chayenne <zhaochenyang@g.ucla.edu>
|
2024-10-30 00:39:41 -07:00 |
|
DanielC12321
|
5e00ddebc0
|
Add new model: Gpt2 (#1833)
|
2024-10-29 17:52:33 -07:00 |
|
HAI
|
54dd3ea122
|
[FP8 KV Cache, Mixtral] Avoid KeyError at loading pre-quantized FP8 m… (#1835)
|
2024-10-29 13:58:03 -07:00 |
|
yizhang2077
|
d04899d7ca
|
stop_str of qwen2-vl template should be a tuple not a str (#1834)
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
|
2024-10-29 20:30:41 +00:00 |
|
HAI
|
5010e0d2ca
|
[3rdparty, document] Add 3rdparty/amd, with profiling and tuning instructions to be added (#1822)
|
2024-10-29 10:51:02 -07:00 |
|
Yanyi Liu
|
5e6c32657e
|
Support setting use_thread in the run_program for easier debugging. (#1823)
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
|
2024-10-29 06:51:47 +00:00 |
|
Byron Hsu
|
680cad2023
|
fix get_memory_pool_size deadlock for DP (#1830)
|
2024-10-28 23:07:14 -07:00 |
|
Byron Hsu
|
0a24eb850a
|
Fix update_weights deadlock for DP (#1825)
|
2024-10-28 12:02:23 -07:00 |
|
Byron Hsu
|
3839be2913
|
[Router] Add a rust-based router (#1790)
|
2024-10-28 09:49:48 -07:00 |
|
Chayenne
|
6e13b650a9
|
Fix docs deploy ci (#1821)
|
2024-10-27 21:03:41 -07:00 |
|
Byron Hsu
|
6fcd6d7d6d
|
Support token ids in engine.generate (#1820)
|
2024-10-27 14:02:34 -07:00 |
|
Ke Bao
|
c77762d57f
|
Fix Triton decode kernel & ut (#1819)
|
2024-10-27 10:54:38 -07:00 |
|
Chayenne
|
51c81e339b
|
Add openAI compatible API (#1810)
Co-authored-by: Chayenne <zhaochenyang@g.ucla.edu>
|
2024-10-27 10:51:42 -07:00 |
|
Lianmin Zheng
|
eaade87a42
|
Fix unit tests (#1817)
|
2024-10-27 03:04:54 -07:00 |
|
Lianmin Zheng
|
86fc0d79d0
|
Add a watch dog thread (#1816)
|
2024-10-27 02:00:50 -07:00 |
|
Lianmin Zheng
|
1be853ee69
|
Update hyperparameter_tuning.md (#1813)
|
2024-10-26 21:08:44 -07:00 |
|
Lianmin Zheng
|
86e0dde555
|
Improve the user control of new_token_ratio (#1811)
|
2024-10-26 16:39:41 -07:00 |
|