Lianmin Zheng 86a2c473b7 [Fix] Fix seq_lens_sum for cuda graph runner in padded cases (#1789)

2024-10-24 21:26:05 -07:00

..

configs

check user-specified model_max_len with hf derived max_model_len (#1778)

2024-10-24 12:40:36 -07:00

constrained

[Bug] Catch any errors caused by parsing json schema (#1776)

2024-10-24 01:54:53 -07:00

layers

Re-introduce get_cuda_graph_seq_len_fill_value (#1783)

2024-10-24 13:30:11 -07:00

lora

Update vllm to 0.6.3 (#1711) (#1720)

2024-10-19 20:45:41 -07:00

managers

Fix the perf regression due to additional_stop_token_ids (#1773)

2024-10-23 16:45:21 -07:00

mem_cache

[Fix] Fix cuda graph padding for triton attention backend (#1782)

2024-10-24 12:33:15 -07:00

model_executor

[Fix] Fix seq_lens_sum for cuda graph runner in padded cases (#1789)

2024-10-24 21:26:05 -07:00

models

Llama3.2 vision model support (#1551)

2024-10-21 15:01:21 -07:00

openai_api

Add matched_stop token or str to distinguish between eos or stop str finish_reason generation (#1684)

2024-10-17 18:06:52 +00:00

sampling

Fix the perf regression due to additional_stop_token_ids (#1773)

2024-10-23 16:45:21 -07:00

conversation.py

Llama3.2 vision model support (#1551)

2024-10-21 15:01:21 -07:00

hf_transformers_utils.py

Fix the perf regression due to additional_stop_token_ids (#1773)

2024-10-23 16:45:21 -07:00

mm_utils.py

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)

2024-08-23 14:11:16 -07:00

server_args.py

Simplify batch result resolution (#1735)

2024-10-20 19:47:14 -07:00

server.py

[API] add get memory pool size (#1760)

2024-10-23 07:02:29 +00:00

utils.py

Llama3.2 vision model support (#1551)

2024-10-21 15:01:21 -07:00