Byron Hsu 551a3a9d38 Provide an offline engine API (#1567)

2024-10-06 20:27:03 -07:00

..

configs

minor: fix config (#1524)

2024-09-27 01:49:16 -07:00

constrained

[FIX] Catch syntax error of Regex Guide to avoid crash (#1521)

2024-09-28 12:42:06 -07:00

layers

[Minor, Performance] Use torch.argmax for greedy sampling (#1589)

2024-10-06 13:15:05 -07:00

lora

[LoRA, Performance] Speedup multi-LoRA serving - Step 1 (#1587)

2024-10-06 10:33:44 -07:00

managers

Fix modality for image inputs (#1592)

2024-10-06 15:43:32 -07:00

mem_cache

[Minor] Improve the style and fix flaky tests (#1584)

2024-10-06 00:10:48 -07:00

model_executor

Make input_ids a torch.Tensor (#1568)

2024-10-04 01:09:59 -07:00

models

Add llama implementation with no tensor parallel linears (#1561)

2024-10-05 11:22:27 -07:00

openai_api

[API, Feature] Support response prefill for openai API (#1490)

2024-09-22 06:46:17 -07:00

sampling

[Minor] Improve the style and fix flaky tests (#1584)

2024-10-06 00:10:48 -07:00

conversation.py

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

hf_transformers_utils.py

[Fix] Fix clean_up_tokenization_spaces in tokenizer (#1510)

2024-09-24 21:37:33 -07:00

mm_utils.py

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)

2024-08-23 14:11:16 -07:00

server_args.py

Use ipc instead of tcp in zmq (#1566)

2024-10-04 00:45:52 -07:00

server.py

Provide an offline engine API (#1567)

2024-10-06 20:27:03 -07:00

utils.py

[Minor] Improve the style and fix flaky tests (#1584)

2024-10-06 00:10:48 -07:00