Lianmin Zheng 58d1082e39 Clean up event loop (#1586)

2024-10-06 03:24:04 -07:00

..

configs

minor: fix config (#1524)

2024-09-27 01:49:16 -07:00

constrained

[FIX] Catch syntax error of Regex Guide to avoid crash (#1521)

2024-09-28 12:42:06 -07:00

layers

[Performance, Hardware] MoE tuning on AMD MI300x GPUs (#1554)

2024-10-02 09:52:46 -07:00

lora

[Fix] Fix AttributeError in Qwen2.5 LoRA: 'Qwen2ForCausalLM' object has no attribute 'get_hidden_dim' (#1536)

2024-10-02 20:41:15 -07:00

managers

Clean up event loop (#1586)

2024-10-06 03:24:04 -07:00

mem_cache

[Minor] Improve the style and fix flaky tests (#1584)

2024-10-06 00:10:48 -07:00

model_executor

Make input_ids a torch.Tensor (#1568)

2024-10-04 01:09:59 -07:00

models

Add llama implementation with no tensor parallel linears (#1561)

2024-10-05 11:22:27 -07:00

openai_api

[API, Feature] Support response prefill for openai API (#1490)

2024-09-22 06:46:17 -07:00

sampling

[Minor] Improve the style and fix flaky tests (#1584)

2024-10-06 00:10:48 -07:00

conversation.py

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

hf_transformers_utils.py

[Fix] Fix clean_up_tokenization_spaces in tokenizer (#1510)

2024-09-24 21:37:33 -07:00

mm_utils.py

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)

2024-08-23 14:11:16 -07:00

server_args.py

Use ipc instead of tcp in zmq (#1566)

2024-10-04 00:45:52 -07:00

server.py

Use ipc instead of tcp in zmq (#1566)

2024-10-04 00:45:52 -07:00

utils.py

[Minor] Improve the style and fix flaky tests (#1584)

2024-10-06 00:10:48 -07:00