Lianmin Zheng 2fa5cec775 Simplify sampler and its error handling (#1441)

2024-09-16 21:23:31 -07:00

..

configs

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

constrained

Add constrained_json_whitespace_pattern to ServerArgs (#1438)

2024-09-16 13:29:18 -07:00

layers

Simplify sampler and its error handling (#1441)

2024-09-16 21:23:31 -07:00

lora

[Feature] Support LoRA path renaming and add LoRA serving benchmarks (#1433)

2024-09-15 12:46:04 -07:00

managers

Simplify sampler and its error handling (#1441)

2024-09-16 21:23:31 -07:00

mem_cache

[Feature] Support fp8 e5m2 kv cache with flashinfer (#1204)

2024-08-25 17:38:11 -07:00

model_executor

Simplify sampler and its error handling (#1441)

2024-09-16 21:23:31 -07:00

models

Add torchao quant for mixtral and qwen_moe (#1418)

2024-09-14 06:46:55 +00:00

openai_api

[Fix] Fix logprob and normalized_logprob (#1428)

2024-09-15 06:36:06 -07:00

sampling

Simplify sampler and its error handling (#1441)

2024-09-16 21:23:31 -07:00

conversation.py

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

hf_transformers_utils.py

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

mm_utils.py

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)

2024-08-23 14:11:16 -07:00

server_args.py

Add constrained_json_whitespace_pattern to ServerArgs (#1438)

2024-09-16 13:29:18 -07:00

server.py

Clean up model loader (#1440)

2024-09-16 18:16:27 -07:00

utils.py

Clean up model loader (#1440)

2024-09-16 18:16:27 -07:00