Ying Sheng c71880f896 Vectorize logprobs computation (#787)

2024-07-28 05:22:14 -07:00

..

constrained

Fix TransformerTokenizer init for chatglm2 & 3 (#761)

2024-07-27 02:44:46 -07:00

layers

Vectorize logprobs computation (#787)

2024-07-28 05:22:14 -07:00

managers

Vectorize logprobs computation (#787)

2024-07-28 05:22:14 -07:00

model_loader

refactor model loader: initial refactor (#664)

2024-07-20 02:18:22 -07:00

models

Rename prefill_token_logprobs -> input_token_logprobs; decode_token_logprobs -> output_token_logprobs (#776)

2024-07-27 19:50:34 -07:00

openai_api

Rename prefill_token_logprobs -> input_token_logprobs; decode_token_logprobs -> output_token_logprobs (#776)

2024-07-27 19:50:34 -07:00

conversation.py

Update OpenAI API (#667)

2024-07-19 23:20:54 -07:00

flush_cache.py

Improve doc strings (#518)

2024-06-08 02:39:32 -07:00

hf_transformers_utils.py

Fix context length (#757)

2024-07-26 18:13:13 -07:00

memory_pool.py

Fix prefill size (#711)

2024-07-24 03:41:15 -07:00

mm_utils.py

Handle grayscale images in expand2square (#97)

2024-01-24 16:23:11 -08:00

model_config.py

Deepseek v2 support (#693)

2024-07-26 17:10:07 -07:00

sampling_params.py

Fix max_tokens for OpenAI chat completion API (#766)

2024-07-27 15:44:27 -07:00

server_args.py

Allow disabling flashinfer sampling kernel (#778)

2024-07-27 20:18:56 -07:00

server.py

Allow disabling flashinfer sampling kernel (#778)

2024-07-27 20:18:56 -07:00

utils.py

fix: small bug for llama-405b fp16 (#733)

2024-07-25 21:14:54 -07:00