Lianmin Zheng 63fbef9876 fix flashinfer & http log level

2024-07-03 23:19:33 -07:00

..

constrained

Increase the number of thread limitation for tp worker managers. (#567)

2024-06-26 09:33:45 -07:00

layers

2x performance improvement for large prefill & Fix workspace conflicts (#579)

2024-07-03 16:14:57 -07:00

managers

2x performance improvement for large prefill & Fix workspace conflicts (#579)

2024-07-03 16:14:57 -07:00

models

support gptj style rope in llama

2024-07-03 22:06:58 +00:00

conversation.py

Higher priority for user input of max_prefill_tokens & format (#540)

2024-06-12 21:48:40 -07:00

flush_cache.py

Improve doc strings (#518)

2024-06-08 02:39:32 -07:00

hf_transformers_utils.py

Increase the number of thread limitation for tp worker managers. (#567)

2024-06-26 09:33:45 -07:00

memory_pool.py

Minor: style improvement of radix_cache and memory_pool (#395)

2024-04-26 01:01:36 +08:00

mm_utils.py

Handle grayscale images in expand2square (#97)

2024-01-24 16:23:11 -08:00

model_config.py

Multi-node Tensor Parallelism (#550)

2024-06-17 20:41:24 -07:00

openai_api_adapter.py

Higher priority for user input of max_prefill_tokens & format (#540)

2024-06-12 21:48:40 -07:00

openai_protocol.py

Improve doc strings (#518)

2024-06-08 02:39:32 -07:00

sampling_params.py

SamplingParams add "spaces_between_special_tokens" argument (#392)

2024-04-30 16:17:12 -07:00

server_args.py

fix flashinfer & http log level

2024-07-03 23:19:33 -07:00

server.py

fix flashinfer & http log level

2024-07-03 23:19:33 -07:00

utils.py

fix flashinfer & http log level

2024-07-03 23:19:33 -07:00