Liangsheng Yin 62757db6f0 Reduce the overhead when cache is disabled (#1010)

2024-08-09 16:36:57 -07:00

..

controller_multi.py

Code structure refactor (#807)

2024-07-29 23:04:48 -07:00

controller_single.py

Code structure refactor (#807)

2024-07-29 23:04:48 -07:00

detokenizer_manager.py

Add skip_tokenizer_init args. (#959)

2024-08-09 12:14:13 -07:00

io_struct.py

Add openai embedding API (#997)

2024-08-09 11:19:18 -07:00

policy_scheduler.py

Reduce the overhead when cache is disabled (#1010)

2024-08-09 16:36:57 -07:00

schedule_batch.py

Reduce the overhead when cache is disabled (#1010)

2024-08-09 16:36:57 -07:00

tokenizer_manager.py

Add skip_tokenizer_init args. (#959)

2024-08-09 12:14:13 -07:00

tp_worker.py

Reduce the overhead when cache is disabled (#1010)

2024-08-09 16:36:57 -07:00