Lianmin Zheng 8f8f96a621 Fix the perf regression due to additional_stop_token_ids (#1773)

2024-10-23 16:45:21 -07:00

..

data_parallel_controller.py

[Fix] Fix abort in dp (#1767)

2024-10-23 10:46:29 -07:00

detokenizer_manager.py

[API] add get memory pool size (#1760)

2024-10-23 07:02:29 +00:00

image_processor.py

Llama3.2 vision model support (#1551)

2024-10-21 15:01:21 -07:00

io_struct.py

[API] add get memory pool size (#1760)

2024-10-23 07:02:29 +00:00

schedule_batch.py

Fix the perf regression due to additional_stop_token_ids (#1773)

2024-10-23 16:45:21 -07:00

schedule_policy.py

Returning a per request metric for number of cached_tokens read (#1599)

2024-10-16 11:49:22 -07:00

scheduler.py

Fix out of memory message. (#1771)

2024-10-23 15:20:39 -07:00

tokenizer_manager.py

[API] add get memory pool size (#1760)

2024-10-23 07:02:29 +00:00

tp_worker_overlap_thread.py

Fuse more ops & Simplify token mapping (#1758)

2024-10-22 23:20:43 -07:00

tp_worker.py

Update max_req_len and max_req_input_len (#1748)

2024-10-21 16:12:04 -07:00