Lianmin Zheng b7a065eae3 Use cuda event wait and synchronization instead of busy waiting (#2089)

2024-11-19 00:21:46 -08:00

..

data_parallel_controller.py

Launch dp ranks in parallel (#2053)

2024-11-16 17:39:39 -08:00

detokenizer_manager.py

Fix the default arguments of bench_offline_throughput.py & simplify detokenizer manager (#2042)

2024-11-15 05:02:44 -08:00

image_processor.py

Unify the model type checking (#1905)

2024-11-03 12:25:39 -08:00

io_struct.py

Fix a typo in io_struct.py (#2008)

2024-11-11 16:34:10 -08:00

schedule_batch.py

Use cuda event wait and synchronization instead of busy waiting (#2089)

2024-11-19 00:21:46 -08:00

schedule_policy.py

Fix chunked prefill with output logprob (#2083)

2024-11-18 13:16:28 -08:00

scheduler.py

Use cuda event wait and synchronization instead of busy waiting (#2089)

2024-11-19 00:21:46 -08:00

tokenizer_manager.py

Simplify prometheus metrics (#1981)

2024-11-10 04:39:32 -08:00

tp_worker_overlap_thread.py

Use cuda event wait and synchronization instead of busy waiting (#2089)

2024-11-19 00:21:46 -08:00

tp_worker.py

Use cuda event wait and synchronization instead of busy waiting (#2089)

2024-11-19 00:21:46 -08:00