Ying Sheng 75b31a2a88 Update run_batch interface and max_prefill_tokens (#574)

2024-06-30 18:26:04 -07:00

..

dp_worker.py

Higher priority for user input of max_prefill_tokens & format (#540)

2024-06-12 21:48:40 -07:00

infer_batch.py

Minor fix in compiler & format (#545)

2024-06-29 23:42:14 -07:00

manager_multi.py

Higher priority for user input of max_prefill_tokens & format (#540)

2024-06-12 21:48:40 -07:00

manager_single.py

Increase the number of thread limitation for tp worker managers. (#567)

2024-06-26 09:33:45 -07:00

model_runner.py

Expose dtype argument (#569)

2024-06-27 23:30:39 -07:00

radix_cache.py

Higher priority for user input of max_prefill_tokens & format (#540)

2024-06-12 21:48:40 -07:00

schedule_heuristic.py

Higher priority for user input of max_prefill_tokens & format (#540)

2024-06-12 21:48:40 -07:00

tp_worker.py

Update run_batch interface and max_prefill_tokens (#574)

2024-06-30 18:26:04 -07:00