Jerry Zhang a7c47e0f02 Add torchao quant (int4/int8/fp8) to llama models (#1341) ...

Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>

2024-09-09 05:32:41 -07:00

..

cuda_graph_runner.py

Fix bugs in sampler with CUDA graph / torch.compile (#1306)

2024-09-02 23:18:48 +00:00

forward_batch_info.py

[Feat] Add modalities for vision server when handling pixel values for llava (#1346)

2024-09-09 02:07:34 -07:00

model_runner.py

Add torchao quant (int4/int8/fp8) to llama models (#1341)

2024-09-09 05:32:41 -07:00