Lianmin Zheng ac2387279e Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988) ...

Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>

2025-03-03 00:12:04 -08:00

..

configs

Tuning Script for Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3922)

2025-02-27 10:59:46 +00:00

__init__.py

Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730)

2025-02-24 05:43:35 -08:00

base_config.py

fix black in pre-commit (#1940)

2024-11-08 07:42:47 +08:00

blockwise_int8.py

Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730)

2025-02-24 05:43:35 -08:00

fp8_kernel.py

Revert "[ROCm] Use tl.range() in block GEMM kernels with `num_stage… (#3632)

2025-02-17 18:01:21 +08:00

fp8_utils.py

add control for cutlass fp8 blockwise gemm (#3727)

2025-02-20 16:10:35 +08:00

fp8.py

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)

2025-03-03 00:12:04 -08:00

int8_kernel.py

Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730)

2025-02-24 05:43:35 -08:00

int8_utils.py

Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730)

2025-02-24 05:43:35 -08:00

modelopt_quant.py

Add support for nvidia modelopt fp8 kv cache (#3223)

2025-02-22 07:04:58 +08:00

w8a8_int8.py

feat: patch linear base (#2915)

2025-01-16 18:00:03 +08:00