fzyzcjy 0b9dfba787 Support dispatch low latency (#10263) ...

Co-authored-by: Kaixi Hou <4001424+kaixih@users.noreply.github.com>

2025-10-02 18:02:19 +08:00

..

compressed_tensors

[model] added support for w8a8int8 used by neuralmagic/Qwen2-0.5B-Ins… (#9642)

2025-09-30 11:26:17 +08:00

configs

Add H20 tuning configs support DeepSeek V3/R1 INT8(block-wise) (#4220)

2025-03-11 12:32:33 -07:00

deep_gemm_wrapper

[Auto Sync] Update load_config.py, model_config.py, configu... (20250923) (#10825)

2025-09-23 16:48:12 -07:00

quark

[Fix] RuntimeError: get_cfg Unsupported input_type:Float4_e2m1fn_x2 in using aiter-mxfp4-moe (#10981)

2025-09-26 22:12:22 -07:00

__init__.py

Revert "enable auto-round quantization model (#6226)" (#10148)

2025-09-07 22:31:11 -07:00

awq_triton.py

[AMD] Add triton awq_dequantize kernel to support AWQ on ROCm (#7661)

2025-07-18 14:27:25 -07:00

awq.py

[7/N] MoE Refactor: the implementation of new framework (#9269)

2025-09-05 21:09:09 -07:00

base_config.py

[7/N] MoE Refactor: the implementation of new framework (#9269)

2025-09-05 21:09:09 -07:00

blockwise_int8.py

[7/N] MoE Refactor: the implementation of new framework (#9269)

2025-09-05 21:09:09 -07:00

fp8_kernel.py

Revert "[1/2] Optimizations and refactors about quant kernel (#9534)" (#10292)

2025-09-10 18:24:23 -07:00

fp8_utils.py

[ROCm] Fix fp8 quantization accuracy issue. (#10558)

2025-09-17 17:44:59 -07:00

fp8.py

fix crash of DeepSeek-V3 update_weights_from_disk (#8863)

2025-09-16 09:45:15 +08:00

fpgemm_fp8.py

fix: support fb fp8 (#9462)

2025-08-21 14:31:43 -07:00

gptq.py

[7/N] MoE Refactor: the implementation of new framework (#9269)

2025-09-05 21:09:09 -07:00

int8_kernel.py

Revert "[1/2] Optimizations and refactors about quant kernel (#9534)" (#10292)

2025-09-10 18:24:23 -07:00

int8_utils.py

Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730)

2025-02-24 05:43:35 -08:00

kv_cache.py

[refactor] slightly tidy fp8 module (#5993)

2025-05-07 17:28:24 -07:00

marlin_utils_fp8.py

[5/n]decouple quantization implementation from vLLM dependency (#9454)

2025-08-21 12:52:07 -07:00

marlin_utils.py

[5/n]decouple quantization implementation from vLLM dependency (#9454)

2025-08-21 12:52:07 -07:00

modelopt_quant.py

Support dispatch low latency (#10263)

2025-10-02 18:02:19 +08:00

moe_wna16.py

[7/N] MoE Refactor: the implementation of new framework (#9269)

2025-09-05 21:09:09 -07:00

mxfp4_tensor.py

[sgl-kernel] misc: update deepgemm version for sgl-kernel (#9340)

2025-08-27 12:01:30 -07:00

mxfp4.py

[Fix] RuntimeError: get_cfg Unsupported input_type:Float4_e2m1fn_x2 in using aiter-mxfp4-moe (#10981)

2025-09-26 22:12:22 -07:00

petit_utils.py

Support NVFP4 quantized dense models on AMD CDNA2/CDNA3 GPUs (#7302)

2025-07-18 19:59:39 -07:00

petit.py

[fix] fix modelopt fp4 on b200 (#8195)

2025-07-20 17:39:57 -07:00

qoq.py

Refactor: move all quantization-related code to srt/layer/quantization (#7989)

2025-07-17 00:47:07 -07:00

rocm_mxfp4_utils.py

Optimized deepseek-v3/r1 model performance on mxfp4 run (#10008)

2025-09-04 15:11:22 -07:00

unquant.py

[Bugfix] Qwen3MoE aclrtMemcpy failed with NPUGraph (#10013)

2025-09-07 21:50:49 -07:00

utils.py

[Model] Support Meituan LongCat-Flash && LongCat-Flash-MTP (#9824)

2025-08-30 23:29:21 -07:00

w4afp8.py

Restruct sgl-kernel benchmark (#10861)

2025-09-25 07:45:25 +08:00

w8a8_fp8.py

[Fix] MoE: fix w8a8_fp8 MoE and add tests to cover this code path (#10429)

2025-09-14 17:34:28 -07:00

w8a8_int8.py

fix: fp8 quantization failure of qwen 2.5 VL 7B model (#10112)

2025-09-27 05:05:23 +00:00