fzyzcjy 349bb2c92a Fix error when disabling new DeepGEMM (#7198)

2025-06-14 19:24:54 -07:00

..

baichuan.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

bert.py

Feat/support encoder model (like bert) (#4887)

2025-04-17 01:50:48 -07:00

chatglm.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

clip.py

Implement Siglip Vision model, and support BNB quantization for gemma3-mm (#5339)

2025-05-20 23:53:46 -07:00

commandr.py

Revert "fix some typos" (#6244)

2025-05-12 12:53:26 -07:00

dbrx.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

deepseek_janus_pro.py

Revert "fix some typos" (#6244)

2025-05-12 12:53:26 -07:00

deepseek_nextn.py

[Refactor] Rename n_share_experts_fusion as num_fused_shared_experts (#6735)

2025-06-03 17:48:24 -07:00

deepseek_v2.py

Fix error when disabling new DeepGEMM (#7198)

2025-06-14 19:24:54 -07:00

deepseek_vl2.py

[Fix]: support deepseek-vl2-tiny model (#5552)

2025-04-26 17:52:53 +08:00

deepseek.py

[Fix]: support deepseek-vl2-tiny model (#5552)

2025-04-26 17:52:53 +08:00

exaone.py

fix: EXAONE when using tie_word_embeddings (#5759)

2025-05-21 11:30:04 -07:00

gemma2_reward.py

[QUANT] Add GPTQModel Dynamic Quantization + lm_head Quantization (#3790)

2025-03-05 01:11:00 -08:00

gemma2.py

Revert "fix some typos" (#6244)

2025-05-12 12:53:26 -07:00

gemma3_causal.py

Support sliding window in triton backend (#6509)

2025-05-30 01:11:53 -07:00

gemma3_mm.py

[Refactor] Multimodal data processing for VLM (#6659)

2025-06-04 11:22:33 -07:00

gemma.py

Revert "fix some typos" (#6244)

2025-05-12 12:53:26 -07:00

gpt2.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

gpt_bigcode.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

granite.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

grok.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

idefics2.py

Fix incorrect LoRA weight loading for fused gate_up_proj (#6734)

2025-05-31 13:41:44 -07:00

internlm2_reward.py

[QUANT] Add GPTQModel Dynamic Quantization + lm_head Quantization (#3790)

2025-03-05 01:11:00 -08:00

internlm2.py

Support InternVL3 (#5350)

2025-05-01 22:38:59 -07:00

internvl.py

vlm: adapt internvl to VisionAttention (#6870)

2025-06-11 01:16:04 -07:00

kimi_vl_moonvit.py

Revert "fix some typos" (#6244)

2025-05-12 12:53:26 -07:00

kimi_vl.py

[Refactor] Multimodal data processing for VLM (#6659)

2025-06-04 11:22:33 -07:00

llama4.py

Disable all two stream overlap on amd (#6475)

2025-05-20 19:06:59 -07:00

llama_classification.py

[QUANT] Add GPTQModel Dynamic Quantization + lm_head Quantization (#3790)

2025-03-05 01:11:00 -08:00

llama_eagle3.py

[PP] Add pipeline parallelism (#5724)

2025-04-30 18:18:07 -07:00

llama_eagle.py

[PP] Add pipeline parallelism (#5724)

2025-04-30 18:18:07 -07:00

llama_embedding.py

[QUANT] Add GPTQModel Dynamic Quantization + lm_head Quantization (#3790)

2025-03-05 01:11:00 -08:00

llama_reward.py

[QUANT] Add GPTQModel Dynamic Quantization + lm_head Quantization (#3790)

2025-03-05 01:11:00 -08:00

llama.py

Slightly improve the sampler to skip unnecessary steps (#6956)

2025-06-08 03:18:54 -07:00

llava.py

model(vlm): mistral 3.1 (#5099)

2025-05-16 18:36:18 -07:00

llavavid.py

refactor: multimodal data (#4754)

2025-03-31 09:57:51 -07:00

mimo_mtp.py

Support XiaomiMiMo inference with mtp (#6059)

2025-05-22 14:14:49 -07:00

mimo.py

Support XiaomiMiMo inference with mtp (#6059)

2025-05-22 14:14:49 -07:00

minicpm3.py

[misc] remove is_cuda_available (#5319)

2025-04-20 18:16:51 -07:00

minicpm.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

minicpmo.py

follow-up: move Idefics2 to a shared location to eliminate unexpected dependency. (#6603)

2025-05-26 19:23:59 -07:00

minicpmv.py

follow-up: move Idefics2 to a shared location to eliminate unexpected dependency. (#6603)

2025-05-26 19:23:59 -07:00

mistral.py

model(vlm): mistral 3.1 (#5099)

2025-05-16 18:36:18 -07:00

mixtral_quant.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

mixtral.py

[PP] Fix init_memory_pool desync & add PP for mixtral (#6223)

2025-05-12 12:38:09 -07:00

mllama4.py

vlm: enable radix cache for qwen-vl models (#5349)

2025-04-23 20:35:05 -07:00

mllama.py

[Fix]Fix capture fail bug for DeepSeek (#6275)

2025-05-21 11:11:20 -07:00

olmo2.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

olmo.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

olmoe.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

phi3_small.py

Fix Phi3 serving which was broke by earlier change (#5991)

2025-05-03 00:28:47 -07:00

phi4mm.py

Fix incorrect LoRA weight loading for fused gate_up_proj (#6734)

2025-05-31 13:41:44 -07:00

pixtral.py

model(vlm): pixtral (#5084)

2025-05-13 00:16:10 -07:00

qwen2_5_vl.py

[Refactor] Multimodal data processing for VLM (#6659)

2025-06-04 11:22:33 -07:00

qwen2_classification.py

[Model] Support Qwen2ForSequenceClassification (#4609)

2025-03-24 19:13:44 -07:00

qwen2_eagle.py

[Bugfix] pipeline parallelism and Eagle Qwen2 (#6910)

2025-06-07 01:58:50 -07:00

qwen2_moe.py

Revert "fix communicator for non-dp lm head (#6662)" (#6677)

2025-05-27 12:22:59 -07:00

qwen2_rm.py

[QUANT] Add GPTQModel Dynamic Quantization + lm_head Quantization (#3790)

2025-03-05 01:11:00 -08:00

qwen2_vl.py

[Refactor] Multimodal data processing for VLM (#6659)

2025-06-04 11:22:33 -07:00

qwen2.py

added support for tied weights in qwen pipeline parallelism (#6546)

2025-05-25 00:00:56 -07:00

qwen3_moe.py

Support both approximate and exact expert distribution collection (#6964)

2025-06-09 20:56:17 -07:00

qwen3.py

support qwen3 emebedding (#6990)

2025-06-09 01:32:49 -07:00

qwen.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

registry.py

[FEAT] Add transformers backend support (#5929)

2025-06-03 21:05:29 -07:00

roberta.py

Revert "fix some typos" (#6244)

2025-05-12 12:53:26 -07:00

siglip.py

Implement Siglip Vision model, and support BNB quantization for gemma3-mm (#5339)

2025-05-20 23:53:46 -07:00

stablelm.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

torch_native_llama.py

Add typo checker in pre-commit (#6179)

2025-05-11 12:55:00 +08:00

transformers.py

[FEAT] Add transformers backend support (#5929)

2025-06-03 21:05:29 -07:00

vila.py

Support VILA models (#6106)

2025-06-11 11:47:25 -07:00

xverse_moe.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

xverse.py

Fix loading KV quantization scale; Enable modelopt kv cache (#4686)

2025-04-08 09:11:35 -07:00

yivl.py

[QUANT] Add GPTQModel Dynamic Quantization + lm_head Quantization (#3790)

2025-03-05 01:11:00 -08:00