EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

[deepseek][bugfix] support deepseek quant (#469)

Browse Source 
- support deepseek quant
  - add w8a8_dynamic quant
see #391

Signed-off-by: MengqingCao <cmq0113@163.com>
Co-authored-by: zzzzwwjj <1183291235@qq.com>
This commit is contained in:
Mengqing Cao
2025-04-07 10:56:12 +08:00
committed by GitHub
parent 3f9752f8ee
commit 344228a5da
3 changed files with 522 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
vllm_ascend/models/__init__.py
View File

@@ -7,3 +7,11 @@ def register_model():
ModelRegistry.register_model(
"Qwen2VLForConditionalGeneration",
"vllm_ascend.models.qwen2_vl:CustomQwen2VLForConditionalGeneration")
ModelRegistry.register_model(
"DeepseekV2ForCausalLM",
"vllm_ascend.models.deepseek_v2:CustomDeepseekV2ForCausalLM")
ModelRegistry.register_model(
"DeepseekV3ForCausalLM",
"vllm_ascend.models.deepseek_v2:CustomDeepseekV3ForCausalLM")