e52ebf8674
### What this PR does / why we need it? - [x] Patch `Qwen2_5_VisionAttention` with `AscendQwen2_5_VisionAttention`. - [x] Replace `AscendQwen2_5_VisionTransformer` with `Qwen2_5_VisionTransformer` in vllm. - [x] Move padding logic (q/k/v and cos/sin) before FA to `forward()` of `Qwen2_5_VisionAttention`. - [x] Covert `cu_seqlens` in `Qwen2_5_VisionAttention` from cumulative form to intervals and move it to cpu (compatible for npu FA). - [x] Remove Qwen2.5-VL modeling files. - [x] Remove Qwen2.5-VL (without padding) modeling files. - [x] Remove related UT. - [x] Make `set_forward_context` pluggable when getting MM embedding. Find more details at https://github.com/vllm-project/vllm/pull/29388. - [x] Simplify padding logic for FA. - [x] Add patch for https://github.com/vllm-project/vllm/pull/28798. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? - [x] Functional test (eager mode) - [x] Functional test (graph mode) - [x] Benchmark - vLLM version: v0.11.2 --------- Signed-off-by: shen-shanshan <467638484@qq.com>
30 lines
1.1 KiB
Python
30 lines
1.1 KiB
Python
from vllm import ModelRegistry
|
|
|
|
|
|
def register_model():
|
|
ModelRegistry.register_model(
|
|
"Qwen2VLForConditionalGeneration",
|
|
"vllm_ascend.models.qwen2_vl:AscendQwen2VLForConditionalGeneration")
|
|
|
|
ModelRegistry.register_model(
|
|
"Qwen3VLMoeForConditionalGeneration",
|
|
"vllm_ascend.models.qwen3_vl:AscendQwen3VLMoeForConditionalGeneration")
|
|
|
|
ModelRegistry.register_model(
|
|
"Qwen3VLForConditionalGeneration",
|
|
"vllm_ascend.models.qwen3_vl:AscendQwen3VLForConditionalGeneration")
|
|
|
|
# There is no PanguProMoEForCausalLM in vLLM, so we should register it before vLLM config initialization
|
|
# to make sure the model can be loaded correctly. This register step can be removed once vLLM support PanguProMoEForCausalLM.
|
|
ModelRegistry.register_model(
|
|
"PanguProMoEForCausalLM",
|
|
"vllm_ascend.torchair.models.torchair_pangu_moe:PanguProMoEForCausalLM"
|
|
)
|
|
|
|
ModelRegistry.register_model(
|
|
"Qwen3NextForCausalLM",
|
|
"vllm_ascend.models.qwen3_next:CustomQwen3NextForCausalLM")
|
|
|
|
ModelRegistry.register_model(
|
|
"Qwen3NextMTP", "vllm_ascend.models.qwen3_next_mtp:CustomQwen3NextMTP")
|