chenxi-hh 42bcad7e9b GMM custom operator optimization in small batch scenarios (#7100) ...

### What this PR does / why we need it?
GMM custom operator optimization in small batch scenarios

### How was this patch tested?

Qwen3-30B input: 4k, output: 1k

batch 1：
TPOT 7.9 ms -> 7.0 ms
Output Token Throughput 125.4651 token/s -> 140.6278 token/s

batch 2：
TPOT 9.4 ms -> 8.8 ms
Output Token Throughput 211.8187 token/s -> 225.2254 token/s

batch 16：
TPOT 13.6 ms -> 13.5 ms
Output Token Throughput 1159.8213 token/s -> 1165.0982 token/s

- vLLM version: v0.16.0
- vLLM main:
4034c3d32e

---------

Signed-off-by: chenxi-hh <chen464822955@163.com>

2026-03-19 16:10:30 +08:00

..

_310p

[Refactor] [310p] Support Mamba Cache and support attn_head_size larger than 128 (#7372)

2026-03-19 09:16:22 +08:00

_cann_ops_custom

[Kernel] add custom op GmmSwigluQuantWeightNzTensorList (#3804)

2025-11-28 18:06:39 +08:00

attention

[bugfix][accuracy] Fix ds indexer accuracy problem caused by k rope (#7341)

2026-03-18 14:20:21 +08:00

compilation

[main2main] upgrade vllm to 0308 (#7213)

2026-03-18 09:24:43 +08:00

core

[bugfix] restore pr-7029 and fix patch error (#7294)

2026-03-16 15:39:42 +08:00

device

GMM custom operator optimization in small batch scenarios (#7100)

2026-03-19 16:10:30 +08:00

device_allocator

[Lint]Style: Convert vllm-ascend/ to ruff format(Batch #2) (#5977)

2026-01-19 08:59:46 +08:00

distributed

[P/D] LayerwiseConnector supports the virtual push functionality on node D. (#7361)

2026-03-18 10:50:02 +08:00

eplb

Support per-step heat collection and enhance FlashLB for multi-stage load balancing (#6477)

2026-03-12 15:49:09 +08:00

kv_offload

Main2main upgrade to vllm 0317 afternoon (#7409)

2026-03-18 23:24:27 +08:00

lora

[Bugfix][LoRA] Fix the bug when runs Qwen3-Reranker-0.6B with LoRA. (#7156)

2026-03-15 17:55:42 +08:00

model_loader

[Lint]Style: Convert vllm-ascend/ to ruff format(Batch #6) (#6001)

2026-01-24 22:08:33 +08:00

ops

GMM custom operator optimization in small batch scenarios (#7100)

2026-03-19 16:10:30 +08:00

patch

[Refactor] [310p] Support Mamba Cache and support attn_head_size larger than 128 (#7372)

2026-03-19 09:16:22 +08:00

quantization

[Bugfix] fix bug about model type of qwen3_vl_8b_instruct_w8a8 (#7383)

2026-03-18 20:30:03 +08:00

sample

[Feature] Add docs of batch invariance and make some extra operators patch (#6910)

2026-03-05 09:12:40 +08:00

spec_decode

[main2main] upgrade vllm to 0308 (#7213)

2026-03-18 09:24:43 +08:00

worker

Main2main upgrade to vllm 0317 afternoon (#7409)

2026-03-18 23:24:27 +08:00

xlite

Main2main upgrade to vllm 0317 afternoon (#7409)

2026-03-18 23:24:27 +08:00

__init__.py

[Lint]Style: Convert vllm-ascend/compilation to ruff format (#5912)

2026-01-16 20:57:46 +08:00

ascend_config.py

[Misc] Drop Prefetch MLP Env (#7357)

2026-03-19 14:27:27 +08:00

ascend_forward_context.py

[BugFix]A2 MOE method&& layerwise MTP bugfix && Mamba gdn_metadata bugfix (#7364)

2026-03-17 23:03:45 +08:00

batch_invariant.py

[Feature] Add docs of batch invariance and make some extra operators patch (#6910)

2026-03-05 09:12:40 +08:00

cpu_binding.py

[CPU binding] Implement global CPU slicing and improve IRQ binding for Ascend NPUs (#6945)

2026-03-03 17:20:52 +08:00

envs.py

[Misc] Drop Prefetch MLP Env (#7357)

2026-03-19 14:27:27 +08:00

flash_common3_context.py

[Lint]Style: Convert vllm-ascend/compilation to ruff format (#5912)

2026-01-16 20:57:46 +08:00

meta_registration.py

[Ops][Refactor] Remove custom rotary_embedding operator (#6523)

2026-02-07 09:24:05 +08:00

platform.py

[Feature][Quant] Reapply auto-detect quantization format and support remote model ID (#7111)

2026-03-13 22:53:25 +08:00

profiling_config.py

[Core][Misc] Clean up ProfileExecuteDuration (#6461)

2026-02-01 20:06:01 +08:00

utils.py

[Feature]Supports DSv3.1 PD separation and C8 quantization (#7222)

2026-03-16 22:49:05 +08:00