【feat】switch for fusion ops gmmswigluquant (#5992)

### What this PR does / why we need it?

Set a additional config parameter to control whether the gmmswigluequant
fuseion operator is enabled; it is enabled by True. / When enabled with
a small number of GPUs, the gmmswigluquant fused operator can cause some
performance degradation.

### Does this PR introduce _any_ user-facing change?

### How was this patch tested?

- vLLM version: v0.13.0
- vLLM main:
2c24bc6996

#### Perf

test model: GLM 4.6(w8a8)
- single A3 node(ep16, tp16),  async-scheduling, mtp, FULL_DECODE_ONLY
- bs=1, input_lens=32000, ouput_lens=1024

Without this PR: TPOT 32.22.ms
With this PR: TPOT 30.23ms

---------

Signed-off-by: zjks98 <zhangjiakang4@huawei.com>
Co-authored-by: zjks98 <zhangjiakang4@huawei.com>
This commit is contained in:
aipaes
2026-01-19 21:19:25 +08:00
committed by GitHub
parent 38cfcd572a
commit f58e110afe
4 changed files with 45 additions and 1 deletions

View File

@@ -43,6 +43,9 @@ class TestAscendConfig(TestBase):
ascend_compilation_config = ascend_config.ascend_compilation_config
self.assertTrue(ascend_compilation_config.fuse_norm_quant)
ascend_fusion_config = ascend_config.ascend_fusion_config
self.assertTrue(ascend_fusion_config.fusion_ops_gmmswigluquant)
@_clean_up_ascend_config
def test_init_ascend_config_with_additional_config(self):
test_vllm_config = VllmConfig()
@@ -50,6 +53,9 @@ class TestAscendConfig(TestBase):
"ascend_compilation_config": {
"fuse_norm_quant": False,
},
"ascend_fusion_config": {
"fusion_ops_gmmswigluquant": False,
},
"multistream_overlap_shared_expert": True,
"eplb_config": {
"num_redundant_experts": 2
@@ -66,6 +72,9 @@ class TestAscendConfig(TestBase):
self.assertFalse(ascend_compilation_config.fuse_norm_quant)
self.assertFalse(ascend_config.enable_kv_nz)
ascend_fusion_config = ascend_config.ascend_fusion_config
self.assertFalse(ascend_fusion_config.fusion_ops_gmmswigluquant)
@_clean_up_ascend_config
def test_init_ascend_config_enable_npugraph_ex(self):
test_vllm_config = VllmConfig()