[main] remove torch.cat and replace it by List[0] (#2153)
### What this PR does / why we need it? torch_npu.npu_grouped_matmul: https://www.hiascend.com/document/detail/zh/Pytorch/710/apiref/torchnpuCustomsapi/context/torch_npu-npu_grouped_matmul.md According to the document, when `split_item` is 2 or 3, `torch_npu.npu_grouped_matmul` will return a list which has one element. Therefore, the `torch.cat` after `torch_npu.npu_grouped_matmul` is unnecessary. ### Does this PR introduce _any_ user-facing change? not involved ### How was this patch tested? ut and e2e covered: `tests/ut/ops/test_fused_ops.py`, `tests/e2e/singlecard/ops/test_fused_moe.py` **performance**: (qwen3 30B, 2k->20k) base: Total Token throughput (tok/s): 667.76 remove cat: Total Token throughput (tok/s): 680.82 - vLLM version: v0.10.0 - vLLM main: https://github.com/vllm-project/vllm/commit/fa00c5d75bc63c87f5822f839db1342f19e4acc8 Signed-off-by: huangxialu <huangxialu1@huawei.com>
This commit is contained in:
1 parent
b2598c3271
commit
dceef080b1
2 files changed
+13
-27
No files matched your search
@@ -112,7 +112,7 @@ def mock_moe_env(mocker: MockerFixture):
|
||||
torch.randn(16, 2)
|
||||
)), \
|
||||
patch("torch_npu.npu_grouped_matmul", return_value=(
|
||||
(torch.randn(8, 2), torch.randn(8, 2))
|
||||
[torch.randn(16, 2)]
|
||||
)), \
|
||||
patch("torch_npu.npu_swiglu", return_value=(
|
||||
torch.randn(16, 2)
|
||||
|
||||
Reference in new issue
Block a user