EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

[Fix] Sorts aclgraph batch sizes in ascending order (#4230)

Browse Source 
### What this PR does / why we need it?
Sorts aclgraph batch sizes in ascending order, corresponding to vLLM
[#26016](https://github.com/vllm-project/vllm/pull/26016)

Ensures batch sizes for aclgraph are sorted ascending when aclgraph mode
is enabled, improving consistency and compatibility with later logic
that may depend on order.

### Does this PR introduce _any_ user-facing change?
None.

### How was this patch tested?
Waiting for #3886 

- vLLM version: v0.11.0
- vLLM main:
2918c1b49c

Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
This commit is contained in:
Yizhou
2025-11-19 09:36:37 +08:00
committed by GitHub
parent e98543267a
commit 63561d6763
1 changed files with 8 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
vllm_ascend/worker/model_runner_v1.py
View File

@@ -521,8 +521,12 @@ class NPUModelRunner(LoRAModelRunnerMixin):
if self.speculative_config else 0) if self.speculative_config else 0)
self.use_aclgraph = self._use_aclgraph() self.use_aclgraph = self._use_aclgraph()
self.aclgraph_batch_sizes = list(
reversed(self.compilation_config.cudagraph_capture_sizes)) # self.aclgraph_batch_sizes sorts in ascending order.
if (self.compilation_config.cudagraph_capture_sizes and
self.compilation_config.cudagraph_mode != CUDAGraphMode.NONE):
self.aclgraph_batch_sizes = sorted(
self.compilation_config.cudagraph_capture_sizes)
self.uniform_decode_query_len = 1 if not self.speculative_config else \ self.uniform_decode_query_len = 1 if not self.speculative_config else \
1 + self.speculative_config.num_speculative_tokens 1 + self.speculative_config.num_speculative_tokens
@@ -4101,7 +4105,8 @@ class NPUModelRunner(LoRAModelRunnerMixin):
if aclgraph_mode.mixed_mode() != CUDAGraphMode.NONE: if aclgraph_mode.mixed_mode() != CUDAGraphMode.NONE:
aclgraph_runtime_mode = aclgraph_mode.mixed_mode() aclgraph_runtime_mode = aclgraph_mode.mixed_mode()
compilation_cases = sorted(self.aclgraph_batch_sizes) # make sure we capture the largest batch size first
compilation_cases = list(reversed(self.aclgraph_batch_sizes))
try: try:
self._capture_aclgraphs( self._capture_aclgraphs(