EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

[Platform] add dispatch key (#17)

Browse Source 
### What this PR does / why we need it?
Add dispatch key for NPU, so that the log could be print correctly.

Now
```
executor_base.py:110] # CPU blocks: 220478, # CPU blocks: 21845
```

After this pr
```
executor_base.py:110] # NPU blocks: 220478, # CPU blocks: 21845
```

### Does this PR introduce _any_ user-facing change?
N/A

### How was this patch tested?
CI passed and log printed as above

Signed-off-by: MengqingCao <cmq0113@163.com>
This commit is contained in:
Mengqing Cao
2025-02-21 17:10:30 +08:00
committed by GitHub
parent 5f465010de
commit dd425d68f8
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
vllm_ascend/platform.py
View File

@@ -57,6 +57,7 @@ class NPUPlatform(Platform):
simple_compile_backend: str = "npu"
ray_device_key: str = "NPU"
device_control_env_var: str = "ASCEND_RT_VISIBLE_DEVICES"
dispatch_key: str = "PrivateUse1"
supported_quantization: list[str] = ["ascend"]