EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

[Bugfix] Add verification for quant_action.choices to avoid TypeError (#1046)

Browse Source 
### What this PR does / why we need it?

When I run vllm-ascend, I get this error msg:

```bash
Traceback (most recent call last):
  File "/home/sss/software/miniconda3/envs/vllm-v1/bin/vllm", line 8, in <module>
    sys.exit(main())
  File "/home/sss/github/vllm-project/vllm/vllm/entrypoints/cli/main.py", line 50, in main
    cmd.subparser_init(subparsers).set_defaults(
  File "/home/sss/github/vllm-project/vllm/vllm/entrypoints/cli/serve.py", line 101, in subparser_init
    serve_parser = make_arg_parser(serve_parser)
  File "/home/sss/github/vllm-project/vllm/vllm/entrypoints/openai/cli_args.py", line 254, in make_arg_parser
    parser = AsyncEngineArgs.add_cli_args(parser)
  File "/home/sss/github/vllm-project/vllm/vllm/engine/arg_utils.py", line 1582, in add_cli_args
    current_platform.pre_register_and_update(parser)
  File "/home/sss/github/vllm-project/vllm-ascend/vllm_ascend/platform.py", line 80, in pre_register_and_update
    if ASCEND_QUATIZATION_METHOD not in quant_action.choices:
TypeError: argument of type 'NoneType' is not iterable
[ERROR] 2025-06-03-02:53:42 (PID:6005, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception
```

This is because the `choices` attribute in `quant_action` can be `None`
and we don't check it.

```bash
# quant_action
_StoreAction(option_strings=['--quantization', '-q'], dest='quantization', nargs=None, const=None, default=None, type=<class 'str'>, choices=None, required=False, help='Method used to quantize the weights. If `None`, we first check the\n`quantization_config` attribute in the model config file. If that is\n`None`, we assume the model weights are not quantized and use `dtype` to\ndetermine the data type of the weights.', metavar=None)
```

Thus, I have added check for the `choices` to handle the scenario of
`choices=None`.

### Does this PR introduce _any_ user-facing change?
yes, vllm server with ascend quantization works now.

### How was this patch tested?
by `vllm server --quantization ascend` command.

Related: https://github.com/vllm-project/vllm/issues/19004

Signed-off-by: shen-shanshan <467638484@qq.com>
Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com>
This commit is contained in:
Shanshan Shen
2025-06-03 11:44:45 +08:00
committed by GitHub
parent 93860574bb
commit 068c3a0167
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
vllm_ascend/platform.py
View File

@@ -75,7 +75,8 @@ class NPUPlatform(Platform):
# and the user can enable quantization using "vllm serve --quantization ascend".
if parser is not None:
quant_action = parser._option_string_actions.get('--quantization')
if quant_action and hasattr(quant_action, 'choices'):
if quant_action and hasattr(quant_action,
'choices') and quant_action.choices:
if ASCEND_QUATIZATION_METHOD not in quant_action.choices:
quant_action.choices.append(ASCEND_QUATIZATION_METHOD)