[Feat][310p] 310P support w8a8s quantization and saving w8a8sc state (#6878)

### What this PR does / why we need it?
This pull request introduces significant enhancements for 310P device
support, primarily by enabling W8A8S quantization and facilitating the
saving of models with W8A8SC state outputs. It provides an example
script for saving sharded and compressed model states, implements the
core W8A8S quantization method, and integrates metadata generation
within the 310P worker to accurately describe the quantization types of
saved parameters. These changes aim to improve efficiency and
compatibility for quantized models on 310P hardware.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
W8A8S accuarcy test and W8A8SC states save.
<img width="886" height="184" alt="image"
src="https://github.com/user-attachments/assets/e9bcac54-1f69-4d3a-a5b8-221a147ef99d"
/>

- vLLM version: v0.16.0
- vLLM main:
15d76f74e2

---------

Signed-off-by: pu-zhe <zpuaa@outlook.com>
This commit is contained in:
pu-zhe
2026-03-02 20:09:15 +08:00
committed by GitHub
parent 68d8d20ca2
commit 5899438a86
8 changed files with 668 additions and 1 deletions

View File

@@ -14,7 +14,6 @@
# limitations under the License.
# This file is a part of the vllm-ascend project.
#
import torch_npu
from vllm.logger import logger
@@ -31,6 +30,23 @@ class NPUWorker310(NPUWorker):
self.model_runner = NPUModelRunner310(self.vllm_config, self.device)
def save_sharded_state(
self,
path: str,
pattern: str | None = None,
max_size: int | None = None,
) -> None:
from vllm_ascend._310p.sharded_state_loader_310p import ShardedStateLoader310
ShardedStateLoader310.save_model(
self.model_runner.model,
path,
pattern=pattern,
max_size=max_size,
)
ShardedStateLoader310.generate_quant_description(self.model_runner.model, path)
def _warm_up_atb(self):
# 310p device do not support torch_npu._npu_matmul_add_fp32 atb ops
logger.info("Skip warm-up atb ops for 310P device.")