[Feat.][310P]: weightNZ feature with quant or unquant. (#6705)
NZ Format Support for Linear Layers: Implemented support for the NZ
(N-dimensional Z-order) format for linear layer weights on Ascend 310P,
enhancing performance for both quantized and unquantized layers.
Unquantized Linear Method for Ascend 310P: Introduced
AscendUnquantizedLinearMethod310 to specifically handle and apply NZ
format casting to unquantized linear layer weights during the loading
process.
MRotaryEmbedding Integration: Extended Rotary Embedding support by
adding AscendMRotaryEmbedding310 to provide an Ascend-specific
implementation for MRotaryEmbedding.
Quantization Method Updates: Updated the w8a8_static quantization method
to directly transpose weights and apply NZ format casting, ensuring
consistency with the new format.
- vLLM version: v0.15.0
- vLLM main:
9562912cea
---------
Signed-off-by: Tflowers-0129 <2906339855@qq.com>
This commit is contained in:
@@ -104,9 +104,9 @@ class AscendModelSlimConfig310(AscendModelSlimConfig):
|
||||
if isinstance(layer, LinearBase):
|
||||
packed = getattr(self, "packed_modules_mapping", {})
|
||||
if self.is_layer_skipped_ascend(prefix, packed):
|
||||
from vllm_ascend.ops.linear import AscendUnquantizedLinearMethod
|
||||
from vllm_ascend._310p.ops.linear import AscendUnquantizedLinearMethod310
|
||||
|
||||
return AscendUnquantizedLinearMethod()
|
||||
return AscendUnquantizedLinearMethod310()
|
||||
|
||||
scheme = create_scheme_for_layer(
|
||||
quant_description=self.quant_description,
|
||||
|
||||
Reference in New Issue
Block a user