[Feat.][310P]: weightNZ feature with quant or unquant. (#6705)

NZ Format Support for Linear Layers: Implemented support for the NZ
(N-dimensional Z-order) format for linear layer weights on Ascend 310P,
enhancing performance for both quantized and unquantized layers.
Unquantized Linear Method for Ascend 310P: Introduced
AscendUnquantizedLinearMethod310 to specifically handle and apply NZ
format casting to unquantized linear layer weights during the loading
process.
MRotaryEmbedding Integration: Extended Rotary Embedding support by
adding AscendMRotaryEmbedding310 to provide an Ascend-specific
implementation for MRotaryEmbedding.
Quantization Method Updates: Updated the w8a8_static quantization method
to directly transpose weights and apply NZ format casting, ensuring
consistency with the new format.
- vLLM version: v0.15.0
- vLLM main:
9562912cea

---------

Signed-off-by: Tflowers-0129 <2906339855@qq.com>

vllm_ascend/_310p/ops/activation.py

@@ -17,12 +17,16 @@
 
 import torch
 import torch.nn.functional as F
+import torch_npu
 
 from vllm_ascend.ops.activation import AscendSiluAndMul
 
 
 class AscendSiluAndMul310(AscendSiluAndMul):
     def forward(self, x: torch.Tensor) -> torch.Tensor:
-        h = x.shape[-1] // 2
-        out = (F.silu(x[..., :h].to(torch.float32)) * x[..., h:].to(torch.float32)).to(torch.float16)
+        if x.shape[-1] % 32 == 0:
+            out = torch_npu.npu_swiglu(x)
+        else:
+            h = x.shape[-1] // 2
+            out = F.silu(x[..., :h]) * x[..., h:]
         return out