[CI/Build][Doc] upgrade torch-npu to 0320 (#392)

### What this PR does / why we need it?
This pr upgrades torch-npu to 0320, so that #321,
https://github.com/vllm-project/vllm-ascend/issues/267#issuecomment-2745045743
could be fixed, and #372 should be reverted after this pr

### Does this PR introduce _any_ user-facing change?
upgrade torch-npu to 0320

### How was this patch tested?
tested locally with long seq inferencing.

---------

Signed-off-by: MengqingCao <cmq0113@163.com>

pytest.ini

@@ -37,6 +37,8 @@ norecursedirs =
     vllm-empty/tests/worker
     vllm-empty/tests/metrics
     vllm-empty/tests/neuron
+  ; fastsafetensors not support npu now
+    vllm-empty/tests/fastsafetensors_loader
 
 addopts = --ignore=vllm-empty/tests/test_utils.py
           --ignore=vllm-empty/tests/test_config.py