Files
xc-llm-ascend/vllm_ascend/quantization
linfeng-yuan 2cd036ee8e [Bugfix] fix accuracy problem for quantized deepseek models (#768)
### What this PR does / why we need it?

The root cause of the bug is that numerical computations involving NaNs
cannot eliminate them. We addressed it by using `masked_fill_` to
eliminate NaNs while avoiding memory-wasting `torch.where` approach.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
This patch was tested with vllm v0.8.5 and vllm-ascend master. I run
deepseek_v3 model with offline inference scripts
(examples/dp_offline/run_dp.sh & data_parallel.py).

Signed-off-by: linfeng-yuan <1102311262@qq.com>
2025-05-06 22:09:56 +08:00
..