diff --git a/docs/source/user_guide/feature_guide/quantization.md b/docs/source/user_guide/feature_guide/quantization.md index 1ffc0bf..abdf344 100644 --- a/docs/source/user_guide/feature_guide/quantization.md +++ b/docs/source/user_guide/feature_guide/quantization.md @@ -105,3 +105,21 @@ submit a issue, maybe some new models need to be adapted. ### 2. How to solve the error "Could not locate the configuration_deepseek.py"? Please convert DeepSeek series models using `modelslim-VLLM-8.1.RC1.b020_001` modelslim, this version has fixed the missing configuration_deepseek.py error. + +### 3. When converting deepseek series models with modelslim, what should you pay attention? + +When using the weight generated by modelslim with the `--dynamic` parameter, if torchair graph mode is enabled, please modify the configuration file in the CANN package to prevent incorrect inference results. + +The operation steps are as follows: + +1. Search in the CANN package directory used, for example: +find /usr/local/Ascend/ -name fusion_config.json + +2. Add `"AddRmsNormDynamicQuantFusionPass":"off",` to the fusion_config.json you find, the location is as follows: + +```bash +{ + "Switch":{ + "GraphFusion":{ + "AddRmsNormDynamicQuantFusionPass":"off", +```