EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

[Doc] update faq about progress bar display issue (#538)

Browse Source 
### What this PR does / why we need it?
update faq about progress bar display issue

Signed-off-by: hfadzxy <starmoon_zhang@163.com>
This commit is contained in:
hfadzxy
2025-04-16 16:07:08 +08:00
committed by GitHub
parent fe13cd9ea5
commit 00de2ee6ad
1 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/source/faqs.md
View File

@@ -113,3 +113,7 @@ Currently, running on v0.7.3, we should run w8a8 with vllm + vllm-ascend + mindi
1. Quantize bf16 DeepSeek, e.g. [unsloth/DeepSeek-R1-BF16](https://modelscope.cn/models/unsloth/DeepSeek-R1-BF16), with msModelSlim to get w8a8 DeepSeek. Find more details in [msModelSlim doc](https://gitee.com/ascend/msit/tree/master/msmodelslim/msmodelslim/pytorch/llm_ptq)
2. Copy the content of `quant_model_description_w8a8_dynamic.json` into the `quantization_config` of `config.json` of the quantized model files.
3. Reference with the quantized DeepSeek model.
### 13. There is not output in log when loading models using vllm-ascend, How to solve it?
If you're using vllm 0.7.3 version, this is a known progress bar display issue in VLLM, which has been resolved in [this PR](https://github.com/vllm-project/vllm/pull/12428), please cherry-pick it locally by yourself. Otherwise, please fill up an issue.