xc-llm-ascend/docs/source/developer_guide/evaluation/index.md

# Evaluation

:::{toctree}
:caption: Accuracy
:maxdepth: 1
using_lm_eval
using_opencompass
using_evalscope
:::

:::{toctree}
:caption: Performance
:maxdepth: 1
performance_benchmark
:::
[Doc]Add developer guide for using OpenCompass (#368) ### What this PR does / why we need it? Add developer guide for using OpenCompass ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? test manually --------- Signed-off-by: hfadzxy <starmoon_zhang@163.com> Signed-off-by: Yikun Jiang <yikunkero@gmail.com> Co-authored-by: Yikun Jiang <yikunkero@gmail.com> 2025-03-31 00:24:25 +08:00			`# Evaluation`

			`:::{toctree}`
			`:caption: Accuracy`
			`:maxdepth: 1`
[Doc]Add developer guide for using lm-eval (#456) ### What this PR does / why we need it? Add developer guide for using lm-eval ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? test manually --------- Signed-off-by: hfadzxy <starmoon_zhang@163.com> Signed-off-by: Yikun Jiang <yikunkero@gmail.com> Co-authored-by: Yikun Jiang <yikunkero@gmail.com> 2025-04-01 23:43:51 +08:00			`using_lm_eval`
[Test] Add accuracy test report workflow (#542) ### What this PR does / why we need it? 1. Provide accuracy test report for development branch release. 2. Models and datasets for accuracy test： \| Model \| datasets \| \|---------------------------- \| --------------------------- \| \| Qwen2.5-7B-Instruct \| ceval-val, gsm8k, mmlu \| \| Qwen3-8B \| ceval-val, gsm8k, mmlu \| \| Llama-3.1-8B-Instruct \| ceval-val, gsm8k, mmlu \| \| Qwen2.5-VL-7B-Instruct \| mmmu_val \| ### Does this PR introduce _any_ user-facing change? This PR will display the accuracy test report of the release versionin docs/source/developer_guide/accuracy_report。 Qwen2.5-7B-Instruct.md Qwen3-8B.md Llama-3.1-8B-Instruct.md Qwen2.5-VL-7B-Instruct .md Signed-off-by: hfadzxy <starmoon_zhang@163.com> 2025-04-30 14:53:58 +08:00			`using_opencompass`
Using EvalScope evaluation (#611) ### What this PR does / why we need it? Using EvalScope to hava a evaluation (include eval and test): - https://evalscope.readthedocs.io/en/latest/user_guides/stress_test/quick_start.html#basic-usage - https://evalscope.readthedocs.io/en/latest/get_started/basic_usage.html#model-api-service-evaluation ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Test locally --------- Signed-off-by: RongRongStudio <82669040+RongRongStudio@users.noreply.github.com> Signed-off-by: Yikun Jiang <yikunkero@gmail.com> Co-authored-by: Yikun Jiang <yikunkero@gmail.com> 2025-04-23 00:50:09 +08:00			`using_evalscope`
			`:::`
[Doc] Add benchmark guide (#635) ### What this PR does / why we need it? Add benchmark developer guide --------- Signed-off-by: wangli <wangli858794774@gmail.com> 2025-04-30 09:17:59 +08:00
			`:::{toctree}`
			`:caption: Performance`
			`:maxdepth: 1`
			`performance_benchmark`
			`:::`