Commit Graph

5 Commits

Author SHA1 Message Date
hfadzxy
affca6f348 [Test] Add accuracy test report workflow (#542)
### What this PR does / why we need it?
1. Provide accuracy test report for development branch release.
2. Models and datasets for accuracy test:
    
| Model | datasets |
|---------------------------- | --------------------------- | 
| Qwen2.5-7B-Instruct        |  ceval-val, gsm8k, mmlu  |
| Qwen3-8B                        |  ceval-val, gsm8k, mmlu  |
| Llama-3.1-8B-Instruct      |  ceval-val, gsm8k, mmlu  |
| Qwen2.5-VL-7B-Instruct  |           mmmu_val             |

### Does this PR introduce _any_ user-facing change?
This PR will display the accuracy test report of the release versionin
docs/source/developer_guide/accuracy_report。
Qwen2.5-7B-Instruct.md
Qwen3-8B.md
Llama-3.1-8B-Instruct.md
Qwen2.5-VL-7B-Instruct .md

Signed-off-by: hfadzxy <starmoon_zhang@163.com>
2025-04-30 14:53:58 +08:00
Li Wang
90aabaeb2e [Doc] Add benchmark guide (#635)
### What this PR does / why we need it?
 Add benchmark developer guide

---------

Signed-off-by: wangli <wangli858794774@gmail.com>
2025-04-30 09:17:59 +08:00
RongRongStudio
848e041a54 Using EvalScope evaluation (#611)
### What this PR does / why we need it?
Using EvalScope to hava a evaluation (include eval and test):
-
https://evalscope.readthedocs.io/en/latest/user_guides/stress_test/quick_start.html#basic-usage
-
https://evalscope.readthedocs.io/en/latest/get_started/basic_usage.html#model-api-service-evaluation

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Test locally

---------

Signed-off-by: RongRongStudio <82669040+RongRongStudio@users.noreply.github.com>
Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
Co-authored-by: Yikun Jiang <yikunkero@gmail.com>
2025-04-23 00:50:09 +08:00
hfadzxy
94bf9c379e [Doc]Add developer guide for using lm-eval (#456)
### What this PR does / why we need it?
Add developer guide for using lm-eval

### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?
test manually

---------

Signed-off-by: hfadzxy <starmoon_zhang@163.com>
Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
Co-authored-by: Yikun Jiang <yikunkero@gmail.com>
2025-04-01 23:43:51 +08:00
hfadzxy
7beb4339dc [Doc]Add developer guide for using OpenCompass (#368)
### What this PR does / why we need it?
Add developer guide for using OpenCompass

### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?

test manually

---------

Signed-off-by: hfadzxy <starmoon_zhang@163.com>
Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
Co-authored-by: Yikun Jiang <yikunkero@gmail.com>
2025-03-31 00:24:25 +08:00