AliceYin/l20-edu-135m
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: AliceYin/l20-edu-135m
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-10 18:43:03 +08:00
commit 9e2ca78a06
88 changed files with 605541 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
eval/comparison.md Normal file
View File

@@ -0,0 +1,22 @@
# lm-eval Comparison
Candidate: `l20-edu-135m-deepthin`
| Task | Metric | l20-edu-135m-deepthin | gpt2-small | opt-125m | gpt-neo-125m | cerebras-gpt-111m | pythia-160m | smollm-135m | smollm2-135m |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| arc_challenge | acc_norm,none | 0.2765 | 0.2261 | 0.2210 | 0.2321 | 0.2099 | 0.2312 | 0.2875 | 0.2969 |
| arc_easy | acc_norm,none | 0.5059 | 0.3973 | 0.3990 | 0.3965 | 0.3506 | 0.3641 | 0.5610 | 0.5854 |
| hellaswag | acc_norm,none | 0.3272 | 0.3138 | 0.3160 | 0.3055 | 0.2720 | 0.3030 | 0.4265 | 0.4301 |
| lambada_openai | acc,none | 0.2540 | 0.3076 | 0.3856 | 0.3765 | 0.1912 | 0.1225 | 0.3757 | 0.4289 |
| piqa | acc_norm,none | 0.6224 | 0.6208 | 0.6202 | 0.6213 | 0.5811 | 0.5979 | 0.6823 | 0.6839 |
| winogrande | acc,none | 0.5099 | 0.5067 | 0.5178 | 0.5099 | 0.4901 | 0.5075 | 0.5272 | 0.5249 |
## Win Rates
- `gpt2-small`: 5/6 = 0.833
- `opt-125m`: 4/6 = 0.667
- `gpt-neo-125m`: 4/6 = 0.667
- `cerebras-gpt-111m`: 6/6 = 1.000
- `pythia-160m`: 6/6 = 1.000
- `smollm-135m`: 0/6 = 0.000
- `smollm2-135m`: 0/6 = 0.000