# SmolLM Target Benchmark

| Task | Metric | ours-stage2 | smollm-135m | smollm2-135m |
| --- | --- | --- | --- | --- |
| ARC-Challenge | acc_norm,none | 0.2807 | 0.2875 | 0.2969 |
| ARC-Easy | acc_norm,none | 0.5059 | 0.5610 | 0.5854 |
| HellaSwag | acc_norm,none | 0.3242 | 0.4265 | 0.4301 |
| LAMBADA OpenAI | acc,none |  | 0.3757 | 0.4289 |
| PIQA | acc_norm,none | 0.6137 | 0.6823 | 0.6839 |
| WinoGrande | acc,none |  | 0.5272 | 0.5249 |

## Means

- `ours-stage2`: 0.4311
- `smollm-135m`: 0.4767
- `smollm2-135m`: 0.4917

## Candidate Gaps

- `ours-stage2` vs `smollm-135m` mean gap: -0.0456
  - HellaSwag: -0.1023
  - PIQA: -0.0686
  - ARC-Easy: -0.0551
  - ARC-Challenge: -0.0068
- `ours-stage2` vs `smollm2-135m` mean gap: -0.0605
  - HellaSwag: -0.1059
  - ARC-Easy: -0.0795
  - PIQA: -0.0702
  - ARC-Challenge: -0.0162