18 lines
422 B
Markdown
18 lines
422 B
Markdown
|
---
|
||
|
|
library_name: transformers
|
||
|
|
tags: []
|
||
|
|
---
|
||
|
|
|
||
|
|
|
||
|
|
# 📊 Benchmark Results
|
||
|
|
|
||
|
|
| Benchmark | Metric | Score |
|
||
|
|
|---|---|---|
|
||
|
|
| ARC-Challenge | acc_norm | 39.25% |
|
||
|
|
| GSM8K | exact_match | 37.45% |
|
||
|
|
| MMLU | acc | 53.75% |
|
||
|
|
|
||
|
|
*Evaluated using `lm-evaluation-harness`.*
|
||
|
|
|
||
|
|
Note: v2 of this model coming soon with stronger reasoning and agentic capabilities. Will fix the "identity confusion" Users may experience with this current model.
|