79 lines
2.6 KiB
Markdown
79 lines
2.6 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
datasets:
|
||
|
|
- meta-math/MetaMathQA
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
metrics:
|
||
|
|
- accuracy
|
||
|
|
---
|
||
|
|
|
||
|
|
see our paper in https://arxiv.org/abs/2401.02415
|
||
|
|
|
||
|
|
View the project page:
|
||
|
|
https://github.com/TencentARC/LLaMA-Pro
|
||
|
|
|
||
|
|
|
||
|
|
## Model Details
|
||
|
|
|
||
|
|
MetaMath-Mistral-Pro is fully fine-tuned on the MetaMathQA datasets and based on the powerful Mistral-Pro model.
|
||
|
|
|
||
|
|
|
||
|
|
## Model Usage
|
||
|
|
|
||
|
|
The model is trained to use the following format (note the newlines):
|
||
|
|
```
|
||
|
|
<|user|>
|
||
|
|
Your message here!
|
||
|
|
<|assistant|>
|
||
|
|
```
|
||
|
|
|
||
|
|
For best results, format all inputs in this manner. **Make sure to include a newline after `<|assistant|>`, this can affect generation quality quite a bit.**
|
||
|
|
|
||
|
|
|
||
|
|
## Experiments
|
||
|
|
|
||
|
|
| Model | GSM8k Pass@1 | MATH Pass@1 |
|
||
|
|
|---------------------|--------------|-------------|
|
||
|
|
| MPT-7B | 6.8 | 3.0 |
|
||
|
|
| Falcon-7B | 6.8 | 2.3 |
|
||
|
|
| LLaMA-1-7B | 11.0 | 2.9 |
|
||
|
|
| LLaMA-2-7B | 14.6 | 2.5 |
|
||
|
|
| MPT-30B | 15.2 | 3.1 |
|
||
|
|
| LLaMA-1-13B | 17.8 | 3.9 |
|
||
|
|
| GPT-Neo-2.7B | 19.5 | -- |
|
||
|
|
| Falcon-40B | 19.6 | 2.5 |
|
||
|
|
| Baichuan-chat-13B | 23.9 | -- |
|
||
|
|
| Vicuna-v1.3-13B | 27.6 | -- |
|
||
|
|
| LLaMA-2-13B | 28.7 | 3.9 |
|
||
|
|
| InternLM-7B | 31.2 | -- |
|
||
|
|
| ChatGLM-2-6B | 32.4 | -- |
|
||
|
|
| GPT-J-6B | 34.9 | -- |
|
||
|
|
| LLaMA-1-33B | 35.6 | 3.9 |
|
||
|
|
| LLaMA-2-34B | 42.2 | 6.24 |
|
||
|
|
| RFT-7B | 50.3 | -- |
|
||
|
|
| LLaMA-1-65B | 50.9 | 10.6 |
|
||
|
|
| Qwen-7B | 51.6 | -- |
|
||
|
|
| WizardMath-7B | 54.9 | 10.7 |
|
||
|
|
| LLaMA-2-70B | 56.8 | 13.5 |
|
||
|
|
| WizardMath-13B | 63.9 | 14.0 |
|
||
|
|
| MAmmoTH-7B (COT) | 50.5 | 10.4 |
|
||
|
|
| MAmmoTH-7B (POT+COT)| 53.6 | 31.5 |
|
||
|
|
| Arithmo-Mistral-7B | 74.7 | 25.3 |
|
||
|
|
| MetaMath-7B | 66.5 | 19.8 |
|
||
|
|
| MetaMath-13B | 72.3 | 22.4 |
|
||
|
|
| MetaMath-Mistral-7B | 77.7 | 28.2 |
|
||
|
|
| MetaMath-Llemma-7B | 69.2 | 30.0 |
|
||
|
|
| 🔥 **MetaMath-Mistral-Pro** | **78.4** | **30.3** |
|
||
|
|
|
||
|
|
|
||
|
|
## Citation
|
||
|
|
|
||
|
|
```bibtex
|
||
|
|
@article{wu2024llama,
|
||
|
|
title={Llama pro: Progressive llama with block expansion},
|
||
|
|
author={Wu, Chengyue and Gan, Yukang and Ge, Yixiao and Lu, Zeyu and Wang, Jiahao and Feng, Ye and Luo, Ping and Shan, Ying},
|
||
|
|
journal={arXiv preprint arXiv:2401.02415},
|
||
|
|
year={2024}
|
||
|
|
}
|
||
|
|
```
|