39 lines
1.1 KiB
Markdown
39 lines
1.1 KiB
Markdown
---
|
|
library_name: transformers
|
|
license: apache-2.0
|
|
datasets:
|
|
- lkevinzc/numia-1.5-qa-concatenated
|
|
base_model:
|
|
- HuggingFaceTB/FineMath-Llama-3B
|
|
---
|
|
|
|
# Llama-3.2-3B-NuminaQA
|
|
|
|
## Links
|
|
|
|
- 📜 [Paper](https://github.com/sail-sg/understand-r1-zero/blob/main/understand-r1-zero.pdf)
|
|
- 💻 [GitHub](https://github.com/sail-sg/understand-r1-zero)
|
|
- 🤗 [Oat-Zero Collection](https://huggingface.co/collections/sail/oat-zero-understanding-r1-zero-like-training-67dcdb07b9f3eb05f1501c4a)
|
|
|
|
## Introduction
|
|
|
|
This model serves as a 3B base in our minimalist R1-Zero recipe.
|
|
|
|
Training details:
|
|
- Base model: [HuggingFaceTB/FineMath-Llama-3B](https://huggingface.co/HuggingFaceTB/FineMath-Llama-3B)
|
|
- Dataset: [lkevinzc/numia-1.5-qa-concatenated](https://huggingface.co/datasets/lkevinzc/numia-1.5-qa-concatenated)
|
|
- Epochs: 2
|
|
- Learning rate: 1e-5
|
|
|
|
|
|
## Citation
|
|
|
|
```latex
|
|
@article{liu2025understanding,
|
|
title={Understanding r1-zero-like training: A critical perspective},
|
|
author={Liu, Zichen and Chen, Changyu and Li, Wenjun and Qi, Penghui and Pang, Tianyu and Du, Chao and Lee, Wee Sun and Lin, Min},
|
|
journal={arXiv preprint arXiv:2503.20783},
|
|
year={2025}
|
|
}
|
|
```
|