tiny-think-dpo-math-stem-dpo-beta0-5-lr2e-6-e1-bs8/README.md

base_model, library_name, model_name, tags, licence

base_model	library_name	model_name	tags	licence
Shekswess/tiny-think-sft-math-stem-loss-nll-bf16-lr2e-5-e2-bs8	transformers	tiny-think-dpo-math-stem-dpo-beta0-5-lr2e-6-e1-bs8	generated_from_trainer trl dpo	license

Model Card for tiny-think-dpo-math-stem-dpo-beta0-5-lr2e-6-e1-bs8

This model is a fine-tuned version of Shekswess/tiny-think-sft-math-stem-loss-nll-bf16-lr2e-5-e2-bs8. It has been trained using TRL.

Training procedure

This model was trained with DPO, a method introduced in Direct Preference Optimization: Your Language Model is Secretly a Reward Model.

Framework versions

• TRL: 0.26.2
• Transformers: 4.57.5
• Pytorch: 2.9.0+cu128
• Datasets: 4.5.0
• Tokenizers: 0.22.2