Qwen2.5-Math-14B-Instruct-Preview/README.md

language, license, tags, base_model, model-index

language	license	tags	base_model	model-index
en	apache-2.0	text-generation-inference transformers unsloth qwen2 trl	unsloth/qwen2.5-14b-instruct-bnb-4bit	name results Qwen2.5-Math-14B-Instruct task dataset metrics source type name text-generation Text Generation name type args IFEval (0-Shot) HuggingFaceH4/ifeval num_few_shot 0 type value name inst_level_strict_acc and prompt_level_strict_acc 60.66 strict accuracy url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Math-14B-Instruct Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type args BBH (3-Shot) BBH num_few_shot 3 type value name acc_norm 47.02 normalized accuracy url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Math-14B-Instruct Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type args MATH Lvl 5 (4-Shot) hendrycks/competition_math num_few_shot 4 type value name exact_match 28.47 exact match url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Math-14B-Instruct Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type args GPQA (0-shot) Idavidrein/gpqa num_few_shot 0 type value name acc_norm 16.33 acc_norm url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Math-14B-Instruct Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type args MuSR (0-shot) TAUR-Lab/MuSR num_few_shot 0 type value name acc_norm 19.63 acc_norm url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Math-14B-Instruct Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args MMLU-PRO (5-shot) TIGER-Lab/MMLU-Pro main test num_few_shot 5 type value name acc 48.12 accuracy url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Math-14B-Instruct Open LLM Leaderboard

Uploaded model

• Developed by: qingy2019
• License: apache-2.0
• Finetuned from model : unsloth/qwen2.5-14b-instruct-bnb-4bit

This Qwen 2.5 model was trained 2x faster with Unsloth and Huggingface's TRL library.

I fine-tuned it for 400 steps on garage-bAInd/Open-Platypus with a batch size of 3.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	36.71
IFEval (0-Shot)	60.66
BBH (3-Shot)	47.02
MATH Lvl 5 (4-Shot)	28.47
GPQA (0-shot)	16.33
MuSR (0-shot)	19.63
MMLU-PRO (5-shot)	48.12