Hathor_Stable-v0.2-L3-8B/README.md

language, license, model-index

language	license	model-index
en	other	name results Hathor_Stable-v0.2-L3-8B task dataset metrics source type name text-generation Text Generation name type args IFEval (0-Shot) HuggingFaceH4/ifeval num_few_shot 0 type value name inst_level_strict_acc and prompt_level_strict_acc 71.75 strict accuracy url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Nitral-AI/Hathor_Stable-v0.2-L3-8B Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type args BBH (3-Shot) BBH num_few_shot 3 type value name acc_norm 32.83 normalized accuracy url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Nitral-AI/Hathor_Stable-v0.2-L3-8B Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type args MATH Lvl 5 (4-Shot) hendrycks/competition_math num_few_shot 4 type value name exact_match 9.21 exact match url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Nitral-AI/Hathor_Stable-v0.2-L3-8B Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type args GPQA (0-shot) Idavidrein/gpqa num_few_shot 0 type value name acc_norm 4.92 acc_norm url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Nitral-AI/Hathor_Stable-v0.2-L3-8B Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type args MuSR (0-shot) TAUR-Lab/MuSR num_few_shot 0 type value name acc_norm 5.56 acc_norm url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Nitral-AI/Hathor_Stable-v0.2-L3-8B Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args MMLU-PRO (5-shot) TIGER-Lab/MMLU-Pro main test num_few_shot 5 type value name acc 29.96 accuracy url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Nitral-AI/Hathor_Stable-v0.2-L3-8B Open LLM Leaderboard

Quants From Bartowski <3: https://huggingface.co/bartowski/Hathor-L3-8B-v.02-GGUF https://huggingface.co/bartowski/Hathor-L3-8B-v.02-exl2

---

Notes: Hathor is trained on 3 epochs of private data, synthetic opus instructions, a mix of light/classical novel data, roleplaying chat pairs over llama 3 8B instruct. (expanded)

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	25.70
IFEval (0-Shot)	71.75
BBH (3-Shot)	32.83
MATH Lvl 5 (4-Shot)	9.21
GPQA (0-shot)	4.92
MuSR (0-shot)	5.56
MMLU-PRO (5-shot)	29.96