open-llama-3b-everythingLM-2048/README.md

language, license, library_name, datasets, model-index

language	license	library_name	datasets	model-index
en	apache-2.0	transformers	totally-not-an-llm/EverythingLM-data-V3	name results open-llama-3b-everythingLM-2048 task dataset metrics source type name text-generation Text Generation name type config split args AI2 Reasoning Challenge (25-Shot) ai2_arc ARC-Challenge test num_few_shot 25 type value name acc_norm 42.75 normalized accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=harborwater/open-llama-3b-everythingLM-2048 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args HellaSwag (10-Shot) hellaswag validation num_few_shot 10 type value name acc_norm 71.72 normalized accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=harborwater/open-llama-3b-everythingLM-2048 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args MMLU (5-Shot) cais/mmlu all test num_few_shot 5 type value name acc 27.16 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=harborwater/open-llama-3b-everythingLM-2048 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args TruthfulQA (0-shot) truthful_qa multiple_choice validation num_few_shot 0 type value mc2 34.26 url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=harborwater/open-llama-3b-everythingLM-2048 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args Winogrande (5-shot) winogrande winogrande_xl validation num_few_shot 5 type value name acc 66.3 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=harborwater/open-llama-3b-everythingLM-2048 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args GSM8k (5-shot) gsm8k main test num_few_shot 5 type value name acc 1.52 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=harborwater/open-llama-3b-everythingLM-2048 Open LLM Leaderboard

Trained on 2 epochs on the EverythingLM-data-V3 dataset.

This model uses the alpaca prompt format:

Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
Instruction

### Input:
Input

### Response:

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	40.62
AI2 Reasoning Challenge (25-Shot)	42.75
HellaSwag (10-Shot)	71.72
MMLU (5-Shot)	27.16
TruthfulQA (0-shot)	34.26
Winogrande (5-shot)	66.30
GSM8k (5-shot)	1.52