language, license, model-index

language	license	model-index
en	apache-2.0	name results mistral-7b-v0.1-layla-v4 task dataset metrics source type name text-generation Text Generation name type config split args AI2 Reasoning Challenge (25-Shot) ai2_arc ARC-Challenge test num_few_shot 25 type value name acc_norm 62.29 normalized accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=l3utterfly/mistral-7b-v0.1-layla-v4 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args HellaSwag (10-Shot) hellaswag validation num_few_shot 10 type value name acc_norm 83.36 normalized accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=l3utterfly/mistral-7b-v0.1-layla-v4 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args MMLU (5-Shot) cais/mmlu all test num_few_shot 5 type value name acc 64.32 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=l3utterfly/mistral-7b-v0.1-layla-v4 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args TruthfulQA (0-shot) truthful_qa multiple_choice validation num_few_shot 0 type value mc2 43.14 url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=l3utterfly/mistral-7b-v0.1-layla-v4 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args Winogrande (5-shot) winogrande winogrande_xl validation num_few_shot 5 type value name acc 79.56 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=l3utterfly/mistral-7b-v0.1-layla-v4 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args GSM8k (5-shot) gsm8k main test num_few_shot 5 type value name acc 55.5 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=l3utterfly/mistral-7b-v0.1-layla-v4 Open LLM Leaderboard

Model Card

Model Description

Mistral 7B fine-tuned by the OpenHermes 2.5 dataset optimised for multi-turn conversation and character impersonation.

The dataset has been pre-processed by doing the following:

1. remove all refusals
2. remove any mention of AI assistant
3. split any multi-turn dialog generated in the dataset into multi-turn conversations records
4. added nfsw generated conversations from the Teatime dataset

• Developed by: l3utterfly
• Funded by: Layla Network
• Model type: Mistral
• Language(s) (NLP): English
• License: Apache-2.0
• Finetuned from model: Mistral 7B

Uses

Base model used by Layla - the offline personal assistant: https://www.layla-network.ai

Help & support: https://discord.gg/x546YJ6nYC

Prompt:

USER:
ASSISTANT:

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	64.69
AI2 Reasoning Challenge (25-Shot)	62.29
HellaSwag (10-Shot)	83.36
MMLU (5-Shot)	64.32
TruthfulQA (0-shot)	43.14
Winogrande (5-shot)	79.56
GSM8k (5-shot)	55.50