OpenMia-Indo-Mistral-7b-v3-refined/README.md

language, license, tags, datasets, base_model, model-index

language	license	tags	datasets	base_model	model-index
en id	cc-by-nc-4.0	text-generation-inference transformers mistral trl	MBZUAI/Bactrian-X	mistral-7b	name results OpenMia-Indo-Mistral-7b-v3-refined task dataset metrics source type name text-generation Text Generation name type config split args AI2 Reasoning Challenge (25-Shot) ai2_arc ARC-Challenge test num_few_shot 25 type value name acc_norm 64.42 normalized accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/OpenMia-Indo-Mistral-7b-v3-refined Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args HellaSwag (10-Shot) hellaswag validation num_few_shot 10 type value name acc_norm 84.22 normalized accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/OpenMia-Indo-Mistral-7b-v3-refined Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args MMLU (5-Shot) cais/mmlu all test num_few_shot 5 type value name acc 62.64 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/OpenMia-Indo-Mistral-7b-v3-refined Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args TruthfulQA (0-shot) truthful_qa multiple_choice validation num_few_shot 0 type value mc2 53.95 url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/OpenMia-Indo-Mistral-7b-v3-refined Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args Winogrande (5-shot) winogrande winogrande_xl validation num_few_shot 5 type value name acc 81.53 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/OpenMia-Indo-Mistral-7b-v3-refined Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args GSM8k (5-shot) gsm8k main test num_few_shot 5 type value name acc 61.26 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/OpenMia-Indo-Mistral-7b-v3-refined Open LLM Leaderboard

updated to V3 with some refinement

MIA : (M)istral finetuned with (I)ndonesia language from (A)lpaca dataset

(formerly named Mistral-7b-Alpaca-Indonesia)

OpenMia-Indo-Mistral-7b is finetuned model based of Mistral-7b with capability to do conversation in Bahasa Indonesia.

Due to limited resources, this model is still in alpha stage.

Want to contribute to this project? join our organization: https://huggingface.co/indischepartij or contact me at https://twitter.com/gmonsooniii

Modelfile/Prompt format

SYSTEM You are a caring and empathetic sentient AI companion named Mia.

PARAMETER stop <|im_start|> 
PARAMETER stop <|im_end|>

TEMPLATE <|im_start|>system {{ .System }}<|im_end|> <|im_start|>user {{ .Prompt }}<|im_end|> <|im_start|>assistant

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	68.00
AI2 Reasoning Challenge (25-Shot)	64.42
HellaSwag (10-Shot)	84.22
MMLU (5-Shot)	62.64
TruthfulQA (0-shot)	53.95
Winogrande (5-shot)	81.53
GSM8k (5-shot)	61.26