license, library_name, tags, base_model, model-index

license	library_name	tags	base_model	model-index
apache-2.0	transformers	mergekit merge	NeuralNovel/Senzu-7B-v0.1-DPO senseable/WestLake-7B-v2	name results WestSenzu-Swap-7B task dataset metrics source type name text-generation Text Generation name type config split args AI2 Reasoning Challenge (25-Shot) ai2_arc ARC-Challenge test num_few_shot 25 type value name acc_norm 68.34 normalized accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Aryanne/WestSenzu-Swap-7B Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args HellaSwag (10-Shot) hellaswag validation num_few_shot 10 type value name acc_norm 85.7 normalized accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Aryanne/WestSenzu-Swap-7B Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args MMLU (5-Shot) cais/mmlu all test num_few_shot 5 type value name acc 64.14 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Aryanne/WestSenzu-Swap-7B Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args TruthfulQA (0-shot) truthful_qa multiple_choice validation num_few_shot 0 type value mc2 50.43 url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Aryanne/WestSenzu-Swap-7B Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args Winogrande (5-shot) winogrande winogrande_xl validation num_few_shot 5 type value name acc 82.48 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Aryanne/WestSenzu-Swap-7B Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args GSM8k (5-shot) gsm8k main test num_few_shot 5 type value name acc 52.62 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Aryanne/WestSenzu-Swap-7B Open LLM Leaderboard

It's experimental, but seems fine for me, I didn't run it deeply yet but should be good for Role-play 😈 considering the two merged models, feel free to leave a suggestion or feedback.

This is a merge of pre-trained language models created using mergekit(my experimental branch swapping here )

Merge Details

Merge Method

This model was merged using the task_swapping merge method using NeuralNovel/Senzu-7B-v0.1-DPO as a base.

Models Merged

The following models were included in the merge:

• senseable/WestLake-7B-v2

Configuration

The following YAML configuration was used to produce this model:

merge_method: task_swapping
base_model: NeuralNovel/Senzu-7B-v0.1-DPO
models:
  - model: senseable/WestLake-7B-v2
    parameters:
      weight: 0.75
      diagonal_offset: 2    #it doesn't do anything when you use random_mask
      random_mask: 0.3333
      random_mask_seed: 98557
dtype: bfloat16

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	67.28
AI2 Reasoning Challenge (25-Shot)	68.34
HellaSwag (10-Shot)	85.70
MMLU (5-Shot)	64.14
TruthfulQA (0-shot)	50.43
Winogrande (5-shot)	82.48
GSM8k (5-shot)	52.62