kidyu/Moza-7B-v1.0
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
1 Commit 1 Branch 0 Tags
783c0dac236fb9c37ccb9bfd33529c3232e3d746
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE

README.md

license, library_name, tags, base_model, inference, model-index
license library_name tags base_model inference model-index
apache-2.0 transformers
mergekit
merge
mistralai/Mistral-7B-v0.1
cognitivecomputations/dolphin-2.2.1-mistral-7b
Open-Orca/Mistral-7B-OpenOrca
openchat/openchat-3.5-0106
mlabonne/NeuralHermes-2.5-Mistral-7B
GreenNode/GreenNode-mini-7B-multilingual-v1olet
berkeley-nest/Starling-LM-7B-alpha
viethq188/LeoScorpius-7B-Chat-DPO
meta-math/MetaMath-Mistral-7B
Intel/neural-chat-7b-v3-3
false
name results
Moza-7B-v1.0
task dataset metrics source
type name
text-generation Text Generation
name type config split args
AI2 Reasoning Challenge (25-Shot) ai2_arc ARC-Challenge test
num_few_shot
25
type value name
acc_norm 66.55 normalized accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=kidyu/Moza-7B-v1.0 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type split args
HellaSwag (10-Shot) hellaswag validation
num_few_shot
10
type value name
acc_norm 83.45 normalized accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=kidyu/Moza-7B-v1.0 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
MMLU (5-Shot) cais/mmlu all test
num_few_shot
5
type value name
acc 62.77 accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=kidyu/Moza-7B-v1.0 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
TruthfulQA (0-shot) truthful_qa multiple_choice validation
num_few_shot
0
type value
mc2 65.16
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=kidyu/Moza-7B-v1.0 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
Winogrande (5-shot) winogrande winogrande_xl validation
num_few_shot
5
type value name
acc 77.51 accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=kidyu/Moza-7B-v1.0 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
GSM8k (5-shot) gsm8k main test
num_few_shot
5
type value name
acc 62.55 accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=kidyu/Moza-7B-v1.0 Open LLM Leaderboard

Moza-7B-v1.0

image/png

This is a meme-merge of pre-trained language models, created using mergekit. Use at your own risk.

Details

Quantized Model

Merge Method

This model was merged using the DARE TIES merge method, using mistralai/Mistral-7B-v0.1 as a base.

The value for density are from this blogpost, and the weight was randomly generated and then assigned to the models, with priority (of using the bigger weight) to NeuralHermes, OpenOrca, and neural-chat. The models themselves are chosen by "vibes".

Models Merged

The following models were included in the merge:

Prompt Format

You can use Alpaca formatting for inference

### Instruction:

### Response:

Configuration

The following YAML configuration was used to produce this model:

base_model: mistralai/Mistral-7B-v0.1
models:
  - model: mlabonne/NeuralHermes-2.5-Mistral-7B
    parameters:
      density: 0.63
      weight: 0.83
  - model: Intel/neural-chat-7b-v3-3
    parameters:
      density: 0.63
      weight: 0.74
  - model: meta-math/MetaMath-Mistral-7B
    parameters:
      density: 0.63
      weight: 0.22
  - model: openchat/openchat-3.5-0106
    parameters:
      density: 0.63
      weight: 0.37
  - model: Open-Orca/Mistral-7B-OpenOrca
    parameters:
      density: 0.63
      weight: 0.76
  - model: cognitivecomputations/dolphin-2.2.1-mistral-7b
    parameters:
      density: 0.63
      weight: 0.69
  - model: viethq188/LeoScorpius-7B-Chat-DPO
    parameters:
      density: 0.63
      weight: 0.38
  - model: GreenNode/GreenNode-mini-7B-multilingual-v1olet
    parameters:
      density: 0.63
      weight: 0.13
  - model: berkeley-nest/Starling-LM-7B-alpha
    parameters:
      density: 0.63
      weight: 0.33
merge_method: dare_ties
parameters:
  normalize: true
  int8_mask: true
dtype: bfloat16

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 69.66
AI2 Reasoning Challenge (25-Shot) 66.55
HellaSwag (10-Shot) 83.45
MMLU (5-Shot) 62.77
TruthfulQA (0-shot) 65.16
Winogrande (5-shot) 77.51
GSM8k (5-shot) 62.55
Description
Model synced from source: kidyu/Moza-7B-v1.0
Readme 564 KiB