QwenCollection/NeuralReyna-Mini-1.8B-v0.3
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
1 Commit 1 Branch 0 Tags
05c356c9d95415937ff5024a4218c7c6ec370a3c
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE

README.md

language, license, datasets, inference, model-index
language license datasets inference model-index
en
apache-2.0
argilla/OpenHermes2.5-dpo-binarized-alpha
Locutusque/Hercules-v3.0
parameters
do_sample temperature top_p top_k min_new_tokens max_new_tokens repetition_penalty
true 0.8 0.95 40 2 250 1.1
name results
NeuralReyna-Mini-1.8B-v0.3
task dataset metrics source
type name
text-generation Text Generation
name type config split args
AI2 Reasoning Challenge (25-Shot) ai2_arc ARC-Challenge test
num_few_shot
25
type value name
acc_norm 35.58 normalized accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=M4-ai/NeuralReyna-Mini-1.8B-v0.3 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type split args
HellaSwag (10-Shot) hellaswag validation
num_few_shot
10
type value name
acc_norm 61.13 normalized accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=M4-ai/NeuralReyna-Mini-1.8B-v0.3 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
MMLU (5-Shot) cais/mmlu all test
num_few_shot
5
type value name
acc 44.22 accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=M4-ai/NeuralReyna-Mini-1.8B-v0.3 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
TruthfulQA (0-shot) truthful_qa multiple_choice validation
num_few_shot
0
type value
mc2 41.99
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=M4-ai/NeuralReyna-Mini-1.8B-v0.3 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
Winogrande (5-shot) winogrande winogrande_xl validation
num_few_shot
5
type value name
acc 60.93 accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=M4-ai/NeuralReyna-Mini-1.8B-v0.3 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
GSM8k (5-shot) gsm8k main test
num_few_shot
5
type value name
acc 6.75 accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=M4-ai/NeuralReyna-Mini-1.8B-v0.3 Open LLM Leaderboard

NeuralReyna-Mini-1.8B-v0.3

Reyna image

Description

Taken aloobun/Reyna-Mini-1.8B-v0.2 and further fine-tuned it using DPO using the argilla/OpenHermes2.5-dpo-binarized-alpha.

This model has capabilities in coding, math, science, roleplay, and function calling.

This model was trained on OpenAI's ChatML prompt format.

Quants

HQQ - https://huggingface.co/twoxfh/NeuralReyna-Mini-hqq-1.8B-v0.3

Evaluation

Coming soon

Contributions

Thanks to @aloobun and @Locutusque for their contributions to this model.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 41.77
AI2 Reasoning Challenge (25-Shot) 35.58
HellaSwag (10-Shot) 61.13
MMLU (5-Shot) 44.22
TruthfulQA (0-shot) 41.99
Winogrande (5-shot) 60.93
GSM8k (5-shot) 6.75
Description
Model synced from source: QwenCollection/NeuralReyna-Mini-1.8B-v0.3
Readme 4.2 MiB