DarkForest-20B-v2.0/README.md

license, tags, license_name, model-index

license	tags	license_name	model-index
other	merge not-for-all-audiences	microsoft-research-license	name results DarkForest-20B-v2.0 task dataset metrics source type name text-generation Text Generation name type config split args AI2 Reasoning Challenge (25-Shot) ai2_arc ARC-Challenge test num_few_shot 25 type value name acc_norm 63.74 normalized accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkForest-20B-v2.0 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args HellaSwag (10-Shot) hellaswag validation num_few_shot 10 type value name acc_norm 86.32 normalized accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkForest-20B-v2.0 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args MMLU (5-Shot) cais/mmlu all test num_few_shot 5 type value name acc 59.79 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkForest-20B-v2.0 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args TruthfulQA (0-shot) truthful_qa multiple_choice validation num_few_shot 0 type value mc2 56.14 url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkForest-20B-v2.0 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args Winogrande (5-shot) winogrande winogrande_xl validation num_few_shot 5 type value name acc 77.9 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkForest-20B-v2.0 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args GSM8k (5-shot) gsm8k main test num_few_shot 5 type value name acc 23.28 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkForest-20B-v2.0 Open LLM Leaderboard

DarkForest 20B v2.0

Model Details

• To create this model two step procedure was used. First a new 20B model was created using microsoft/Orca-2-13b and KoboldAI/LLaMA2-13B-Erebus-v3 , deatils of the merge in darkforest_v2_step1.yml
• then jebcarter/psyonic-cetacean-20B
• and TeeZee/BigMaid-20B-v1.0 was used to produce the final model, merge config in darkforest_v2_step2.yml
• The resulting model has approximately 20 billion parameters.

Warning: This model can produce NSFW content!

Results

• main difference to v1.0 - model has much better sense of humor.
• produces SFW nad NSFW content without issues, switches context seamlessly.
• good at following instructions.
• good at tracking multiple characters in one scene.
• very creative, scenarios produced are mature and complicated, model doesn't shy from writing about PTSD, mental issues or complicated relationships.
• NSFW output is more creative and suprising than typical limaRP output.
• definitely for mature audiences, not only because of vivid NSFW content but also because of overall maturity of stories it produces.
• This is NOT Harry Potter level storytelling.

All comments are greatly appreciated, download, test and if you appreciate my work, consider buying me my fuel:

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	61.19
AI2 Reasoning Challenge (25-Shot)	63.74
HellaSwag (10-Shot)	86.32
MMLU (5-Shot)	59.79
TruthfulQA (0-shot)	56.14
Winogrande (5-shot)	77.90
GSM8k (5-shot)	23.28