151 lines
4.1 KiB
Markdown
151 lines
4.1 KiB
Markdown
---
|
||
language:
|
||
- en
|
||
license: cc-by-nc-sa-4.0
|
||
tags:
|
||
- merge
|
||
- slerp
|
||
model-index:
|
||
- name: CarbonVillain-en-10.7B-v4
|
||
results:
|
||
- task:
|
||
type: text-generation
|
||
name: Text Generation
|
||
dataset:
|
||
name: AI2 Reasoning Challenge (25-Shot)
|
||
type: ai2_arc
|
||
config: ARC-Challenge
|
||
split: test
|
||
args:
|
||
num_few_shot: 25
|
||
metrics:
|
||
- type: acc_norm
|
||
value: 71.25
|
||
name: normalized accuracy
|
||
source:
|
||
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=jeonsworld/CarbonVillain-en-10.7B-v4
|
||
name: Open LLM Leaderboard
|
||
- task:
|
||
type: text-generation
|
||
name: Text Generation
|
||
dataset:
|
||
name: HellaSwag (10-Shot)
|
||
type: hellaswag
|
||
split: validation
|
||
args:
|
||
num_few_shot: 10
|
||
metrics:
|
||
- type: acc_norm
|
||
value: 88.48
|
||
name: normalized accuracy
|
||
source:
|
||
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=jeonsworld/CarbonVillain-en-10.7B-v4
|
||
name: Open LLM Leaderboard
|
||
- task:
|
||
type: text-generation
|
||
name: Text Generation
|
||
dataset:
|
||
name: MMLU (5-Shot)
|
||
type: cais/mmlu
|
||
config: all
|
||
split: test
|
||
args:
|
||
num_few_shot: 5
|
||
metrics:
|
||
- type: acc
|
||
value: 66.27
|
||
name: accuracy
|
||
source:
|
||
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=jeonsworld/CarbonVillain-en-10.7B-v4
|
||
name: Open LLM Leaderboard
|
||
- task:
|
||
type: text-generation
|
||
name: Text Generation
|
||
dataset:
|
||
name: TruthfulQA (0-shot)
|
||
type: truthful_qa
|
||
config: multiple_choice
|
||
split: validation
|
||
args:
|
||
num_few_shot: 0
|
||
metrics:
|
||
- type: mc2
|
||
value: 71.95
|
||
source:
|
||
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=jeonsworld/CarbonVillain-en-10.7B-v4
|
||
name: Open LLM Leaderboard
|
||
- task:
|
||
type: text-generation
|
||
name: Text Generation
|
||
dataset:
|
||
name: Winogrande (5-shot)
|
||
type: winogrande
|
||
config: winogrande_xl
|
||
split: validation
|
||
args:
|
||
num_few_shot: 5
|
||
metrics:
|
||
- type: acc
|
||
value: 83.58
|
||
name: accuracy
|
||
source:
|
||
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=jeonsworld/CarbonVillain-en-10.7B-v4
|
||
name: Open LLM Leaderboard
|
||
- task:
|
||
type: text-generation
|
||
name: Text Generation
|
||
dataset:
|
||
name: GSM8k (5-shot)
|
||
type: gsm8k
|
||
config: main
|
||
split: test
|
||
args:
|
||
num_few_shot: 5
|
||
metrics:
|
||
- type: acc
|
||
value: 65.58
|
||
name: accuracy
|
||
source:
|
||
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=jeonsworld/CarbonVillain-en-10.7B-v4
|
||
name: Open LLM Leaderboard
|
||
---
|
||
|
||
# CarbonVillain
|
||
**This is a model created without learning to oppose indiscriminate carbon emissions.**
|
||
|
||
This model is an experimental version created using [mergekit](https://github.com/cg123/mergekit).
|
||
- merge models
|
||
- [jeonsworld/CarbonVillain-en-10.7B-v1](https://huggingface.co/jeonsworld/CarbonVillain-en-10.7B-v1)
|
||
- [jeonsworld/CarbonVillain-en-10.7B-v2](https://huggingface.co/jeonsworld/CarbonVillain-en-10.7B-v2)
|
||
- method: slerp
|
||
|
||
*Ranked 1st 2023.12.30*
|
||
|
||
|
||
> thanks to [KyujinHan](https://huggingface.co/kyujinpy), [Yağız Çalık](https://huggingface.co/Weyaxi) and [VAGO solutions](https://huggingface.co/VAGOsolutions)
|
||
|
||
|
||
# Prompt Template(s)
|
||
|
||
```
|
||
### User:
|
||
{user}
|
||
|
||
### Assistant:
|
||
{asistant}
|
||
```
|
||
|
||
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
||
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_jeonsworld__CarbonVillain-en-10.7B-v4)
|
||
|
||
| Metric |Value|
|
||
|---------------------------------|----:|
|
||
|Avg. |74.52|
|
||
|AI2 Reasoning Challenge (25-Shot)|71.25|
|
||
|HellaSwag (10-Shot) |88.48|
|
||
|MMLU (5-Shot) |66.27|
|
||
|TruthfulQA (0-shot) |71.95|
|
||
|Winogrande (5-shot) |83.58|
|
||
|GSM8k (5-shot) |65.58|
|
||
|