35 lines
1.2 KiB
Markdown
35 lines
1.2 KiB
Markdown
---
|
|
license: other
|
|
language:
|
|
- en
|
|
pipeline_tag: text-generation
|
|
inference: false
|
|
tags:
|
|
- transformers
|
|
- gguf
|
|
- imatrix
|
|
- Delexa-Instruct-V0.1-7b
|
|
---
|
|
Quantizations of https://huggingface.co/lex-hue/Delexa-Instruct-V0.1-7b
|
|
|
|
|
|
# From original readme
|
|
|
|
## Delexa-V0.1-Instruct-7b: Our Newest and Best Model Yet!
|
|
|
|
We are excited to announce the release of Delexa-V0.1-Instruct-7b, our newest and best model yet! Delexa-V0.1-Instruct-7b has shown excellent performance on a variety of tasks, and we are confident that it will be a valuable asset to the research community.
|
|
|
|
### Eval Results
|
|
|
|
Delexa-V0.1-Instruct-7b was evaluated on a dataset of question-answer pairs. The model was given a single question and three different answer choices, and it was tasked with selecting the best answer. Delexa-V0.1-Instruct-7b achieved an average score of 8.27 on this task.
|
|
|
|
Here is a table showing the detailed eval results:
|
|
|
|
| Model | Turn 1 | Turn 2 | Average |
|
|
|---|---|---|---|
|
|
| gpt-4 | 8.95625 | 9.0250 | 8.990625 |
|
|
| Delexa-V0.1-Instruct-7b | 8.57500 | 7.9500 | 8.268750 |
|
|
| claude-v1 | 8.15000 | 7.6500 | 7.900000 |
|
|
| gpt-3.5-turbo | 8.07500 | 7.8125 | 7.943750 |
|
|
| vicuna-13b-v1.3 | 6.81250 | 5.9625 | 6.387500 |
|
|
| palm-2-chat-bison-001 | 6.71250 | 6.0875 | 6.400000 | |