509 lines
7.8 KiB
Markdown
509 lines
7.8 KiB
Markdown
---
|
||
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
|
||
inference: false
|
||
pipeline_tag: text-generation
|
||
language:
|
||
- en
|
||
license: other
|
||
license_name: llama3.1
|
||
license_link: https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/blob/main/LICENSE
|
||
model_creator: meta-llama
|
||
model_name: Meta-Llama-3.1-8B-Instruct
|
||
model_type: llama
|
||
tags:
|
||
- facebook
|
||
- meta
|
||
- pytorch
|
||
- llama
|
||
- llama-3
|
||
- llama-3.1
|
||
quantized_by: brittlewis12
|
||
|
||
---
|
||
|
||
# Llama 3.1 8B Instruct GGUF
|
||
|
||
** *Updated as of 2024-07-27* **
|
||
|
||
**Original model**: [Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)
|
||
|
||
**Model creator**: [Meta](https://huggingface.co/meta-llama)
|
||
|
||
> The Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction tuned generative models in 8B, 70B and 405B sizes (text in/text out). The Llama 3.1 instruction tuned text only models (8B, 70B, 405B) are optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.
|
||
|
||
This repo contains GGUF format model files for Meta’s Llama 3.1 8B Instruct,
|
||
**updated as of 2024-07-27** to incorporate [long context improvements](https://github.com/ggerganov/llama.cpp/pull/8676), as well as changes to the huggingface model itself.
|
||
|
||
Learn more on Meta’s [Llama 3.1 page](https://llama.meta.com).
|
||
|
||
### What is GGUF?
|
||
|
||
GGUF is a file format for representing AI models. It is the third version of the format,
|
||
introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp.
|
||
Converted with llama.cpp build 3472 (revision [b5e9546](https://github.com/ggerganov/llama.cpp/commits/b5e95468b1676e1e5c9d80d1eeeb26f542a38f42)),
|
||
using [autogguf](https://github.com/brittlewis12/autogguf).
|
||
|
||
### Prompt template
|
||
|
||
```
|
||
<|start_header_id|>system<|end_header_id|>
|
||
|
||
{{system_prompt}}<|eot_id|><|start_header_id|>user<|end_header_id|>
|
||
|
||
{{prompt}}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
|
||
|
||
|
||
```
|
||
|
||
---
|
||
|
||
## Download & run with [cnvrs](https://twitter.com/cnvrsai) on iPhone, iPad, and Mac!
|
||
|
||
|
||
|
||
[cnvrs](https://testflight.apple.com/join/sFWReS7K) is the best app for private, local AI on your device:
|
||
- create & save **Characters** with custom system prompts & temperature settings
|
||
- download and experiment with any **GGUF model** you can [find on HuggingFace](https://huggingface.co/models?library=gguf)!
|
||
- make it your own with custom **Theme colors**
|
||
- powered by Metal ⚡️ & [Llama.cpp](https://github.com/ggerganov/llama.cpp), with **haptics** during response streaming!
|
||
- **try it out** yourself today, on [Testflight](https://testflight.apple.com/join/sFWReS7K)!
|
||
- follow [cnvrs on twitter](https://twitter.com/cnvrsai) to stay up to date
|
||
|
||
---
|
||
|
||
## Original Model Evaluation
|
||
|
||
<table>
|
||
<tr>
|
||
<td><strong>Category</strong>
|
||
</td>
|
||
<td><strong>Benchmark</strong>
|
||
</td>
|
||
<td><strong># Shots</strong>
|
||
</td>
|
||
<td><strong>Metric</strong>
|
||
</td>
|
||
<td><strong>Llama 3 8B Instruct</strong>
|
||
</td>
|
||
<td><strong>Llama 3.1 8B Instruct</strong>
|
||
</td>
|
||
<td><strong>Llama 3 70B Instruct</strong>
|
||
</td>
|
||
<td><strong>Llama 3.1 70B Instruct</strong>
|
||
</td>
|
||
<td><strong>Llama 3.1 405B Instruct</strong>
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td rowspan="4" >General
|
||
</td>
|
||
<td>MMLU
|
||
</td>
|
||
<td>5
|
||
</td>
|
||
<td>macro_avg/acc
|
||
</td>
|
||
<td>68.5
|
||
</td>
|
||
<td>69.4
|
||
</td>
|
||
<td>82.0
|
||
</td>
|
||
<td>83.6
|
||
</td>
|
||
<td>87.3
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>MMLU (CoT)
|
||
</td>
|
||
<td>0
|
||
</td>
|
||
<td>macro_avg/acc
|
||
</td>
|
||
<td>65.3
|
||
</td>
|
||
<td>73.0
|
||
</td>
|
||
<td>80.9
|
||
</td>
|
||
<td>86.0
|
||
</td>
|
||
<td>88.6
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>MMLU-Pro (CoT)
|
||
</td>
|
||
<td>5
|
||
</td>
|
||
<td>micro_avg/acc_char
|
||
</td>
|
||
<td>45.5
|
||
</td>
|
||
<td>48.3
|
||
</td>
|
||
<td>63.4
|
||
</td>
|
||
<td>66.4
|
||
</td>
|
||
<td>73.3
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>IFEval
|
||
</td>
|
||
<td>
|
||
</td>
|
||
<td>
|
||
</td>
|
||
<td>76.8
|
||
</td>
|
||
<td>80.4
|
||
</td>
|
||
<td>82.9
|
||
</td>
|
||
<td>87.5
|
||
</td>
|
||
<td>88.6
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td rowspan="2" >Reasoning
|
||
</td>
|
||
<td>ARC-C
|
||
</td>
|
||
<td>0
|
||
</td>
|
||
<td>acc
|
||
</td>
|
||
<td>82.4
|
||
</td>
|
||
<td>83.4
|
||
</td>
|
||
<td>94.4
|
||
</td>
|
||
<td>94.8
|
||
</td>
|
||
<td>96.9
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>GPQA
|
||
</td>
|
||
<td>0
|
||
</td>
|
||
<td>em
|
||
</td>
|
||
<td>34.6
|
||
</td>
|
||
<td>30.4
|
||
</td>
|
||
<td>39.5
|
||
</td>
|
||
<td>41.7
|
||
</td>
|
||
<td>50.7
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td rowspan="4" >Code
|
||
</td>
|
||
<td>HumanEval
|
||
</td>
|
||
<td>0
|
||
</td>
|
||
<td>pass@1
|
||
</td>
|
||
<td>60.4
|
||
</td>
|
||
<td>72.6
|
||
</td>
|
||
<td>81.7
|
||
</td>
|
||
<td>80.5
|
||
</td>
|
||
<td>89.0
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>MBPP ++ base version
|
||
</td>
|
||
<td>0
|
||
</td>
|
||
<td>pass@1
|
||
</td>
|
||
<td>70.6
|
||
</td>
|
||
<td>72.8
|
||
</td>
|
||
<td>82.5
|
||
</td>
|
||
<td>86.0
|
||
</td>
|
||
<td>88.6
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>Multipl-E HumanEval
|
||
</td>
|
||
<td>0
|
||
</td>
|
||
<td>pass@1
|
||
</td>
|
||
<td>-
|
||
</td>
|
||
<td>50.8
|
||
</td>
|
||
<td>-
|
||
</td>
|
||
<td>65.5
|
||
</td>
|
||
<td>75.2
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>Multipl-E MBPP
|
||
</td>
|
||
<td>0
|
||
</td>
|
||
<td>pass@1
|
||
</td>
|
||
<td>-
|
||
</td>
|
||
<td>52.4
|
||
</td>
|
||
<td>-
|
||
</td>
|
||
<td>62.0
|
||
</td>
|
||
<td>65.7
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td rowspan="2" >Math
|
||
</td>
|
||
<td>GSM-8K (CoT)
|
||
</td>
|
||
<td>8
|
||
</td>
|
||
<td>em_maj1@1
|
||
</td>
|
||
<td>80.6
|
||
</td>
|
||
<td>84.5
|
||
</td>
|
||
<td>93.0
|
||
</td>
|
||
<td>95.1
|
||
</td>
|
||
<td>96.8
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>MATH (CoT)
|
||
</td>
|
||
<td>0
|
||
</td>
|
||
<td>final_em
|
||
</td>
|
||
<td>29.1
|
||
</td>
|
||
<td>51.9
|
||
</td>
|
||
<td>51.0
|
||
</td>
|
||
<td>68.0
|
||
</td>
|
||
<td>73.8
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td rowspan="4" >Tool Use
|
||
</td>
|
||
<td>API-Bank
|
||
</td>
|
||
<td>0
|
||
</td>
|
||
<td>acc
|
||
</td>
|
||
<td>48.3
|
||
</td>
|
||
<td>82.6
|
||
</td>
|
||
<td>85.1
|
||
</td>
|
||
<td>90.0
|
||
</td>
|
||
<td>92.0
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>BFCL
|
||
</td>
|
||
<td>0
|
||
</td>
|
||
<td>acc
|
||
</td>
|
||
<td>60.3
|
||
</td>
|
||
<td>76.1
|
||
</td>
|
||
<td>83.0
|
||
</td>
|
||
<td>84.8
|
||
</td>
|
||
<td>88.5
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>Gorilla Benchmark API Bench
|
||
</td>
|
||
<td>0
|
||
</td>
|
||
<td>acc
|
||
</td>
|
||
<td>1.7
|
||
</td>
|
||
<td>8.2
|
||
</td>
|
||
<td>14.7
|
||
</td>
|
||
<td>29.7
|
||
</td>
|
||
<td>35.3
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>Nexus (0-shot)
|
||
</td>
|
||
<td>0
|
||
</td>
|
||
<td>macro_avg/acc
|
||
</td>
|
||
<td>18.1
|
||
</td>
|
||
<td>38.5
|
||
</td>
|
||
<td>47.8
|
||
</td>
|
||
<td>56.7
|
||
</td>
|
||
<td>58.7
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>Multilingual
|
||
</td>
|
||
<td>Multilingual MGSM (CoT)
|
||
</td>
|
||
<td>0
|
||
</td>
|
||
<td>em
|
||
</td>
|
||
<td>-
|
||
</td>
|
||
<td>68.9
|
||
</td>
|
||
<td>-
|
||
</td>
|
||
<td>86.9
|
||
</td>
|
||
<td>91.6
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
#### Multilingual benchmarks
|
||
|
||
<table>
|
||
<tr>
|
||
<td><strong>Category</strong>
|
||
</td>
|
||
<td><strong>Benchmark</strong>
|
||
</td>
|
||
<td><strong>Language</strong>
|
||
</td>
|
||
<td><strong>Llama 3.1 8B</strong>
|
||
</td>
|
||
<td><strong>Llama 3.1 70B</strong>
|
||
</td>
|
||
<td><strong>Llama 3.1 405B</strong>
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td rowspan="9" ><strong>General</strong>
|
||
</td>
|
||
<td rowspan="9" ><strong>MMLU (5-shot, macro_avg/acc)</strong>
|
||
</td>
|
||
<td>Portuguese
|
||
</td>
|
||
<td>62.12
|
||
</td>
|
||
<td>80.13
|
||
</td>
|
||
<td>84.95
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>Spanish
|
||
</td>
|
||
<td>62.45
|
||
</td>
|
||
<td>80.05
|
||
</td>
|
||
<td>85.08
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>Italian
|
||
</td>
|
||
<td>61.63
|
||
</td>
|
||
<td>80.4
|
||
</td>
|
||
<td>85.04
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>German
|
||
</td>
|
||
<td>60.59
|
||
</td>
|
||
<td>79.27
|
||
</td>
|
||
<td>84.36
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>French
|
||
</td>
|
||
<td>62.34
|
||
</td>
|
||
<td>79.82
|
||
</td>
|
||
<td>84.66
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>Hindi
|
||
</td>
|
||
<td>50.88
|
||
</td>
|
||
<td>74.52
|
||
</td>
|
||
<td>80.31
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td>Thai
|
||
</td>
|
||
<td>50.32
|
||
</td>
|
||
<td>72.95
|
||
</td>
|
||
<td>78.21
|
||
</td>
|
||
</tr>
|
||
</table>
|