brittlewis12/Meta-Llama-3.1-8B-Instruct-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: brittlewis12/Meta-Llama-3.1-8B-Instruct-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-25 12:15:26 +08:00
commit 646f8a2fa5
27 changed files with 643 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
.gitattributes vendored Normal file
View File

@@ -0,0 +1,60 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
Meta-Llama-3.1-8B-Instruct.imatrix filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
meta-llama-3.1-8b-instruct.fp16.gguf filter=lfs diff=lfs merge=lfs -text

3
Meta-Llama-3.1-8B-Instruct.imatrix Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5fec733e485da89d0e41824f198e48bcba4d9e75d6f2192ddb28ec5df342453f
size 4988151

508
README.md Normal file
View File

@@ -0,0 +1,508 @@
---
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
inference: false
pipeline_tag: text-generation
language:
- en
license: other
license_name: llama3.1
license_link: https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/blob/main/LICENSE
model_creator: meta-llama
model_name: Meta-Llama-3.1-8B-Instruct
model_type: llama
tags:
- facebook
- meta
- pytorch
- llama
- llama-3
- llama-3.1
quantized_by: brittlewis12
---
# Llama 3.1 8B Instruct GGUF
** *Updated as of 2024-07-27* **
**Original model**: [Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)
**Model creator**: [Meta](https://huggingface.co/meta-llama)
> The Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction tuned generative models in 8B, 70B and 405B sizes (text in/text out). The Llama 3.1 instruction tuned text only models (8B, 70B, 405B) are optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.
This repo contains GGUF format model files for Meta’s Llama 3.1 8B Instruct,
**updated as of 2024-07-27** to incorporate [long context improvements](https://github.com/ggerganov/llama.cpp/pull/8676), as well as changes to the huggingface model itself.
Learn more on Meta’s [Llama 3.1 page](https://llama.meta.com).
### What is GGUF?
GGUF is a file format for representing AI models. It is the third version of the format,
introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp.
Converted with llama.cpp build 3472 (revision [b5e9546](https://github.com/ggerganov/llama.cpp/commits/b5e95468b1676e1e5c9d80d1eeeb26f542a38f42)),
using [autogguf](https://github.com/brittlewis12/autogguf).
### Prompt template
```
<|start_header_id|>system<|end_header_id|>
{{system_prompt}}<|eot_id|><|start_header_id|>user<|end_header_id|>
{{prompt}}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
```
---
## Download & run with [cnvrs](https://twitter.com/cnvrsai) on iPhone, iPad, and Mac!
![cnvrs.ai](https://pbs.twimg.com/profile_images/1744049151241797632/0mIP-P9e_400x400.jpg)
[cnvrs](https://testflight.apple.com/join/sFWReS7K) is the best app for private, local AI on your device:
- create & save **Characters** with custom system prompts & temperature settings
- download and experiment with any **GGUF model** you can [find on HuggingFace](https://huggingface.co/models?library=gguf)!
- make it your own with custom **Theme colors**
- powered by Metal ⚡️ & [Llama.cpp](https://github.com/ggerganov/llama.cpp), with **haptics** during response streaming!
- **try it out** yourself today, on [Testflight](https://testflight.apple.com/join/sFWReS7K)!
- follow [cnvrs on twitter](https://twitter.com/cnvrsai) to stay up to date
---
## Original Model Evaluation
<table>
<tr>
<td><strong>Category</strong>
</td>
<td><strong>Benchmark</strong>
</td>
<td><strong># Shots</strong>
</td>
<td><strong>Metric</strong>
</td>
<td><strong>Llama 3 8B Instruct</strong>
</td>
<td><strong>Llama 3.1 8B Instruct</strong>
</td>
<td><strong>Llama 3 70B Instruct</strong>
</td>
<td><strong>Llama 3.1 70B Instruct</strong>
</td>
<td><strong>Llama 3.1 405B Instruct</strong>
</td>
</tr>
<tr>
<td rowspan="4" >General
</td>
<td>MMLU
</td>
<td>5
</td>
<td>macro_avg/acc
</td>
<td>68.5
</td>
<td>69.4
</td>
<td>82.0
</td>
<td>83.6
</td>
<td>87.3
</td>
</tr>
<tr>
<td>MMLU (CoT)
</td>
<td>0
</td>
<td>macro_avg/acc
</td>
<td>65.3
</td>
<td>73.0
</td>
<td>80.9
</td>
<td>86.0
</td>
<td>88.6
</td>
</tr>
<tr>
<td>MMLU-Pro (CoT)
</td>
<td>5
</td>
<td>micro_avg/acc_char
</td>
<td>45.5
</td>
<td>48.3
</td>
<td>63.4
</td>
<td>66.4
</td>
<td>73.3
</td>
</tr>
<tr>
<td>IFEval
</td>
<td>
</td>
<td>
</td>
<td>76.8
</td>
<td>80.4
</td>
<td>82.9
</td>
<td>87.5
</td>
<td>88.6
</td>
</tr>
<tr>
<td rowspan="2" >Reasoning
</td>
<td>ARC-C
</td>
<td>0
</td>
<td>acc
</td>
<td>82.4
</td>
<td>83.4
</td>
<td>94.4
</td>
<td>94.8
</td>
<td>96.9
</td>
</tr>
<tr>
<td>GPQA
</td>
<td>0
</td>
<td>em
</td>
<td>34.6
</td>
<td>30.4
</td>
<td>39.5
</td>
<td>41.7
</td>
<td>50.7
</td>
</tr>
<tr>
<td rowspan="4" >Code
</td>
<td>HumanEval
</td>
<td>0
</td>
<td>pass@1
</td>
<td>60.4
</td>
<td>72.6
</td>
<td>81.7
</td>
<td>80.5
</td>
<td>89.0
</td>
</tr>
<tr>
<td>MBPP ++ base version
</td>
<td>0
</td>
<td>pass@1
</td>
<td>70.6
</td>
<td>72.8
</td>
<td>82.5
</td>
<td>86.0
</td>
<td>88.6
</td>
</tr>
<tr>
<td>Multipl-E HumanEval
</td>
<td>0
</td>
<td>pass@1
</td>
<td>-
</td>
<td>50.8
</td>
<td>-
</td>
<td>65.5
</td>
<td>75.2
</td>
</tr>
<tr>
<td>Multipl-E MBPP
</td>
<td>0
</td>
<td>pass@1
</td>
<td>-
</td>
<td>52.4
</td>
<td>-
</td>
<td>62.0
</td>
<td>65.7
</td>
</tr>
<tr>
<td rowspan="2" >Math
</td>
<td>GSM-8K (CoT)
</td>
<td>8
</td>
<td>em_maj1@1
</td>
<td>80.6
</td>
<td>84.5
</td>
<td>93.0
</td>
<td>95.1
</td>
<td>96.8
</td>
</tr>
<tr>
<td>MATH (CoT)
</td>
<td>0
</td>
<td>final_em
</td>
<td>29.1
</td>
<td>51.9
</td>
<td>51.0
</td>
<td>68.0
</td>
<td>73.8
</td>
</tr>
<tr>
<td rowspan="4" >Tool Use
</td>
<td>API-Bank
</td>
<td>0
</td>
<td>acc
</td>
<td>48.3
</td>
<td>82.6
</td>
<td>85.1
</td>
<td>90.0
</td>
<td>92.0
</td>
</tr>
<tr>
<td>BFCL
</td>
<td>0
</td>
<td>acc
</td>
<td>60.3
</td>
<td>76.1
</td>
<td>83.0
</td>
<td>84.8
</td>
<td>88.5
</td>
</tr>
<tr>
<td>Gorilla Benchmark API Bench
</td>
<td>0
</td>
<td>acc
</td>
<td>1.7
</td>
<td>8.2
</td>
<td>14.7
</td>
<td>29.7
</td>
<td>35.3
</td>
</tr>
<tr>
<td>Nexus (0-shot)
</td>
<td>0
</td>
<td>macro_avg/acc
</td>
<td>18.1
</td>
<td>38.5
</td>
<td>47.8
</td>
<td>56.7
</td>
<td>58.7
</td>
</tr>
<tr>
<td>Multilingual
</td>
<td>Multilingual MGSM (CoT)
</td>
<td>0
</td>
<td>em
</td>
<td>-
</td>
<td>68.9
</td>
<td>-
</td>
<td>86.9
</td>
<td>91.6
</td>
</tr>
</table>
#### Multilingual benchmarks
<table>
<tr>
<td><strong>Category</strong>
</td>
<td><strong>Benchmark</strong>
</td>
<td><strong>Language</strong>
</td>
<td><strong>Llama 3.1 8B</strong>
</td>
<td><strong>Llama 3.1 70B</strong>
</td>
<td><strong>Llama 3.1 405B</strong>
</td>
</tr>
<tr>
<td rowspan="9" ><strong>General</strong>
</td>
<td rowspan="9" ><strong>MMLU (5-shot, macro_avg/acc)</strong>
</td>
<td>Portuguese
</td>
<td>62.12
</td>
<td>80.13
</td>
<td>84.95
</td>
</tr>
<tr>
<td>Spanish
</td>
<td>62.45
</td>
<td>80.05
</td>
<td>85.08
</td>
</tr>
<tr>
<td>Italian
</td>
<td>61.63
</td>
<td>80.4
</td>
<td>85.04
</td>
</tr>
<tr>
<td>German
</td>
<td>60.59
</td>
<td>79.27
</td>
<td>84.36
</td>
</tr>
<tr>
<td>French
</td>
<td>62.34
</td>
<td>79.82
</td>
<td>84.66
</td>
</tr>
<tr>
<td>Hindi
</td>
<td>50.88
</td>
<td>74.52
</td>
<td>80.31
</td>
</tr>
<tr>
<td>Thai
</td>
<td>50.32
</td>
<td>72.95
</td>
<td>78.21
</td>
</tr>
</table>

3
meta-llama-3.1-8b-instruct.IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ddb4ddb9ca755f86b0b9e7076436429017d6a6a11ea521e70f9850249cf3fda0
size 2161972416

3
meta-llama-3.1-8b-instruct.IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1429cc7d7547f107adca1e48b42f35aab5cd0d06993c22f464a5adba0288839d
size 2019628224

3
meta-llama-3.1-8b-instruct.IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3d01c536ea33569094a943dc45536ba0fd14cb4a85c357472867a2080a559531
size 2948281536

3
meta-llama-3.1-8b-instruct.IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7c21b6e1668f000e997ef81645b7609d57cf4d4e9af19891c82cfecce33748f3
size 2758489280

3
meta-llama-3.1-8b-instruct.IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d57ab8d0dbc48ba3f8ad06424aed5813772f3b48560d45981f52166b262f1630
size 2605782208

3
meta-llama-3.1-8b-instruct.IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:929c4848203162f98aad970d5c32b0d50eb6a216c1ee9f2725890c4a1ff0db14
size 2399212736

3
meta-llama-3.1-8b-instruct.IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:07a28e4818d89b1ebb79d0ccf1c2bf7ebea528406ed4553388ffcbdf8bf05d98
size 3784824000

3
meta-llama-3.1-8b-instruct.IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ec2da35288bb17fbfb40f67290bcb3685f3460271d5f11fd322b0cd04fc6ca37
size 3682325696

3
meta-llama-3.1-8b-instruct.IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:86461ed871343578eeb4f5e261e573deac8fa02330a832091ddc1ffdf5229a74
size 3518747840

3
meta-llama-3.1-8b-instruct.IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a3593312865deddc6255047a97751641f8d347b9a5f0044ba2418c379f471dc5
size 3274912960

3
meta-llama-3.1-8b-instruct.IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d8966dab0ef6bb637a1ac855992a892d9018d037deddf2698c122b9824a09014
size 4677989568

3
meta-llama-3.1-8b-instruct.IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:99563886353e8eecfcc1effbfb20669934948fe1ec9d0ef16e8030938210f7d4
size 4447663296

3
meta-llama-3.1-8b-instruct.Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3dab9b5f2958df30064c1ef8d859c45c8d2466d4718b0b65b5920c874232a35c
size 3179131840

3
meta-llama-3.1-8b-instruct.Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a859d2fb26fc0c79f487129c7c7ca2140b98d4f1e3e28f2316e5952a639f54c3
size 2988815552

3
meta-llama-3.1-8b-instruct.Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8b8a277383de7631e4d84e94da73940fb3519ac91cab97b86667c0f92c5ccdc6
size 4321956800

3
meta-llama-3.1-8b-instruct.Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cc82b68601c3e0b928f36dcde0e6b9993366bcc4f14ca6eb23a69dde745d0212
size 4018918336

3
meta-llama-3.1-8b-instruct.Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f0b56c8dc159ed4d70040a785555628b1f126026933d69f605d306ab6f602512
size 3664499648

3
meta-llama-3.1-8b-instruct.Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:606ffa92823cf0563d5247613497cd670cad26019e6fc48e156bd945d7bca50c
size 4920734656

3
meta-llama-3.1-8b-instruct.Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5ceec06d1371f163e32dea6d7a4b41a7f7f880e78f5880e863c088eee969f364
size 4692669376

3
meta-llama-3.1-8b-instruct.Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ea18bb992104097d9d57e96e56c4b17eeb84df7099ebb975bc40cac315278f09
size 5732987840

3
meta-llama-3.1-8b-instruct.Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d7db344eb82766ad257f92f5a9b1f259fdcc25443788213660fc641b54718581
size 5599294400

3
meta-llama-3.1-8b-instruct.Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a5dfbb3a8ce68994e7f5b03ea2b1da585f0766ffc699e752d1a4822fc4150a77
size 6596006848

3
meta-llama-3.1-8b-instruct.Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:610f1831567e040a8240e21e5644eeede5aee46807cab73a59afaa8fd054a72c
size 8540771264

3
meta-llama-3.1-8b-instruct.fp16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a32b1dce9cbb6bcc366a2fa4ce83b5e319b58f0c5cb34f65a57942ab636515f7
size 16068891584