commit 4313ded27d4e3192b85af7da9d4e20c9d0801f6e Author: ModelHub XC Date: Sun Sep 6 07:56:16 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: brittlewis12/Meta-Llama-3-8B-Instruct-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..ed07cb8 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,59 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.fp16.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +meta-llama-3-8b-instruct.Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..28b484a --- /dev/null +++ b/README.md @@ -0,0 +1,136 @@ +--- +base_model: meta-llama/Meta-Llama-3-8B-Instruct +inference: false +pipeline_tag: text-generation +language: +- en +license: other +license_name: llama3 +license_link: https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/blob/main/LICENSE +model_creator: meta-llama +model_name: Meta-Llama-3-8B-Instruct +model_type: llama +tags: +- facebook +- meta +- pytorch +- llama +- llama-3 +quantized_by: brittlewis12 + +--- + +# Meta-Llama-3-8B-Instruct GGUF + +**Original model**: [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) + +**Model creator**: [Meta](https://huggingface.co/meta-llama) + +> Meta developed and released the Meta Llama 3 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. The Llama 3 instruction tuned models are optimized for dialogue use cases and outperform many of the available open source chat models on common industry benchmarks. Further, in developing these models, we took great care to optimize helpfulness and safety. + +> Llama 3 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety. + +This repo contains GGUF format model files for Meta’s Llama-3-8B-Instruct, +**updated as of 2024-04-29** to incorporate [tokenization improvements](https://github.com/ggerganov/llama.cpp/pull/6920), as well as previous interventions to handle the `<|eot_id|>` special token as EOS token. + +Learn more on Meta’s [Llama 3 page](https://llama.meta.com/llama3). + +### What is GGUF? + +GGUF is a file format for representing AI models. It is the third version of the format, +introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. +Converted with llama.cpp build 2763 (revision [ffe666](https://github.com/ggerganov/llama.cpp/commits/ffe666572f98a686b17a2cd1dbf4c0a982e5ac0a)), +using [autogguf](https://github.com/brittlewis12/autogguf). + +### Prompt template + +``` +<|start_header_id|>system<|end_header_id|> + +{{system_prompt}}<|eot_id|><|start_header_id|>user<|end_header_id|> + +{{prompt}}<|eot_id|><|start_header_id|>assistant<|end_header_id|> + + +``` + +--- + +## Download & run with [cnvrs](https://twitter.com/cnvrsai) on iPhone, iPad, and Mac! + +![cnvrs.ai](https://pbs.twimg.com/profile_images/1744049151241797632/0mIP-P9e_400x400.jpg) + +[cnvrs](https://testflight.apple.com/join/sFWReS7K) is the best app for private, local AI on your device: +- create & save **Characters** with custom system prompts & temperature settings +- download and experiment with any **GGUF model** you can [find on HuggingFace](https://huggingface.co/models?library=gguf)! +- make it your own with custom **Theme colors** +- powered by Metal ⚡️ & [Llama.cpp](https://github.com/ggerganov/llama.cpp), with **haptics** during response streaming! +- **try it out** yourself today, on [Testflight](https://testflight.apple.com/join/sFWReS7K)! +- follow [cnvrs on twitter](https://twitter.com/cnvrsai) to stay up to date + +--- + +## Original Model Evaluation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Benchmark + Llama 3 8B + Llama 2 7B + Llama 2 13B +
MMLU (5-shot) + 68.4 + 34.1 + 47.8 +
GPQA (0-shot) + 34.2 + 21.7 + 22.3 +
HumanEval (0-shot) + 62.2 + 7.9 + 14.0 +
GSM-8K (8-shot, CoT) + 79.6 + 25.7 + 77.4 +
MATH (4-shot, CoT) + 30.0 + 3.8 + 6.7 +
diff --git a/meta-llama-3-8b-instruct.IQ1_M.gguf b/meta-llama-3-8b-instruct.IQ1_M.gguf new file mode 100644 index 0000000..58aa911 --- /dev/null +++ b/meta-llama-3-8b-instruct.IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cde141c4b3606f95c8fd6fb6da989acfcd3f99f35e754245faad3f708b9ffd70 +size 2162484704 diff --git a/meta-llama-3-8b-instruct.IQ1_S.gguf b/meta-llama-3-8b-instruct.IQ1_S.gguf new file mode 100644 index 0000000..ecc738c --- /dev/null +++ b/meta-llama-3-8b-instruct.IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ecab5634dc449a5e3ae11190fadd5279c02a80c0dd1b1e243fd67f6a192dd7ed +size 2020140512 diff --git a/meta-llama-3-8b-instruct.IQ2_M.gguf b/meta-llama-3-8b-instruct.IQ2_M.gguf new file mode 100644 index 0000000..a6e23d5 --- /dev/null +++ b/meta-llama-3-8b-instruct.IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:45b01cc79f508d521c0a94836dbec0db5bbff0760aab15e0edbfbd1f413907d0 +size 2948793824 diff --git a/meta-llama-3-8b-instruct.IQ2_S.gguf b/meta-llama-3-8b-instruct.IQ2_S.gguf new file mode 100644 index 0000000..52c3a38 --- /dev/null +++ b/meta-llama-3-8b-instruct.IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d99178b6c148047b32dacfd7e77a5339658d5b7e23f443ee9be226a9c259140a +size 2759001568 diff --git a/meta-llama-3-8b-instruct.IQ2_XS.gguf b/meta-llama-3-8b-instruct.IQ2_XS.gguf new file mode 100644 index 0000000..0ee7326 --- /dev/null +++ b/meta-llama-3-8b-instruct.IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dca3ff48676bc7fd4ce69e8b5a26ba8a3ae71dedefd597296e3f62da3995cf4f +size 2606294496 diff --git a/meta-llama-3-8b-instruct.IQ2_XXS.gguf b/meta-llama-3-8b-instruct.IQ2_XXS.gguf new file mode 100644 index 0000000..a48c2e1 --- /dev/null +++ b/meta-llama-3-8b-instruct.IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2b0b171bc3313eebc357e30878584313ba475ecad034845af1c50c3c304a85c4 +size 2399725024 diff --git a/meta-llama-3-8b-instruct.IQ3_M.gguf b/meta-llama-3-8b-instruct.IQ3_M.gguf new file mode 100644 index 0000000..dffa532 --- /dev/null +++ b/meta-llama-3-8b-instruct.IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:140777c14d4ed18eb953747062af754e97fe5aace3dc1c39a700d545904f7d43 +size 3785336288 diff --git a/meta-llama-3-8b-instruct.IQ3_S.gguf b/meta-llama-3-8b-instruct.IQ3_S.gguf new file mode 100644 index 0000000..738a55d --- /dev/null +++ b/meta-llama-3-8b-instruct.IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8950d3841b3b862ab8f0c9590b3c8c15918cc63cef4e5779a66a31b3ed01c61b +size 3682837984 diff --git a/meta-llama-3-8b-instruct.IQ3_XS.gguf b/meta-llama-3-8b-instruct.IQ3_XS.gguf new file mode 100644 index 0000000..b0b210e --- /dev/null +++ b/meta-llama-3-8b-instruct.IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ecf56dd4c61f7e84fc20dfe499e96c63760b62270d089ad3e184b3c7ea0187b1 +size 3519260128 diff --git a/meta-llama-3-8b-instruct.IQ3_XXS.gguf b/meta-llama-3-8b-instruct.IQ3_XXS.gguf new file mode 100644 index 0000000..e56ff16 --- /dev/null +++ b/meta-llama-3-8b-instruct.IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:77fa6537122bf711af6ef0909dfe932dbfcd3d2022abe89665e29efdd2011482 +size 3275425248 diff --git a/meta-llama-3-8b-instruct.IQ4_NL.gguf b/meta-llama-3-8b-instruct.IQ4_NL.gguf new file mode 100644 index 0000000..d24a0ae --- /dev/null +++ b/meta-llama-3-8b-instruct.IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9c49cfd894c2cb66b9e20b47a7ed96ea57a5b086fbf14b05b1fefcfd079d685d +size 4678501856 diff --git a/meta-llama-3-8b-instruct.IQ4_XS.gguf b/meta-llama-3-8b-instruct.IQ4_XS.gguf new file mode 100644 index 0000000..e7490a6 --- /dev/null +++ b/meta-llama-3-8b-instruct.IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:09f1be1d9c427485daa4f6889799aed700aba61d3406ae02bce1a4968426d1ad +size 4448175584 diff --git a/meta-llama-3-8b-instruct.Q2_K.gguf b/meta-llama-3-8b-instruct.Q2_K.gguf new file mode 100644 index 0000000..7d18252 --- /dev/null +++ b/meta-llama-3-8b-instruct.Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8a0af1b5d6613351a1b662f3d585b945141789944209504026a5e4e211757ca0 +size 3179644160 diff --git a/meta-llama-3-8b-instruct.Q2_K_S.gguf b/meta-llama-3-8b-instruct.Q2_K_S.gguf new file mode 100644 index 0000000..cfb3617 --- /dev/null +++ b/meta-llama-3-8b-instruct.Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:928c449f8831f796ff184a0412d1fbec85ffc6ef7328b48e50ad9bd0476ba3a1 +size 2989327840 diff --git a/meta-llama-3-8b-instruct.Q3_K_L.gguf b/meta-llama-3-8b-instruct.Q3_K_L.gguf new file mode 100644 index 0000000..c2a26ca --- /dev/null +++ b/meta-llama-3-8b-instruct.Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2c0563672db237b12a3276ff7a2aef337978b663e52a79c504e92ce10c5a4f2f +size 4322469120 diff --git a/meta-llama-3-8b-instruct.Q3_K_M.gguf b/meta-llama-3-8b-instruct.Q3_K_M.gguf new file mode 100644 index 0000000..81e0030 --- /dev/null +++ b/meta-llama-3-8b-instruct.Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9107ede285a15dd7126a1b6a63cea4deca9df2e8c4a9abba634130b7829c29c2 +size 4019430656 diff --git a/meta-llama-3-8b-instruct.Q3_K_S.gguf b/meta-llama-3-8b-instruct.Q3_K_S.gguf new file mode 100644 index 0000000..18ed4b9 --- /dev/null +++ b/meta-llama-3-8b-instruct.Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e45390b26626a95b9e96b463a244818f9ea9a5ce672b8b179ec819a46cf98fbf +size 3665011968 diff --git a/meta-llama-3-8b-instruct.Q4_K_M.gguf b/meta-llama-3-8b-instruct.Q4_K_M.gguf new file mode 100644 index 0000000..6a3e1dd --- /dev/null +++ b/meta-llama-3-8b-instruct.Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:80b72ca18456cb66672f238c84a7f7b56561c583fef81586acf6959d549281a2 +size 4921246976 diff --git a/meta-llama-3-8b-instruct.Q4_K_S.gguf b/meta-llama-3-8b-instruct.Q4_K_S.gguf new file mode 100644 index 0000000..de9528c --- /dev/null +++ b/meta-llama-3-8b-instruct.Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e15b6178885003c376b9f2fc3d848a279200a058d7468f13e2e4fdf69bffb915 +size 4693181696 diff --git a/meta-llama-3-8b-instruct.Q5_K_M.gguf b/meta-llama-3-8b-instruct.Q5_K_M.gguf new file mode 100644 index 0000000..502ff67 --- /dev/null +++ b/meta-llama-3-8b-instruct.Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2f169b7f83309bd38c3c33df3a8c76be57c781be24b3bafebea52c905b633b34 +size 5733500160 diff --git a/meta-llama-3-8b-instruct.Q5_K_S.gguf b/meta-llama-3-8b-instruct.Q5_K_S.gguf new file mode 100644 index 0000000..d49d742 --- /dev/null +++ b/meta-llama-3-8b-instruct.Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f50580a201f34edd2a0a44ed67aa2940e1518b071d6a90aaefc6131125ab5f53 +size 5599806720 diff --git a/meta-llama-3-8b-instruct.Q6_K.gguf b/meta-llama-3-8b-instruct.Q6_K.gguf new file mode 100644 index 0000000..42ab2f8 --- /dev/null +++ b/meta-llama-3-8b-instruct.Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d7a7c387d6e9dc7634d113c9d711181e520f360455e96cee8c36665262c7d554 +size 6596519168 diff --git a/meta-llama-3-8b-instruct.Q8_0.gguf b/meta-llama-3-8b-instruct.Q8_0.gguf new file mode 100644 index 0000000..f27b6f1 --- /dev/null +++ b/meta-llama-3-8b-instruct.Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d9a1b2567ae72e98194d6aa3776b3471fe13c026e06d0566b1e5d5b41fc9bbcf +size 8541283584 diff --git a/meta-llama-3-8b-instruct.fp16.gguf b/meta-llama-3-8b-instruct.fp16.gguf new file mode 100644 index 0000000..f65bb82 --- /dev/null +++ b/meta-llama-3-8b-instruct.fp16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9bbe82c8d149f218f9c1e4dadd05a792c1f40d10e91ec9740ec059ecc3ef7758 +size 16069403872