74 lines
7.1 KiB
Markdown
74 lines
7.1 KiB
Markdown
---
|
|
language:
|
|
- en
|
|
tags:
|
|
- facebook
|
|
- meta
|
|
- pytorch
|
|
- llama
|
|
model_name: Llama 13B
|
|
inference: false
|
|
model_creator: Meta Llama
|
|
model_type: llama
|
|
pipeline_tag: text-generation
|
|
prompt_template: '[INST] <<SYS>>
|
|
|
|
You are a helpful, respectful and honest assistant. Always answer as helpfully as
|
|
possible, while being safe. Your answers should not include any harmful, unethical,
|
|
racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses
|
|
are socially unbiased and positive in nature. If a question does not make any sense,
|
|
or is not factually coherent, explain why instead of answering something not correct.
|
|
If you don''t know the answer to a question, please don''t share false information.
|
|
|
|
<</SYS>>
|
|
|
|
{prompt}[/INST]
|
|
|
|
'
|
|
quantized_by: PhilLovesCats
|
|
---
|
|
|
|
# LLaMA 1 13B - GGUF
|
|
- Model creator: [Meta Llama](https://huggingface.co/meta-llama)
|
|
- **The 'Provided files'-section layout, as well as the metadata layout are heavily inspired from [TheBloke](https://huggingface.co/TheBloke)**
|
|
- **The 'License'-section, aswell as the 'LICENSE'-file are burrowed by [huggyllama](https://huggingface.co/huggyllama)**
|
|
|
|
Please tell me when i have violated your copyright, so that i can remove the belonging section!
|
|
|
|
|
|
## Description
|
|
Due the fact that the old LLaMA (1) models are in a deprecated format and really hard to use with modern tools, i converted them using the [transformers](https://huggingface.co/docs/transformers/index) library's `convert_llama_weights_to_hf.py` script to convert them into a [PyTorch](https://pytorch.org/get-started/locally/) model.
|
|
|
|
Further i used the `convert_hf_to_gguf.py` script from [llama.cpp](https://github.com/ggml-org/llama.cpp/tree/master) and to bring them into the well-known GGUF file format. Afterwards, i've build llama.cpp and ran `llama-quantize.exe` to downscale the model files.
|
|
|
|
|
|
## Provided files
|
|
|
|
The following table only lists the quantized (`.gguf`) files (without the PyTorch (`.bin`) ones).
|
|
|
|
| Name | Quant method | Bits | Size | MD5-Hash | Use case |
|
|
|---------------------------------------------------------------------------------------------------------------------|--------------|------|----------|----------------------------------|---------------------------------------------------------------------------|
|
|
| [llama-1-13b.TQ1_0.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.TQ1_0.gguf) | TQ1_0 | 1 | 3.10 GB | BB2D9B5BEB7D980AA1EB94B625F816D9 | smallest, significant quality loss - not recommended for most purposes |
|
|
| [llama-1-13b.TQ2_0.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.TQ2_0.gguf) | TQ2_0 | 2 | 3.65 GB | 74D4D83AB488334FAD875DD97EBCB5FB | very small, significant quality loss - not recommended for most purposes |
|
|
| [llama-1-13b.Q2_K.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.Q2_K.gguf) | Q2_K | 2 | 4.52 GB | 6E0108775665BBA35BE4999A3DA83B37 | very small, significant quality loss - not recommended for most purposes |
|
|
| [llama-1-13b.Q3_K_S.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.Q3_K_S.gguf) | Q3_K_S | 3 | 5.26 GB | 73D94231E96AA3E7F37BB766D73FF182 | very small, high quality loss |
|
|
| [llama-1-13b.Q3_K_M.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.Q3_K_M.gguf) | Q3_K_M | 3 | 5.90 GB | 8504BFC10C64316F7BAB09291A16771C | very small, high quality loss |
|
|
| [llama-1-13b.Q3_K_L.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.Q3_K_L.gguf) | Q3_K_L | 3 | 6.45 GB | 5FE671BF75A5BEAA74090F49267E3F06 | small, substantial quality loss |
|
|
| [llama-1-13b.Q4_0.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.Q4_0.gguf) | Q4_0 | 4 | 6.85 GB | CE4B66B8D4F14B113F81792A25646017 | legacy; small, very high quality loss - prefer using Q3_K_M |
|
|
| [llama-1-13b.Q4_K_S.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.Q4_K_S.gguf) | Q4_K_S | 4 | 6.91 GB | 44EE8F65AB911C9C83658BE9EC00C52F | small, greater quality loss |
|
|
| [llama-1-13b.Q4_K_M.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.Q4_K_M.gguf) | Q4_K_M | 4 | 7.32 GB | 32A087FEAECB1DC6001E0C9AC4A011DB | medium, balanced quality - recommended |
|
|
| [llama-1-13b.Q5_0.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.Q5_0.gguf) | Q5_0 | 5 | 8.35 GB | E52DFB01E0F7D93E8A69A062EAF0D476 | legacy; medium, balanced quality - prefer using Q4_K_M |
|
|
| [llama-1-13b.Q5_K_S.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.Q5_K_S.gguf) | Q5_K_S | 5 | 8.35 GB | 49ADA56DE54587C08BBE0703A47CEA0C | large, low quality loss - recommended |
|
|
| [llama-1-13b.Q5_K_M.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.Q5_K_M.gguf) | Q5_K_M | 5 | 8.59 GB | 5B96E6112A19E65C554333FD127AD6C3 | large, very low quality loss - recommended |
|
|
| [llama-1-13b.Q6_K.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.Q6_K.gguf) | Q6_K | 6 | 9.94 GB | A8CEF707AD9B7BD30547CF8BE197AE33 | very large, extremely low quality loss |
|
|
| [llama-1-13b.Q8_0.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.Q8_0.gguf) | Q8_0 | 8 | 12.80 GB | 4B08E60E0F3CF1EA95111855FCA8FFF9 | very large, extremely low quality loss - not recommended |
|
|
| [llama-1-13b.F16.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.F16.gguf) | F16 | 16 | 24.20 GB | 15C88027F1ED223D05D94D26CD59EAF7 | original, no quality loss - not recommended |
|
|
| [llama-1-13b.BF16.gguf](https://huggingface.co/PhilLovesCats/LLaMA-1-13B-GGUF/blob/main/llama-1-13b.BF16.gguf) | BF16 | 16 | 24.20 GB | 656E54161D5AF6A5D4FFB4C2E65B3ED7 | original, no quality loss - not recommended |
|
|
|
|
**Notice:** Hashes were calculated with `Get-FileHash .\FILENAME -Algorithm MD5` on Microsoft Windows 11 64-bit (x86_x64).
|
|
|
|
## License
|
|
|
|
This model is under a non-commercial license (see the LICENSE file). You should only use this repository if you have been granted access to the model by filling out [this form](https://docs.google.com/forms/d/e/1FAIpQLSfqNECQnMkycAp2jP4Z9TFX0cGR4uf13b_fBxjY_OjhJILlKGA/viewform) but either lost your copy of the weights or got some trouble converting them to the Transformers format.
|
|
|