language, tags, model_name, inference, model_creator, model_type, pipeline_tag, prompt_template, quantized_by

language	tags	model_name	inference	model_creator	model_type	pipeline_tag	prompt_template	quantized_by
en	facebook meta pytorch llama	Llama 13B	false	Meta Llama	llama	text-generation	[INST] <<SYS>> You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature. If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information. <</SYS>> {prompt}[/INST]	PhilLovesCats

LLaMA 1 13B - GGUF

• Model creator: Meta Llama
• The 'Provided files'-section layout, as well as the metadata layout are heavily inspired from TheBloke
• The 'License'-section, aswell as the 'LICENSE'-file are burrowed by huggyllama

Please tell me when i have violated your copyright, so that i can remove the belonging section!

Description

Due the fact that the old LLaMA (1) models are in a deprecated format and really hard to use with modern tools, i converted them using the transformers library's convert_llama_weights_to_hf.py script to convert them into a PyTorch model.

Further i used the convert_hf_to_gguf.py script from llama.cpp and to bring them into the well-known GGUF file format. Afterwards, i've build llama.cpp and ran llama-quantize.exe to downscale the model files.

Provided files

The following table only lists the quantized (.gguf) files (without the PyTorch (.bin) ones).

Name	Quant method	Bits	Size	MD5-Hash	Use case
llama-1-13b.TQ1_0.gguf	TQ1_0	1	3.10 GB	BB2D9B5BEB7D980AA1EB94B625F816D9	smallest, significant quality loss - not recommended for most purposes
llama-1-13b.TQ2_0.gguf	TQ2_0	2	3.65 GB	74D4D83AB488334FAD875DD97EBCB5FB	very small, significant quality loss - not recommended for most purposes
llama-1-13b.Q2_K.gguf	Q2_K	2	4.52 GB	6E0108775665BBA35BE4999A3DA83B37	very small, significant quality loss - not recommended for most purposes
llama-1-13b.Q3_K_S.gguf	Q3_K_S	3	5.26 GB	73D94231E96AA3E7F37BB766D73FF182	very small, high quality loss
llama-1-13b.Q3_K_M.gguf	Q3_K_M	3	5.90 GB	8504BFC10C64316F7BAB09291A16771C	very small, high quality loss
llama-1-13b.Q3_K_L.gguf	Q3_K_L	3	6.45 GB	5FE671BF75A5BEAA74090F49267E3F06	small, substantial quality loss
llama-1-13b.Q4_0.gguf	Q4_0	4	6.85 GB	CE4B66B8D4F14B113F81792A25646017	legacy; small, very high quality loss - prefer using Q3_K_M
llama-1-13b.Q4_K_S.gguf	Q4_K_S	4	6.91 GB	44EE8F65AB911C9C83658BE9EC00C52F	small, greater quality loss
llama-1-13b.Q4_K_M.gguf	Q4_K_M	4	7.32 GB	32A087FEAECB1DC6001E0C9AC4A011DB	medium, balanced quality - recommended
llama-1-13b.Q5_0.gguf	Q5_0	5	8.35 GB	E52DFB01E0F7D93E8A69A062EAF0D476	legacy; medium, balanced quality - prefer using Q4_K_M
llama-1-13b.Q5_K_S.gguf	Q5_K_S	5	8.35 GB	49ADA56DE54587C08BBE0703A47CEA0C	large, low quality loss - recommended
llama-1-13b.Q5_K_M.gguf	Q5_K_M	5	8.59 GB	5B96E6112A19E65C554333FD127AD6C3	large, very low quality loss - recommended
llama-1-13b.Q6_K.gguf	Q6_K	6	9.94 GB	A8CEF707AD9B7BD30547CF8BE197AE33	very large, extremely low quality loss
llama-1-13b.Q8_0.gguf	Q8_0	8	12.80 GB	4B08E60E0F3CF1EA95111855FCA8FFF9	very large, extremely low quality loss - not recommended
llama-1-13b.F16.gguf	F16	16	24.20 GB	15C88027F1ED223D05D94D26CD59EAF7	original, no quality loss - not recommended
llama-1-13b.BF16.gguf	BF16	16	24.20 GB	656E54161D5AF6A5D4FFB4C2E65B3ED7	original, no quality loss - not recommended

Notice: Hashes were calculated with Get-FileHash .\FILENAME -Algorithm MD5 on Microsoft Windows 11 64-bit (x86_x64).

License

This model is under a non-commercial license (see the LICENSE file). You should only use this repository if you have been granted access to the model by filling out this form but either lost your copy of the weights or got some trouble converting them to the Transformers format.