aya-23-8B-GGUF/README.md

base_model, extra_gated_fields, extra_gated_prompt, language, library_name, license, quantized_by

base_model	extra_gated_fields	extra_gated_prompt	language	library_name	license	quantized_by
CohereLabs/aya-23-8B	Affiliation Country I agree to use this model for non-commercial use ONLY Name text options type Aruba Afghanistan Angola Anguilla Åland Islands Albania Andorra United Arab Emirates Argentina Armenia American Samoa Antarctica French Southern Territories Antigua and Barbuda Australia Austria Azerbaijan Burundi Belgium Benin Bonaire Sint Eustatius and Saba Burkina Faso Bangladesh Bulgaria Bahrain Bahamas Bosnia and Herzegovina Saint Barthélemy Belarus Belize Bermuda Plurinational State of Bolivia Brazil Barbados Brunei-Darussalam Bhutan Bouvet-Island Botswana Central African Republic Canada Cocos (Keeling) Islands Switzerland Chile China Côte-dIvoire Cameroon Democratic Republic of the Congo Cook Islands Colombia Comoros Cabo Verde Costa Rica Cuba Curaçao Christmas Island Cayman Islands Cyprus Czechia Germany Djibouti Dominica Denmark Dominican Republic Algeria Ecuador Egypt Eritrea Western Sahara Spain Estonia Ethiopia Finland Fiji Falkland Islands (Malvinas) France Faroe Islands Federated States of Micronesia Gabon United Kingdom Georgia Guernsey Ghana Gibraltar Guinea Guadeloupe Gambia Guinea Bissau Equatorial Guinea Greece Grenada Greenland Guatemala French Guiana Guam Guyana Hong Kong Heard Island and McDonald Islands Honduras Croatia Haiti Hungary Indonesia Isle of Man India British Indian Ocean Territory Ireland Islamic Republic of Iran Iraq Iceland Israel Italy Jamaica Jersey Jordan Japan Kazakhstan Kenya Kyrgyzstan Cambodia Kiribati Saint-Kitts-and-Nevis South Korea Kuwait Lao-Peoples-Democratic-Republic Lebanon Liberia Libya Saint-Lucia Liechtenstein Sri Lanka Lesotho Lithuania Luxembourg Latvia Macao Saint Martin (French-part) Morocco Monaco Republic of Moldova Madagascar Maldives Mexico Marshall Islands North Macedonia Mali Malta Myanmar Montenegro Mongolia Northern Mariana Islands Mozambique Mauritania Montserrat Martinique Mauritius Malawi Malaysia Mayotte Namibia New Caledonia Niger Norfolk Island Nigeria Nicaragua Niue Netherlands Norway Nepal Nauru New Zealand Oman Pakistan Panama Pitcairn Peru Philippines Palau Papua New Guinea Poland Puerto Rico North Korea Portugal Paraguay State of Palestine French Polynesia Qatar Réunion Romania Russia Rwanda Saudi Arabia Sudan Senegal Singapore South Georgia and the South Sandwich Islands Saint Helena Ascension and Tristan da Cunha Svalbard and Jan Mayen Solomon Islands Sierra Leone El Salvador San Marino Somalia Saint Pierre and Miquelon Serbia South Sudan Sao Tome and Principe Suriname Slovakia Slovenia Sweden Eswatini Sint Maarten (Dutch-part) Seychelles Syrian Arab Republic Turks and Caicos Islands Chad Togo Thailand Tajikistan Tokelau Turkmenistan Timor Leste Tonga Trinidad and Tobago Tunisia Turkey Tuvalu Taiwan United Republic of Tanzania Uganda Ukraine United States Minor Outlying Islands Uruguay United-States Uzbekistan Holy See (Vatican City State) Saint Vincent and the Grenadines Bolivarian Republic of Venezuela Virgin Islands British Virgin Islands U.S. VietNam Vanuatu Wallis and Futuna Samoa Yemen South Africa Zambia Zimbabwe select checkbox text	By submitting this form, you agree to the [License Agreement](https://cohere.com/c4ai-cc-by-nc-license) and acknowledge that the information you provide will be collected, used, and shared in accordance with Cohere’s [Privacy Policy]( https://cohere.com/privacy). You’ll receive email updates about Cohere Labs and Cohere research, events, products and services. You can unsubscribe at any time.	en fr de es it pt ja ko zh ar el fa pl id cs he hi nl ro ru tr uk vi	transformers	cc-by-nc-4.0	mradermacher

About

static quants of https://huggingface.co/CohereLabs/aya-23-8B

weighted/imatrix quants are available at https://huggingface.co/mradermacher/aya-23-8B-i1-GGUF

Usage

If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files.

Provided Quants

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Link	Type	Size/GB	Notes
GGUF	Q2_K	3.5
GGUF	IQ3_XS	3.8
GGUF	Q3_K_S	4.0
GGUF	IQ3_S	4.0	beats Q3_K*
GGUF	IQ3_M	4.1
GGUF	Q3_K_M	4.3	lower quality
GGUF	Q3_K_L	4.6
GGUF	IQ4_XS	4.7
GGUF	Q4_K_S	4.9	fast, recommended
GGUF	Q4_K_M	5.2	fast, recommended
GGUF	Q5_K_S	5.8
GGUF	Q5_K_M	5.9
GGUF	Q6_K	6.7	very good quality
GGUF	Q8_0	8.6	fast, best quality
GGUF	f16	16.2	16 bpw, overkill

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9

FAQ / Model Request

See https://huggingface.co/mradermacher/model_requests for some answers to questions you might have and/or if you want some other model quantized.

Thanks

I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.