349 lines
8.3 KiB
Markdown
349 lines
8.3 KiB
Markdown
---
|
||
base_model: CohereLabs/aya-23-8B
|
||
extra_gated_fields:
|
||
Affiliation: text
|
||
Country:
|
||
options:
|
||
- Aruba
|
||
- Afghanistan
|
||
- Angola
|
||
- Anguilla
|
||
- Åland Islands
|
||
- Albania
|
||
- Andorra
|
||
- United Arab Emirates
|
||
- Argentina
|
||
- Armenia
|
||
- American Samoa
|
||
- Antarctica
|
||
- French Southern Territories
|
||
- Antigua and Barbuda
|
||
- Australia
|
||
- Austria
|
||
- Azerbaijan
|
||
- Burundi
|
||
- Belgium
|
||
- Benin
|
||
- Bonaire Sint Eustatius and Saba
|
||
- Burkina Faso
|
||
- Bangladesh
|
||
- Bulgaria
|
||
- Bahrain
|
||
- Bahamas
|
||
- Bosnia and Herzegovina
|
||
- Saint Barthélemy
|
||
- Belarus
|
||
- Belize
|
||
- Bermuda
|
||
- Plurinational State of Bolivia
|
||
- Brazil
|
||
- Barbados
|
||
- Brunei-Darussalam
|
||
- Bhutan
|
||
- Bouvet-Island
|
||
- Botswana
|
||
- Central African Republic
|
||
- Canada
|
||
- Cocos (Keeling) Islands
|
||
- Switzerland
|
||
- Chile
|
||
- China
|
||
- Côte-dIvoire
|
||
- Cameroon
|
||
- Democratic Republic of the Congo
|
||
- Cook Islands
|
||
- Colombia
|
||
- Comoros
|
||
- Cabo Verde
|
||
- Costa Rica
|
||
- Cuba
|
||
- Curaçao
|
||
- Christmas Island
|
||
- Cayman Islands
|
||
- Cyprus
|
||
- Czechia
|
||
- Germany
|
||
- Djibouti
|
||
- Dominica
|
||
- Denmark
|
||
- Dominican Republic
|
||
- Algeria
|
||
- Ecuador
|
||
- Egypt
|
||
- Eritrea
|
||
- Western Sahara
|
||
- Spain
|
||
- Estonia
|
||
- Ethiopia
|
||
- Finland
|
||
- Fiji
|
||
- Falkland Islands (Malvinas)
|
||
- France
|
||
- Faroe Islands
|
||
- Federated States of Micronesia
|
||
- Gabon
|
||
- United Kingdom
|
||
- Georgia
|
||
- Guernsey
|
||
- Ghana
|
||
- Gibraltar
|
||
- Guinea
|
||
- Guadeloupe
|
||
- Gambia
|
||
- Guinea Bissau
|
||
- Equatorial Guinea
|
||
- Greece
|
||
- Grenada
|
||
- Greenland
|
||
- Guatemala
|
||
- French Guiana
|
||
- Guam
|
||
- Guyana
|
||
- Hong Kong
|
||
- Heard Island and McDonald Islands
|
||
- Honduras
|
||
- Croatia
|
||
- Haiti
|
||
- Hungary
|
||
- Indonesia
|
||
- Isle of Man
|
||
- India
|
||
- British Indian Ocean Territory
|
||
- Ireland
|
||
- Islamic Republic of Iran
|
||
- Iraq
|
||
- Iceland
|
||
- Israel
|
||
- Italy
|
||
- Jamaica
|
||
- Jersey
|
||
- Jordan
|
||
- Japan
|
||
- Kazakhstan
|
||
- Kenya
|
||
- Kyrgyzstan
|
||
- Cambodia
|
||
- Kiribati
|
||
- Saint-Kitts-and-Nevis
|
||
- South Korea
|
||
- Kuwait
|
||
- Lao-Peoples-Democratic-Republic
|
||
- Lebanon
|
||
- Liberia
|
||
- Libya
|
||
- Saint-Lucia
|
||
- Liechtenstein
|
||
- Sri Lanka
|
||
- Lesotho
|
||
- Lithuania
|
||
- Luxembourg
|
||
- Latvia
|
||
- Macao
|
||
- Saint Martin (French-part)
|
||
- Morocco
|
||
- Monaco
|
||
- Republic of Moldova
|
||
- Madagascar
|
||
- Maldives
|
||
- Mexico
|
||
- Marshall Islands
|
||
- North Macedonia
|
||
- Mali
|
||
- Malta
|
||
- Myanmar
|
||
- Montenegro
|
||
- Mongolia
|
||
- Northern Mariana Islands
|
||
- Mozambique
|
||
- Mauritania
|
||
- Montserrat
|
||
- Martinique
|
||
- Mauritius
|
||
- Malawi
|
||
- Malaysia
|
||
- Mayotte
|
||
- Namibia
|
||
- New Caledonia
|
||
- Niger
|
||
- Norfolk Island
|
||
- Nigeria
|
||
- Nicaragua
|
||
- Niue
|
||
- Netherlands
|
||
- Norway
|
||
- Nepal
|
||
- Nauru
|
||
- New Zealand
|
||
- Oman
|
||
- Pakistan
|
||
- Panama
|
||
- Pitcairn
|
||
- Peru
|
||
- Philippines
|
||
- Palau
|
||
- Papua New Guinea
|
||
- Poland
|
||
- Puerto Rico
|
||
- North Korea
|
||
- Portugal
|
||
- Paraguay
|
||
- State of Palestine
|
||
- French Polynesia
|
||
- Qatar
|
||
- Réunion
|
||
- Romania
|
||
- Russia
|
||
- Rwanda
|
||
- Saudi Arabia
|
||
- Sudan
|
||
- Senegal
|
||
- Singapore
|
||
- South Georgia and the South Sandwich Islands
|
||
- Saint Helena Ascension and Tristan da Cunha
|
||
- Svalbard and Jan Mayen
|
||
- Solomon Islands
|
||
- Sierra Leone
|
||
- El Salvador
|
||
- San Marino
|
||
- Somalia
|
||
- Saint Pierre and Miquelon
|
||
- Serbia
|
||
- South Sudan
|
||
- Sao Tome and Principe
|
||
- Suriname
|
||
- Slovakia
|
||
- Slovenia
|
||
- Sweden
|
||
- Eswatini
|
||
- Sint Maarten (Dutch-part)
|
||
- Seychelles
|
||
- Syrian Arab Republic
|
||
- Turks and Caicos Islands
|
||
- Chad
|
||
- Togo
|
||
- Thailand
|
||
- Tajikistan
|
||
- Tokelau
|
||
- Turkmenistan
|
||
- Timor Leste
|
||
- Tonga
|
||
- Trinidad and Tobago
|
||
- Tunisia
|
||
- Turkey
|
||
- Tuvalu
|
||
- Taiwan
|
||
- United Republic of Tanzania
|
||
- Uganda
|
||
- Ukraine
|
||
- United States Minor Outlying Islands
|
||
- Uruguay
|
||
- United-States
|
||
- Uzbekistan
|
||
- Holy See (Vatican City State)
|
||
- Saint Vincent and the Grenadines
|
||
- Bolivarian Republic of Venezuela
|
||
- Virgin Islands British
|
||
- Virgin Islands U.S.
|
||
- VietNam
|
||
- Vanuatu
|
||
- Wallis and Futuna
|
||
- Samoa
|
||
- Yemen
|
||
- South Africa
|
||
- Zambia
|
||
- Zimbabwe
|
||
type: select
|
||
I agree to use this model for non-commercial use ONLY: checkbox
|
||
Name: text
|
||
extra_gated_prompt: By submitting this form, you agree to the [License Agreement](https://cohere.com/c4ai-cc-by-nc-license) and
|
||
acknowledge that the information you provide will be collected, used, and shared
|
||
in accordance with Cohere’s [Privacy Policy]( https://cohere.com/privacy). You’ll
|
||
receive email updates about Cohere Labs and Cohere research, events, products and
|
||
services. You can unsubscribe at any time.
|
||
language:
|
||
- en
|
||
- fr
|
||
- de
|
||
- es
|
||
- it
|
||
- pt
|
||
- ja
|
||
- ko
|
||
- zh
|
||
- ar
|
||
- el
|
||
- fa
|
||
- pl
|
||
- id
|
||
- cs
|
||
- he
|
||
- hi
|
||
- nl
|
||
- ro
|
||
- ru
|
||
- tr
|
||
- uk
|
||
- vi
|
||
library_name: transformers
|
||
license: cc-by-nc-4.0
|
||
quantized_by: mradermacher
|
||
---
|
||
## About
|
||
|
||
<!-- ### quantize_version: 2 -->
|
||
<!-- ### output_tensor_quantised: 1 -->
|
||
<!-- ### convert_type: hf -->
|
||
<!-- ### vocab_type: -->
|
||
static quants of https://huggingface.co/CohereLabs/aya-23-8B
|
||
|
||
<!-- provided-files -->
|
||
weighted/imatrix quants are available at https://huggingface.co/mradermacher/aya-23-8B-i1-GGUF
|
||
## Usage
|
||
|
||
If you are unsure how to use GGUF files, refer to one of [TheBloke's
|
||
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
|
||
more details, including on how to concatenate multi-part files.
|
||
|
||
## Provided Quants
|
||
|
||
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
|
||
|
||
| Link | Type | Size/GB | Notes |
|
||
|:-----|:-----|--------:|:------|
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.Q2_K.gguf) | Q2_K | 3.5 | |
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.IQ3_XS.gguf) | IQ3_XS | 3.8 | |
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.Q3_K_S.gguf) | Q3_K_S | 4.0 | |
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.IQ3_S.gguf) | IQ3_S | 4.0 | beats Q3_K* |
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.IQ3_M.gguf) | IQ3_M | 4.1 | |
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.Q3_K_M.gguf) | Q3_K_M | 4.3 | lower quality |
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.Q3_K_L.gguf) | Q3_K_L | 4.6 | |
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.IQ4_XS.gguf) | IQ4_XS | 4.7 | |
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.Q4_K_S.gguf) | Q4_K_S | 4.9 | fast, recommended |
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.Q4_K_M.gguf) | Q4_K_M | 5.2 | fast, recommended |
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.Q5_K_S.gguf) | Q5_K_S | 5.8 | |
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.Q5_K_M.gguf) | Q5_K_M | 5.9 | |
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.Q6_K.gguf) | Q6_K | 6.7 | very good quality |
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.Q8_0.gguf) | Q8_0 | 8.6 | fast, best quality |
|
||
| [GGUF](https://huggingface.co/mradermacher/aya-23-8B-GGUF/resolve/main/aya-23-8B.f16.gguf) | f16 | 16.2 | 16 bpw, overkill |
|
||
|
||
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
||
types (lower is better):
|
||
|
||
|
||
|
||
And here are Artefact2's thoughts on the matter:
|
||
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
|
||
|
||
## FAQ / Model Request
|
||
|
||
See https://huggingface.co/mradermacher/model_requests for some answers to
|
||
questions you might have and/or if you want some other model quantized.
|
||
|
||
## Thanks
|
||
|
||
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
|
||
me use its servers and providing upgrades to my workstation to enable
|
||
this work in my free time.
|
||
|
||
<!-- end -->
|