初始化项目,由ModelHub XC社区提供模型
Model: mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF Source: Original Platform
This commit is contained in:
59
.gitattributes
vendored
Normal file
59
.gitattributes
vendored
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen2.5-0.5B-Instruct-ITA.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_M.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:d235a4a2ad4f06cf27e91c002737e21837cff8588d6e8d8a912e7525e8f4dcf2
|
||||||
|
size 394551264
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_S.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:3cd03674b70ef5e88bb4b1ceafde871573ff0b017b72fef89cade262bbe7c867
|
||||||
|
size 392406240
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_M.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:940e83fcc1decd7728220fb1b18205353067131593718f385cc6040898e66f16
|
||||||
|
size 405174240
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_S.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:2520c0db4774df947f82c9d90ef9c8294a7c7fcb761eced330338824d6c4db98
|
||||||
|
size 402314208
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XS.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b673701db5035ab442bf2624671a2fdfb14720b7995084789c61f35127f24c3a
|
||||||
|
size 400986336
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XXS.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:c2cfef51340ac3fefad0defe6969693331103657db2b338715172cd489455bea
|
||||||
|
size 398126304
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_M.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:ce48d1edca01913791c4b3b11a3f6950f669026f2b9fab31899316880d518135
|
||||||
|
size 419329248
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_S.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:bd875291cf7703e78c2822e36ded38aff55d2f506af181f64f284820b5c868ca
|
||||||
|
size 415184352
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XS.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:0629af06b829e35e9946ab5031bb9d8a277455acb6327569a0d1be582ca4027f
|
||||||
|
size 415184352
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XXS.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:10c198452c01b69679964e5c71f4b7784a82c0b30b3906d54b2064f083fe5584
|
||||||
|
size 410281440
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_NL.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_NL.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:37bd71a20fd9a8ce0656a4bee679fd2424a8a031733590d519edfea407d97188
|
||||||
|
size 429247968
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_XS.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:5bc08b848af16a73cc119f5dcfa201182a39c0140fdd50d39b7b9d7b8d6829fa
|
||||||
|
size 425979360
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:3a33bca8cc13d18b30f7a9baacc34b0ffc63bfbf79635ddead924c02641248ab
|
||||||
|
size 415184352
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K_S.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b669a29e4a8f2676f313e453a9f7b754dfdd7db8a3892aeb0f0d60b62185fdb5
|
||||||
|
size 407625696
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_L.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_L.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:c18d7a865e44c492c82c445cfc9d49718896a9a8a4d23dd6d5c56f488c08c4cf
|
||||||
|
size 445935072
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_M.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:d2f7f2ba45782b5aa1cc4b635222967e6e380c02b13389700b53b45ae8668aee
|
||||||
|
size 432043488
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_S.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:225570ff37c4898e632be4468cd80e4eed9db8e833d95d0230b31f4c29c80ebe
|
||||||
|
size 414840288
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q4_0.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:328f9a17e75bf717c888dfda385e2ac03ee34f48a7d0476c74176d1ecc4d376f
|
||||||
|
size 429549024
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q4_1.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q4_1.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:1d5dac316a942051c931a31fc8de520514ad03498e81b8b02a5c74ca9102e374
|
||||||
|
size 459604448
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_M.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:01c26571a4f025eec43d11d5e56a494555f3f164b7c764faf5e6ac4752fc3c70
|
||||||
|
size 491401696
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_S.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:faed76df76cf57c70069d54e34048c976a653b93f1075870d143be4aced37126
|
||||||
|
size 479065568
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_M.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:f228ddd9c7d60e9f754196e201a59de6905f9dbab50dd69196921978d2003e78
|
||||||
|
size 522188256
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_S.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:bd4fced78aac2b76acd048747098a644c93eea161de7222f3fb7acd8ac0ccaad
|
||||||
|
size 514812384
|
||||||
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q6_K.gguf
Normal file
3
Qwen2.5-0.5B-Instruct-ITA.i1-Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:90b122328a2744c8ffc0d2a8de90da2cb62238ce9d6f00e97df52a2c69168d3a
|
||||||
|
size 650380768
|
||||||
83
README.md
Normal file
83
README.md
Normal file
@@ -0,0 +1,83 @@
|
|||||||
|
---
|
||||||
|
base_model: ReDiX/Qwen2.5-0.5B-Instruct-ITA
|
||||||
|
datasets:
|
||||||
|
- ReDiX/everyday-conversations-ita
|
||||||
|
- ReDiX/dataforge-cleaned
|
||||||
|
language:
|
||||||
|
- it
|
||||||
|
- en
|
||||||
|
library_name: transformers
|
||||||
|
license: apache-2.0
|
||||||
|
quantized_by: mradermacher
|
||||||
|
tags:
|
||||||
|
- generated_from_trainer
|
||||||
|
- axolotl
|
||||||
|
---
|
||||||
|
## About
|
||||||
|
|
||||||
|
<!-- ### quantize_version: 2 -->
|
||||||
|
<!-- ### output_tensor_quantised: 1 -->
|
||||||
|
<!-- ### convert_type: hf -->
|
||||||
|
<!-- ### vocab_type: -->
|
||||||
|
<!-- ### tags: nicoboss -->
|
||||||
|
weighted/imatrix quants of https://huggingface.co/ReDiX/Qwen2.5-0.5B-Instruct-ITA
|
||||||
|
|
||||||
|
<!-- provided-files -->
|
||||||
|
static quants are available at https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-GGUF
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
If you are unsure how to use GGUF files, refer to one of [TheBloke's
|
||||||
|
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
|
||||||
|
more details, including on how to concatenate multi-part files.
|
||||||
|
|
||||||
|
## Provided Quants
|
||||||
|
|
||||||
|
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
|
||||||
|
|
||||||
|
| Link | Type | Size/GB | Notes |
|
||||||
|
|:-----|:-----|--------:|:------|
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_S.gguf) | i1-IQ1_S | 0.5 | for the desperate |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_M.gguf) | i1-IQ1_M | 0.5 | mostly desperate |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 0.5 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XS.gguf) | i1-IQ2_XS | 0.5 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_S.gguf) | i1-IQ2_S | 0.5 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_M.gguf) | i1-IQ2_M | 0.5 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K_S.gguf) | i1-Q2_K_S | 0.5 | very low quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 0.5 | lower quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_S.gguf) | i1-Q3_K_S | 0.5 | IQ3_XS probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_S.gguf) | i1-IQ3_S | 0.5 | beats Q3_K* |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XS.gguf) | i1-IQ3_XS | 0.5 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K.gguf) | i1-Q2_K | 0.5 | IQ3_XXS probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_M.gguf) | i1-IQ3_M | 0.5 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_XS.gguf) | i1-IQ4_XS | 0.5 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_NL.gguf) | i1-IQ4_NL | 0.5 | prefer IQ4_XS |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_0.gguf) | i1-Q4_0 | 0.5 | fast, low quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_M.gguf) | i1-Q3_K_M | 0.5 | IQ3_S probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_L.gguf) | i1-Q3_K_L | 0.5 | IQ3_M probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_1.gguf) | i1-Q4_1 | 0.6 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_S.gguf) | i1-Q4_K_S | 0.6 | optimal size/speed/quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_M.gguf) | i1-Q4_K_M | 0.6 | fast, recommended |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_S.gguf) | i1-Q5_K_S | 0.6 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_M.gguf) | i1-Q5_K_M | 0.6 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q6_K.gguf) | i1-Q6_K | 0.8 | practically like static Q6_K |
|
||||||
|
|
||||||
|
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
||||||
|
types (lower is better):
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
And here are Artefact2's thoughts on the matter:
|
||||||
|
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
|
||||||
|
|
||||||
|
## FAQ / Model Request
|
||||||
|
|
||||||
|
See https://huggingface.co/mradermacher/model_requests for some answers to
|
||||||
|
questions you might have and/or if you want some other model quantized.
|
||||||
|
|
||||||
|
## Thanks
|
||||||
|
|
||||||
|
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
|
||||||
|
me use its servers and providing upgrades to my workstation to enable
|
||||||
|
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
|
||||||
|
|
||||||
|
<!-- end -->
|
||||||
BIN
imatrix.dat
Normal file
BIN
imatrix.dat
Normal file
Binary file not shown.
Reference in New Issue
Block a user