commit ecf80c94c9c0b67e7f5e28fcfe97e806b7106ff8 Author: ModelHub XC Date: Sat Apr 11 22:47:59 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..2b99d67 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,59 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-0.5B-Instruct-ITA.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_M.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_M.gguf new file mode 100644 index 0000000..8c87e8d --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d235a4a2ad4f06cf27e91c002737e21837cff8588d6e8d8a912e7525e8f4dcf2 +size 394551264 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_S.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_S.gguf new file mode 100644 index 0000000..a99ee56 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3cd03674b70ef5e88bb4b1ceafde871573ff0b017b72fef89cade262bbe7c867 +size 392406240 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_M.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_M.gguf new file mode 100644 index 0000000..db15eb1 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:940e83fcc1decd7728220fb1b18205353067131593718f385cc6040898e66f16 +size 405174240 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_S.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_S.gguf new file mode 100644 index 0000000..336ee84 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2520c0db4774df947f82c9d90ef9c8294a7c7fcb761eced330338824d6c4db98 +size 402314208 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XS.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XS.gguf new file mode 100644 index 0000000..e513720 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b673701db5035ab442bf2624671a2fdfb14720b7995084789c61f35127f24c3a +size 400986336 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XXS.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..158e573 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2cfef51340ac3fefad0defe6969693331103657db2b338715172cd489455bea +size 398126304 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_M.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_M.gguf new file mode 100644 index 0000000..87d2af3 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ce48d1edca01913791c4b3b11a3f6950f669026f2b9fab31899316880d518135 +size 419329248 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_S.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_S.gguf new file mode 100644 index 0000000..2462f2d --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd875291cf7703e78c2822e36ded38aff55d2f506af181f64f284820b5c868ca +size 415184352 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XS.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XS.gguf new file mode 100644 index 0000000..c0ed785 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0629af06b829e35e9946ab5031bb9d8a277455acb6327569a0d1be582ca4027f +size 415184352 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XXS.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..2f4044c --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:10c198452c01b69679964e5c71f4b7784a82c0b30b3906d54b2064f083fe5584 +size 410281440 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_NL.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_NL.gguf new file mode 100644 index 0000000..6137d69 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37bd71a20fd9a8ce0656a4bee679fd2424a8a031733590d519edfea407d97188 +size 429247968 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_XS.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_XS.gguf new file mode 100644 index 0000000..89bd544 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5bc08b848af16a73cc119f5dcfa201182a39c0140fdd50d39b7b9d7b8d6829fa +size 425979360 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K.gguf new file mode 100644 index 0000000..b6fe0e8 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3a33bca8cc13d18b30f7a9baacc34b0ffc63bfbf79635ddead924c02641248ab +size 415184352 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K_S.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K_S.gguf new file mode 100644 index 0000000..3d084a5 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b669a29e4a8f2676f313e453a9f7b754dfdd7db8a3892aeb0f0d60b62185fdb5 +size 407625696 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_L.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_L.gguf new file mode 100644 index 0000000..8f5c150 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c18d7a865e44c492c82c445cfc9d49718896a9a8a4d23dd6d5c56f488c08c4cf +size 445935072 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_M.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_M.gguf new file mode 100644 index 0000000..4c6bd67 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d2f7f2ba45782b5aa1cc4b635222967e6e380c02b13389700b53b45ae8668aee +size 432043488 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_S.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_S.gguf new file mode 100644 index 0000000..0741180 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:225570ff37c4898e632be4468cd80e4eed9db8e833d95d0230b31f4c29c80ebe +size 414840288 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_0.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_0.gguf new file mode 100644 index 0000000..5d45fb5 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:328f9a17e75bf717c888dfda385e2ac03ee34f48a7d0476c74176d1ecc4d376f +size 429549024 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_1.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_1.gguf new file mode 100644 index 0000000..cd165b2 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1d5dac316a942051c931a31fc8de520514ad03498e81b8b02a5c74ca9102e374 +size 459604448 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_M.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_M.gguf new file mode 100644 index 0000000..3b17c90 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:01c26571a4f025eec43d11d5e56a494555f3f164b7c764faf5e6ac4752fc3c70 +size 491401696 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_S.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_S.gguf new file mode 100644 index 0000000..418e743 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:faed76df76cf57c70069d54e34048c976a653b93f1075870d143be4aced37126 +size 479065568 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_M.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_M.gguf new file mode 100644 index 0000000..ad5372f --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f228ddd9c7d60e9f754196e201a59de6905f9dbab50dd69196921978d2003e78 +size 522188256 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_S.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_S.gguf new file mode 100644 index 0000000..387ae59 --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd4fced78aac2b76acd048747098a644c93eea161de7222f3fb7acd8ac0ccaad +size 514812384 diff --git a/Qwen2.5-0.5B-Instruct-ITA.i1-Q6_K.gguf b/Qwen2.5-0.5B-Instruct-ITA.i1-Q6_K.gguf new file mode 100644 index 0000000..547b7ed --- /dev/null +++ b/Qwen2.5-0.5B-Instruct-ITA.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:90b122328a2744c8ffc0d2a8de90da2cb62238ce9d6f00e97df52a2c69168d3a +size 650380768 diff --git a/README.md b/README.md new file mode 100644 index 0000000..f04ddcb --- /dev/null +++ b/README.md @@ -0,0 +1,83 @@ +--- +base_model: ReDiX/Qwen2.5-0.5B-Instruct-ITA +datasets: +- ReDiX/everyday-conversations-ita +- ReDiX/dataforge-cleaned +language: +- it +- en +library_name: transformers +license: apache-2.0 +quantized_by: mradermacher +tags: +- generated_from_trainer +- axolotl +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/ReDiX/Qwen2.5-0.5B-Instruct-ITA + + +static quants are available at https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_S.gguf) | i1-IQ1_S | 0.5 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ1_M.gguf) | i1-IQ1_M | 0.5 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 0.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_XS.gguf) | i1-IQ2_XS | 0.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_S.gguf) | i1-IQ2_S | 0.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ2_M.gguf) | i1-IQ2_M | 0.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K_S.gguf) | i1-Q2_K_S | 0.5 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 0.5 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_S.gguf) | i1-Q3_K_S | 0.5 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_S.gguf) | i1-IQ3_S | 0.5 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_XS.gguf) | i1-IQ3_XS | 0.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q2_K.gguf) | i1-Q2_K | 0.5 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ3_M.gguf) | i1-IQ3_M | 0.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_XS.gguf) | i1-IQ4_XS | 0.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-IQ4_NL.gguf) | i1-IQ4_NL | 0.5 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_0.gguf) | i1-Q4_0 | 0.5 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_M.gguf) | i1-Q3_K_M | 0.5 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q3_K_L.gguf) | i1-Q3_K_L | 0.5 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_1.gguf) | i1-Q4_1 | 0.6 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_S.gguf) | i1-Q4_K_S | 0.6 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q4_K_M.gguf) | i1-Q4_K_M | 0.6 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_S.gguf) | i1-Q5_K_S | 0.6 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q5_K_M.gguf) | i1-Q5_K_M | 0.6 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-0.5B-Instruct-ITA-i1-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-ITA.i1-Q6_K.gguf) | i1-Q6_K | 0.8 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..f7cc7b0 Binary files /dev/null and b/imatrix.dat differ