commit 525fb0fc92397f161004849c4d528ce4d564c438 Author: ModelHub XC Date: Sat May 9 11:55:47 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..35707c2 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +GeneralChat-Llama3.2-3B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/GeneralChat-Llama3.2-3B.i1-IQ1_M.gguf b/GeneralChat-Llama3.2-3B.i1-IQ1_M.gguf new file mode 100644 index 0000000..64288fa --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:66ec9c9fdbb4bd89253da89203ee060e16fcaf6a76e5e9fdb1eca7de8de4d269 +size 924193024 diff --git a/GeneralChat-Llama3.2-3B.i1-IQ1_S.gguf b/GeneralChat-Llama3.2-3B.i1-IQ1_S.gguf new file mode 100644 index 0000000..4793e50 --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37095fdaac08f53c40568b9ee64e677d9e02244729cffdca9c538c30ca39b2f7 +size 868159744 diff --git a/GeneralChat-Llama3.2-3B.i1-IQ2_M.gguf b/GeneralChat-Llama3.2-3B.i1-IQ2_M.gguf new file mode 100644 index 0000000..1349170 --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b7313d8ac0f5019544a02c7fa8b89b3d2e0cc35aaf9e9920380f44625ac71002 +size 1229033728 diff --git a/GeneralChat-Llama3.2-3B.i1-IQ2_S.gguf b/GeneralChat-Llama3.2-3B.i1-IQ2_S.gguf new file mode 100644 index 0000000..40cacd6 --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:63284b7a7c8832ab03b5d546968f9a97a733a13a19888735936052bbec1878b8 +size 1154322688 diff --git a/GeneralChat-Llama3.2-3B.i1-IQ2_XS.gguf b/GeneralChat-Llama3.2-3B.i1-IQ2_XS.gguf new file mode 100644 index 0000000..f20349a --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:be7a7f0e2644123fd8922f92334afc51c6a16d153f2075e0a5e71b13dcaae6e4 +size 1100550400 diff --git a/GeneralChat-Llama3.2-3B.i1-IQ2_XXS.gguf b/GeneralChat-Llama3.2-3B.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..c3222cf --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1e9ef5f7b59e81b08b79bdbc88ed7a551ed4e218834f2176679521082d66ecbc +size 1017581824 diff --git a/GeneralChat-Llama3.2-3B.i1-IQ3_M.gguf b/GeneralChat-Llama3.2-3B.i1-IQ3_M.gguf new file mode 100644 index 0000000..3a6d5be --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:adc8de9cb84f73c4720d2de86fca27d313a1139428b76c8cffba5e1acc51de21 +size 1599670528 diff --git a/GeneralChat-Llama3.2-3B.i1-IQ3_S.gguf b/GeneralChat-Llama3.2-3B.i1-IQ3_S.gguf new file mode 100644 index 0000000..f603aed --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5d594a61c4e7f1f082db0313c936cbd157c1c9ab24a74c4130cefcef30c4b7d2 +size 1542850816 diff --git a/GeneralChat-Llama3.2-3B.i1-IQ3_XS.gguf b/GeneralChat-Llama3.2-3B.i1-IQ3_XS.gguf new file mode 100644 index 0000000..e54ac9a --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1bf158d72fd49af17530ac1e6dc671bef3ae268e91660b4c09a3b62cf3c6de49 +size 1476790528 diff --git a/GeneralChat-Llama3.2-3B.i1-IQ3_XXS.gguf b/GeneralChat-Llama3.2-3B.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..679f853 --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2a6a456536b5dd4b40fa27c54202c8966f536c532d19a774183c1652a60ef96e +size 1348768000 diff --git a/GeneralChat-Llama3.2-3B.i1-IQ4_NL.gguf b/GeneralChat-Llama3.2-3B.i1-IQ4_NL.gguf new file mode 100644 index 0000000..7a2a321 --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0deb603e72c282c255c2c8781dfa1caa85edd244ec0f715eb0c1090c6b3f1b2b +size 1917192448 diff --git a/GeneralChat-Llama3.2-3B.i1-IQ4_XS.gguf b/GeneralChat-Llama3.2-3B.i1-IQ4_XS.gguf new file mode 100644 index 0000000..ade9544 --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8ed5053293614a6fb8c01afcec39ffe007d14ddc196f1bfa2490ce73a56f9288 +size 1829112064 diff --git a/GeneralChat-Llama3.2-3B.i1-Q2_K.gguf b/GeneralChat-Llama3.2-3B.i1-Q2_K.gguf new file mode 100644 index 0000000..fb9987f --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:78abc6fd2695b107859ef6f9dac5ab1c7c29ac3dce1d3582fbe5eaec4d4120b7 +size 1363937536 diff --git a/GeneralChat-Llama3.2-3B.i1-Q2_K_S.gguf b/GeneralChat-Llama3.2-3B.i1-Q2_K_S.gguf new file mode 100644 index 0000000..b95bde7 --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9b4dad1c3d73da33193e98f5660936da6469970251a562a60c34320e1ac01809 +size 1274284288 diff --git a/GeneralChat-Llama3.2-3B.i1-Q3_K_L.gguf b/GeneralChat-Llama3.2-3B.i1-Q3_K_L.gguf new file mode 100644 index 0000000..1b7085a --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:09c309c52e506ca7bc95c7281e2e889eec2ba1ac0a67d81c7a7d2ba98f0e5122 +size 1815349504 diff --git a/GeneralChat-Llama3.2-3B.i1-Q3_K_M.gguf b/GeneralChat-Llama3.2-3B.i1-Q3_K_M.gguf new file mode 100644 index 0000000..d458c4b --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cbdf51742ab7bfa8fec9a397afa8fecd15d0b6c6fe955677a37b47f41bb9ba25 +size 1687161088 diff --git a/GeneralChat-Llama3.2-3B.i1-Q3_K_S.gguf b/GeneralChat-Llama3.2-3B.i1-Q3_K_S.gguf new file mode 100644 index 0000000..925023f --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:43d3c302142cea2f720083630ab61aeabfe0ebf5b3ad5fb349e1e77177a44bbe +size 1542850816 diff --git a/GeneralChat-Llama3.2-3B.i1-Q4_0.gguf b/GeneralChat-Llama3.2-3B.i1-Q4_0.gguf new file mode 100644 index 0000000..6fd0498 --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:872cdd3eb73f287e9ea67ff54ce4d4b1a01af455148b97ad0b437338000d96ac +size 1921911040 diff --git a/GeneralChat-Llama3.2-3B.i1-Q4_1.gguf b/GeneralChat-Llama3.2-3B.i1-Q4_1.gguf new file mode 100644 index 0000000..5de6b82 --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0a711e54b3420292c311f94cbf7dae880d5f68463bc9a3ca96802a8e7bb1035b +size 2093353216 diff --git a/GeneralChat-Llama3.2-3B.i1-Q4_K_M.gguf b/GeneralChat-Llama3.2-3B.i1-Q4_K_M.gguf new file mode 100644 index 0000000..3ec3971 --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f2be557ada60f9bff090f5c855d864182bfa16aa885eaa24045a6d2e7a852157 +size 2019379456 diff --git a/GeneralChat-Llama3.2-3B.i1-Q4_K_S.gguf b/GeneralChat-Llama3.2-3B.i1-Q4_K_S.gguf new file mode 100644 index 0000000..d2889e4 --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:54add20c7e7acf0ad9092371dfe493a99a438d43c62f5abf21f238503f2b97c8 +size 1928202496 diff --git a/GeneralChat-Llama3.2-3B.i1-Q5_K_M.gguf b/GeneralChat-Llama3.2-3B.i1-Q5_K_M.gguf new file mode 100644 index 0000000..3ea2bdb --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:00a96121f0bf7a58907904fba403b723a29992e9e83962b3b208bd503820bf16 +size 2322155776 diff --git a/GeneralChat-Llama3.2-3B.i1-Q5_K_S.gguf b/GeneralChat-Llama3.2-3B.i1-Q5_K_S.gguf new file mode 100644 index 0000000..9166a2d --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:54d5ed1b7fd324e73680b889d78fd905dc0dae50caa150a87cdf2fd40fa6c164 +size 2269513984 diff --git a/GeneralChat-Llama3.2-3B.i1-Q6_K.gguf b/GeneralChat-Llama3.2-3B.i1-Q6_K.gguf new file mode 100644 index 0000000..6ff5802 --- /dev/null +++ b/GeneralChat-Llama3.2-3B.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a7eb2f2677dc46afb4ab9d3f6eae99640b1cc692ead6804603ce5f39f78089f9 +size 2643855616 diff --git a/GeneralChat-Llama3.2-3B.imatrix.gguf b/GeneralChat-Llama3.2-3B.imatrix.gguf new file mode 100644 index 0000000..b64ec79 --- /dev/null +++ b/GeneralChat-Llama3.2-3B.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4e2d90d5abdf5be8ec121185861bcb558792d113021f14ec199617e08180deab +size 3012064 diff --git a/README.md b/README.md new file mode 100644 index 0000000..725754d --- /dev/null +++ b/README.md @@ -0,0 +1,92 @@ +--- +base_model: theprint/GeneralChat-Llama3.2-3B +datasets: +- theprint/GeneralChat-GPT +language: +- en +library_name: transformers +license: llama3.2 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- text-generation-inference +- transformers +- unsloth +- llama +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/theprint/GeneralChat-Llama3.2-3B + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#GeneralChat-Llama3.2-3B-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-IQ1_S.gguf) | i1-IQ1_S | 1.0 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-IQ1_M.gguf) | i1-IQ1_M | 1.0 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.1 | | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.2 | | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-IQ2_S.gguf) | i1-IQ2_S | 1.3 | | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-IQ2_M.gguf) | i1-IQ2_M | 1.3 | | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.4 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.4 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-Q2_K.gguf) | i1-Q2_K | 1.5 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-IQ3_S.gguf) | i1-IQ3_S | 1.6 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 1.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-IQ3_M.gguf) | i1-IQ3_M | 1.7 | | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 1.8 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 1.9 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 1.9 | | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.0 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-Q4_0.gguf) | i1-Q4_0 | 2.0 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.0 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.1 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-Q4_1.gguf) | i1-Q4_1 | 2.2 | | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/GeneralChat-Llama3.2-3B-i1-GGUF/resolve/main/GeneralChat-Llama3.2-3B.i1-Q6_K.gguf) | i1-Q6_K | 2.7 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +