commit 5d119e45cd5697f4cefde663c1b6350d44ef0012 Author: ModelHub XC Date: Tue May 19 23:41:06 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..63fe12a --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +DistilGPT-OSS-qwen3-4B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/DistilGPT-OSS-qwen3-4B.i1-IQ1_M.gguf b/DistilGPT-OSS-qwen3-4B.i1-IQ1_M.gguf new file mode 100644 index 0000000..6d2efd8 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bde9e9ddd70e4a743c23554f2d53e56115286a413656c9d2568b0109a6af8dfd +size 1127016704 diff --git a/DistilGPT-OSS-qwen3-4B.i1-IQ1_S.gguf b/DistilGPT-OSS-qwen3-4B.i1-IQ1_S.gguf new file mode 100644 index 0000000..fb804c7 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:11221de9755175130725e11d86ed301d87663925cb85186ee8507fe9c80f81d9 +size 1055254784 diff --git a/DistilGPT-OSS-qwen3-4B.i1-IQ2_M.gguf b/DistilGPT-OSS-qwen3-4B.i1-IQ2_M.gguf new file mode 100644 index 0000000..2991c41 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a5af9dc684123f0539179e23531c22f9c42cc3f7191fadb5d29e55a62dbb7744 +size 1512982784 diff --git a/DistilGPT-OSS-qwen3-4B.i1-IQ2_S.gguf b/DistilGPT-OSS-qwen3-4B.i1-IQ2_S.gguf new file mode 100644 index 0000000..c97036e --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3c8e137e2d011c9be2d5575225ddff7a6c4c586f5a52d4ba244072aa81f59f8c +size 1417300224 diff --git a/DistilGPT-OSS-qwen3-4B.i1-IQ2_XS.gguf b/DistilGPT-OSS-qwen3-4B.i1-IQ2_XS.gguf new file mode 100644 index 0000000..2ef878e --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:891b1512c7b13040d5d84c9b67f5c6d8d35b45cd139954b5edaff1b25bf7aa74 +size 1354098944 diff --git a/DistilGPT-OSS-qwen3-4B.i1-IQ2_XXS.gguf b/DistilGPT-OSS-qwen3-4B.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..4a26725 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c6c32c1b01b347a115fc69a40cfafb81c207a15a82fca29f18272222c4703381 +size 1246619904 diff --git a/DistilGPT-OSS-qwen3-4B.i1-IQ3_M.gguf b/DistilGPT-OSS-qwen3-4B.i1-IQ3_M.gguf new file mode 100644 index 0000000..77b4103 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a0898d5a6ddf012a59182e0b4199ac24c254dfa1226372fb72e1587e682f2624 +size 1962895104 diff --git a/DistilGPT-OSS-qwen3-4B.i1-IQ3_S.gguf b/DistilGPT-OSS-qwen3-4B.i1-IQ3_S.gguf new file mode 100644 index 0000000..9dc4102 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e5d938929137356dfa3ad6b15aadc87a5d30ae42eff5978a215fd1b5a3770a40 +size 1899529984 diff --git a/DistilGPT-OSS-qwen3-4B.i1-IQ3_XS.gguf b/DistilGPT-OSS-qwen3-4B.i1-IQ3_XS.gguf new file mode 100644 index 0000000..86c7080 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa4c1fda0a120a76e673f6f94230b993d0f008b34c4c1c44cbc0c24f7e4ae97f +size 1814374144 diff --git a/DistilGPT-OSS-qwen3-4B.i1-IQ3_XXS.gguf b/DistilGPT-OSS-qwen3-4B.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..4259656 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5eb89443184cf0b85a843161691bacaacf501bf52a3b41f2f8f721ca7e7f19e9 +size 1670187264 diff --git a/DistilGPT-OSS-qwen3-4B.i1-IQ4_NL.gguf b/DistilGPT-OSS-qwen3-4B.i1-IQ4_NL.gguf new file mode 100644 index 0000000..1a923f8 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c8e1ce7bebb5e6ecfe8387da81a0f871a07a00757e05f2e55ac4cbe21e8bb685 +size 2381342464 diff --git a/DistilGPT-OSS-qwen3-4B.i1-IQ4_XS.gguf b/DistilGPT-OSS-qwen3-4B.i1-IQ4_XS.gguf new file mode 100644 index 0000000..0e7b212 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da687b3e783564562903987d376702e76dcb8aaa63629ff12d40c665c8868486 +size 2270750464 diff --git a/DistilGPT-OSS-qwen3-4B.i1-Q2_K.gguf b/DistilGPT-OSS-qwen3-4B.i1-Q2_K.gguf new file mode 100644 index 0000000..c04c458 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eccd1273df7be7545571b4a08f3d45f4d8405d03d4a77c8f5b948c90ffd2e330 +size 1669498624 diff --git a/DistilGPT-OSS-qwen3-4B.i1-Q2_K_S.gguf b/DistilGPT-OSS-qwen3-4B.i1-Q2_K_S.gguf new file mode 100644 index 0000000..c807314 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dd234dca8a23a5105e6dc1062153c74484b1dc9cf724c56043bf6ab10d4a3db0 +size 1563453184 diff --git a/DistilGPT-OSS-qwen3-4B.i1-Q3_K_L.gguf b/DistilGPT-OSS-qwen3-4B.i1-Q3_K_L.gguf new file mode 100644 index 0000000..55cf51c --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0669ec4c309e0d10f8acff57a2d792c045d60b7933bfc89c3f4c4c32cb47973a +size 2239784704 diff --git a/DistilGPT-OSS-qwen3-4B.i1-Q3_K_M.gguf b/DistilGPT-OSS-qwen3-4B.i1-Q3_K_M.gguf new file mode 100644 index 0000000..afd44a0 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3819e52f717c5c58d896056af7c293139502e42689079c14c73da4c2069280f +size 2075617024 diff --git a/DistilGPT-OSS-qwen3-4B.i1-Q3_K_S.gguf b/DistilGPT-OSS-qwen3-4B.i1-Q3_K_S.gguf new file mode 100644 index 0000000..6fa596c --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4517ab646e58de263532a88e589d91e1df5d2a567c6df24c894414aad4f8dd41 +size 1886996224 diff --git a/DistilGPT-OSS-qwen3-4B.i1-Q4_0.gguf b/DistilGPT-OSS-qwen3-4B.i1-Q4_0.gguf new file mode 100644 index 0000000..ffd9851 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:afaad2900fc205d3e52303813ebe628e8145122c10d0163f3c50ff02a738f55a +size 2375771904 diff --git a/DistilGPT-OSS-qwen3-4B.i1-Q4_1.gguf b/DistilGPT-OSS-qwen3-4B.i1-Q4_1.gguf new file mode 100644 index 0000000..3cbe8f3 --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2bcb6fb67722342cc90268cd02f82c0be5ed9b757b886a1b5eda0222116c5a18 +size 2596628224 diff --git a/DistilGPT-OSS-qwen3-4B.i1-Q4_K_M.gguf b/DistilGPT-OSS-qwen3-4B.i1-Q4_K_M.gguf new file mode 100644 index 0000000..9745e7e --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8d89008f9c932075a4efa95317ad8ff22dc9ae77e8d4a6b60cbb47c54c7cae92 +size 2497279744 diff --git a/DistilGPT-OSS-qwen3-4B.i1-Q4_K_S.gguf b/DistilGPT-OSS-qwen3-4B.i1-Q4_K_S.gguf new file mode 100644 index 0000000..678801b --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bc46c0776221738496c1ffb317267ee26ce1f47cfa4d821efa1febe28f011222 +size 2383308544 diff --git a/DistilGPT-OSS-qwen3-4B.i1-Q5_K_M.gguf b/DistilGPT-OSS-qwen3-4B.i1-Q5_K_M.gguf new file mode 100644 index 0000000..0df23be --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2150c3fae144c4af27251e825a4f7bb22215a1fc771df5df94ae5f1ae8d71eab +size 2889512704 diff --git a/DistilGPT-OSS-qwen3-4B.i1-Q5_K_S.gguf b/DistilGPT-OSS-qwen3-4B.i1-Q5_K_S.gguf new file mode 100644 index 0000000..56597ab --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c9fca800c6ca4f10072471bd5f697315a257bf0a71bd738079c74de39374d66f +size 2823710464 diff --git a/DistilGPT-OSS-qwen3-4B.i1-Q6_K.gguf b/DistilGPT-OSS-qwen3-4B.i1-Q6_K.gguf new file mode 100644 index 0000000..4ab1faa --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f06a67f5a50d57a11175a700f3cbf36be9dd223fa3877495ce751f8e63bdbe6d +size 3306260224 diff --git a/DistilGPT-OSS-qwen3-4B.imatrix.gguf b/DistilGPT-OSS-qwen3-4B.imatrix.gguf new file mode 100644 index 0000000..05e34eb --- /dev/null +++ b/DistilGPT-OSS-qwen3-4B.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a1a6c09df1a6d144cd8269e8046d184b5f0a32db9bdc3f8d6835c1597128f134 +size 3872640 diff --git a/README.md b/README.md new file mode 100644 index 0000000..cea4579 --- /dev/null +++ b/README.md @@ -0,0 +1,94 @@ +--- +base_model: Pinkstack/DistilGPT-OSS-qwen3-4B +datasets: +- Pinkstack/gpt_oss_sft +language: +- en +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- cot +- code +- gpt_oss +- conversational +- distillation +- math +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/Pinkstack/DistilGPT-OSS-qwen3-4B + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#DistilGPT-OSS-qwen3-4B-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-IQ1_S.gguf) | i1-IQ1_S | 1.2 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-IQ1_M.gguf) | i1-IQ1_M | 1.2 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.3 | | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-IQ2_S.gguf) | i1-IQ2_S | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-IQ2_M.gguf) | i1-IQ2_M | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.7 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-Q2_K.gguf) | i1-Q2_K | 1.8 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.8 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.9 | | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 2.0 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-IQ3_S.gguf) | i1-IQ3_S | 2.0 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-IQ3_M.gguf) | i1-IQ3_M | 2.1 | | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 2.2 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 2.3 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-Q4_0.gguf) | i1-Q4_0 | 2.5 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.5 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.5 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.6 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-Q4_1.gguf) | i1-Q4_1 | 2.7 | | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 2.9 | | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 3.0 | | +| [GGUF](https://huggingface.co/mradermacher/DistilGPT-OSS-qwen3-4B-i1-GGUF/resolve/main/DistilGPT-OSS-qwen3-4B.i1-Q6_K.gguf) | i1-Q6_K | 3.4 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +