commit 19618ebc60c8ca9665e315a403d4360915910b89 Author: ModelHub XC Date: Sat Jun 6 18:18:21 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..a85cca7 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Nemotron-15b-Thinker-instruct.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Nemotron-15b-Thinker-instruct.i1-IQ1_M.gguf b/Nemotron-15b-Thinker-instruct.i1-IQ1_M.gguf new file mode 100644 index 0000000..d831ac1 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:35de5934b76f0618293207a9481c33089d8a71c097da59a4361faf18dcdc2707 +size 3852660096 diff --git a/Nemotron-15b-Thinker-instruct.i1-IQ1_S.gguf b/Nemotron-15b-Thinker-instruct.i1-IQ1_S.gguf new file mode 100644 index 0000000..b9d0c2d --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:79343c87d02de62b47a7a55f058a2e3f5bf468bf3bc541981c6221782fcf5ec1 +size 3574214016 diff --git a/Nemotron-15b-Thinker-instruct.i1-IQ2_M.gguf b/Nemotron-15b-Thinker-instruct.i1-IQ2_M.gguf new file mode 100644 index 0000000..9ed5204 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:49fd0a5c0195cc5d2d608aa1c4e978a21d08bdb9db66828d1b87d88a85f35e45 +size 5352369536 diff --git a/Nemotron-15b-Thinker-instruct.i1-IQ2_S.gguf b/Nemotron-15b-Thinker-instruct.i1-IQ2_S.gguf new file mode 100644 index 0000000..991d0a6 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:475be6118d2732a79bdf674df0aed2fd1260ca6173a20f5d3a4c5e54c297526b +size 4981108096 diff --git a/Nemotron-15b-Thinker-instruct.i1-IQ2_XS.gguf b/Nemotron-15b-Thinker-instruct.i1-IQ2_XS.gguf new file mode 100644 index 0000000..ef118f5 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ce2751ae7ee2a8e7c027547b7d9f2b6f7869fff8e6ab18d54e52521ba20a0109 +size 4720766336 diff --git a/Nemotron-15b-Thinker-instruct.i1-IQ2_XXS.gguf b/Nemotron-15b-Thinker-instruct.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..ae2acda --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6ba883d5116e6d7bcff08f4244a084d91d72004c3ede15c182b76d989a9e7d4d +size 4316736896 diff --git a/Nemotron-15b-Thinker-instruct.i1-IQ3_M.gguf b/Nemotron-15b-Thinker-instruct.i1-IQ3_M.gguf new file mode 100644 index 0000000..8e0f334 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ae9292d992b7cfff8d26c702163ba6b766b176dccf8d10a648324d9fe699f95 +size 6938668416 diff --git a/Nemotron-15b-Thinker-instruct.i1-IQ3_S.gguf b/Nemotron-15b-Thinker-instruct.i1-IQ3_S.gguf new file mode 100644 index 0000000..b28421e --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ea393b3d1c7c61e249c328c3932802b2e9827947c959b9c8abc6d6b9eff928a0 +size 6740913536 diff --git a/Nemotron-15b-Thinker-instruct.i1-IQ3_XS.gguf b/Nemotron-15b-Thinker-instruct.i1-IQ3_XS.gguf new file mode 100644 index 0000000..911547c --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a692b6b25839dcf2eb53a05cb5584edc784d0326584234c118d5d6a9f166fc89 +size 6424866176 diff --git a/Nemotron-15b-Thinker-instruct.i1-IQ3_XXS.gguf b/Nemotron-15b-Thinker-instruct.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..37bce05 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3c0e5448ea9012ab2f54901808cbab500d29c1798d889958c7c2ca6139f86408 +size 5992328576 diff --git a/Nemotron-15b-Thinker-instruct.i1-IQ4_NL.gguf b/Nemotron-15b-Thinker-instruct.i1-IQ4_NL.gguf new file mode 100644 index 0000000..2175e6b --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7fcff0a5e68ddc7ff668b74201010c26d12623ec9ced3c1e73109ed6184246b4 +size 8638426496 diff --git a/Nemotron-15b-Thinker-instruct.i1-IQ4_XS.gguf b/Nemotron-15b-Thinker-instruct.i1-IQ4_XS.gguf new file mode 100644 index 0000000..f1c8c06 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a2b9e8a58e0b0253fc4a9ebc7d818c840a8df748821cc6b5b3ba1580b4651738 +size 8199662976 diff --git a/Nemotron-15b-Thinker-instruct.i1-Q2_K.gguf b/Nemotron-15b-Thinker-instruct.i1-Q2_K.gguf new file mode 100644 index 0000000..b00495b --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:88fbca60f66e991c322b46f4ae920eb3d2fdde6d3d217e406e32d909b50d0bee +size 5794164096 diff --git a/Nemotron-15b-Thinker-instruct.i1-Q2_K_S.gguf b/Nemotron-15b-Thinker-instruct.i1-Q2_K_S.gguf new file mode 100644 index 0000000..8660322 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:49a3c3ee45d4cda9281e691a9bbff4ef6c1f204b954767cf7f13e02b5f7571d5 +size 5418151296 diff --git a/Nemotron-15b-Thinker-instruct.i1-Q3_K_L.gguf b/Nemotron-15b-Thinker-instruct.i1-Q3_K_L.gguf new file mode 100644 index 0000000..d54dd06 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6f4c63173e1bcd2ea86417b0a5e7b0a210e964abd6e4a90ab3bf052096cda687 +size 7990193536 diff --git a/Nemotron-15b-Thinker-instruct.i1-Q3_K_M.gguf b/Nemotron-15b-Thinker-instruct.i1-Q3_K_M.gguf new file mode 100644 index 0000000..b13cfb3 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:16cbca85e4f4e6223b540d68bee4317565ae7f903c29feb6767d0fec31f3d15a +size 7396437376 diff --git a/Nemotron-15b-Thinker-instruct.i1-Q3_K_S.gguf b/Nemotron-15b-Thinker-instruct.i1-Q3_K_S.gguf new file mode 100644 index 0000000..4c96e61 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:72f05e2a99e6f80b6fc605754f1ca4ccf1921968641479ce5f680ef14a5f2caa +size 6706097536 diff --git a/Nemotron-15b-Thinker-instruct.i1-Q4_0.gguf b/Nemotron-15b-Thinker-instruct.i1-Q4_0.gguf new file mode 100644 index 0000000..c4fa2b8 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9a5351c35ec61ec61433b340cc1ef56153b56066619437614e2a10d98b7b9916 +size 8633183616 diff --git a/Nemotron-15b-Thinker-instruct.i1-Q4_1.gguf b/Nemotron-15b-Thinker-instruct.i1-Q4_1.gguf new file mode 100644 index 0000000..9033c56 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:17fae86e0b0b8e4ac3b320ef111ad02c1b6a85cd6c00e594852acb0952a40ee6 +size 9499569536 diff --git a/Nemotron-15b-Thinker-instruct.i1-Q4_K_M.gguf b/Nemotron-15b-Thinker-instruct.i1-Q4_K_M.gguf new file mode 100644 index 0000000..b126f01 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a5dd5575aea76b2c958507d032b0d7f8a11e62dfe32cda9db297a49cd5448039 +size 9112538496 diff --git a/Nemotron-15b-Thinker-instruct.i1-Q4_K_S.gguf b/Nemotron-15b-Thinker-instruct.i1-Q4_K_S.gguf new file mode 100644 index 0000000..8daad37 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aa875404c24a51b0235363fd74fa360ab6e5aad69282e477c8a2f8ce1666d0f0 +size 8663330176 diff --git a/Nemotron-15b-Thinker-instruct.i1-Q5_K_M.gguf b/Nemotron-15b-Thinker-instruct.i1-Q5_K_M.gguf new file mode 100644 index 0000000..7a7f211 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:079c8377d4d1e36506a05fac4c8086b692a195e96a85b42c025f2a3c9c80fbcb +size 10654600576 diff --git a/Nemotron-15b-Thinker-instruct.i1-Q5_K_S.gguf b/Nemotron-15b-Thinker-instruct.i1-Q5_K_S.gguf new file mode 100644 index 0000000..2e76783 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:628841a8e2ac77a58fc5364db7c43d4391c01e82ed5e06551a4bd09b5161844f +size 10393480576 diff --git a/Nemotron-15b-Thinker-instruct.i1-Q6_K.gguf b/Nemotron-15b-Thinker-instruct.i1-Q6_K.gguf new file mode 100644 index 0000000..9ad7a3d --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:59e5f5f089868686f2580366896bf8d35dbeb1d087ffcb23512b862b1c6de07c +size 12293041536 diff --git a/Nemotron-15b-Thinker-instruct.imatrix.gguf b/Nemotron-15b-Thinker-instruct.imatrix.gguf new file mode 100644 index 0000000..5693321 --- /dev/null +++ b/Nemotron-15b-Thinker-instruct.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fde3d53db7ee460925ef87a8abcb45cb5d67de9dd5119fd3dae13f243df7cf33 +size 8860224 diff --git a/README.md b/README.md new file mode 100644 index 0000000..4bc0477 --- /dev/null +++ b/README.md @@ -0,0 +1,85 @@ +--- +base_model: Nitral-Archive/Nemotron-15b-Thinker-instruct +language: +- en +library_name: transformers +license: other +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/Nitral-Archive/Nemotron-15b-Thinker-instruct + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Nemotron-15b-Thinker-instruct-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-IQ1_S.gguf) | i1-IQ1_S | 3.7 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-IQ1_M.gguf) | i1-IQ1_M | 4.0 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 4.4 | | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.8 | | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-IQ2_S.gguf) | i1-IQ2_S | 5.1 | | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-IQ2_M.gguf) | i1-IQ2_M | 5.5 | | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-Q2_K_S.gguf) | i1-Q2_K_S | 5.5 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-Q2_K.gguf) | i1-Q2_K | 5.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 6.1 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-IQ3_XS.gguf) | i1-IQ3_XS | 6.5 | | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-Q3_K_S.gguf) | i1-Q3_K_S | 6.8 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-IQ3_S.gguf) | i1-IQ3_S | 6.8 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-IQ3_M.gguf) | i1-IQ3_M | 7.0 | | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-Q3_K_M.gguf) | i1-Q3_K_M | 7.5 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-Q3_K_L.gguf) | i1-Q3_K_L | 8.1 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-IQ4_XS.gguf) | i1-IQ4_XS | 8.3 | | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-Q4_0.gguf) | i1-Q4_0 | 8.7 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-IQ4_NL.gguf) | i1-IQ4_NL | 8.7 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-Q4_K_S.gguf) | i1-Q4_K_S | 8.8 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-Q4_K_M.gguf) | i1-Q4_K_M | 9.2 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-Q4_1.gguf) | i1-Q4_1 | 9.6 | | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-Q5_K_S.gguf) | i1-Q5_K_S | 10.5 | | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-Q5_K_M.gguf) | i1-Q5_K_M | 10.8 | | +| [GGUF](https://huggingface.co/mradermacher/Nemotron-15b-Thinker-instruct-i1-GGUF/resolve/main/Nemotron-15b-Thinker-instruct.i1-Q6_K.gguf) | i1-Q6_K | 12.4 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +