commit a32781cefc917f6445d1b1abf5ba8643634b494e Author: ModelHub XC Date: Mon Apr 13 12:07:05 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..931ae8c --- /dev/null +++ b/.gitattributes @@ -0,0 +1,59 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Dolphin3.0-Llama3.1-1B-abliterated.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ1_M.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ1_M.gguf new file mode 100644 index 0000000..948b4c9 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:508827436246141df9ebce89e7ea627bed6754c800643a2c1c3d0d1f02996fc2 +size 413605728 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ1_S.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ1_S.gguf new file mode 100644 index 0000000..f1d2342 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e88cc62e837f93efb15822039ec90131747ec48d5d324409af7f4bec9a933f1c +size 393551712 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_M.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_M.gguf new file mode 100644 index 0000000..d5daca0 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0bcb4efc88144e3360388941243db9351ce2ce3ce2318934f774663a16083320 +size 515448672 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_S.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_S.gguf new file mode 100644 index 0000000..d17007d --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5f82efbab00573e40ac9bcf215df0a907ae24b9e1e145645aa668a702757c0dc +size 488709984 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_XS.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_XS.gguf new file mode 100644 index 0000000..ed5d4d9 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c3b95ade42d61fbc577378b78cab02a7216ccb87f913351ec63813fd4c7b8b5c +size 475864928 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_XXS.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..ee4186e --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5482a9dc67b54976aeaf77f12d0b8eff0257152e86a2cf04f698c5025922e7f3 +size 447029088 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_M.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_M.gguf new file mode 100644 index 0000000..971e4ed --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4e1c18510bf6c96a2d01696846a17287f4986aaaadc9a59531591a6d63abc066 +size 657289600 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_S.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_S.gguf new file mode 100644 index 0000000..29c90aa --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de436e1ba01a7a1dbe2122156b87a1cc8028c2c5aa582f9b3ed4eaa0b83a078f +size 643920256 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_XS.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_XS.gguf new file mode 100644 index 0000000..74a04c6 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5a5e1363bd7117fd296a270eb322cadb3777f1d5da90e1b344dc421fea7abdfa +size 621113728 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_XXS.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..8f70055 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:deb06ac29cdc6cba1866ce9e8594a676dd161b27f90ee42e58b6d601a0aaca1d +size 562110304 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ4_NL.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ4_NL.gguf new file mode 100644 index 0000000..5062900 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8dd66031c2db7b8675f161d1b1f7aade62c4c1fb72dffdc4c161736dcd2e5ba5 +size 773026176 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ4_XS.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ4_XS.gguf new file mode 100644 index 0000000..ec3a752 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4f68a789530ee3bb7657a331481e58af1384bfd988fe0087bd1f065af4521527 +size 743141760 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q2_K.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q2_K.gguf new file mode 100644 index 0000000..1ad71b5 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e265f6284554b196d675c1e8363aa7bf24598b1ad9c71a430d4c8dab91ab4a9d +size 580874624 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q2_K_S.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q2_K_S.gguf new file mode 100644 index 0000000..963fdac --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c58c00bdc028fed1dc7377a2bd6f8d3099b63fb15b5b4a57b44ddaf4d409fccb +size 554660224 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_L.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_L.gguf new file mode 100644 index 0000000..2e718e4 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a7e7e278237875e61d47a39ecf38f1a910819d3a6bb34f91e7936ee1644f99e8 +size 732524928 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_M.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_M.gguf new file mode 100644 index 0000000..5ac10c8 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9767eb2ac02c3c4685a69acdaf55c206e654f178f9c8db960dc9ef98eb8f142c +size 690844032 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_S.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_S.gguf new file mode 100644 index 0000000..8fd8d5d --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c094fc91fd3e136504f40522ef70295f1a63508fee5295c156f6e2d9989665b3 +size 641692032 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_0.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_0.gguf new file mode 100644 index 0000000..e8b577d --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cc801426ad363141749c8532812a231b2f1c46355d0b51f4cf9567670e815498 +size 773026176 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_1.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_1.gguf new file mode 100644 index 0000000..f8cc35d --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:81e6345dca4d785d1a41bfd866075ab8c52b0e54b69f45261e3d95ac5fb38131 +size 831746432 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_K_M.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_K_M.gguf new file mode 100644 index 0000000..057cace --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5dfa0e11bdc8b6aafb546bc57af1e0e6976df0e65fa05c11ae736686bb3cb6e3 +size 807694720 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_K_S.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_K_S.gguf new file mode 100644 index 0000000..9cd5485 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c02a941001ae389d6e013c5f5f0c181c3111515809bc33961bb5717931ec3b84 +size 775647616 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q5_K_M.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q5_K_M.gguf new file mode 100644 index 0000000..157f308 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8bb7d5c8fd1b9dc78b7def4d67d1616c78b4c083554962b8cdb2b683bf97f44b +size 911503744 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q5_K_S.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q5_K_S.gguf new file mode 100644 index 0000000..bb1de13 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b3088e3108e0ea9c2dd602326d62dda47127a77047124e193bd98829afa47693 +size 892563840 diff --git a/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q6_K.gguf b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q6_K.gguf new file mode 100644 index 0000000..1358033 --- /dev/null +++ b/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5a4fcb4c470736ffccb8dbd48e11b5d346e55582976bb073ab4d0bbe208cb4f0 +size 1021800832 diff --git a/README.md b/README.md new file mode 100644 index 0000000..f3fbf96 --- /dev/null +++ b/README.md @@ -0,0 +1,76 @@ +--- +base_model: Nexesenex/Dolphin3.0-Llama3.1-1B-abliterated +language: +- en +library_name: transformers +license: llama3.1 +quantized_by: mradermacher +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/Nexesenex/Dolphin3.0-Llama3.1-1B-abliterated + + +static quants are available at https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ1_S.gguf) | i1-IQ1_S | 0.5 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ1_M.gguf) | i1-IQ1_M | 0.5 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 0.5 | | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_XS.gguf) | i1-IQ2_XS | 0.6 | | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_S.gguf) | i1-IQ2_S | 0.6 | | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ2_M.gguf) | i1-IQ2_M | 0.6 | | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q2_K_S.gguf) | i1-Q2_K_S | 0.7 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 0.7 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q2_K.gguf) | i1-Q2_K | 0.7 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_XS.gguf) | i1-IQ3_XS | 0.7 | | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_S.gguf) | i1-Q3_K_S | 0.7 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_S.gguf) | i1-IQ3_S | 0.7 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ3_M.gguf) | i1-IQ3_M | 0.8 | | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_M.gguf) | i1-Q3_K_M | 0.8 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q3_K_L.gguf) | i1-Q3_K_L | 0.8 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ4_XS.gguf) | i1-IQ4_XS | 0.8 | | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-IQ4_NL.gguf) | i1-IQ4_NL | 0.9 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_0.gguf) | i1-Q4_0 | 0.9 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_K_S.gguf) | i1-Q4_K_S | 0.9 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_K_M.gguf) | i1-Q4_K_M | 0.9 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q4_1.gguf) | i1-Q4_1 | 0.9 | | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q5_K_S.gguf) | i1-Q5_K_S | 1.0 | | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q5_K_M.gguf) | i1-Q5_K_M | 1.0 | | +| [GGUF](https://huggingface.co/mradermacher/Dolphin3.0-Llama3.1-1B-abliterated-i1-GGUF/resolve/main/Dolphin3.0-Llama3.1-1B-abliterated.i1-Q6_K.gguf) | i1-Q6_K | 1.1 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +