commit d129d3e5c31b0dfcece6b32cdffc6aa8f5d8d7a9 Author: ModelHub XC Date: Mon May 25 00:43:49 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..05ccbe3 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,58 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ1_M.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ1_M.gguf new file mode 100644 index 0000000..9a9c4ea --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ddcdc0a632e7ee63b9acd2464fdccb732fc9a4773d9d1bcc9d4f2428a6a0839e +size 317975264 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ1_S.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ1_S.gguf new file mode 100644 index 0000000..27c6efe --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7d1668120f722a5988264013d8e3509ebf4d409fe69f0084aa72c5bfe895e6e3 +size 315830240 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_M.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_M.gguf new file mode 100644 index 0000000..a05f0ca --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:00e04b49f1b892010f6e050e79ba6fceb9aece9bc0ccab0b62df2fb725cef04d +size 328598240 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_S.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_S.gguf new file mode 100644 index 0000000..3835b95 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0503ab53632223890deddc47af2d2541eab80ba0fa1f29c952edc5e0bc7bf417 +size 325738208 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_XS.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_XS.gguf new file mode 100644 index 0000000..ab9cb8e --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6f908e4a6873d730a28757002cf69f3dcb09ba2584ea6a80267e93a0f13b8d65 +size 324410336 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_XXS.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..f798638 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:97d8c98d2533fb9886a2893239f6c0651a3b2dc116dda3899cd01d77c959f3fa +size 321550304 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_M.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_M.gguf new file mode 100644 index 0000000..a179f23 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a24f2f14947a78ca46b9536e0ba54c6ef316a4ed854c32ab273b83bb1c8be9a4 +size 342753248 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_S.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_S.gguf new file mode 100644 index 0000000..c30a47a --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0c609d155eef6f34ae5dc137d05b38f006dc12b2fa2040c4773edbdf9b965076 +size 338608352 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_XS.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_XS.gguf new file mode 100644 index 0000000..7ac9ef4 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:02ebd8bbd94e0df7490b76efffaee7ebc8abbf97ce763e18e8128cf267b9e23f +size 338608352 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_XXS.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..8231470 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:15e16bf7d2a35df4b7a0383fe5ee0f30e58d1b3462133f05879656a76230dc1e +size 333705440 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ4_XS.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ4_XS.gguf new file mode 100644 index 0000000..7292029 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37ac3e835b9561515f9c75d61557720e2cf0120706365b1a9127ca7f4d778a82 +size 349403360 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q2_K.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q2_K.gguf new file mode 100644 index 0000000..7edf0e8 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c55264c9f598dec874fdc138be2385c7f43b527bcf08cbe5080b0acf2a11b2fd +size 338608352 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_L.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_L.gguf new file mode 100644 index 0000000..f420d18 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f7a57d02d436fbe08e26b2b1d5ccd051885e1cdf8bb70e7c5fd1f4857882797b +size 369359072 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_M.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_M.gguf new file mode 100644 index 0000000..734c43b --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:30c89d87f65fad321178a3a94f8e0fc06169005b707802aaef2b5a7adf56e9a5 +size 355467488 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_S.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_S.gguf new file mode 100644 index 0000000..91d36ce --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:805cf0629878359380f035286fe046e73c6325319935a7c4af80aaa349c08c72 +size 338264288 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0.gguf new file mode 100644 index 0000000..3b9ec56 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6691d78fc44e1af185b1e962ea5fe85370c2a501e5b59d4a9b08de07a7adbae9 +size 352973024 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0_4_4.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0_4_4.gguf new file mode 100644 index 0000000..55e02d2 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0_4_4.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ce185a280e98c9868be7d07684b195bda1e59c1bf31e7eabae77aea7f18fd1eb +size 352155872 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0_4_8.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0_4_8.gguf new file mode 100644 index 0000000..e9eace5 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0_4_8.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:895cc747cb23edc1b55f8c38dcc969bc2266630ce179078d5eb71465691d47bc +size 352155872 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0_8_8.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0_8_8.gguf new file mode 100644 index 0000000..584fdfd --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0_8_8.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:711eb4c1f75e1dcf0702b8f6ff630f3c62b508055ec96ce1b0f1bb8dfb841b8f +size 352155872 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_K_M.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_K_M.gguf new file mode 100644 index 0000000..c35e7c0 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:29eb0d2ffed4286746caa25ce22b29e7128e6957749733adf4410f50730d1a68 +size 397808864 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_K_S.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_K_S.gguf new file mode 100644 index 0000000..a130897 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fb1a2ce0714527c088f63f62b0c7514d1874f6af200b027697fc1f0acccd1974 +size 385472736 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q5_K_M.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q5_K_M.gguf new file mode 100644 index 0000000..a295a69 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:13dc661c0a3ca8011aaf58de67abb7287a8be584ebbfa9a4d0f412498b52c5bb +size 420087008 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q5_K_S.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q5_K_S.gguf new file mode 100644 index 0000000..28f11a9 --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:53df4704eb6120558bc0684deca1ac27bf73e592d59e4bee7bb2a09b7cbd98df +size 412711136 diff --git a/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q6_K.gguf b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q6_K.gguf new file mode 100644 index 0000000..6eecbcb --- /dev/null +++ b/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:74c2d3058a29e420c189951f8169e49d1c86b1a106f6870a68fa21afa005bd9b +size 505737440 diff --git a/README.md b/README.md new file mode 100644 index 0000000..afc798b --- /dev/null +++ b/README.md @@ -0,0 +1,85 @@ +--- +base_model: huihui-ai/Qwen2.5-Coder-0.5B-Instruct-abliterated +language: +- en +library_name: transformers +license: apache-2.0 +license_link: https://huggingface.co/huihui-ai/Qwen2.5-Coder-0.5-Instruct-abliterate/blob/main/LICENSE +quantized_by: mradermacher +tags: +- code +- codeqwen +- chat +- qwen +- qwen-coder +- abliterated +- uncensored +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/huihui-ai/Qwen2.5-Coder-0.5B-Instruct-abliterated + + +static quants are available at https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ1_S.gguf) | i1-IQ1_S | 0.4 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ1_M.gguf) | i1-IQ1_M | 0.4 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 0.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_XS.gguf) | i1-IQ2_XS | 0.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_S.gguf) | i1-IQ2_S | 0.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ2_M.gguf) | i1-IQ2_M | 0.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 0.4 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_S.gguf) | i1-Q3_K_S | 0.4 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_S.gguf) | i1-IQ3_S | 0.4 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_XS.gguf) | i1-IQ3_XS | 0.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q2_K.gguf) | i1-Q2_K | 0.4 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ3_M.gguf) | i1-IQ3_M | 0.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-IQ4_XS.gguf) | i1-IQ4_XS | 0.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0_4_4.gguf) | i1-Q4_0_4_4 | 0.5 | fast on arm, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0_4_8.gguf) | i1-Q4_0_4_8 | 0.5 | fast on arm+i8mm, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0_8_8.gguf) | i1-Q4_0_8_8 | 0.5 | fast on arm+sve, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_0.gguf) | i1-Q4_0 | 0.5 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_M.gguf) | i1-Q3_K_M | 0.5 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q3_K_L.gguf) | i1-Q3_K_L | 0.5 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_K_S.gguf) | i1-Q4_K_S | 0.5 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q4_K_M.gguf) | i1-Q4_K_M | 0.5 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q5_K_S.gguf) | i1-Q5_K_S | 0.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q5_K_M.gguf) | i1-Q5_K_M | 0.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-0.5B-Instruct-abliterated-i1-GGUF/resolve/main/Qwen2.5-Coder-0.5B-Instruct-abliterated.i1-Q6_K.gguf) | i1-Q6_K | 0.6 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..c33034a Binary files /dev/null and b/imatrix.dat differ