commit da3dfa8a2b3c25046b0f628097fe0ee9a31ca6f9 Author: ModelHub XC Date: Thu Jun 4 23:32:18 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/QiMing-Socratic-12B-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..cc68a48 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +QiMing-Socratic-12B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/QiMing-Socratic-12B.i1-IQ1_M.gguf b/QiMing-Socratic-12B.i1-IQ1_M.gguf new file mode 100644 index 0000000..0169e5c --- /dev/null +++ b/QiMing-Socratic-12B.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:28d62607fb0332b8ee68d30b599ba2add2f5e6d43ebb373c618940b8a48eda98 +size 3164728544 diff --git a/QiMing-Socratic-12B.i1-IQ1_S.gguf b/QiMing-Socratic-12B.i1-IQ1_S.gguf new file mode 100644 index 0000000..9d45bb0 --- /dev/null +++ b/QiMing-Socratic-12B.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f23425367d29f4bf3ccd14ee974940fff78df29b969c85166c4b9fadde7cf766 +size 2947415264 diff --git a/QiMing-Socratic-12B.i1-IQ2_M.gguf b/QiMing-Socratic-12B.i1-IQ2_M.gguf new file mode 100644 index 0000000..ac895fb --- /dev/null +++ b/QiMing-Socratic-12B.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7768112fe3c6d7f1157349f9fe5dca3c0331362c6211bf900690e2e28478c5f0 +size 4310461664 diff --git a/QiMing-Socratic-12B.i1-IQ2_S.gguf b/QiMing-Socratic-12B.i1-IQ2_S.gguf new file mode 100644 index 0000000..a2bfa08 --- /dev/null +++ b/QiMing-Socratic-12B.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8a7a96e837125fa2fd9c9c708bc4a76791c8db650f2e6ad021accee30a22a812 +size 4020710624 diff --git a/QiMing-Socratic-12B.i1-IQ2_XS.gguf b/QiMing-Socratic-12B.i1-IQ2_XS.gguf new file mode 100644 index 0000000..62dad72 --- /dev/null +++ b/QiMing-Socratic-12B.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:27ab4ca33be6cbbc2b07c271d4676e63308c686d7a412f9d84a41200aa3b345a +size 3840261344 diff --git a/QiMing-Socratic-12B.i1-IQ2_XXS.gguf b/QiMing-Socratic-12B.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..462f2d4 --- /dev/null +++ b/QiMing-Socratic-12B.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3aa6d525d0c745c54eccc849b4910f81273c30525107f7dab567792a29164b36 +size 3526917344 diff --git a/QiMing-Socratic-12B.i1-IQ3_M.gguf b/QiMing-Socratic-12B.i1-IQ3_M.gguf new file mode 100644 index 0000000..d9f75d8 --- /dev/null +++ b/QiMing-Socratic-12B.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c24d64da53e1300a18f6dd6c982e82fe440fae98c7fdb2eb77e4e2737ebc2113 +size 5655723104 diff --git a/QiMing-Socratic-12B.i1-IQ3_S.gguf b/QiMing-Socratic-12B.i1-IQ3_S.gguf new file mode 100644 index 0000000..3a4b4f0 --- /dev/null +++ b/QiMing-Socratic-12B.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:805a9c63883794447e1e5ee506757f86d1bb49ae701e0fcd4c4c662f3e0f52d3 +size 5458316384 diff --git a/QiMing-Socratic-12B.i1-IQ3_XS.gguf b/QiMing-Socratic-12B.i1-IQ3_XS.gguf new file mode 100644 index 0000000..c5e6ee4 --- /dev/null +++ b/QiMing-Socratic-12B.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:560d762bda223c3e397f904a90eac40f1aa8504274e33ed8788ee77069f33476 +size 5206166624 diff --git a/QiMing-Socratic-12B.i1-IQ3_XXS.gguf b/QiMing-Socratic-12B.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..0fd1831 --- /dev/null +++ b/QiMing-Socratic-12B.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5a56107da7e4dafe0218366e750872e94923787d150b327e068cd53354dfce6f +size 4784901344 diff --git a/QiMing-Socratic-12B.i1-IQ4_NL.gguf b/QiMing-Socratic-12B.i1-IQ4_NL.gguf new file mode 100644 index 0000000..0ba7ce6 --- /dev/null +++ b/QiMing-Socratic-12B.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4b50d27daf3c002bde11fbb2f166cb294c51857d0767ed8dd50b0a65b9d1e1b8 +size 6887165024 diff --git a/QiMing-Socratic-12B.i1-IQ4_XS.gguf b/QiMing-Socratic-12B.i1-IQ4_XS.gguf new file mode 100644 index 0000000..39db1d1 --- /dev/null +++ b/QiMing-Socratic-12B.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c0f68bf7e3540fd44e679327e96ae09bb7e75dc1039bbabfe856d156acb3ee04 +size 6550965344 diff --git a/QiMing-Socratic-12B.i1-Q2_K.gguf b/QiMing-Socratic-12B.i1-Q2_K.gguf new file mode 100644 index 0000000..8afb816 --- /dev/null +++ b/QiMing-Socratic-12B.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:90e6a6bb8d82984ef5306eb94c05e3e4dc886d358c943040a23f3ef46dfea31e +size 4768222304 diff --git a/QiMing-Socratic-12B.i1-Q2_K_S.gguf b/QiMing-Socratic-12B.i1-Q2_K_S.gguf new file mode 100644 index 0000000..0179c11 --- /dev/null +++ b/QiMing-Socratic-12B.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b26c9a64212329dbe796cf8faae5f39cbefaaffd7f291d21c50c0ea7aa44b814 +size 4448611424 diff --git a/QiMing-Socratic-12B.i1-Q3_K_L.gguf b/QiMing-Socratic-12B.i1-Q3_K_L.gguf new file mode 100644 index 0000000..e249e43 --- /dev/null +++ b/QiMing-Socratic-12B.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:91cb1a410d2afcc52cfefaaca043951ea29051f152f9e2a76e43e1f62dde8ccc +size 6480186464 diff --git a/QiMing-Socratic-12B.i1-Q3_K_M.gguf b/QiMing-Socratic-12B.i1-Q3_K_M.gguf new file mode 100644 index 0000000..86fd7c0 --- /dev/null +++ b/QiMing-Socratic-12B.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4469339b5e10dce3c27b224d87082754bfb42116c0acdde423a8ffb499ee1aec +size 6008818784 diff --git a/QiMing-Socratic-12B.i1-Q3_K_S.gguf b/QiMing-Socratic-12B.i1-Q3_K_S.gguf new file mode 100644 index 0000000..1ff36e1 --- /dev/null +++ b/QiMing-Socratic-12B.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44eddae37e8820733a394577443f1a10fad18f8b69e7057a1392a06943043b5a +size 5458316384 diff --git a/QiMing-Socratic-12B.i1-Q4_0.gguf b/QiMing-Socratic-12B.i1-Q4_0.gguf new file mode 100644 index 0000000..146a0e6 --- /dev/null +++ b/QiMing-Socratic-12B.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bda377be0c0f72f0a9b32fcc585ba88b6df1c9303e3da3391dad01b07a1aa692 +size 6909283424 diff --git a/QiMing-Socratic-12B.i1-Q4_1.gguf b/QiMing-Socratic-12B.i1-Q4_1.gguf new file mode 100644 index 0000000..3b486a8 --- /dev/null +++ b/QiMing-Socratic-12B.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:25243bc7d94f7ec18656a95d17e5c1fcf850d6d77c6678f0b8a2db00b40f9572 +size 7559564384 diff --git a/QiMing-Socratic-12B.i1-Q4_K_M.gguf b/QiMing-Socratic-12B.i1-Q4_K_M.gguf new file mode 100644 index 0000000..9f3077f --- /dev/null +++ b/QiMing-Socratic-12B.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e74351a0ede73b1072a35131f9d6f3b7c5e8a087ed4ffb9b2476c84a6532df33 +size 7300779104 diff --git a/QiMing-Socratic-12B.i1-Q4_K_S.gguf b/QiMing-Socratic-12B.i1-Q4_K_S.gguf new file mode 100644 index 0000000..675b805 --- /dev/null +++ b/QiMing-Socratic-12B.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:34ebf1712fc304c1700ab5e0509aea2fbfca6da6e30da2a7552ef48d95144e76 +size 6935333984 diff --git a/QiMing-Socratic-12B.i1-Q5_K_M.gguf b/QiMing-Socratic-12B.i1-Q5_K_M.gguf new file mode 100644 index 0000000..7ac3079 --- /dev/null +++ b/QiMing-Socratic-12B.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1a6152dcd3e605adcb5c9cb0c888205cc1c30814f2bac4f0682c459adfb5944d +size 8445037664 diff --git a/QiMing-Socratic-12B.i1-Q5_K_S.gguf b/QiMing-Socratic-12B.i1-Q5_K_S.gguf new file mode 100644 index 0000000..134f808 --- /dev/null +++ b/QiMing-Socratic-12B.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:41810013fd033b4b3f5cd9bcc1bd0f079fce50c1c30792c00f086c7817b6e21b +size 8231963744 diff --git a/QiMing-Socratic-12B.i1-Q6_K.gguf b/QiMing-Socratic-12B.i1-Q6_K.gguf new file mode 100644 index 0000000..59deb03 --- /dev/null +++ b/QiMing-Socratic-12B.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:88fda9e954061dbdcfe6bbb2489f90fe971eadf3fc3bbe6b9057767379f38260 +size 9660812384 diff --git a/QiMing-Socratic-12B.imatrix.gguf b/QiMing-Socratic-12B.imatrix.gguf new file mode 100644 index 0000000..83b5302 --- /dev/null +++ b/QiMing-Socratic-12B.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:738aa327c1eb7b88a61c4e9d0b1178ba890d29cdb4edbb4a12597f728ab7354b +size 7473632 diff --git a/README.md b/README.md new file mode 100644 index 0000000..2a19ed7 --- /dev/null +++ b/README.md @@ -0,0 +1,99 @@ +--- +base_model: aifeifei798/QiMing-Socratic-12B +language: +- zh +- en +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- gemma +- gemma3 +- unsloth +- QiMing +- QiMing-holos +- bagua +- decision-making +- strategic-analysis +- cognitive-architecture +- chat +- lora +- philosophy-driven-ai +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/aifeifei798/QiMing-Socratic-12B + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#QiMing-Socratic-12B-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/QiMing-Socratic-12B-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-IQ1_S.gguf) | i1-IQ1_S | 3.0 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-IQ1_M.gguf) | i1-IQ1_M | 3.3 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 3.6 | | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 3.9 | | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-IQ2_S.gguf) | i1-IQ2_S | 4.1 | | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-IQ2_M.gguf) | i1-IQ2_M | 4.4 | | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 4.5 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-Q2_K.gguf) | i1-Q2_K | 4.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 4.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 5.3 | | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-IQ3_S.gguf) | i1-IQ3_S | 5.6 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 5.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-IQ3_M.gguf) | i1-IQ3_M | 5.8 | | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 6.1 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 6.6 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 6.7 | | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 7.0 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-Q4_0.gguf) | i1-Q4_0 | 7.0 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 7.0 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 7.4 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-Q4_1.gguf) | i1-Q4_1 | 7.7 | | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 8.3 | | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 8.5 | | +| [GGUF](https://huggingface.co/mradermacher/QiMing-Socratic-12B-i1-GGUF/resolve/main/QiMing-Socratic-12B.i1-Q6_K.gguf) | i1-Q6_K | 9.8 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +