commit 91c4a8b3a9ffb93e1fe615da364bd8fa9f60a117 Author: ModelHub XC Date: Thu Aug 6 19:39:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..579d31c --- /dev/null +++ b/.gitattributes @@ -0,0 +1,57 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ1_M.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ1_M.gguf new file mode 100644 index 0000000..eb96f2f --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0c52c8f189823322118c51375e6f1b20e15f7c4d69852f6b93ef54c3021b4a69 +size 5532994080 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ1_S.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ1_S.gguf new file mode 100644 index 0000000..f13e8c7 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:16c78cf75bc1da2c93f455a2e88c4ac5fa254510710660b70a6b3559ed35f15a +size 5010803232 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_M.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_M.gguf new file mode 100644 index 0000000..08b3633 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6e62f9cd70d6dfde87a957c323cbf598bb12e782eabd79bd7e922d8d93c48589 +size 7996828192 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_S.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_S.gguf new file mode 100644 index 0000000..cf4f2f6 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:68f33d834794b7f97de3b203e1fc4402f8df35885e2bb6fa0f35b19e3c316a7e +size 7300573728 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_XS.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_XS.gguf new file mode 100644 index 0000000..f7ad442 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1d02e0c0b54dac46edfc806b9e1e872a2f0cd4123cec0cfb8db3406e4042067e +size 7116343840 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_XXS.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..e0a260e --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1a2a1fe5250ae6252bb1e9a84a111aff3cf0bd4a04721d099c32a0ba379a0bcd +size 6403312160 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_M.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_M.gguf new file mode 100644 index 0000000..493a229 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6370478943b8ab20d792cd4f2a665955d954d827143fa8cb3ba5553ab0425ad3 +size 10647210528 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_S.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_S.gguf new file mode 100644 index 0000000..4f022b2 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4977fb2280ecb7ae1cf23bbac7d7f1bf5503012a019f755942cc87f35f62c24f +size 10451126816 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_XS.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_XS.gguf new file mode 100644 index 0000000..efe30ac --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6764802653e52207a8e7a0529bb1284db3dd3d8f18409ab143b0022ff8d19841 +size 9891187232 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_XXS.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..a74355f --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0ca04f9dba8c495c485e1c6f824ab7e0d63e93ff6dfb4bde75bc9318359f4a41 +size 9303353888 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ4_XS.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ4_XS.gguf new file mode 100644 index 0000000..942791b --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:15b641ffd6818aff707fa1fbb9e5d3c313d8ec6e3d6df277915cec81fa392305 +size 12893989408 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q2_K.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q2_K.gguf new file mode 100644 index 0000000..7f5cca8 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6aae6545d5409f3dfda108cb9d0476e10c77c561a345bb99e4e69b8453a09d94 +size 8842930720 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_L.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_L.gguf new file mode 100644 index 0000000..1fab8e6 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a3708e1f9b1c0aa9cbf7ec825105657f0aa6c1f68de31a648cf905d7e1adb953 +size 12544084512 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_M.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_M.gguf new file mode 100644 index 0000000..de9f0f4 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a3de63a2d1c30f2bc901d2c17568a2914cdee6d8b87ebe64ca2fbb2f693eb486 +size 11580443168 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_S.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_S.gguf new file mode 100644 index 0000000..7806f52 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1e135e18922925379bbf8b961ee495748756a32caed5a8834cd0a5011bbd6b1d +size 10433301024 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_0.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_0.gguf new file mode 100644 index 0000000..d169f62 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4a3a78d1f5c0e776f80c0e52d4b41b5a76f3dfdc89662c9257155150b843bbc3 +size 13682420256 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_K_M.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_K_M.gguf new file mode 100644 index 0000000..a52d1d8 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1c3ba63b6eb2ce44a9299d1f8acffb499441013d71ac692463fe69f59dff3d69 +size 14609885728 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_K_S.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_K_S.gguf new file mode 100644 index 0000000..f87763a --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:15bf779958fb17b0241fdacd1af7cafeee5fdb45e9c0423aca58850512999455 +size 13743237664 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q5_K_M.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q5_K_M.gguf new file mode 100644 index 0000000..98110d1 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:af585da9633b1f1972c501ff1ef90a21adcdef331d8f97ebee84f77d7be455b5 +size 17134463520 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q5_K_S.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q5_K_S.gguf new file mode 100644 index 0000000..22bdb20 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0baff028445d7d5b2a7cd6cdea56eace213427d5703eec469239b364552db8e5 +size 16626428448 diff --git a/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q6_K.gguf b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q6_K.gguf new file mode 100644 index 0000000..48c8606 --- /dev/null +++ b/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:939ed46b051bd70302d43d30d0e1a0a4f6308bff7086418b5f21b70201da1a81 +size 19816827424 diff --git a/README.md b/README.md new file mode 100644 index 0000000..7b8f34a --- /dev/null +++ b/README.md @@ -0,0 +1,75 @@ +--- +base_model: xxx777xxxASD/NeuralKunoichi-EroSumika-4x7B-128k +language: +- en +library_name: transformers +license: apache-2.0 +quantized_by: mradermacher +tags: +- merge +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/xxx777xxxASD/NeuralKunoichi-EroSumika-4x7B-128k + + +static quants are available at https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ1_S.gguf) | i1-IQ1_S | 5.1 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ1_M.gguf) | i1-IQ1_M | 5.6 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 6.5 | | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_XS.gguf) | i1-IQ2_XS | 7.2 | | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_S.gguf) | i1-IQ2_S | 7.4 | | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ2_M.gguf) | i1-IQ2_M | 8.1 | | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q2_K.gguf) | i1-Q2_K | 8.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 9.4 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_XS.gguf) | i1-IQ3_XS | 10.0 | | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_S.gguf) | i1-Q3_K_S | 10.5 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_S.gguf) | i1-IQ3_S | 10.6 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ3_M.gguf) | i1-IQ3_M | 10.7 | | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_M.gguf) | i1-Q3_K_M | 11.7 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q3_K_L.gguf) | i1-Q3_K_L | 12.6 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-IQ4_XS.gguf) | i1-IQ4_XS | 13.0 | | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_0.gguf) | i1-Q4_0 | 13.8 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_K_S.gguf) | i1-Q4_K_S | 13.8 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q4_K_M.gguf) | i1-Q4_K_M | 14.7 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q5_K_S.gguf) | i1-Q5_K_S | 16.7 | | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q5_K_M.gguf) | i1-Q5_K_M | 17.2 | | +| [GGUF](https://huggingface.co/mradermacher/NeuralKunoichi-EroSumika-4x7B-128k-i1-GGUF/resolve/main/NeuralKunoichi-EroSumika-4x7B-128k.i1-Q6_K.gguf) | i1-Q6_K | 19.9 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..522c042 --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d151a1716b097e5a1c167fd2d02f203b5fe5be53c3b599047e54395caf202242 +size 14164883