commit b5ad311899040584cf2fae8225c01557b81131d7 Author: ModelHub XC Date: Sun May 31 02:43:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..626e76e --- /dev/null +++ b/.gitattributes @@ -0,0 +1,59 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Peach-2.0-9B-8k-Roleplay.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Peach-2.0-9B-8k-Roleplay.i1-IQ1_M.gguf b/Peach-2.0-9B-8k-Roleplay.i1-IQ1_M.gguf new file mode 100644 index 0000000..62baa5c --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:beb8c7e8cb91c0284e1e0d32eedf1e2680691b1091902c5bdd85eafbe5f7b0b3 +size 2181641632 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-IQ1_S.gguf b/Peach-2.0-9B-8k-Roleplay.i1-IQ1_S.gguf new file mode 100644 index 0000000..9f71ca1 --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b881beaaa570486346d820f15889fd068b4796058531222eb8628a0d5f8d9a1b +size 2014573984 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-IQ2_M.gguf b/Peach-2.0-9B-8k-Roleplay.i1-IQ2_M.gguf new file mode 100644 index 0000000..ec6cbe9 --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f67a4c4d4447ef25c6db279b89ee21de38e944e0bc2e962e8ae16f740e97dc74 +size 3098113440 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-IQ2_S.gguf b/Peach-2.0-9B-8k-Roleplay.i1-IQ2_S.gguf new file mode 100644 index 0000000..38d223a --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:961c38a61c2d10202ee19c1813c13b1b004a0a91fc575f87cfa1633b37f36eb7 +size 2875356576 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-IQ2_XS.gguf b/Peach-2.0-9B-8k-Roleplay.i1-IQ2_XS.gguf new file mode 100644 index 0000000..fdd461d --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:13ae1b0b752238bc6ec074794a7320e13f123caa89bb9a8c4f05ed52aeeab7b3 +size 2708010400 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-IQ2_XXS.gguf b/Peach-2.0-9B-8k-Roleplay.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..1a6c1b2 --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d6a7d7da0c018f53b5aca69393710f3121688ceb6250e58f8cfb8f822dba7966 +size 2460087712 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-IQ3_M.gguf b/Peach-2.0-9B-8k-Roleplay.i1-IQ3_M.gguf new file mode 100644 index 0000000..6e02cdd --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc48bd668e780b72afb0ff60441dd53a43a1d1f8644e999d6ffc7d88da0dc956 +size 4055463328 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-IQ3_S.gguf b/Peach-2.0-9B-8k-Roleplay.i1-IQ3_S.gguf new file mode 100644 index 0000000..3900904 --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9d4b67a98c7a47ef6349889196032f9c7a788fba5f557b37aeeefd7edcbf5014 +size 3912578464 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-IQ3_XS.gguf b/Peach-2.0-9B-8k-Roleplay.i1-IQ3_XS.gguf new file mode 100644 index 0000000..a6d5db2 --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5d7f598b56e2d2164bf7aab70e58c84088e6106e7eb9077983f1ad64cea2c8f0 +size 3717936544 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-IQ3_XXS.gguf b/Peach-2.0-9B-8k-Roleplay.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..1434ee9 --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:75477197d8398bcb7238649c0a42aca1acb68a76a77ea53b0f133aad7bef7ce3 +size 3474322848 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-IQ4_NL.gguf b/Peach-2.0-9B-8k-Roleplay.i1-IQ4_NL.gguf new file mode 100644 index 0000000..20c511e --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3f9dab457c037ff21a1f5bc4b0b8be2f9cebb211c2152f60f4dba99919915ec9 +size 5049578912 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-IQ4_XS.gguf b/Peach-2.0-9B-8k-Roleplay.i1-IQ4_XS.gguf new file mode 100644 index 0000000..ec9c570 --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:382e65e7740a61fb134b52354297397b21fd25ff19a1d65bc9d238f534731bf7 +size 4785010080 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-Q2_K.gguf b/Peach-2.0-9B-8k-Roleplay.i1-Q2_K.gguf new file mode 100644 index 0000000..1556cfa --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3b3e086f3c2d442d68ad93574f37cb9bcb91e3690f28831977742f99e29b1955 +size 3354326432 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-Q2_K_S.gguf b/Peach-2.0-9B-8k-Roleplay.i1-Q2_K_S.gguf new file mode 100644 index 0000000..58e1024 --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:05c9d8b0ea53854adaef4664b2de179a30806c13ff90d247690eadbdb5cfa73c +size 3116135840 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-Q3_K_L.gguf b/Peach-2.0-9B-8k-Roleplay.i1-Q3_K_L.gguf new file mode 100644 index 0000000..7860eac --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7a7cebe81c3806e702c5f64d9945c49e266270a625acbed549e72bda09e906b8 +size 4690752928 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-Q3_K_M.gguf b/Peach-2.0-9B-8k-Roleplay.i1-Q3_K_M.gguf new file mode 100644 index 0000000..7b068fd --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bc8e0667f3c9994d1dc07e2012d6a43799f91ba8903aea3e0242e9ecac1cd457 +size 4324406688 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-Q3_K_S.gguf b/Peach-2.0-9B-8k-Roleplay.i1-Q3_K_S.gguf new file mode 100644 index 0000000..54d461c --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44d52e65f6f7423ff88141b2ac0c7eee6ef2cf5bcc98be1b39ef8ec2ad2dc120 +size 3899209120 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-Q4_0.gguf b/Peach-2.0-9B-8k-Roleplay.i1-Q4_0.gguf new file mode 100644 index 0000000..8b52dac --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5ae2d86b390eb23d48c0e21df292e10fad6d61f9d29c4c8da6a6745fc6631417 +size 5053904288 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-Q4_1.gguf b/Peach-2.0-9B-8k-Roleplay.i1-Q4_1.gguf new file mode 100644 index 0000000..2da89c4 --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4c066e120c4e4d46c7dbf4c033159524bdd5d20fd15478239bd10a4b2bd8dd84 +size 5572425120 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-Q4_K_M.gguf b/Peach-2.0-9B-8k-Roleplay.i1-Q4_K_M.gguf new file mode 100644 index 0000000..321bc68 --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e2514d2c1ef8717cb033697679b234d870ce21d3b9787a63f078386c9ac224ac +size 5328958880 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-Q4_K_S.gguf b/Peach-2.0-9B-8k-Roleplay.i1-Q4_K_S.gguf new file mode 100644 index 0000000..bd6904f --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:473f22fae05bb13fbbba4da30bea55587516105e3615f4e95f53f93f7655f9ff +size 5071861152 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-Q5_K_M.gguf b/Peach-2.0-9B-8k-Roleplay.i1-Q5_K_M.gguf new file mode 100644 index 0000000..ceaa92e --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:50e8a5d799b69bb33128f2dc56df526c2de59f3e8aca32eea962705daeb9c8a5 +size 6258259360 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-Q5_K_S.gguf b/Peach-2.0-9B-8k-Roleplay.i1-Q5_K_S.gguf new file mode 100644 index 0000000..7659b5e --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3a563c1a437ce550df32813a33af2449e28c53a65d4f00a8ebada45b95a8dc53 +size 6107854240 diff --git a/Peach-2.0-9B-8k-Roleplay.i1-Q6_K.gguf b/Peach-2.0-9B-8k-Roleplay.i1-Q6_K.gguf new file mode 100644 index 0000000..70fcffa --- /dev/null +++ b/Peach-2.0-9B-8k-Roleplay.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2bec1d5c142d14c2959b429b9684101809fbaf99cc018b9a911a471458c7914 +size 7245641120 diff --git a/README.md b/README.md new file mode 100644 index 0000000..8980ebf --- /dev/null +++ b/README.md @@ -0,0 +1,81 @@ +--- +base_model: ClosedCharacter/Peach-2.0-9B-8k-Roleplay +language: +- zh +- en +library_name: transformers +license: mit +quantized_by: mradermacher +tags: +- roleplay +- rp +- character +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/ClosedCharacter/Peach-2.0-9B-8k-Roleplay + + +static quants are available at https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-IQ1_S.gguf) | i1-IQ1_S | 2.1 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-IQ1_M.gguf) | i1-IQ1_M | 2.3 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.6 | | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.8 | | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-IQ2_S.gguf) | i1-IQ2_S | 3.0 | | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-IQ2_M.gguf) | i1-IQ2_M | 3.2 | | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.2 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-Q2_K.gguf) | i1-Q2_K | 3.5 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.6 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.8 | | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-Q3_K_S.gguf) | i1-Q3_K_S | 4.0 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-IQ3_S.gguf) | i1-IQ3_S | 4.0 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-IQ3_M.gguf) | i1-IQ3_M | 4.2 | | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.4 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.8 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.9 | | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-IQ4_NL.gguf) | i1-IQ4_NL | 5.1 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-Q4_0.gguf) | i1-Q4_0 | 5.2 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-Q4_K_S.gguf) | i1-Q4_K_S | 5.2 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.4 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-Q4_1.gguf) | i1-Q4_1 | 5.7 | | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-Q5_K_S.gguf) | i1-Q5_K_S | 6.2 | | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-Q5_K_M.gguf) | i1-Q5_K_M | 6.4 | | +| [GGUF](https://huggingface.co/mradermacher/Peach-2.0-9B-8k-Roleplay-i1-GGUF/resolve/main/Peach-2.0-9B-8k-Roleplay.i1-Q6_K.gguf) | i1-Q6_K | 7.3 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +