commit a9c911218fa0ec5581eac9b99b93efc724bb923f Author: ModelHub XC Date: Fri May 15 07:16:05 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..7a4880d --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +IntentRL-Ambig-CoQA-4B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/IntentRL-Ambig-CoQA-4B.i1-IQ1_M.gguf b/IntentRL-Ambig-CoQA-4B.i1-IQ1_M.gguf new file mode 100644 index 0000000..13b7021 --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:826cc12303637d9c8f2c0074c44804035c674fc55c888960fde4b116fc39752e +size 1254643360 diff --git a/IntentRL-Ambig-CoQA-4B.i1-IQ1_S.gguf b/IntentRL-Ambig-CoQA-4B.i1-IQ1_S.gguf new file mode 100644 index 0000000..37a60ee --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:172ca5a297f9a8a4ea601778c58b40d3c74b3072fd362297afdc4d04c63a3b51 +size 1182881440 diff --git a/IntentRL-Ambig-CoQA-4B.i1-IQ2_M.gguf b/IntentRL-Ambig-CoQA-4B.i1-IQ2_M.gguf new file mode 100644 index 0000000..d7d6913 --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:36ac3c8582e969e668bcebd549b6ba058fa40964fa3f132a55af1a16e77bfa3a +size 1680112800 diff --git a/IntentRL-Ambig-CoQA-4B.i1-IQ2_S.gguf b/IntentRL-Ambig-CoQA-4B.i1-IQ2_S.gguf new file mode 100644 index 0000000..5c53d5f --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d64896293941d1a0fdfe24406baeb6ede8c338324e996ff1e011fa22e4bb9328 +size 1584430240 diff --git a/IntentRL-Ambig-CoQA-4B.i1-IQ2_XS.gguf b/IntentRL-Ambig-CoQA-4B.i1-IQ2_XS.gguf new file mode 100644 index 0000000..51efdbb --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:10b9b01976d20cd6c5076b30ec3ae764e58792754531ec98b9877fe79c5aaf70 +size 1481725600 diff --git a/IntentRL-Ambig-CoQA-4B.i1-IQ2_XXS.gguf b/IntentRL-Ambig-CoQA-4B.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..281c987 --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8e125c74e29ac90109cd9662f9db92a974c5f39b0c4f8265df9f915f6016b817 +size 1374246560 diff --git a/IntentRL-Ambig-CoQA-4B.i1-IQ3_M.gguf b/IntentRL-Ambig-CoQA-4B.i1-IQ3_M.gguf new file mode 100644 index 0000000..5e02ef7 --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d2adab63e004c1b3e7d6f66838fb00c81a4550670ea552c8a44442dc44cad4a3 +size 2130025120 diff --git a/IntentRL-Ambig-CoQA-4B.i1-IQ3_S.gguf b/IntentRL-Ambig-CoQA-4B.i1-IQ3_S.gguf new file mode 100644 index 0000000..1225cdb --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5ec6132b71c2c1e5db16dfbd8e86ace3a8a66a645588b5b59dabaf86ccd79f52 +size 2066660000 diff --git a/IntentRL-Ambig-CoQA-4B.i1-IQ3_XS.gguf b/IntentRL-Ambig-CoQA-4B.i1-IQ3_XS.gguf new file mode 100644 index 0000000..48536e0 --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:79e86a4a57eb1fa77cefd282cc2a58116997a0d718a4096ec440b335c9e1b1ee +size 1981504160 diff --git a/IntentRL-Ambig-CoQA-4B.i1-IQ3_XXS.gguf b/IntentRL-Ambig-CoQA-4B.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..c0234fb --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:98465d3688a14e37b8eab9aaa68bd1e849b69f190d92124ae3ccbdbc43c8552c +size 1837317280 diff --git a/IntentRL-Ambig-CoQA-4B.i1-IQ4_NL.gguf b/IntentRL-Ambig-CoQA-4B.i1-IQ4_NL.gguf new file mode 100644 index 0000000..da2f7bc --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1531f74c0cf8f8bf4626306e294eb0e9546dd99e640e21fbd50e1691d47b1b15 +size 2600130720 diff --git a/IntentRL-Ambig-CoQA-4B.i1-IQ4_XS.gguf b/IntentRL-Ambig-CoQA-4B.i1-IQ4_XS.gguf new file mode 100644 index 0000000..d27c83c --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:87bf0258c50406dfe4e3c68ec29add9404c86e1742e161417b79dcadc0420172 +size 2477383840 diff --git a/IntentRL-Ambig-CoQA-4B.i1-Q2_K.gguf b/IntentRL-Ambig-CoQA-4B.i1-Q2_K.gguf new file mode 100644 index 0000000..193eace --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:747f571d3499555556502be22a64b16ab43348a5a717955153b2263f46b08273 +size 1797125280 diff --git a/IntentRL-Ambig-CoQA-4B.i1-Q2_K_S.gguf b/IntentRL-Ambig-CoQA-4B.i1-Q2_K_S.gguf new file mode 100644 index 0000000..352acb2 --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6f33998a11a122532f38a2f43f761e0dc032f834f92aaa6daff33193749ff83e +size 1691079840 diff --git a/IntentRL-Ambig-CoQA-4B.i1-Q3_K_L.gguf b/IntentRL-Ambig-CoQA-4B.i1-Q3_K_L.gguf new file mode 100644 index 0000000..d8cc5c8 --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:59c5c92c26e2a46798759d8179ac0c45e4884335065e2a080cd387a415e149c8 +size 2406914720 diff --git a/IntentRL-Ambig-CoQA-4B.i1-Q3_K_M.gguf b/IntentRL-Ambig-CoQA-4B.i1-Q3_K_M.gguf new file mode 100644 index 0000000..de9d73a --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b1a192dda9782b8e460a7555cd4cda37caf802cfbf91e52a270c4b01a3dc0265 +size 2242747040 diff --git a/IntentRL-Ambig-CoQA-4B.i1-Q3_K_S.gguf b/IntentRL-Ambig-CoQA-4B.i1-Q3_K_S.gguf new file mode 100644 index 0000000..05e1d67 --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:196f049ff1caef14104969c9930b0d78010ea183157b4b9a0b9cd6411f0c3881 +size 2054126240 diff --git a/IntentRL-Ambig-CoQA-4B.i1-Q4_0.gguf b/IntentRL-Ambig-CoQA-4B.i1-Q4_0.gguf new file mode 100644 index 0000000..5ccb374 --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0aa72707bcd00092c030ed6d4afdc3c4f4bf7ccf31d8ddc9d2c33144f1e8961c +size 2594560160 diff --git a/IntentRL-Ambig-CoQA-4B.i1-Q4_1.gguf b/IntentRL-Ambig-CoQA-4B.i1-Q4_1.gguf new file mode 100644 index 0000000..383c445 --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7c5b212a50c0984f1538a93d21f1a53f3ff2462f84c23c29cbdd524886818006 +size 2839726240 diff --git a/IntentRL-Ambig-CoQA-4B.i1-Q4_K_M.gguf b/IntentRL-Ambig-CoQA-4B.i1-Q4_K_M.gguf new file mode 100644 index 0000000..be278b7 --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:db34acc4dc78aed1b35a3d184d4a068dc541ae55a3ad865a9ad5a4e06098a745 +size 2716068000 diff --git a/IntentRL-Ambig-CoQA-4B.i1-Q4_K_S.gguf b/IntentRL-Ambig-CoQA-4B.i1-Q4_K_S.gguf new file mode 100644 index 0000000..321264d --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ed359cfe9247561b90db9c831b95a17ca2f78fb90b434475d8aa30f7c7b9f6b2 +size 2602096800 diff --git a/IntentRL-Ambig-CoQA-4B.i1-Q5_K_M.gguf b/IntentRL-Ambig-CoQA-4B.i1-Q5_K_M.gguf new file mode 100644 index 0000000..39a0cdb --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:63774b3c5d16ba8695e8dd4294dc476b3014826c1b3eff04a3e40ffc8722b291 +size 3156920480 diff --git a/IntentRL-Ambig-CoQA-4B.i1-Q5_K_S.gguf b/IntentRL-Ambig-CoQA-4B.i1-Q5_K_S.gguf new file mode 100644 index 0000000..d65919d --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:796900b8ba9cbd84f3d2b266c33f50b4f23292045466b2f0f20aade07ee40832 +size 3091118240 diff --git a/IntentRL-Ambig-CoQA-4B.i1-Q6_K.gguf b/IntentRL-Ambig-CoQA-4B.i1-Q6_K.gguf new file mode 100644 index 0000000..aebc2d8 --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4def80d9525bc561e8a02b1f36c7c8e782223573a2bedfbe4bbfb437758081d4 +size 3625326240 diff --git a/IntentRL-Ambig-CoQA-4B.imatrix.gguf b/IntentRL-Ambig-CoQA-4B.imatrix.gguf new file mode 100644 index 0000000..b189605 --- /dev/null +++ b/IntentRL-Ambig-CoQA-4B.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f4c7236772b21a1b943f524f7612b97b8ef501819fc3ea0226dfd7dd778ae7a4 +size 3872640 diff --git a/README.md b/README.md new file mode 100644 index 0000000..586f820 --- /dev/null +++ b/README.md @@ -0,0 +1,91 @@ +--- +base_model: irisaparina/IntentRL-Ambig-CoQA-4B +language: +- en +library_name: transformers +license: mit +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- question-answering +- conversational-qa +- ambiguity +- reinforcement-learning +- grpo +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/irisaparina/IntentRL-Ambig-CoQA-4B + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#IntentRL-Ambig-CoQA-4B-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-IQ1_S.gguf) | i1-IQ1_S | 1.3 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-IQ1_M.gguf) | i1-IQ1_M | 1.4 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-IQ2_S.gguf) | i1-IQ2_S | 1.7 | | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-IQ2_M.gguf) | i1-IQ2_M | 1.8 | | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.8 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-Q2_K.gguf) | i1-Q2_K | 1.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 2.1 | | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 2.2 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-IQ3_S.gguf) | i1-IQ3_S | 2.2 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-IQ3_M.gguf) | i1-IQ3_M | 2.2 | | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 2.3 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 2.5 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 2.6 | | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-Q4_0.gguf) | i1-Q4_0 | 2.7 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.7 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.7 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.8 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-Q4_1.gguf) | i1-Q4_1 | 2.9 | | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 3.2 | | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 3.3 | | +| [GGUF](https://huggingface.co/mradermacher/IntentRL-Ambig-CoQA-4B-i1-GGUF/resolve/main/IntentRL-Ambig-CoQA-4B.i1-Q6_K.gguf) | i1-Q6_K | 3.7 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +