commit 0141705bbfafb24006165e663f24c331a765ce65 Author: ModelHub XC Date: Fri May 22 16:48:16 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/NuExtract-2.0-8B-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..660c735 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +NuExtract-2.0-8B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/NuExtract-2.0-8B.i1-IQ1_M.gguf b/NuExtract-2.0-8B.i1-IQ1_M.gguf new file mode 100644 index 0000000..fbb7dbd --- /dev/null +++ b/NuExtract-2.0-8B.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d39e223846263cba0d22b32507d49cbe7eff5a85ec988cb731a69243d27f68ed +size 2042200064 diff --git a/NuExtract-2.0-8B.i1-IQ1_S.gguf b/NuExtract-2.0-8B.i1-IQ1_S.gguf new file mode 100644 index 0000000..0258726 --- /dev/null +++ b/NuExtract-2.0-8B.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c7abc8c07fc3f9441fc08d4f4bc4c1d41b8d90b2bd2809834f3df912d853ca8e +size 1903671296 diff --git a/NuExtract-2.0-8B.i1-IQ2_M.gguf b/NuExtract-2.0-8B.i1-IQ2_M.gguf new file mode 100644 index 0000000..918d97c --- /dev/null +++ b/NuExtract-2.0-8B.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa3b26f8d22ce97dbacbfadbc8351514e7652c3dc8f9e80753fb75d03cec6c57 +size 2780346368 diff --git a/NuExtract-2.0-8B.i1-IQ2_S.gguf b/NuExtract-2.0-8B.i1-IQ2_S.gguf new file mode 100644 index 0000000..adee18e --- /dev/null +++ b/NuExtract-2.0-8B.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aed47d473a2514039c5bc95d40228c7e854d789c109d73eecb84db22471f0e9a +size 2595641344 diff --git a/NuExtract-2.0-8B.i1-IQ2_XS.gguf b/NuExtract-2.0-8B.i1-IQ2_XS.gguf new file mode 100644 index 0000000..cbdce06 --- /dev/null +++ b/NuExtract-2.0-8B.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e02abe9bd4e29e2e2f405b1e5bd8d6a7b85c7fc1d7a5f26adbf31d16830bb8af +size 2469025792 diff --git a/NuExtract-2.0-8B.i1-IQ2_XXS.gguf b/NuExtract-2.0-8B.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..aef2b82 --- /dev/null +++ b/NuExtract-2.0-8B.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7dfc11b1eea8db17fbbeca8384f3047f5df314db98dba573285a2ffc3f3e2c07 +size 2273081344 diff --git a/NuExtract-2.0-8B.i1-IQ3_M.gguf b/NuExtract-2.0-8B.i1-IQ3_M.gguf new file mode 100644 index 0000000..a4a574d --- /dev/null +++ b/NuExtract-2.0-8B.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0afefb17b0ffbe5fac08bbe5b3b458e05116950a8de30462444b933d43f7b5e2 +size 3574016000 diff --git a/NuExtract-2.0-8B.i1-IQ3_S.gguf b/NuExtract-2.0-8B.i1-IQ3_S.gguf new file mode 100644 index 0000000..f3cbaf0 --- /dev/null +++ b/NuExtract-2.0-8B.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:472181a4affb2766c087b2d9ca18a48a0e9472fcc0cdf2ca9edc30de20bfd9cd +size 3499196416 diff --git a/NuExtract-2.0-8B.i1-IQ3_XS.gguf b/NuExtract-2.0-8B.i1-IQ3_XS.gguf new file mode 100644 index 0000000..b4ee861 --- /dev/null +++ b/NuExtract-2.0-8B.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6e5bdfd50bf00a559a8784e8a7a0a95d5ed98769a7d4f6329cf8e5de837703f9 +size 3346259968 diff --git a/NuExtract-2.0-8B.i1-IQ3_XXS.gguf b/NuExtract-2.0-8B.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..15a0c8c --- /dev/null +++ b/NuExtract-2.0-8B.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:90ddd14e97deb0bd774fa00f28d4a625ce8074d55625e64a852c5a18dc31c576 +size 3114518528 diff --git a/NuExtract-2.0-8B.i1-IQ4_NL.gguf b/NuExtract-2.0-8B.i1-IQ4_NL.gguf new file mode 100644 index 0000000..50d6eb5 --- /dev/null +++ b/NuExtract-2.0-8B.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:481e6f1ed3a3879040596d3755e4d189fa7947b04c2591905c9339d79cc35ed2 +size 4437817344 diff --git a/NuExtract-2.0-8B.i1-IQ4_XS.gguf b/NuExtract-2.0-8B.i1-IQ4_XS.gguf new file mode 100644 index 0000000..4c0e55d --- /dev/null +++ b/NuExtract-2.0-8B.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:65541facbda8cd25eb7053f4ba437cb4243afc0d1eb19fff8242e908a8229a1f +size 4218476544 diff --git a/NuExtract-2.0-8B.i1-Q2_K.gguf b/NuExtract-2.0-8B.i1-Q2_K.gguf new file mode 100644 index 0000000..872c43f --- /dev/null +++ b/NuExtract-2.0-8B.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c44c432b04557e38222e3b05e67d5947462a470646d17564b92f16b261920259 +size 3015944192 diff --git a/NuExtract-2.0-8B.i1-Q2_K_S.gguf b/NuExtract-2.0-8B.i1-Q2_K_S.gguf new file mode 100644 index 0000000..72c3b28 --- /dev/null +++ b/NuExtract-2.0-8B.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:096f434b2b42a50ffeb41d74e7b2a01e4fda451d3375de005d61a8dffc49f983 +size 2834077696 diff --git a/NuExtract-2.0-8B.i1-Q3_K_L.gguf b/NuExtract-2.0-8B.i1-Q3_K_L.gguf new file mode 100644 index 0000000..6481461 --- /dev/null +++ b/NuExtract-2.0-8B.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2bfcfedc2e486d7e9a051814267c59a4312cbe27a48c60909dbe5d8995d01f62 +size 4088463360 diff --git a/NuExtract-2.0-8B.i1-Q3_K_M.gguf b/NuExtract-2.0-8B.i1-Q3_K_M.gguf new file mode 100644 index 0000000..5425b8d --- /dev/null +++ b/NuExtract-2.0-8B.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6b278f35f81ff19b5942739dda6d275b11d061c7362640f3b1928d4266f6c64d +size 3808395264 diff --git a/NuExtract-2.0-8B.i1-Q3_K_S.gguf b/NuExtract-2.0-8B.i1-Q3_K_S.gguf new file mode 100644 index 0000000..18682fa --- /dev/null +++ b/NuExtract-2.0-8B.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1c091e0f298772ed42d519d538cdcd0e62f6313513b4a84d65eae6019ef19773 +size 3492372480 diff --git a/NuExtract-2.0-8B.i1-Q4_0.gguf b/NuExtract-2.0-8B.i1-Q4_0.gguf new file mode 100644 index 0000000..428f930 --- /dev/null +++ b/NuExtract-2.0-8B.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:33117a0574160cb31ee16b2f1af5ef1f6cafb6f2b3140e6acb4ced1c4986e8e0 +size 4444125184 diff --git a/NuExtract-2.0-8B.i1-Q4_1.gguf b/NuExtract-2.0-8B.i1-Q4_1.gguf new file mode 100644 index 0000000..a8277af --- /dev/null +++ b/NuExtract-2.0-8B.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6a974b216ade1cfa96f0230513ee2004f50fb0ab5650c842e1f31e7e7e6fe31a +size 4873287680 diff --git a/NuExtract-2.0-8B.i1-Q4_K_M.gguf b/NuExtract-2.0-8B.i1-Q4_K_M.gguf new file mode 100644 index 0000000..ce10ff1 --- /dev/null +++ b/NuExtract-2.0-8B.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:af0ec2ec0e409a16827f37454ae3cc99df72c7b00b1c208f66d0f93eae397223 +size 4683077632 diff --git a/NuExtract-2.0-8B.i1-Q4_K_S.gguf b/NuExtract-2.0-8B.i1-Q4_K_S.gguf new file mode 100644 index 0000000..0f7dc95 --- /dev/null +++ b/NuExtract-2.0-8B.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:32073b1e09dc69639cdf970b9a894c9cc263147ce1d79459917219711f57b6cf +size 4457773056 diff --git a/NuExtract-2.0-8B.i1-Q5_K_M.gguf b/NuExtract-2.0-8B.i1-Q5_K_M.gguf new file mode 100644 index 0000000..1e46697 --- /dev/null +++ b/NuExtract-2.0-8B.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1aa8c9f9578860e0d674ce863d312a3004792a48d4956153d724a77d18125e4f +size 5444835328 diff --git a/NuExtract-2.0-8B.i1-Q5_K_S.gguf b/NuExtract-2.0-8B.i1-Q5_K_S.gguf new file mode 100644 index 0000000..62e07c5 --- /dev/null +++ b/NuExtract-2.0-8B.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b13a876811b578727edca114d3cdfc6b0ca4c2869d655b7ba807cd1de37277a5 +size 5315180544 diff --git a/NuExtract-2.0-8B.i1-Q6_K.gguf b/NuExtract-2.0-8B.i1-Q6_K.gguf new file mode 100644 index 0000000..55ed275 --- /dev/null +++ b/NuExtract-2.0-8B.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9a16f197095b837a76504419bf092299e2626e7769fc4240fe8dbbaf5be456e9 +size 6254202880 diff --git a/README.md b/README.md new file mode 100644 index 0000000..61035be --- /dev/null +++ b/README.md @@ -0,0 +1,83 @@ +--- +base_model: numind/NuExtract-2.0-8B +language: +- en +library_name: transformers +license: mit +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/numind/NuExtract-2.0-8B + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#NuExtract-2.0-8B-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/NuExtract-2.0-8B-GGUF + +**This is a vision model - mmproj files (if any) will be in the [static repository](https://huggingface.co/mradermacher/NuExtract-2.0-8B-GGUF).** +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-IQ1_S.gguf) | i1-IQ1_S | 2.0 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-IQ1_M.gguf) | i1-IQ1_M | 2.1 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.6 | | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-IQ2_S.gguf) | i1-IQ2_S | 2.7 | | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-IQ2_M.gguf) | i1-IQ2_M | 2.9 | | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 2.9 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-Q2_K.gguf) | i1-Q2_K | 3.1 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.2 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.4 | | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-IQ3_S.gguf) | i1-IQ3_S | 3.6 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-IQ3_M.gguf) | i1-IQ3_M | 3.7 | | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 3.9 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.2 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.3 | | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 4.5 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-Q4_0.gguf) | i1-Q4_0 | 4.5 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.6 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 4.8 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-Q4_1.gguf) | i1-Q4_1 | 5.0 | | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.4 | | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.5 | | +| [GGUF](https://huggingface.co/mradermacher/NuExtract-2.0-8B-i1-GGUF/resolve/main/NuExtract-2.0-8B.i1-Q6_K.gguf) | i1-Q6_K | 6.4 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..b696b7a --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:00997e59ce75637f59d71224156a921e03d4e24c932b2556e0d98aadbca2b5e7 +size 4536665