commit 08c59d67aea71c90d1fc8d1cac9d333f2599aaab Author: ModelHub XC Date: Tue Oct 6 10:05:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/CodeLlama-7B-KStack-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..f26efa1 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-Q4_0_4_8.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-7B-KStack.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/CodeLlama-7B-KStack.i1-IQ1_M.gguf b/CodeLlama-7B-KStack.i1-IQ1_M.gguf new file mode 100644 index 0000000..7df50fa --- /dev/null +++ b/CodeLlama-7B-KStack.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b480940fe3057b58047ae5fb9faeabb34d35bd62f5d5325e8256c36ab7f90f73 +size 1651038752 diff --git a/CodeLlama-7B-KStack.i1-IQ1_S.gguf b/CodeLlama-7B-KStack.i1-IQ1_S.gguf new file mode 100644 index 0000000..79378de --- /dev/null +++ b/CodeLlama-7B-KStack.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:add0e53acae43de872a0ebbdf70eb8970928c74813385129fc1f57fd6042a86e +size 1528650272 diff --git a/CodeLlama-7B-KStack.i1-IQ2_M.gguf b/CodeLlama-7B-KStack.i1-IQ2_M.gguf new file mode 100644 index 0000000..7d7c6d9 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c108603e4bad41eb49fb7c04a207e4966dd3add33c9a3367c1663b04469338b1 +size 2359825440 diff --git a/CodeLlama-7B-KStack.i1-IQ2_S.gguf b/CodeLlama-7B-KStack.i1-IQ2_S.gguf new file mode 100644 index 0000000..2ac3302 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ec42ad4e9707b98f9924e0b8c523cab2d3ca7751ffca0b4d322697075472731 +size 2196640800 diff --git a/CodeLlama-7B-KStack.i1-IQ2_XS.gguf b/CodeLlama-7B-KStack.i1-IQ2_XS.gguf new file mode 100644 index 0000000..ac982b5 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da2e2d2b406a86c601a67122b3067505c555f9e4a80b0535d1f70c3352fb4847 +size 2034981408 diff --git a/CodeLlama-7B-KStack.i1-IQ2_XXS.gguf b/CodeLlama-7B-KStack.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..d321901 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f61567f1eb4dd3b05ec01d874136328501bf17f30b0e89d646eb29e1d316cf81 +size 1855019552 diff --git a/CodeLlama-7B-KStack.i1-IQ3_M.gguf b/CodeLlama-7B-KStack.i1-IQ3_M.gguf new file mode 100644 index 0000000..b181a00 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5aef12a21f3a9c69fdf253432ecda1eb8ef3f01446d820bd7dafa489721b3d1d +size 3114948128 diff --git a/CodeLlama-7B-KStack.i1-IQ3_S.gguf b/CodeLlama-7B-KStack.i1-IQ3_S.gguf new file mode 100644 index 0000000..8276bc6 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6c21f4576f39abca2b7dd0096bea597c23333bd7f1854def8110d1b85baac3a0 +size 2948388384 diff --git a/CodeLlama-7B-KStack.i1-IQ3_XS.gguf b/CodeLlama-7B-KStack.i1-IQ3_XS.gguf new file mode 100644 index 0000000..f319958 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7e7ad5dcca884f2cf58cdb2d16a19cc68d47feb575b9b06fd830402f5b375acc +size 2796607008 diff --git a/CodeLlama-7B-KStack.i1-IQ3_XXS.gguf b/CodeLlama-7B-KStack.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..3d5aa7b --- /dev/null +++ b/CodeLlama-7B-KStack.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3b3c591f30aba6d566471e19000b3cfdb872cc7eaed032cdb5d1c43395081e9c +size 2585465888 diff --git a/CodeLlama-7B-KStack.i1-IQ4_XS.gguf b/CodeLlama-7B-KStack.i1-IQ4_XS.gguf new file mode 100644 index 0000000..74ca2a0 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e2cd20b5df7ed223fdd5b4e0eee618d03b554ee22ed8847cb7dc639ca561701f +size 3619426336 diff --git a/CodeLlama-7B-KStack.i1-Q2_K.gguf b/CodeLlama-7B-KStack.i1-Q2_K.gguf new file mode 100644 index 0000000..ffc4553 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:26576ac46b92ab90460ffcfd3c3beee7892a301b2d27a14e7cc425bd9f68a7f4 +size 2532940832 diff --git a/CodeLlama-7B-KStack.i1-Q3_K_L.gguf b/CodeLlama-7B-KStack.i1-Q3_K_L.gguf new file mode 100644 index 0000000..2c552d7 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a590170a2ac2f40b4d94d49c3e2903c6b6abcf0807f47db0f68318316582fee0 +size 3597194784 diff --git a/CodeLlama-7B-KStack.i1-Q3_K_M.gguf b/CodeLlama-7B-KStack.i1-Q3_K_M.gguf new file mode 100644 index 0000000..ba7c5b5 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a29ea0bd46c68e69201b8036274b655a93a91e787338fabb3f9e757ef57cbb8a +size 3298088480 diff --git a/CodeLlama-7B-KStack.i1-Q3_K_S.gguf b/CodeLlama-7B-KStack.i1-Q3_K_S.gguf new file mode 100644 index 0000000..2f875c9 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d23d404bb75f5dd71e5b98af060d53daf89a69b64f25909acee8becf262de4b5 +size 2948388384 diff --git a/CodeLlama-7B-KStack.i1-Q4_0.gguf b/CodeLlama-7B-KStack.i1-Q4_0.gguf new file mode 100644 index 0000000..79a472f --- /dev/null +++ b/CodeLlama-7B-KStack.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9efa7ae627f6fc8979d51b5f0387dbc264267f504490ed4ad160b0f2ac880627 +size 3837171744 diff --git a/CodeLlama-7B-KStack.i1-Q4_0_4_4.gguf b/CodeLlama-7B-KStack.i1-Q4_0_4_4.gguf new file mode 100644 index 0000000..aebd87d --- /dev/null +++ b/CodeLlama-7B-KStack.i1-Q4_0_4_4.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7e300773ebbac8def91b7a271cfaf921ee58323f81e23403953e1829ed984567 +size 3825899552 diff --git a/CodeLlama-7B-KStack.i1-Q4_0_4_8.gguf b/CodeLlama-7B-KStack.i1-Q4_0_4_8.gguf new file mode 100644 index 0000000..d129e9a --- /dev/null +++ b/CodeLlama-7B-KStack.i1-Q4_0_4_8.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:24488b665ca4f4194b65aea2dfe59b3ce0e775526c2f7a1cd3e4c4c115ab5373 +size 3825899552 diff --git a/CodeLlama-7B-KStack.i1-Q4_0_8_8.gguf b/CodeLlama-7B-KStack.i1-Q4_0_8_8.gguf new file mode 100644 index 0000000..c779317 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-Q4_0_8_8.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:58018bc57228b792338ec3a0a4e742636352c6a5450120d6dd7df453e972fd03 +size 3825899552 diff --git a/CodeLlama-7B-KStack.i1-Q4_K_M.gguf b/CodeLlama-7B-KStack.i1-Q4_K_M.gguf new file mode 100644 index 0000000..ca63a30 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:88fd0b2a6c4e5c0c1b94d87364c2a07fca00f6b861e8d27f2abadf7d82e9f6d8 +size 4081096736 diff --git a/CodeLlama-7B-KStack.i1-Q4_K_S.gguf b/CodeLlama-7B-KStack.i1-Q4_K_S.gguf new file mode 100644 index 0000000..0e510cd --- /dev/null +++ b/CodeLlama-7B-KStack.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f61c4a5307fe4aa9e9aeeb012fec42321b1fe3573d7292dcb9286e8b7a8f59ea +size 3856832544 diff --git a/CodeLlama-7B-KStack.i1-Q5_K_M.gguf b/CodeLlama-7B-KStack.i1-Q5_K_M.gguf new file mode 100644 index 0000000..b1199b1 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6ad6bc6ac533e39d711623b3830ee958fffb26b0e45dc659858ae5040486f7d9 +size 4783257632 diff --git a/CodeLlama-7B-KStack.i1-Q5_K_S.gguf b/CodeLlama-7B-KStack.i1-Q5_K_S.gguf new file mode 100644 index 0000000..38b3f07 --- /dev/null +++ b/CodeLlama-7B-KStack.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2c5c3946fd547ed489737234c0570698110de0bad862e5925c974b611eeebba1 +size 4651792416 diff --git a/CodeLlama-7B-KStack.i1-Q6_K.gguf b/CodeLlama-7B-KStack.i1-Q6_K.gguf new file mode 100644 index 0000000..25a732d --- /dev/null +++ b/CodeLlama-7B-KStack.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6bd463d5f080cd0d8485fe377666de599d67bedc4bc285fececb7d63aed811e7 +size 5529303584 diff --git a/README.md b/README.md new file mode 100644 index 0000000..e96f68e --- /dev/null +++ b/README.md @@ -0,0 +1,80 @@ +--- +base_model: JetBrains/CodeLlama-7B-KStack +datasets: +- JetBrains/KStack +language: +- en +library_name: transformers +license: apache-2.0 +quantized_by: mradermacher +tags: +- code +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/JetBrains/CodeLlama-7B-KStack + + +static quants are available at https://huggingface.co/mradermacher/CodeLlama-7B-KStack-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-IQ1_S.gguf) | i1-IQ1_S | 1.6 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-IQ1_M.gguf) | i1-IQ1_M | 1.8 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.0 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.1 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-IQ2_S.gguf) | i1-IQ2_S | 2.3 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-IQ2_M.gguf) | i1-IQ2_M | 2.5 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-Q2_K.gguf) | i1-Q2_K | 2.6 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 2.7 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-IQ3_XS.gguf) | i1-IQ3_XS | 2.9 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-IQ3_S.gguf) | i1-IQ3_S | 3.0 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.0 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-IQ3_M.gguf) | i1-IQ3_M | 3.2 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-Q3_K_M.gguf) | i1-Q3_K_M | 3.4 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-Q3_K_L.gguf) | i1-Q3_K_L | 3.7 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-IQ4_XS.gguf) | i1-IQ4_XS | 3.7 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-Q4_0_4_4.gguf) | i1-Q4_0_4_4 | 3.9 | fast on arm, low quality | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-Q4_0_4_8.gguf) | i1-Q4_0_4_8 | 3.9 | fast on arm+i8mm, low quality | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-Q4_0_8_8.gguf) | i1-Q4_0_8_8 | 3.9 | fast on arm+sve, low quality | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-Q4_0.gguf) | i1-Q4_0 | 3.9 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.0 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-Q4_K_M.gguf) | i1-Q4_K_M | 4.2 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-Q5_K_S.gguf) | i1-Q5_K_S | 4.8 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-Q5_K_M.gguf) | i1-Q5_K_M | 4.9 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-7B-KStack-i1-GGUF/resolve/main/CodeLlama-7B-KStack.i1-Q6_K.gguf) | i1-Q6_K | 5.6 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..65fa935 --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f88e022b69891843edb2d8b97783ad6ecaa9c579e158c2a243de57e2cf005b55 +size 4562173