commit 0bee346a18c66c170e07e30cf6381d713a6dcc57 Author: ModelHub XC Date: Mon Oct 19 22:56:21 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/CodeLlama-13b-Python-hf-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..a235fb3 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-Q4_0_4_8.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text +CodeLlama-13b-Python-hf.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/CodeLlama-13b-Python-hf.i1-IQ1_M.gguf b/CodeLlama-13b-Python-hf.i1-IQ1_M.gguf new file mode 100644 index 0000000..5368b60 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:55a41f3a569b5103e3f92a138f4bf2718290540e3e4fed116b6ea62441fba49f +size 3138610304 diff --git a/CodeLlama-13b-Python-hf.i1-IQ1_S.gguf b/CodeLlama-13b-Python-hf.i1-IQ1_S.gguf new file mode 100644 index 0000000..806b162 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:85881a37e58c36b0f7523da643132f2020ca1ce23205ed0b4a0c64f0fc27c662 +size 2898687104 diff --git a/CodeLlama-13b-Python-hf.i1-IQ2_M.gguf b/CodeLlama-13b-Python-hf.i1-IQ2_M.gguf new file mode 100644 index 0000000..3e377a7 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3c7b34436ed42fc732fa2c8b625d34203c2ca1a40969502fe255c3933459a0c6 +size 4517579904 diff --git a/CodeLlama-13b-Python-hf.i1-IQ2_S.gguf b/CodeLlama-13b-Python-hf.i1-IQ2_S.gguf new file mode 100644 index 0000000..6579d18 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37585188852d5935e1ea792413ffbdec37ef8b629d60985a6cca387ed39d03ab +size 4197682304 diff --git a/CodeLlama-13b-Python-hf.i1-IQ2_XS.gguf b/CodeLlama-13b-Python-hf.i1-IQ2_XS.gguf new file mode 100644 index 0000000..2411735 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e49099b3fdcc9512327e8b70f2cd975d35cdfdd9ac0c487a3aabc89d259d660d +size 3891147904 diff --git a/CodeLlama-13b-Python-hf.i1-IQ2_XXS.gguf b/CodeLlama-13b-Python-hf.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..aff5a20 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b1cc18b645bf012b332d6070925d53f7ba4dd6ca5e4864876930fb521c5d7b01 +size 3538482304 diff --git a/CodeLlama-13b-Python-hf.i1-IQ3_M.gguf b/CodeLlama-13b-Python-hf.i1-IQ3_M.gguf new file mode 100644 index 0000000..290bc89 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a9aba1add2cf7cb9c787a1420b5a40b681e9ee3b61c9ff126ee81dedcf72f021 +size 5984511104 diff --git a/CodeLlama-13b-Python-hf.i1-IQ3_S.gguf b/CodeLlama-13b-Python-hf.i1-IQ3_S.gguf new file mode 100644 index 0000000..026e692 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d718d0e51d87c9fbcb35d5b756d0ebc53292b09eb8e9a8a11c5d64f618cb2bef +size 5658981504 diff --git a/CodeLlama-13b-Python-hf.i1-IQ3_XS.gguf b/CodeLlama-13b-Python-hf.i1-IQ3_XS.gguf new file mode 100644 index 0000000..c667425 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd85e614545b465c2af6fe9de60eeb9bcb994b9394463c2df3bbac49b0c38cc4 +size 5361611904 diff --git a/CodeLlama-13b-Python-hf.i1-IQ3_XXS.gguf b/CodeLlama-13b-Python-hf.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..de09fee --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ec89d8835707f91eb906b56214ccba4392f76c1e790c06dee139b60ec59e8407 +size 4960562304 diff --git a/CodeLlama-13b-Python-hf.i1-IQ4_XS.gguf b/CodeLlama-13b-Python-hf.i1-IQ4_XS.gguf new file mode 100644 index 0000000..f132712 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f2206d474faf23d137fd81b4a10030aee3910d3468cbb08a9bec49537c494247 +size 6964223104 diff --git a/CodeLlama-13b-Python-hf.i1-Q2_K.gguf b/CodeLlama-13b-Python-hf.i1-Q2_K.gguf new file mode 100644 index 0000000..cb659e0 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:66b1be887ae9a40a141650938089cd75a38203973f74f43b6b90b4edb37c70e6 +size 4854271104 diff --git a/CodeLlama-13b-Python-hf.i1-Q3_K_L.gguf b/CodeLlama-13b-Python-hf.i1-Q3_K_L.gguf new file mode 100644 index 0000000..137d08f --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aab71c95cd789e54f883e5a4ca99c3d368ebc4603ceb7296c9105b15114f4155 +size 6929560704 diff --git a/CodeLlama-13b-Python-hf.i1-Q3_K_M.gguf b/CodeLlama-13b-Python-hf.i1-Q3_K_M.gguf new file mode 100644 index 0000000..4daf02a --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa54acef171de3998b18faee568bbb6bb54ca718e8a9fabdfd90cda22e3f5cab +size 6337770624 diff --git a/CodeLlama-13b-Python-hf.i1-Q3_K_S.gguf b/CodeLlama-13b-Python-hf.i1-Q3_K_S.gguf new file mode 100644 index 0000000..5424fb9 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6208a91b91d7045e44202eade89fb7ca5d7c8b91d8867da2555c52fadeb1578e +size 5658981504 diff --git a/CodeLlama-13b-Python-hf.i1-Q4_0.gguf b/CodeLlama-13b-Python-hf.i1-Q4_0.gguf new file mode 100644 index 0000000..bd65ec7 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a1bf7664a0cf3131f2a6c446f74f3deefabb431516bbf608ae5412cc7aa90e64 +size 7387954304 diff --git a/CodeLlama-13b-Python-hf.i1-Q4_0_4_4.gguf b/CodeLlama-13b-Python-hf.i1-Q4_0_4_4.gguf new file mode 100644 index 0000000..2739ddb --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-Q4_0_4_4.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7469fac000ce7f83d79ae4058d082a04f9f36e01ca4a0607ebc82c015d2521f1 +size 7365835904 diff --git a/CodeLlama-13b-Python-hf.i1-Q4_0_4_8.gguf b/CodeLlama-13b-Python-hf.i1-Q4_0_4_8.gguf new file mode 100644 index 0000000..cfa8e12 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-Q4_0_4_8.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:946380ac74f88960e72d0e154a8d0052b5b325e24a17cbcf16fa154232e9817f +size 7365835904 diff --git a/CodeLlama-13b-Python-hf.i1-Q4_0_8_8.gguf b/CodeLlama-13b-Python-hf.i1-Q4_0_8_8.gguf new file mode 100644 index 0000000..7076525 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-Q4_0_8_8.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:05077b8ea57f82015f6bd1d8814e214482207f90251213a29579458016e0a1f6 +size 7365835904 diff --git a/CodeLlama-13b-Python-hf.i1-Q4_K_M.gguf b/CodeLlama-13b-Python-hf.i1-Q4_K_M.gguf new file mode 100644 index 0000000..14cebcc --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:14a74318fe7990af523d7213aafe1c8db61d2070b47cba4625e2b43f7ffe4d98 +size 7865957504 diff --git a/CodeLlama-13b-Python-hf.i1-Q4_K_S.gguf b/CodeLlama-13b-Python-hf.i1-Q4_K_S.gguf new file mode 100644 index 0000000..fb27b16 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e7fbe030aa57378c9e5ee562f6e92cdd38ae9f0bfcde2b7c2911b1e9b6001675 +size 7423179904 diff --git a/CodeLlama-13b-Python-hf.i1-Q5_K_M.gguf b/CodeLlama-13b-Python-hf.i1-Q5_K_M.gguf new file mode 100644 index 0000000..1c9fdc3 --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5e936b359f5a52a6b6de2e4cc3418c41f41f8a7c11628d0201c88f90be3b6bf2 +size 9229925504 diff --git a/CodeLlama-13b-Python-hf.i1-Q5_K_S.gguf b/CodeLlama-13b-Python-hf.i1-Q5_K_S.gguf new file mode 100644 index 0000000..a838f1e --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1072ed3a187f598f4955102a3e486ab5c319dc42a925fce97b4805c19edd8db1 +size 8972287104 diff --git a/CodeLlama-13b-Python-hf.i1-Q6_K.gguf b/CodeLlama-13b-Python-hf.i1-Q6_K.gguf new file mode 100644 index 0000000..9a138af --- /dev/null +++ b/CodeLlama-13b-Python-hf.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5f90883f654f9f6e1a2ae49b983ac78e786fda3f596ed8eae87ceb7d43ff9de4 +size 10679141504 diff --git a/README.md b/README.md new file mode 100644 index 0000000..fad6f67 --- /dev/null +++ b/README.md @@ -0,0 +1,78 @@ +--- +base_model: codellama/CodeLlama-13b-Python-hf +language: +- code +library_name: transformers +license: llama2 +quantized_by: mradermacher +tags: +- llama-2 +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/codellama/CodeLlama-13b-Python-hf + + +static quants are available at https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-IQ1_S.gguf) | i1-IQ1_S | 3.0 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-IQ1_M.gguf) | i1-IQ1_M | 3.2 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 3.6 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.0 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-IQ2_S.gguf) | i1-IQ2_S | 4.3 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-IQ2_M.gguf) | i1-IQ2_M | 4.6 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-Q2_K.gguf) | i1-Q2_K | 5.0 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 5.1 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-IQ3_XS.gguf) | i1-IQ3_XS | 5.5 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-IQ3_S.gguf) | i1-IQ3_S | 5.8 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-Q3_K_S.gguf) | i1-Q3_K_S | 5.8 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-IQ3_M.gguf) | i1-IQ3_M | 6.1 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-Q3_K_M.gguf) | i1-Q3_K_M | 6.4 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-Q3_K_L.gguf) | i1-Q3_K_L | 7.0 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-IQ4_XS.gguf) | i1-IQ4_XS | 7.1 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-Q4_0_4_4.gguf) | i1-Q4_0_4_4 | 7.5 | fast on arm, low quality | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-Q4_0_4_8.gguf) | i1-Q4_0_4_8 | 7.5 | fast on arm+i8mm, low quality | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-Q4_0_8_8.gguf) | i1-Q4_0_8_8 | 7.5 | fast on arm+sve, low quality | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-Q4_0.gguf) | i1-Q4_0 | 7.5 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-Q4_K_S.gguf) | i1-Q4_K_S | 7.5 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-Q4_K_M.gguf) | i1-Q4_K_M | 8.0 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-Q5_K_S.gguf) | i1-Q5_K_S | 9.1 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-Q5_K_M.gguf) | i1-Q5_K_M | 9.3 | | +| [GGUF](https://huggingface.co/mradermacher/CodeLlama-13b-Python-hf-i1-GGUF/resolve/main/CodeLlama-13b-Python-hf.i1-Q6_K.gguf) | i1-Q6_K | 10.8 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..a95c655 --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:22c6cd0c3b535e1cd2b5b29794d04cf637751410f35b04631a8143cc7bb89119 +size 7136325