初始化项目,由ModelHub XC社区提供模型
Model: mradermacher/KodaLite-1.3B-i1-GGUF Source: Original Platform
This commit is contained in:
60
.gitattributes
vendored
Normal file
60
.gitattributes
vendored
Normal file
@@ -0,0 +1,60 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
KodaLite-1.3B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
3
KodaLite-1.3B.i1-IQ1_M.gguf
Normal file
3
KodaLite-1.3B.i1-IQ1_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:6d129f18db9520bbc2f215eca17d8e07b55acf1f70e60071ca8724b4ce1a66fb
|
||||||
|
size 444883296
|
||||||
3
KodaLite-1.3B.i1-IQ1_S.gguf
Normal file
3
KodaLite-1.3B.i1-IQ1_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b4ff2699d249ca49b9544907797ae59ef64fbfcc223d9347023e12257ffe02a1
|
||||||
|
size 429252960
|
||||||
3
KodaLite-1.3B.i1-IQ2_M.gguf
Normal file
3
KodaLite-1.3B.i1-IQ2_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:3b50724ed2a2ea3b06ee7422d4d494ef5de5598926379312ac95e8a214f83b6b
|
||||||
|
size 540369728
|
||||||
3
KodaLite-1.3B.i1-IQ2_S.gguf
Normal file
3
KodaLite-1.3B.i1-IQ2_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:79fe484e2a9aab2a9588fa856e03fa43bed93f1e493729aa481c12b4744ec01e
|
||||||
|
size 519529280
|
||||||
3
KodaLite-1.3B.i1-IQ2_XS.gguf
Normal file
3
KodaLite-1.3B.i1-IQ2_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:3a66d0373ad94f227669e67566dc0d4bd0848375168ca6f3c3f961ee142009a9
|
||||||
|
size 494920032
|
||||||
3
KodaLite-1.3B.i1-IQ2_XXS.gguf
Normal file
3
KodaLite-1.3B.i1-IQ2_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:a0bd81e103bd1231a7d4a3613f677b46bd1a81f6f63170c84e8dfe87d7faa268
|
||||||
|
size 470933856
|
||||||
3
KodaLite-1.3B.i1-IQ3_M.gguf
Normal file
3
KodaLite-1.3B.i1-IQ3_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:05bd4c576c78843d95eca9533398c526ad9e02520bb884a2e74428d8f2e70552
|
||||||
|
size 644577632
|
||||||
3
KodaLite-1.3B.i1-IQ3_S.gguf
Normal file
3
KodaLite-1.3B.i1-IQ3_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b10327b144132ee8067e7e9b586fabf1d595151ae3664c271d3a46314c60e073
|
||||||
|
size 626981216
|
||||||
3
KodaLite-1.3B.i1-IQ3_XS.gguf
Normal file
3
KodaLite-1.3B.i1-IQ3_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:774acd74c78fa963e3902bc1b89cff816118e1ae126e00bb463fd534196d3675
|
||||||
|
size 602061152
|
||||||
3
KodaLite-1.3B.i1-IQ3_XXS.gguf
Normal file
3
KodaLite-1.3B.i1-IQ3_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:048ef495f7d262bb3f69755e7804d63988284c0471e3cd96074adcce444e9e29
|
||||||
|
size 574186304
|
||||||
3
KodaLite-1.3B.i1-IQ4_NL.gguf
Normal file
3
KodaLite-1.3B.i1-IQ4_NL.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:38f5030f09c56af6ff286f6a3d39d43876c423474f0d7a6de0b6e821390bccdf
|
||||||
|
size 745738080
|
||||||
3
KodaLite-1.3B.i1-IQ4_XS.gguf
Normal file
3
KodaLite-1.3B.i1-IQ4_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:8ff4c599d167efb35f600f5fba67df4662a518850cae92a389dd2c81d24d8045
|
||||||
|
size 718535456
|
||||||
3
KodaLite-1.3B.i1-Q2_K.gguf
Normal file
3
KodaLite-1.3B.i1-Q2_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:55e563260a1374a46ea73ff4889b5813298be32d7a5aa699728d77921ca79992
|
||||||
|
size 548796288
|
||||||
3
KodaLite-1.3B.i1-Q2_K_S.gguf
Normal file
3
KodaLite-1.3B.i1-Q2_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b362eaff9e41afc5f22047a18f0b5559ba4c777d6f84ff0be10966b7a9f949c2
|
||||||
|
size 542799744
|
||||||
3
KodaLite-1.3B.i1-Q3_K_L.gguf
Normal file
3
KodaLite-1.3B.i1-Q3_K_L.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:6262ab08169cf15337bc503adbefc433f44cee80b6e26e542be163ed3b9dd0c5
|
||||||
|
size 706804064
|
||||||
3
KodaLite-1.3B.i1-Q3_K_M.gguf
Normal file
3
KodaLite-1.3B.i1-Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:93c3f51d19df4d8eca505e68f6034488c31390faf0b77c78a243ac8980543a39
|
||||||
|
size 675133792
|
||||||
3
KodaLite-1.3B.i1-Q3_K_S.gguf
Normal file
3
KodaLite-1.3B.i1-Q3_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:2bc564f1e66f3138bce290078b0615e4c6de5a260807c42962b42185de5bb3fb
|
||||||
|
size 623638880
|
||||||
3
KodaLite-1.3B.i1-Q4_0.gguf
Normal file
3
KodaLite-1.3B.i1-Q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:ddb35ff940cc5bd55b25c388bd68482f2895afe5401dd4c4b42f16f94fdc15b5
|
||||||
|
size 744705888
|
||||||
3
KodaLite-1.3B.i1-Q4_1.gguf
Normal file
3
KodaLite-1.3B.i1-Q4_1.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:d84d1f0bf5b7baf048c5553d128a60aa0c2242214aa4843ae4f20a1d9c864675
|
||||||
|
size 815478752
|
||||||
3
KodaLite-1.3B.i1-Q4_K_M.gguf
Normal file
3
KodaLite-1.3B.i1-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:a634cc0b27934a789bb7b341f23850a6d891a7c3acbefe379193f31b52a29b76
|
||||||
|
size 830377824
|
||||||
3
KodaLite-1.3B.i1-Q4_K_S.gguf
Normal file
3
KodaLite-1.3B.i1-Q4_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:96ccadbc2b8449df76b3d5872f1557356427cb0e906f4d55bdcd0e094c069daf
|
||||||
|
size 779046752
|
||||||
3
KodaLite-1.3B.i1-Q5_K_M.gguf
Normal file
3
KodaLite-1.3B.i1-Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:ff193e4d7b63961187b485083005d27a8e6c0411ff2d303d7b605b6329ac6df4
|
||||||
|
size 949215328
|
||||||
3
KodaLite-1.3B.i1-Q5_K_S.gguf
Normal file
3
KodaLite-1.3B.i1-Q5_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:206370cb5de15b5db03225bbab086052010536e3407257863ee9f03333f20274
|
||||||
|
size 905273440
|
||||||
3
KodaLite-1.3B.i1-Q6_K.gguf
Normal file
3
KodaLite-1.3B.i1-Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:204089151a6c0d7c83bbbb23a6b61cd654ab9cfac0656f4f8113b23b115571e2
|
||||||
|
size 1108768384
|
||||||
3
KodaLite-1.3B.imatrix.gguf
Normal file
3
KodaLite-1.3B.imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:dac95868104bc85fa3c629485dbc2a0952c8d35a243e112601152df15d2e2e0d
|
||||||
|
size 1733920
|
||||||
89
README.md
Normal file
89
README.md
Normal file
@@ -0,0 +1,89 @@
|
|||||||
|
---
|
||||||
|
base_model: YoAbriel/KodaLite-1.3B
|
||||||
|
language: en
|
||||||
|
library_name: transformers
|
||||||
|
license: apache-2.0
|
||||||
|
mradermacher:
|
||||||
|
readme_rev: 1
|
||||||
|
quantized_by: mradermacher
|
||||||
|
tags:
|
||||||
|
- text-generation
|
||||||
|
- llama
|
||||||
|
- from-scratch
|
||||||
|
- jax
|
||||||
|
---
|
||||||
|
## About
|
||||||
|
|
||||||
|
<!-- ### quantize_version: 2 -->
|
||||||
|
<!-- ### output_tensor_quantised: 1 -->
|
||||||
|
<!-- ### convert_type: hf -->
|
||||||
|
<!-- ### vocab_type: -->
|
||||||
|
<!-- ### tags: nicoboss -->
|
||||||
|
<!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S -->
|
||||||
|
<!-- ### quants_skip: -->
|
||||||
|
<!-- ### skip_mmproj: -->
|
||||||
|
weighted/imatrix quants of https://huggingface.co/YoAbriel/KodaLite-1.3B
|
||||||
|
|
||||||
|
<!-- provided-files -->
|
||||||
|
|
||||||
|
***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#KodaLite-1.3B-i1-GGUF).***
|
||||||
|
|
||||||
|
static quants are available at https://huggingface.co/mradermacher/KodaLite-1.3B-GGUF
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
If you are unsure how to use GGUF files, refer to one of [TheBloke's
|
||||||
|
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
|
||||||
|
more details, including on how to concatenate multi-part files.
|
||||||
|
|
||||||
|
## Provided Quants
|
||||||
|
|
||||||
|
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
|
||||||
|
|
||||||
|
| Link | Type | Size/GB | Notes |
|
||||||
|
|:-----|:-----|--------:|:------|
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-IQ1_S.gguf) | i1-IQ1_S | 0.5 | for the desperate |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-IQ1_M.gguf) | i1-IQ1_M | 0.5 | mostly desperate |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 0.6 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 0.6 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-IQ2_S.gguf) | i1-IQ2_S | 0.6 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-IQ2_M.gguf) | i1-IQ2_M | 0.6 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 0.6 | very low quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-Q2_K.gguf) | i1-Q2_K | 0.6 | IQ3_XXS probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 0.7 | lower quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 0.7 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 0.7 | IQ3_XS probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-IQ3_S.gguf) | i1-IQ3_S | 0.7 | beats Q3_K* |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-IQ3_M.gguf) | i1-IQ3_M | 0.7 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 0.8 | IQ3_S probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 0.8 | IQ3_M probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 0.8 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-Q4_0.gguf) | i1-Q4_0 | 0.8 | fast, low quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 0.8 | prefer IQ4_XS |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 0.9 | optimal size/speed/quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-Q4_1.gguf) | i1-Q4_1 | 0.9 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 0.9 | fast, recommended |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 1.0 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 1.0 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/KodaLite-1.3B-i1-GGUF/resolve/main/KodaLite-1.3B.i1-Q6_K.gguf) | i1-Q6_K | 1.2 | practically like static Q6_K |
|
||||||
|
|
||||||
|
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
||||||
|
types (lower is better):
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
And here are Artefact2's thoughts on the matter:
|
||||||
|
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
|
||||||
|
|
||||||
|
## FAQ / Model Request
|
||||||
|
|
||||||
|
See https://huggingface.co/mradermacher/model_requests for some answers to
|
||||||
|
questions you might have and/or if you want some other model quantized.
|
||||||
|
|
||||||
|
## Thanks
|
||||||
|
|
||||||
|
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
|
||||||
|
me use its servers and providing upgrades to my workstation to enable
|
||||||
|
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
|
||||||
|
|
||||||
|
<!-- end -->
|
||||||
Reference in New Issue
Block a user