commit e8196b2a2831634fd88520b094a0783633a91fe3 Author: ModelHub XC Date: Sat May 9 02:30:45 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/GemmaColRAC-AeroExpert-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..f4b58f8 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +GemmaColRAC-AeroExpert.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/GemmaColRAC-AeroExpert.i1-IQ1_M.gguf b/GemmaColRAC-AeroExpert.i1-IQ1_M.gguf new file mode 100644 index 0000000..4361484 --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cf08df9542c7560dfe6559672b6d63c48248969c902338c0bfb8bc31f0b15c13 +size 813844736 diff --git a/GemmaColRAC-AeroExpert.i1-IQ1_S.gguf b/GemmaColRAC-AeroExpert.i1-IQ1_S.gguf new file mode 100644 index 0000000..231cc04 --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cf451757ff2a3ba2694be65a4e9016c9e376751cb745aff92879cd2d0077b3db +size 770959616 diff --git a/GemmaColRAC-AeroExpert.i1-IQ2_M.gguf b/GemmaColRAC-AeroExpert.i1-IQ2_M.gguf new file mode 100644 index 0000000..2cbc2be --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:85bcb4ad0c8b99c3d8d7ef890584ded0f4704e1471ff703b33755c64af99948f +size 1019472128 diff --git a/GemmaColRAC-AeroExpert.i1-IQ2_S.gguf b/GemmaColRAC-AeroExpert.i1-IQ2_S.gguf new file mode 100644 index 0000000..8971f1d --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8ce5d0f566ff3db984eda7f146311224ff5f918f80e24f7416164511c504add9 +size 962291968 diff --git a/GemmaColRAC-AeroExpert.i1-IQ2_XS.gguf b/GemmaColRAC-AeroExpert.i1-IQ2_XS.gguf new file mode 100644 index 0000000..4e19de5 --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b29140c001d091a8b57e9c728d053836d6e720a6fee6f0bbf09f7124716338be +size 944859392 diff --git a/GemmaColRAC-AeroExpert.i1-IQ2_XXS.gguf b/GemmaColRAC-AeroExpert.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..3fd4ceb --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cbc611c9a7a9cb63a6a6ec668d62d3297ababef602a6f974bd935b8c0e5e07e3 +size 885319936 diff --git a/GemmaColRAC-AeroExpert.i1-IQ3_M.gguf b/GemmaColRAC-AeroExpert.i1-IQ3_M.gguf new file mode 100644 index 0000000..649574c --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:52b2c8fb1995e3ebad9d5370c85ce573ac027c1687465946a6a137d928c791cb +size 1308174592 diff --git a/GemmaColRAC-AeroExpert.i1-IQ3_S.gguf b/GemmaColRAC-AeroExpert.i1-IQ3_S.gguf new file mode 100644 index 0000000..25d0daf --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:55a97cf38a2305074f1487925a643d543614e39001c61dc89b574ffc989d9968 +size 1289234688 diff --git a/GemmaColRAC-AeroExpert.i1-IQ3_XS.gguf b/GemmaColRAC-AeroExpert.i1-IQ3_XS.gguf new file mode 100644 index 0000000..318ab65 --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b1a0459f78a197cb3d45f09b39518bcebe687b3cfbc46578f9397c54aa636a7f +size 1244358912 diff --git a/GemmaColRAC-AeroExpert.i1-IQ3_XXS.gguf b/GemmaColRAC-AeroExpert.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..37078bd --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:443aa49721812a88df165edf52b45fd8fec6221ead8f4e7ebc6779fdb90324f7 +size 1125378304 diff --git a/GemmaColRAC-AeroExpert.i1-IQ4_NL.gguf b/GemmaColRAC-AeroExpert.i1-IQ4_NL.gguf new file mode 100644 index 0000000..f92b00e --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:25cd29830228d5453212171a66bc643dae7b0b71bd82d8e6a19ae4c1e5505894 +size 1552369920 diff --git a/GemmaColRAC-AeroExpert.i1-IQ4_XS.gguf b/GemmaColRAC-AeroExpert.i1-IQ4_XS.gguf new file mode 100644 index 0000000..1cbc9ed --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a55ad9822e1b84a30bd93f7f68d19aabece0fea40912d88f340cc526ad2df7a6 +size 1490733312 diff --git a/GemmaColRAC-AeroExpert.i1-Q2_K.gguf b/GemmaColRAC-AeroExpert.i1-Q2_K.gguf new file mode 100644 index 0000000..ebcd711 --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:22ca6094d4ca4b132668e1e7725d8ccf5e2fda76fa6c982459912e6307a36ce5 +size 1157925120 diff --git a/GemmaColRAC-AeroExpert.i1-Q2_K_S.gguf b/GemmaColRAC-AeroExpert.i1-Q2_K_S.gguf new file mode 100644 index 0000000..73ef815 --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9c105e68600f28ff0800ba7a48b8ac01d6f811a35ddbd6d10a8de65d8b75bb55 +size 1104644352 diff --git a/GemmaColRAC-AeroExpert.i1-Q3_K_L.gguf b/GemmaColRAC-AeroExpert.i1-Q3_K_L.gguf new file mode 100644 index 0000000..a6b6034 --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f768e45cac3ee5d0f9df0f80d052114a06cb2df76367fc08d0a8a229f3e9e9a8 +size 1465592064 diff --git a/GemmaColRAC-AeroExpert.i1-Q3_K_M.gguf b/GemmaColRAC-AeroExpert.i1-Q3_K_M.gguf new file mode 100644 index 0000000..f59fd09 --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:11c235713e4f98e4b979a3628d0e170fd82c0f26dc162a9f850948c354717015 +size 1383803136 diff --git a/GemmaColRAC-AeroExpert.i1-Q3_K_S.gguf b/GemmaColRAC-AeroExpert.i1-Q3_K_S.gguf new file mode 100644 index 0000000..ed803f3 --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3d355279d849a8c907bad8498ecc0ef13b5929003ccf5917463607b75bef1255 +size 1287981312 diff --git a/GemmaColRAC-AeroExpert.i1-Q4_0.gguf b/GemmaColRAC-AeroExpert.i1-Q4_0.gguf new file mode 100644 index 0000000..e4f2122 --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ec1e53e52a3104ea0463b3df0d05fb22c6c485583c47745e83f4484f65e089ce +size 1555384576 diff --git a/GemmaColRAC-AeroExpert.i1-Q4_1.gguf b/GemmaColRAC-AeroExpert.i1-Q4_1.gguf new file mode 100644 index 0000000..99afc0a --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0c17d6d2c50bc783f87058d144fb2d5c88044c9b8db51ff68f1adec57b22037f +size 1675053312 diff --git a/GemmaColRAC-AeroExpert.i1-Q4_K_M.gguf b/GemmaColRAC-AeroExpert.i1-Q4_K_M.gguf new file mode 100644 index 0000000..59c1192 --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a67e553f374cd749862e15f535fa48b7ca1e1a51221b0cae88819419c2f3fed6 +size 1630263552 diff --git a/GemmaColRAC-AeroExpert.i1-Q4_K_S.gguf b/GemmaColRAC-AeroExpert.i1-Q4_K_S.gguf new file mode 100644 index 0000000..6d608d4 --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:adb35312c5f56d546e33adf6b3ae290e626860987f53965856866767a90bc48c +size 1559841024 diff --git a/GemmaColRAC-AeroExpert.i1-Q5_K_M.gguf b/GemmaColRAC-AeroExpert.i1-Q5_K_M.gguf new file mode 100644 index 0000000..63e7b2d --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7e9daf9db3f7f8bd009a144d6803c2d8e5cd323cd79f5c88cc28b0e009fff123 +size 1839651072 diff --git a/GemmaColRAC-AeroExpert.i1-Q5_K_S.gguf b/GemmaColRAC-AeroExpert.i1-Q5_K_S.gguf new file mode 100644 index 0000000..dddfdd1 --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ad8456b991a79816472302e77d1b120e812fc5cfdde102f18eabe9b0efa9de59 +size 1798916352 diff --git a/GemmaColRAC-AeroExpert.i1-Q6_K.gguf b/GemmaColRAC-AeroExpert.i1-Q6_K.gguf new file mode 100644 index 0000000..0c87241 --- /dev/null +++ b/GemmaColRAC-AeroExpert.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a6093e1bce8a5b03558636dcb68f4be46ff07feb7ef335ca32b54a0f8dfe5032 +size 2062125312 diff --git a/README.md b/README.md new file mode 100644 index 0000000..e45900d --- /dev/null +++ b/README.md @@ -0,0 +1,88 @@ +--- +base_model: somosnlp/GemmaColRAC-AeroExpert +datasets: +- somosnlp/ColombiaRAC_FullyCurated +language: +- es +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- unsloth +- LLMs-Aviation +- AI-Regulatory-Compliance +- RAC-AI-Colombia +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/somosnlp/GemmaColRAC-AeroExpert + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#GemmaColRAC-AeroExpert-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-IQ1_S.gguf) | i1-IQ1_S | 0.9 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-IQ1_M.gguf) | i1-IQ1_M | 0.9 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.0 | | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.0 | | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-IQ2_S.gguf) | i1-IQ2_S | 1.1 | | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-IQ2_M.gguf) | i1-IQ2_M | 1.1 | | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.2 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.2 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-Q2_K.gguf) | i1-Q2_K | 1.3 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.3 | | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-Q3_K_S.gguf) | i1-Q3_K_S | 1.4 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-IQ3_S.gguf) | i1-IQ3_S | 1.4 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-IQ3_M.gguf) | i1-IQ3_M | 1.4 | | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-Q3_K_M.gguf) | i1-Q3_K_M | 1.5 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-Q3_K_L.gguf) | i1-Q3_K_L | 1.6 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-IQ4_XS.gguf) | i1-IQ4_XS | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-IQ4_NL.gguf) | i1-IQ4_NL | 1.7 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-Q4_0.gguf) | i1-Q4_0 | 1.7 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-Q4_K_S.gguf) | i1-Q4_K_S | 1.7 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-Q4_K_M.gguf) | i1-Q4_K_M | 1.7 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-Q4_1.gguf) | i1-Q4_1 | 1.8 | | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-Q5_K_S.gguf) | i1-Q5_K_S | 1.9 | | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-Q5_K_M.gguf) | i1-Q5_K_M | 1.9 | | +| [GGUF](https://huggingface.co/mradermacher/GemmaColRAC-AeroExpert-i1-GGUF/resolve/main/GemmaColRAC-AeroExpert.i1-Q6_K.gguf) | i1-Q6_K | 2.2 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..a3be022 --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:66bcd0caf28f9d43f06a926e28178f0c2a2431ac971dc96425e4b2ebf544cbcb +size 2068543