commit c46ff036b116d2a1325d08d40f0efdc48bc53518 Author: ModelHub XC Date: Wed Oct 7 12:37:41 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Aura-12B-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..4d98342 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Aura-12B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Aura-12B.i1-IQ1_M.gguf b/Aura-12B.i1-IQ1_M.gguf new file mode 100644 index 0000000..4ea2e75 --- /dev/null +++ b/Aura-12B.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e570b5f50bc74b6d9931ce883ab90e972923f849d7d6868612e957c8340e78c0 +size 3221624800 diff --git a/Aura-12B.i1-IQ1_S.gguf b/Aura-12B.i1-IQ1_S.gguf new file mode 100644 index 0000000..205ee98 --- /dev/null +++ b/Aura-12B.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:95f4512b1f14c291ee11433c93004c5931f54b59816938eea069e17f24b9b421 +size 2999212000 diff --git a/Aura-12B.i1-IQ2_M.gguf b/Aura-12B.i1-IQ2_M.gguf new file mode 100644 index 0000000..160422e --- /dev/null +++ b/Aura-12B.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8caa7bb698fdef84fb20ad305db053ec9328d7cd97c9cf4591a82b881d1b5bc4 +size 4435023840 diff --git a/Aura-12B.i1-IQ2_S.gguf b/Aura-12B.i1-IQ2_S.gguf new file mode 100644 index 0000000..66a5090 --- /dev/null +++ b/Aura-12B.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:942e30d5009faaf3e0c2e6d07889e3f05300d85ee1d29cf0cce1785bc62a0848 +size 4138473440 diff --git a/Aura-12B.i1-IQ2_XS.gguf b/Aura-12B.i1-IQ2_XS.gguf new file mode 100644 index 0000000..49792b4 --- /dev/null +++ b/Aura-12B.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cf812faf37843ebae2f1099bca6103fd0b4548253b842e04815e29efbafe6fd7 +size 3915077600 diff --git a/Aura-12B.i1-IQ2_XXS.gguf b/Aura-12B.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..863f358 --- /dev/null +++ b/Aura-12B.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a9630e4bed0bbe198f1d0d9a9328d2e3409c9b80d8fbef6f0e75a8e0b2be1994 +size 3592312800 diff --git a/Aura-12B.i1-IQ3_M.gguf b/Aura-12B.i1-IQ3_M.gguf new file mode 100644 index 0000000..284e0ac --- /dev/null +++ b/Aura-12B.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:908e589a6d366617d8d0ed3af5632d8035b7b218cdab1ab2c4ef08219b22b3f3 +size 5722232800 diff --git a/Aura-12B.i1-IQ3_S.gguf b/Aura-12B.i1-IQ3_S.gguf new file mode 100644 index 0000000..f6017b0 --- /dev/null +++ b/Aura-12B.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:882f60d13fa0c0130387c413a9e05b0f8e0e45b4d4a0119c8981b1847de49ec2 +size 5562079200 diff --git a/Aura-12B.i1-IQ3_XS.gguf b/Aura-12B.i1-IQ3_XS.gguf new file mode 100644 index 0000000..eb998ac --- /dev/null +++ b/Aura-12B.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:87637660619acfc6d1890dca9788eef84729b4370baf470ba6a76baf24697eb5 +size 5306488800 diff --git a/Aura-12B.i1-IQ3_XXS.gguf b/Aura-12B.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..10a6ba3 --- /dev/null +++ b/Aura-12B.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b58f91c8f173d4952fbfe789e0a6fde972c77041fa2f819971f1e077adc79b34 +size 4945385440 diff --git a/Aura-12B.i1-IQ4_NL.gguf b/Aura-12B.i1-IQ4_NL.gguf new file mode 100644 index 0000000..23dad72 --- /dev/null +++ b/Aura-12B.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:12a1b83c76f9aa93fca1de32e53847953801420937af674be4438238ec3af4ba +size 7097915360 diff --git a/Aura-12B.i1-IQ4_XS.gguf b/Aura-12B.i1-IQ4_XS.gguf new file mode 100644 index 0000000..a98ebe8 --- /dev/null +++ b/Aura-12B.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3987147ca5bdbb1891c358f32eb9aad44087f427071f5813729bdb078c290618 +size 6742710240 diff --git a/Aura-12B.i1-Q2_K.gguf b/Aura-12B.i1-Q2_K.gguf new file mode 100644 index 0000000..5254d0e --- /dev/null +++ b/Aura-12B.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4dc19b9b04d92bca7fe4869a5d2738a411bebaffff20b7dfe6d85aaf478401a4 +size 4791048160 diff --git a/Aura-12B.i1-Q2_K_S.gguf b/Aura-12B.i1-Q2_K_S.gguf new file mode 100644 index 0000000..eaf7367 --- /dev/null +++ b/Aura-12B.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bf7c00415f17ec38395bcfb0efcf098824405957bd35de95263127e409bb9b01 +size 4493678560 diff --git a/Aura-12B.i1-Q3_K_L.gguf b/Aura-12B.i1-Q3_K_L.gguf new file mode 100644 index 0000000..a650b26 --- /dev/null +++ b/Aura-12B.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a4160c51ff43736daca9aa8cc34fa2cb1916bb31846b6c84dd9037a849567e28 +size 6561503200 diff --git a/Aura-12B.i1-Q3_K_M.gguf b/Aura-12B.i1-Q3_K_M.gguf new file mode 100644 index 0000000..d6290ba --- /dev/null +++ b/Aura-12B.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:280a67d7fab76db2b3e872c2e39d3a074a96144527140d9123cfe2cc3a9c5065 +size 6083090400 diff --git a/Aura-12B.i1-Q3_K_S.gguf b/Aura-12B.i1-Q3_K_S.gguf new file mode 100644 index 0000000..b1e9bc2 --- /dev/null +++ b/Aura-12B.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:248d4301bfa90b17f54f4af728135b297aa9944da6898ee88d92f48ed5497ebb +size 5534226400 diff --git a/Aura-12B.i1-Q4_0.gguf b/Aura-12B.i1-Q4_0.gguf new file mode 100644 index 0000000..1e777b7 --- /dev/null +++ b/Aura-12B.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6e365f9a03295c5f10f0f1226a4cdf2cd3b7430d3a6503ee4d6f2e0af36cd14a +size 7094638560 diff --git a/Aura-12B.i1-Q4_1.gguf b/Aura-12B.i1-Q4_1.gguf new file mode 100644 index 0000000..94914f8 --- /dev/null +++ b/Aura-12B.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:84fb4ca9078c59b7486e1af6cd299e0d14d84c32aac6aef9321ea255bddfb9ea +size 7795218400 diff --git a/Aura-12B.i1-Q4_K_M.gguf b/Aura-12B.i1-Q4_K_M.gguf new file mode 100644 index 0000000..67aaee6 --- /dev/null +++ b/Aura-12B.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6612cf6c39a20d01b118914daca805caac41b41905eaac23fb36834f37c7e119 +size 7477204960 diff --git a/Aura-12B.i1-Q4_K_S.gguf b/Aura-12B.i1-Q4_K_S.gguf new file mode 100644 index 0000000..1cabd30 --- /dev/null +++ b/Aura-12B.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d191ccc523b934bfc2826494493a0c98b46a71f54a93ace16eb3881d4c5ac4c2 +size 7120197600 diff --git a/Aura-12B.i1-Q5_K_M.gguf b/Aura-12B.i1-Q5_K_M.gguf new file mode 100644 index 0000000..c4607bc --- /dev/null +++ b/Aura-12B.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a5418e4843b984555e12d3bbee021c548576f83e63e5fdb9604209eae93ec108 +size 8727631840 diff --git a/Aura-12B.i1-Q5_K_S.gguf b/Aura-12B.i1-Q5_K_S.gguf new file mode 100644 index 0000000..fab6723 --- /dev/null +++ b/Aura-12B.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cd9b55a4fc5f814ee7de6445c9eea198b8924c5eae6852842c8543c3a89d789f +size 8518735840 diff --git a/Aura-12B.i1-Q6_K.gguf b/Aura-12B.i1-Q6_K.gguf new file mode 100644 index 0000000..56c232a --- /dev/null +++ b/Aura-12B.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:084ce06bfdef6b894d8f7b0894d1ed1296f22242d52d0f8fa98b18fcd03646ad +size 10056210400 diff --git a/README.md b/README.md new file mode 100644 index 0000000..29f9fae --- /dev/null +++ b/README.md @@ -0,0 +1,83 @@ +--- +base_model: hardlyworking/Aura-12B +language: +- en +library_name: transformers +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- mergekit +- merge +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/hardlyworking/Aura-12B + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Aura-12B-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Aura-12B-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-IQ1_S.gguf) | i1-IQ1_S | 3.1 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-IQ1_M.gguf) | i1-IQ1_M | 3.3 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 3.7 | | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.0 | | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-IQ2_S.gguf) | i1-IQ2_S | 4.2 | | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-IQ2_M.gguf) | i1-IQ2_M | 4.5 | | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 4.6 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-Q2_K.gguf) | i1-Q2_K | 4.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 5.0 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 5.4 | | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 5.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-IQ3_S.gguf) | i1-IQ3_S | 5.7 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-IQ3_M.gguf) | i1-IQ3_M | 5.8 | | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 6.2 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 6.7 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 6.8 | | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-Q4_0.gguf) | i1-Q4_0 | 7.2 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 7.2 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 7.2 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 7.6 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-Q4_1.gguf) | i1-Q4_1 | 7.9 | | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 8.6 | | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 8.8 | | +| [GGUF](https://huggingface.co/mradermacher/Aura-12B-i1-GGUF/resolve/main/Aura-12B.i1-Q6_K.gguf) | i1-Q6_K | 10.2 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..faa00ad --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:934c7fcdcddb0dc36c95f56451221e3e4bb8172d9717b5a8f1be4ea5b2bdf92b +size 7054405