初始化项目，由ModelHub XC社区提供模型
Model: mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF Source: Original Platform
This commit is contained in:
ModelHub XC
60
.gitattributes
vendored
Normal file
60
.gitattributes
vendored
Normal file
@@ -0,0 +1,60 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
imatrix.dat filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Forgotten-Abomination-12B-v4.0.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-IQ1_M.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-IQ1_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a60f36216da9829f90970a45870121726fbe0846c0d677b1422151b77654552a
|
||||
size 3221628576
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-IQ1_S.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-IQ1_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4c416a216d2a81999a333326f5aef5656a1f68c4043bc0bb09d3a246c5ebc1fb
|
||||
size 2999215776
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-IQ2_M.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-IQ2_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d8c4f057b34d8f15e34c58880582280e921d8362ab6e8006aaf6fa4ae88d8859
|
||||
size 4435027616
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-IQ2_S.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-IQ2_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b73181f0bbc63ad77b8ef486d414f2cd75c545daf655472fde1c1834fa6dc712
|
||||
size 4138477216
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-IQ2_XS.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-IQ2_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b06633b4f03e2f2852afb9158bea3de45f42477a54a66ff93d6542e577e0c8a9
|
||||
size 3915081376
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-IQ2_XXS.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-IQ2_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7b637936486bd576bac467980efe0bd41bdc1d2195db9f937ca7b2b65e498556
|
||||
size 3592316576
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-IQ3_M.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-IQ3_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:75ffb6beb2de9c9ee353d69628d6e87f1e435f0c0fd5ae26ef822472e3f8cebd
|
||||
size 5722236576
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-IQ3_S.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-IQ3_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:540f230740f2f59287cfb87108e7fbf7592dd3ad679f53f11039dbd480750c5b
|
||||
size 5562082976
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-IQ3_XS.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-IQ3_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d6872bfc06c7dedaa918e031027e29ef9d8764396f7d7750c21508ed21a273b4
|
||||
size 5306492576
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-IQ3_XXS.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-IQ3_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:eef585143f0d9eaf54409eb1ef9984464c821a63068c62075499b40f7cf8b10b
|
||||
size 4945389216
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-IQ4_NL.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-IQ4_NL.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:072abbcad7cb168869bb0ba6c8d1f5a92dd51523e1f5017bb3c73ef4e04ca75b
|
||||
size 7097919136
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-IQ4_XS.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-IQ4_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:90bc8a682c7725dbbd451a55cff024a08617168374ff2886403fb9476eb3ba26
|
||||
size 6742714016
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-Q2_K.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-Q2_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:912c667b992c1f436859b3c4dc9357db7ec7cc2d818614816792b968b5ad7224
|
||||
size 4791051936
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-Q2_K_S.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-Q2_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:61597a66e1f9c26632256f104dc0b2578b8f1ff5f2b0de490c89aa1fd2b3adab
|
||||
size 4493682336
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-Q3_K_L.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-Q3_K_L.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3ccc27ed3092c8a4c54a5329d8ec8a36c1e2be9688d2273e5e4dfc8857872790
|
||||
size 6561506976
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-Q3_K_M.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:26b38b46824d7bb6d637f13858d721693e87af8636f271715fdcf2aae3845337
|
||||
size 6083094176
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-Q3_K_S.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-Q3_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6b24fd3e6641d1d63c3b20a9c761afb6ad761dd56ce6203ba82449323c6dc821
|
||||
size 5534230176
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-Q4_0.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-Q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:086ff825e5cf8581fe6dff1ba1145d3eb69bc5607b22a40c3c6699749c758ac9
|
||||
size 7094642336
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-Q4_1.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-Q4_1.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f950e0178d3a97f6ab1f309c90ab50401fc671288cf0ab0f1fecaa57a1294538
|
||||
size 7795222176
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-Q4_K_M.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:622d80ae880136cc5fb5dec71513fbe1dee3798173379d573307739210c23465
|
||||
size 7477208736
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-Q4_K_S.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-Q4_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a367f7ef7bc40921fed7ca58160e536ded947c2a70c50b08032dc85006835e93
|
||||
size 7120201376
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-Q5_K_M.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d3c9c7d7b71ed903524f3280469208ebf5030e347c4cbc764d8554fe6ebb7b5c
|
||||
size 8727635616
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-Q5_K_S.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-Q5_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1db0adb59cdd61cb1e380e70650cafac3675cf6736b1af6d647749ac8537681d
|
||||
size 8518739616
|
||||
3
Forgotten-Abomination-12B-v4.0.i1-Q6_K.gguf
Normal file
3
Forgotten-Abomination-12B-v4.0.i1-Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:26e87c49ccc08f80ce931dc29160bb02d14dddf930e58c2120d5caee175f0508
|
||||
size 10056214176
|
||||
88
README.md
Normal file
88
README.md
Normal file
@@ -0,0 +1,88 @@
|
||||
---
|
||||
base_model: ReadyArt/Forgotten-Abomination-12B-v4.0
|
||||
language:
|
||||
- en
|
||||
library_name: transformers
|
||||
license: apache-2.0
|
||||
mradermacher:
|
||||
readme_rev: 1
|
||||
quantized_by: mradermacher
|
||||
tags:
|
||||
- nsfw
|
||||
- explicit
|
||||
- roleplay
|
||||
- unaligned
|
||||
- dangerous
|
||||
- ERP
|
||||
---
|
||||
## About
|
||||
|
||||
<!-- ### quantize_version: 2 -->
|
||||
<!-- ### output_tensor_quantised: 1 -->
|
||||
<!-- ### convert_type: hf -->
|
||||
<!-- ### vocab_type: -->
|
||||
<!-- ### tags: nicoboss -->
|
||||
weighted/imatrix quants of https://huggingface.co/ReadyArt/Forgotten-Abomination-12B-v4.0
|
||||
|
||||
<!-- provided-files -->
|
||||
|
||||
***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Forgotten-Abomination-12B-v4.0-i1-GGUF).***
|
||||
|
||||
static quants are available at https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-GGUF
|
||||
## Usage
|
||||
|
||||
If you are unsure how to use GGUF files, refer to one of [TheBloke's
|
||||
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
|
||||
more details, including on how to concatenate multi-part files.
|
||||
|
||||
## Provided Quants
|
||||
|
||||
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
|
||||
|
||||
| Link | Type | Size/GB | Notes |
|
||||
|:-----|:-----|--------:|:------|
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-IQ1_S.gguf) | i1-IQ1_S | 3.1 | for the desperate |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-IQ1_M.gguf) | i1-IQ1_M | 3.3 | mostly desperate |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 3.7 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.0 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-IQ2_S.gguf) | i1-IQ2_S | 4.2 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-IQ2_M.gguf) | i1-IQ2_M | 4.5 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-Q2_K_S.gguf) | i1-Q2_K_S | 4.6 | very low quality |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-Q2_K.gguf) | i1-Q2_K | 4.9 | IQ3_XXS probably better |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 5.0 | lower quality |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-IQ3_XS.gguf) | i1-IQ3_XS | 5.4 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-Q3_K_S.gguf) | i1-Q3_K_S | 5.6 | IQ3_XS probably better |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-IQ3_S.gguf) | i1-IQ3_S | 5.7 | beats Q3_K* |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-IQ3_M.gguf) | i1-IQ3_M | 5.8 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-Q3_K_M.gguf) | i1-Q3_K_M | 6.2 | IQ3_S probably better |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-Q3_K_L.gguf) | i1-Q3_K_L | 6.7 | IQ3_M probably better |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-IQ4_XS.gguf) | i1-IQ4_XS | 6.8 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-Q4_0.gguf) | i1-Q4_0 | 7.2 | fast, low quality |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-IQ4_NL.gguf) | i1-IQ4_NL | 7.2 | prefer IQ4_XS |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-Q4_K_S.gguf) | i1-Q4_K_S | 7.2 | optimal size/speed/quality |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-Q4_K_M.gguf) | i1-Q4_K_M | 7.6 | fast, recommended |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-Q4_1.gguf) | i1-Q4_1 | 7.9 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-Q5_K_S.gguf) | i1-Q5_K_S | 8.6 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-Q5_K_M.gguf) | i1-Q5_K_M | 8.8 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Forgotten-Abomination-12B-v4.0-i1-GGUF/resolve/main/Forgotten-Abomination-12B-v4.0.i1-Q6_K.gguf) | i1-Q6_K | 10.2 | practically like static Q6_K |
|
||||
|
||||
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
||||
types (lower is better):
|
||||
|
||||
|
||||
|
||||
And here are Artefact2's thoughts on the matter:
|
||||
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
|
||||
|
||||
## FAQ / Model Request
|
||||
|
||||
See https://huggingface.co/mradermacher/model_requests for some answers to
|
||||
questions you might have and/or if you want some other model quantized.
|
||||
|
||||
## Thanks
|
||||
|
||||
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
|
||||
me use its servers and providing upgrades to my workstation to enable
|
||||
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
|
||||
|
||||
<!-- end -->
|
||||
3
imatrix.dat
Normal file
3
imatrix.dat
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4ace77a648d44d8683e0d52cf4ec663d8569c405fc95e72ccc7d36326ad58c0b
|
||||
size 7054405
|
||||
Reference in New Issue
Block a user