初始化项目,由ModelHub XC社区提供模型
Model: mradermacher/Hexis-Pure-Soul-24B-i1-GGUF Source: Original Platform
This commit is contained in:
59
.gitattributes
vendored
Normal file
59
.gitattributes
vendored
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Hexis-Pure-Soul-24B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
3
Hexis-Pure-Soul-24B.i1-IQ1_M.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-IQ1_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:eb5c39df1220b3b5e0fbab73a554fc76fd286f1296a8dcc9ffa080fee0cede67
|
||||||
|
size 5750505600
|
||||||
3
Hexis-Pure-Soul-24B.i1-IQ1_S.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-IQ1_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:6385cedcd57b39aa66564b233947b066ccd3a37ca3ba6a3ac7fe93de6f5684d8
|
||||||
|
size 5273731200
|
||||||
3
Hexis-Pure-Soul-24B.i1-IQ2_M.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-IQ2_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:91fca3bdc5ca4cc1e311e26489aa967c4d60d8dce9a427fb4e27b1bd85f9a36d
|
||||||
|
size 8114062496
|
||||||
3
Hexis-Pure-Soul-24B.i1-IQ2_S.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-IQ2_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:88d6b9a22e9774b38bb6ae0749e7efe20a8e64fd12eb914ac5c8f5e5d8e1fe53
|
||||||
|
size 7478363296
|
||||||
3
Hexis-Pure-Soul-24B.i1-IQ2_XS.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-IQ2_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b5e9958a31cdb917dff24456db70edacbbb029312f1841eb47b2d43dec57c6d2
|
||||||
|
size 7207043200
|
||||||
3
Hexis-Pure-Soul-24B.i1-IQ2_XXS.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-IQ2_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:202df2bb407a8010f1a8b78271ceeee4417bd1c09cc4cfee53256d414ba12984
|
||||||
|
size 6545129600
|
||||||
3
Hexis-Pure-Soul-24B.i1-IQ3_M.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-IQ3_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:127f02136304aa515b4c288a9ec4549139c4837e1a9a883ccfd56ec9a8743117
|
||||||
|
size 10650962432
|
||||||
3
Hexis-Pure-Soul-24B.i1-IQ3_S.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-IQ3_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:bb1d57e3ec1400e58eff61c94cb8be19413c1d00d080f2b37cba399075efc49c
|
||||||
|
size 10428140032
|
||||||
3
Hexis-Pure-Soul-24B.i1-IQ3_XS.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-IQ3_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:ae6f6323f0447b45934b4d7b9dd0f0f57265779c1426c4f77ee16078b78e1b7c
|
||||||
|
size 9907128832
|
||||||
3
Hexis-Pure-Soul-24B.i1-IQ3_XXS.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-IQ3_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:fe7cf6079b5d52296f5b1d04795252caa9149cc226789a6f76ce9d3db6dd7fe6
|
||||||
|
size 9280603296
|
||||||
3
Hexis-Pure-Soul-24B.i1-IQ4_XS.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-IQ4_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:4f3a94d51ff3d31310bb87ec58360b46a09e1fa77cdc45934200e3eee574ccd5
|
||||||
|
size 12758928896
|
||||||
3
Hexis-Pure-Soul-24B.i1-Q2_K.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-Q2_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:01ffb8774350178d7e3c1f1be739c721a86a4cd33033f9c705ee5c83efdf935e
|
||||||
|
size 8890336736
|
||||||
3
Hexis-Pure-Soul-24B.i1-Q2_K_S.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-Q2_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:952fad3887f72ff277ae7f6eb5b0dfc1419ef979a8fcdc1f828fd4b07ae94012
|
||||||
|
size 8320173536
|
||||||
3
Hexis-Pure-Soul-24B.i1-Q3_K_L.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-Q3_K_L.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:0444cbcc9ed4de7a9e4f9219c6dc5b7196e2a62388555040394b57b39a1348a6
|
||||||
|
size 12400773632
|
||||||
3
Hexis-Pure-Soul-24B.i1-Q3_K_M.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:7b1162089ab0ab69fc7f1cde0a552800a563f6b0557d43c75f124f97a73e3fea
|
||||||
|
size 11474094592
|
||||||
3
Hexis-Pure-Soul-24B.i1-Q3_K_S.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-Q3_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:147f494f901fa348c35d84c38198e037de7347c060670cec0ba2f8ba06247c19
|
||||||
|
size 10400287232
|
||||||
3
Hexis-Pure-Soul-24B.i1-Q4_0.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-Q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:751001141a8b61f007a3d8fe9af03d46737c3e43361d5e873648b454a5ee1283
|
||||||
|
size 13494243136
|
||||||
3
Hexis-Pure-Soul-24B.i1-Q4_1.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-Q4_1.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:66c7f6f3a4d14ad0df65c0bb4df48219fc0bdc548a9f889a302fbdc07eee5f94
|
||||||
|
size 14873121216
|
||||||
3
Hexis-Pure-Soul-24B.i1-Q4_K_M.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:63c8fad5af8d530d806e1bb0f9e1a821e80b140c2d17fe91eca4727a47f491eb
|
||||||
|
size 14333923136
|
||||||
3
Hexis-Pure-Soul-24B.i1-Q4_K_S.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-Q4_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:63f84bffc1fe14c064d04e818e98b2a47a5645d77052cf96e07c63feb4ced87e
|
||||||
|
size 13549293376
|
||||||
3
Hexis-Pure-Soul-24B.i1-Q5_K_M.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:d7f1870e5408a3667005ae6266c7bf8a50e0132dbc6b27e6a15403d82c88681e
|
||||||
|
size 16763999296
|
||||||
3
Hexis-Pure-Soul-24B.i1-Q5_K_S.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-Q5_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:0c3e7d2b379f66e343f770c68f3f39fcfb85f4c9dbb01cf22fbe62749f0eb74d
|
||||||
|
size 16304428096
|
||||||
3
Hexis-Pure-Soul-24B.i1-Q6_K.gguf
Normal file
3
Hexis-Pure-Soul-24B.i1-Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:e59fb7f5bc7b095bce86888d699ba9d90fbb1d64f8b8912988f5830c7f6bd3c0
|
||||||
|
size 19345955232
|
||||||
3
Hexis-Pure-Soul-24B.imatrix.gguf
Normal file
3
Hexis-Pure-Soul-24B.imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:4d365f837e9076e6bbf917f022f10a348d3d08e12aee2a8a39c615a8d8cb1ab2
|
||||||
|
size 10037312
|
||||||
101
README.md
Normal file
101
README.md
Normal file
@@ -0,0 +1,101 @@
|
|||||||
|
---
|
||||||
|
base_model: WasamiKirua/Hexis-Pure-Soul-24B
|
||||||
|
language:
|
||||||
|
- it
|
||||||
|
- en
|
||||||
|
library_name: transformers
|
||||||
|
model_name: Hexis-Pure-Soul-24B
|
||||||
|
mradermacher:
|
||||||
|
readme_rev: 1
|
||||||
|
quantized_by: mradermacher
|
||||||
|
tags:
|
||||||
|
- mistral
|
||||||
|
- mistral-small
|
||||||
|
- text-generation
|
||||||
|
- roleplay
|
||||||
|
- sentient
|
||||||
|
- pure-soul
|
||||||
|
- intimate
|
||||||
|
- sensory-obsessed
|
||||||
|
- melancholic
|
||||||
|
- chatml
|
||||||
|
- mergekit
|
||||||
|
- ties
|
||||||
|
- uncensored
|
||||||
|
- it
|
||||||
|
- en
|
||||||
|
---
|
||||||
|
## About
|
||||||
|
|
||||||
|
<!-- ### quantize_version: 2 -->
|
||||||
|
<!-- ### output_tensor_quantised: 1 -->
|
||||||
|
<!-- ### convert_type: hf -->
|
||||||
|
<!-- ### vocab_type: -->
|
||||||
|
<!-- ### tags: nicoboss -->
|
||||||
|
<!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S -->
|
||||||
|
<!-- ### quants_skip: -->
|
||||||
|
<!-- ### skip_mmproj: -->
|
||||||
|
weighted/imatrix quants of https://huggingface.co/WasamiKirua/Hexis-Pure-Soul-24B
|
||||||
|
|
||||||
|
<!-- provided-files -->
|
||||||
|
|
||||||
|
***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Hexis-Pure-Soul-24B-i1-GGUF).***
|
||||||
|
|
||||||
|
static quants are available at https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-GGUF
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
If you are unsure how to use GGUF files, refer to one of [TheBloke's
|
||||||
|
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
|
||||||
|
more details, including on how to concatenate multi-part files.
|
||||||
|
|
||||||
|
## Provided Quants
|
||||||
|
|
||||||
|
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
|
||||||
|
|
||||||
|
| Link | Type | Size/GB | Notes |
|
||||||
|
|:-----|:-----|--------:|:------|
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-IQ1_S.gguf) | i1-IQ1_S | 5.4 | for the desperate |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-IQ1_M.gguf) | i1-IQ1_M | 5.9 | mostly desperate |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 6.6 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 7.3 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-IQ2_S.gguf) | i1-IQ2_S | 7.6 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-IQ2_M.gguf) | i1-IQ2_M | 8.2 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 8.4 | very low quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-Q2_K.gguf) | i1-Q2_K | 9.0 | IQ3_XXS probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 9.4 | lower quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 10.0 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 10.5 | IQ3_XS probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-IQ3_S.gguf) | i1-IQ3_S | 10.5 | beats Q3_K* |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-IQ3_M.gguf) | i1-IQ3_M | 10.8 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 11.6 | IQ3_S probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 12.5 | IQ3_M probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 12.9 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-Q4_0.gguf) | i1-Q4_0 | 13.6 | fast, low quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 13.6 | optimal size/speed/quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 14.4 | fast, recommended |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-Q4_1.gguf) | i1-Q4_1 | 15.0 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 16.4 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 16.9 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Hexis-Pure-Soul-24B-i1-GGUF/resolve/main/Hexis-Pure-Soul-24B.i1-Q6_K.gguf) | i1-Q6_K | 19.4 | practically like static Q6_K |
|
||||||
|
|
||||||
|
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
||||||
|
types (lower is better):
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
And here are Artefact2's thoughts on the matter:
|
||||||
|
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
|
||||||
|
|
||||||
|
## FAQ / Model Request
|
||||||
|
|
||||||
|
See https://huggingface.co/mradermacher/model_requests for some answers to
|
||||||
|
questions you might have and/or if you want some other model quantized.
|
||||||
|
|
||||||
|
## Thanks
|
||||||
|
|
||||||
|
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
|
||||||
|
me use its servers and providing upgrades to my workstation to enable
|
||||||
|
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
|
||||||
|
|
||||||
|
<!-- end -->
|
||||||
Reference in New Issue
Block a user