初始化项目,由ModelHub XC社区提供模型
Model: mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF Source: Original Platform
This commit is contained in:
59
.gitattributes
vendored
Normal file
59
.gitattributes
vendored
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ1_M.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ1_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:baa605307ca7e3b9ad22f40c02dd26d2cc616f1e0792385db2b3be3ab09a2ae1
|
||||||
|
size 7660218592
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ1_S.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ1_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:04664f98802f53ebe86303d764e2c642beb9239b1ab37ccd3815966f7e7d3dea
|
||||||
|
size 7002564832
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_M.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:a35fe39b65fe482c7f9ac05865aae5df11a16abff322acd024d72c16928893eb
|
||||||
|
size 10965570208
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_S.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:aa1288cc341ebaadb26785a428a9f2da050b7d80d00ebe5589503e100ff615b4
|
||||||
|
size 10088698528
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_XS.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:a5c11f21a094c5b14695484093d07e2ce1ceb48e3b67c4bcce794ab03fec02cf
|
||||||
|
size 9685608672
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_XXS.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:33c852de10bfa980fb5223b087e1cf412357625a8e2d9753d14ade6ce7d5805f
|
||||||
|
size 8756308192
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_M.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:f665fdd8726df6e41c475ee59ad3a9b5e03eab8119e24b031df707fc317e8ab1
|
||||||
|
size 14476037664
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_S.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:46950650a1f74de40f6177d8de1ab69f0a59f00a531b036049fea6b1ec156967
|
||||||
|
size 14102810144
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_XS.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:035bfea6eb30631e2c543fcde14ecf8eee6c1e121b16837cc766199e8d7bdadc
|
||||||
|
size 13371428384
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_XXS.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:683989ff4dffc17dba6361b5d26fc9f8a03d509cb8d01b4a3ac623fd3ec8b5c1
|
||||||
|
size 12540400288
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ4_XS.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ4_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:33729b54d08d56e1dd3b6a09784be8dfd7576d6116791f3d48ef3e0325bd7cd5
|
||||||
|
size 17332139968
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q2_K.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q2_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:8fb285f1da64ce8a47480c6df4a11372cad82fc1e52c631699ba049f140394b0
|
||||||
|
size 12005942368
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q2_K_S.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q2_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:91ba5556ae29f393d87c68ad9acff871b2fd546e5631b5b5460fd15c3370dffd
|
||||||
|
size 11180844128
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q3_K_L.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q3_K_L.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:a74e9ee77797fca6c7b59f03aae36393daadc317944237e0f51b418b203a7802
|
||||||
|
size 16912993824
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q3_K_M.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b2a6df90462d0cdf7ff5e414feb45fbf9b9561c55766940fdc561710d339b0fd
|
||||||
|
size 15600963104
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q3_K_S.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q3_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b638bd91d4b5af2e0229264bc63836fa4731e9aa3004f32491cfb278cd816b59
|
||||||
|
size 14058245664
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_0.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:c18797fe59508bc7ea24209f2c6c2fff5e97f34adcb31fede3dbb454c4e120fc
|
||||||
|
size 18341710208
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_1.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_1.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:4748140d63bde4e1bb51869e87667c3a6e92d8d66ad5a73fb5c45541a99b8fed
|
||||||
|
size 20253371648
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_K_M.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:e94512adaa42397bdc302a26ca85d8f0192bc91f657d35af2f52916899f382af
|
||||||
|
size 19482036608
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_K_S.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:ff86b7670ba586bbeff648de35122114c4673866a4f4aad562e3d297c224f50a
|
||||||
|
size 18415110528
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q5_K_M.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:ccefa5c5a8d37545ce5288d1b36b90481a702bb84aca07869935bb7f7c0d648a
|
||||||
|
size 22859714688
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q5_K_S.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q5_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:cd64c976abf3f8e440626baee11d0ef0937afcd9646ab1e77dc3b094ed87745c
|
||||||
|
size 22235811968
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q6_K.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:7be30fb4195ca2eb819740d31052d8f5eaa525b03bd446ccd7329d4f80a537ac
|
||||||
|
size 26448497664
|
||||||
3
Olmo-3.1-32B-Instruct-impotent-heresy.imatrix.gguf
Normal file
3
Olmo-3.1-32B-Instruct-impotent-heresy.imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:ee5d1936fdcb701517b6e7939c8aed7b21d10d0131bf33ed0c9ace92effa2fe1
|
||||||
|
size 15011072
|
||||||
91
README.md
Normal file
91
README.md
Normal file
@@ -0,0 +1,91 @@
|
|||||||
|
---
|
||||||
|
base_model: MuXodious/Olmo-3.1-32B-Instruct-impotent-heresy
|
||||||
|
datasets:
|
||||||
|
- allenai/Dolci-Instruct-RL
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
library_name: transformers
|
||||||
|
license: apache-2.0
|
||||||
|
mradermacher:
|
||||||
|
readme_rev: 1
|
||||||
|
quantized_by: mradermacher
|
||||||
|
tags:
|
||||||
|
- heretic
|
||||||
|
- uncensored
|
||||||
|
- decensored
|
||||||
|
- abliterated
|
||||||
|
---
|
||||||
|
## About
|
||||||
|
|
||||||
|
<!-- ### quantize_version: 2 -->
|
||||||
|
<!-- ### output_tensor_quantised: 1 -->
|
||||||
|
<!-- ### convert_type: hf -->
|
||||||
|
<!-- ### vocab_type: -->
|
||||||
|
<!-- ### tags: nicoboss -->
|
||||||
|
<!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S -->
|
||||||
|
<!-- ### quants_skip: -->
|
||||||
|
<!-- ### skip_mmproj: -->
|
||||||
|
weighted/imatrix quants of https://huggingface.co/MuXodious/Olmo-3.1-32B-Instruct-impotent-heresy
|
||||||
|
|
||||||
|
<!-- provided-files -->
|
||||||
|
|
||||||
|
***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF).***
|
||||||
|
|
||||||
|
static quants are available at https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-GGUF
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
If you are unsure how to use GGUF files, refer to one of [TheBloke's
|
||||||
|
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
|
||||||
|
more details, including on how to concatenate multi-part files.
|
||||||
|
|
||||||
|
## Provided Quants
|
||||||
|
|
||||||
|
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
|
||||||
|
|
||||||
|
| Link | Type | Size/GB | Notes |
|
||||||
|
|:-----|:-----|--------:|:------|
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ1_S.gguf) | i1-IQ1_S | 7.1 | for the desperate |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ1_M.gguf) | i1-IQ1_M | 7.8 | mostly desperate |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 8.9 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_XS.gguf) | i1-IQ2_XS | 9.8 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_S.gguf) | i1-IQ2_S | 10.2 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ2_M.gguf) | i1-IQ2_M | 11.1 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q2_K_S.gguf) | i1-Q2_K_S | 11.3 | very low quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q2_K.gguf) | i1-Q2_K | 12.1 | IQ3_XXS probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 12.6 | lower quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_XS.gguf) | i1-IQ3_XS | 13.5 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q3_K_S.gguf) | i1-Q3_K_S | 14.2 | IQ3_XS probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_S.gguf) | i1-IQ3_S | 14.2 | beats Q3_K* |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ3_M.gguf) | i1-IQ3_M | 14.6 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q3_K_M.gguf) | i1-Q3_K_M | 15.7 | IQ3_S probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q3_K_L.gguf) | i1-Q3_K_L | 17.0 | IQ3_M probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-IQ4_XS.gguf) | i1-IQ4_XS | 17.4 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_0.gguf) | i1-Q4_0 | 18.4 | fast, low quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_K_S.gguf) | i1-Q4_K_S | 18.5 | optimal size/speed/quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_K_M.gguf) | i1-Q4_K_M | 19.6 | fast, recommended |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q4_1.gguf) | i1-Q4_1 | 20.4 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q5_K_S.gguf) | i1-Q5_K_S | 22.3 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q5_K_M.gguf) | i1-Q5_K_M | 23.0 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Olmo-3.1-32B-Instruct-impotent-heresy-i1-GGUF/resolve/main/Olmo-3.1-32B-Instruct-impotent-heresy.i1-Q6_K.gguf) | i1-Q6_K | 26.5 | practically like static Q6_K |
|
||||||
|
|
||||||
|
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
||||||
|
types (lower is better):
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
And here are Artefact2's thoughts on the matter:
|
||||||
|
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
|
||||||
|
|
||||||
|
## FAQ / Model Request
|
||||||
|
|
||||||
|
See https://huggingface.co/mradermacher/model_requests for some answers to
|
||||||
|
questions you might have and/or if you want some other model quantized.
|
||||||
|
|
||||||
|
## Thanks
|
||||||
|
|
||||||
|
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
|
||||||
|
me use its servers and providing upgrades to my workstation to enable
|
||||||
|
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
|
||||||
|
|
||||||
|
<!-- end -->
|
||||||
Reference in New Issue
Block a user