mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-12 21:43:31 +08:00
commit 43c1f5b1be
52 changed files with 502 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

85
.gitattributes vendored Normal file
View File

@@ -0,0 +1,85 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-heretic.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-Instruct-Heretic.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text

3
Llama-3.1-8B-Instruct-Heretic.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9725dd7cec72c753e0589b2963730b6ef7a4b2754e16a4f9da756db0703dfa37
size 2161977760

3
Llama-3.1-8B-Instruct-Heretic.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ae79306c9d496e8dab0456b1191a33d31df4511d1ea01f3343541e3640b82b97
size 2019633568

3
Llama-3.1-8B-Instruct-Heretic.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bcd4da7292d56808ab6a4fe4183360a276f7f957c7cea8b05f8aede927524f2f
size 2948286880

3
Llama-3.1-8B-Instruct-Heretic.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e3983b075968bd96220c4e994602b165e5f6428ccb10c92ff66fcf27b709c45d
size 2758494624

3
Llama-3.1-8B-Instruct-Heretic.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5a979f881d2b8ccf238814a02ed69aae8b4a6c4c37a534e00b966b8808a9b2f9
size 2605787552

3
Llama-3.1-8B-Instruct-Heretic.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:07ca159de7d79af14f356b60891d38743ef03aaebab85588b843a279fce3be55
size 2399218080

3
Llama-3.1-8B-Instruct-Heretic.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c32008687c4ede9b3e8a1c112685e44506a302d7700815481fab09e602f0a6bc
size 3784829344

3
Llama-3.1-8B-Instruct-Heretic.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6d5bc2b03ecdea33302a8a2a5ca51f01e1f2418405687546ee48c3f7dcb9a50a
size 3682331040

3
Llama-3.1-8B-Instruct-Heretic.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2f82155d5cb9408d40e083713dd8d777a6ece403b7a8373f893592b5d283e7a1
size 3518753184

3
Llama-3.1-8B-Instruct-Heretic.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4f1ad033320d5d3fb46f61d7950f34e1720c8e93212629400bdd824a41182bb9
size 3274918304

3
Llama-3.1-8B-Instruct-Heretic.i1-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d349189df6877d9f4e1880bdd101faa9b97c12262d78facb08ed0bbb9ba71585
size 4677994912

3
Llama-3.1-8B-Instruct-Heretic.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:33288e065aab0b43441a3187a5985a963837a700252136a692aa45174733daf8
size 4447668640

3
Llama-3.1-8B-Instruct-Heretic.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:923ef88ba74eccd8a6ffbfb6b35d48e7eee56932069e3fa4e4adba0008baeae0
size 3179137440

3
Llama-3.1-8B-Instruct-Heretic.i1-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4df11766b8796609ad88f3725f6365e08b0a242ff9a494521b2b7841912625be
size 2988820896

3
Llama-3.1-8B-Instruct-Heretic.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:aa54dee3243508a6c932621cfab50784fcefed56effd2745f8747ed834a4aaaf
size 4321962400

3
Llama-3.1-8B-Instruct-Heretic.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0a61e72a72ffffc9b715ffb78135d059be19af2ec24de8517625ef4d566b5561
size 4018923936

3
Llama-3.1-8B-Instruct-Heretic.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:aec3670bc0541ce73b1a80c715e5e4c4e286affbd9c20e16e6e5da2385903f21
size 3664505248

3
Llama-3.1-8B-Instruct-Heretic.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e69302efac7147666d92ff9426ce00f277c8374bc817888601bcf2653fd06936
size 4675897760

3
Llama-3.1-8B-Instruct-Heretic.i1-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:381ecc76687a523c1c66673cbe76e345ada13337d70d112d0dda71d0ca5f5594
size 5130258848

3
Llama-3.1-8B-Instruct-Heretic.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c885db9581cf45e46338d0e5bdefb2d7bf05688581890046ba7cbf40cddc66b4
size 4920740256

3
Llama-3.1-8B-Instruct-Heretic.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:edfe1a2099e30e2a5c9286532d891d1d7dca6e18ebba2f25d0443b67b76ebf5d
size 4692674976

3
Llama-3.1-8B-Instruct-Heretic.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0aaa23f543ccb42e11a336929df39e959cb1b912f1cf5ff641605498f5fc4a80
size 5732993440

3
Llama-3.1-8B-Instruct-Heretic.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c016dd51b8babbf8e1fe26f6ea62c1909ca89156ef724f1fb0c14fe81899d687
size 5599300000

3
Llama-3.1-8B-Instruct-Heretic.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:78b39ad7579434784984565f68c07b3a94cb0ba1dbd24d29c2803dc9063346d0
size 6596012448

3
Llama-3.1-8B-Instruct-Heretic.imatrix.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c2826757cc5c700a7fa083e037535fa5ec6c31909e1ab67e15f7527ee49ed821
size 5015200

3
Llama-3.1-8B-Instruct-heretic.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2388fbb39f97e35ae6f97fa139afef112fdd3256afd58272add798d496218791
size 2161977760

3
Llama-3.1-8B-Instruct-heretic.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6fea684daa1977fa95ca2fb759358deb5d017aea74f41c5185427bfd4c16f003
size 2019633568

3
Llama-3.1-8B-Instruct-heretic.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:620c72c49e8092f71cf47ce7da3b25bb6a58efeedb1dc16237ab0827db181730
size 2948286880

3
Llama-3.1-8B-Instruct-heretic.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6441878d6d2bcf8fbe34aa13437d4b91e8a6f3b1c885e5aeb2fbc9d09d788bb0
size 2758494624

3
Llama-3.1-8B-Instruct-heretic.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8ac11c5f1a113e1bd27686fbfc80a1be694e9d915efb74b473293e6f57ae7362
size 2605787552

3
Llama-3.1-8B-Instruct-heretic.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ca6b13cf82049b33331fce6a780f312d1b77b863ba9d465fff385a1ac266eed9
size 2399218080

3
Llama-3.1-8B-Instruct-heretic.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0445252d3324f98a02295cd1ef008a3627b7642f459844431e4bd9913f3b4118
size 3784829344

3
Llama-3.1-8B-Instruct-heretic.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e7d9cd1b84eee34396f94ea32c456ec722611c6c8f8c4af022ce2c85e4bcd525
size 3682331040

3
Llama-3.1-8B-Instruct-heretic.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:76b294b52a30bf9ea914ec2770c5acd31c4b3573ec4b4f7a52361211b1675af7
size 3518753184

3
Llama-3.1-8B-Instruct-heretic.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:50a4c39f4c97a55f9a1ccf0477ab0f8c82ff04aea3a9b7111f88743d0f38fa70
size 3274918304

3
Llama-3.1-8B-Instruct-heretic.i1-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1dff24bba7a8413ee6265f7d9f97634fd82577377cc80fb026a07543914420dc
size 4677994912

3
Llama-3.1-8B-Instruct-heretic.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7e53df7c8ec971cccd9c399a9488ef60471ea28e7ee4f57693f86e784f9456ba
size 4447668640

3
Llama-3.1-8B-Instruct-heretic.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:97c9f9868a3e66d6faafa06549656d12a8afbca821822d2f7c14b8b2369d8df0
size 3179137440

3
Llama-3.1-8B-Instruct-heretic.i1-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5af427530ce9879443280165916e027818104c13021f4f380828e853a4782844
size 2988820896

3
Llama-3.1-8B-Instruct-heretic.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:43158519b29806adcd3b8c61f65d4e62aa03cb7791cf2747394e0a1dfa1dbf48
size 4321962400

3
Llama-3.1-8B-Instruct-heretic.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:80063c8cbc270a30465ebeb3cee77b66d8221329701ffce0c888297171b9e419
size 4018923936

3
Llama-3.1-8B-Instruct-heretic.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2f454665316441b54754a9727f2bc6e6b6c198ba6d2164145724e9fb9b31e34a
size 3664505248

3
Llama-3.1-8B-Instruct-heretic.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3a3e1d09b14caf5094ba1daee4531a14833614fbf1c87977447b20170ae9eca9
size 4675897760

3
Llama-3.1-8B-Instruct-heretic.i1-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d1e16815fc548a0ead6417f568f5ce3b5138669f8e74c3245f810a0184ae3fe5
size 5130258848

3
Llama-3.1-8B-Instruct-heretic.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:eb1568f011249d9214d4de1552e4115cc6635852de73121317707f015a4952f7
size 4920740256

3
Llama-3.1-8B-Instruct-heretic.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4b72ca49f4d4a85a5a0ea46209c21e9c89ef906b8d56c8cd303ba1ac67e12225
size 4692674976

3
Llama-3.1-8B-Instruct-heretic.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:013293c3752744888a1e55bb4be92e2a5f32746ec999211e204bf1a6a4d8c0d9
size 5732993440

3
Llama-3.1-8B-Instruct-heretic.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:46e859dded6de66827663ff1da4058d667b4804af34e85184d231f64ce31c58b
size 5599300000

3
Llama-3.1-8B-Instruct-heretic.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5312e8dc98a66ea3c2e1a4c2fd8e6fe0e2d69ce7839b81e7a105b303a804917b
size 6596012448

3
Llama-3.1-8B-Instruct-heretic.imatrix.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:279ff559a6c29e0483cc153d70d693346715729f961fb4f043796242dc4c80fb
size 5015200

267
README.md Normal file
View File

@@ -0,0 +1,267 @@
---
base_model: p-e-w/Llama-3.1-8B-Instruct-heretic
extra_gated_button_content: Submit
extra_gated_fields:
Affiliation: text
? By clicking Submit below I accept the terms of the license and acknowledge that
the information I provide will be collected stored processed and shared in accordance
with the Meta Privacy Policy
: checkbox
Country: country
Date of birth: date_picker
First Name: text
Job title:
options:
- Student
- Research Graduate
- AI researcher
- AI developer/engineer
- Reporter
- Other
type: select
Last Name: text
geo: ip_location
extra_gated_prompt: "### LLAMA 3.1 COMMUNITY LICENSE AGREEMENT\nLlama 3.1 Version
Release Date: July 23, 2024\n\"Agreement\" means the terms and conditions for use,
reproduction, distribution and modification of the Llama Materials set forth herein.\n\"Documentation\"
means the specifications, manuals and documentation accompanying Llama 3.1 distributed
by Meta at https://llama.meta.com/doc/overview.\n\"Licensee\" or \"you\" means you,
or your employer or any other person or entity (if you are entering into this Agreement
on such person or entity’s behalf), of the age required under applicable laws, rules
or regulations to provide legal consent and that has legal authority to bind your
employer or such other person or entity if you are entering in this Agreement on
their behalf.\n\"Llama 3.1\" means the foundational large language models and software
and algorithms, including machine-learning model code, trained model weights, inference-enabling
code, training-enabling code, fine-tuning enabling code and other elements of the
foregoing distributed by Meta at https://llama.meta.com/llama-downloads.\n\"Llama
Materials\" means, collectively, Meta’s proprietary Llama 3.1 and Documentation
(and any portion thereof) made available under this Agreement.\n\"Meta\" or \"we\"
means Meta Platforms Ireland Limited (if you are located in or, if you are an entity,
your principal place of business is in the EEA or Switzerland) and Meta Platforms,
Inc. (if you are located outside of the EEA or Switzerland).\n \n1. License Rights
and Redistribution.\na. Grant of Rights. You are granted a non-exclusive, worldwide,
non-transferable and royalty-free limited license under Meta’s intellectual property
or other rights owned by Meta embodied in the Llama Materials to use, reproduce,
distribute, copy, create derivative works of, and make modifications to the Llama
Materials.\nb. Redistribution and Use.\ni. If you distribute or make available the
Llama Materials (or any derivative works thereof), or a product or service (including
another AI model) that contains any of them, you shall (A) provide a copy of this
Agreement with any such Llama Materials; and (B) prominently display “Built with
Llama” on a related website, user interface, blogpost, about page, or product documentation.
If you use the Llama Materials or any outputs or results of the Llama Materials
to create, train, fine tune, or otherwise improve an AI model, which is distributed
or made available, you shall also include “Llama” at the beginning of any such AI
model name.\nii. If you receive Llama Materials, or any derivative works thereof,
from a Licensee as part of an integrated end user product, then Section 2 of this
Agreement will not apply to you.\niii. You must retain in all copies of the Llama
Materials that you distribute the following attribution notice within a “Notice”
text file distributed as a part of such copies: “Llama 3.1 is licensed under the
Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.”\niv.
Your use of the Llama Materials must comply with applicable laws and regulations
(including trade compliance laws and regulations) and adhere to the Acceptable Use
Policy for the Llama Materials (available at https://llama.meta.com/llama3_1/use-policy),
which is hereby incorporated by reference into this Agreement.\n2. Additional Commercial
Terms. If, on the Llama 3.1 version release date, the monthly active users of the
products or services made available by or for Licensee, or Licensee’s affiliates,
is greater than 700 million monthly active users in the preceding calendar month,
you must request a license from Meta, which Meta may grant to you in its sole discretion,
and you are not authorized to exercise any of the rights under this Agreement unless
or until Meta otherwise expressly grants you such rights.\n3. Disclaimer of Warranty.
UNLESS REQUIRED BY APPLICABLE LAW, THE LLAMA MATERIALS AND ANY OUTPUT AND RESULTS
THEREFROM ARE PROVIDED ON AN “AS IS” BASIS, WITHOUT WARRANTIES OF ANY KIND, AND
META DISCLAIMS ALL WARRANTIES OF ANY KIND, BOTH EXPRESS AND IMPLIED, INCLUDING,
WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY,
OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING
THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE LLAMA MATERIALS AND ASSUME ANY
RISKS ASSOCIATED WITH YOUR USE OF THE LLAMA MATERIALS AND ANY OUTPUT AND RESULTS.\n4.
Limitation of Liability. IN NO EVENT WILL META OR ITS AFFILIATES BE LIABLE UNDER
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY,
OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS OR ANY INDIRECT,
SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN IF META
OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.\n5.
Intellectual Property.\na. No trademark licenses are granted under this Agreement,
and in connection with the Llama Materials, neither Meta nor Licensee may use any
name or mark owned by or associated with the other or any of its affiliates, except
as required for reasonable and customary use in describing and redistributing the
Llama Materials or as set forth in this Section 5(a). Meta hereby grants you a license
to use “Llama” (the “Mark”) solely as required to comply with the last sentence
of Section 1.b.i. You will comply with Meta’s brand guidelines (currently accessible
at https://about.meta.com/brand/resources/meta/company-brand/ ). All goodwill arising
out of your use of the Mark will inure to the benefit of Meta.\nb. Subject to Meta’s
ownership of Llama Materials and derivatives made by or for Meta, with respect to
any derivative works and modifications of the Llama Materials that are made by you,
as between you and Meta, you are and will be the owner of such derivative works
and modifications.\nc. If you institute litigation or other proceedings against
Meta or any entity (including a cross-claim or counterclaim in a lawsuit) alleging
that the Llama Materials or Llama 3.1 outputs or results, or any portion of any
of the foregoing, constitutes infringement of intellectual property or other rights
owned or licensable by you, then any licenses granted to you under this Agreement
shall terminate as of the date such litigation or claim is filed or instituted.
You will indemnify and hold harmless Meta from and against any claim by any third
party arising out of or related to your use or distribution of the Llama Materials.\n6.
Term and Termination. The term of this Agreement will commence upon your acceptance
of this Agreement or access to the Llama Materials and will continue in full force
and effect until terminated in accordance with the terms and conditions herein.
Meta may terminate this Agreement if you are in breach of any term or condition
of this Agreement. Upon termination of this Agreement, you shall delete and cease
use of the Llama Materials. Sections 3, 4 and 7 shall survive the termination of
this Agreement.\n7. Governing Law and Jurisdiction. This Agreement will be governed
and construed under the laws of the State of California without regard to choice
of law principles, and the UN Convention on Contracts for the International Sale
of Goods does not apply to this Agreement. The courts of California shall have exclusive
jurisdiction of any dispute arising out of this Agreement.\n### Llama 3.1 Acceptable
Use Policy\nMeta is committed to promoting safe and fair use of its tools and features,
including Llama 3.1. If you access or use Llama 3.1, you agree to this Acceptable
Use Policy (“Policy”). The most recent copy of this policy can be found at [https://llama.meta.com/llama3_1/use-policy](https://llama.meta.com/llama3_1/use-policy)\n####
Prohibited Uses\nWe want everyone to use Llama 3.1 safely and responsibly. You agree
you will not use, or allow others to use, Llama 3.1 to:\n 1. Violate the law or
others’ rights, including to:\n 1. Engage in, promote, generate, contribute to,
encourage, plan, incite, or further illegal or unlawful activity or content, such
as:\n 1. Violence or terrorism\n 2. Exploitation or harm to children,
including the solicitation, creation, acquisition, or dissemination of child exploitative
content or failure to report Child Sexual Abuse Material\n 3. Human trafficking,
exploitation, and sexual violence\n 4. The illegal distribution of information
or materials to minors, including obscene materials, or failure to employ legally
required age-gating in connection with such information or materials.\n 5.
Sexual solicitation\n 6. Any other criminal activity\n 3. Engage in, promote,
incite, or facilitate the harassment, abuse, threatening, or bullying of individuals
or groups of individuals\n 4. Engage in, promote, incite, or facilitate discrimination
or other unlawful or harmful conduct in the provision of employment, employment
benefits, credit, housing, other economic benefits, or other essential goods and
services\n 5. Engage in the unauthorized or unlicensed practice of any profession
including, but not limited to, financial, legal, medical/health, or related professional
practices\n 6. Collect, process, disclose, generate, or infer health, demographic,
or other sensitive personal or private information about individuals without rights
and consents required by applicable laws\n 7. Engage in or facilitate any action
or generate any content that infringes, misappropriates, or otherwise violates any
third-party rights, including the outputs or results of any products or services
using the Llama Materials\n 8. Create, generate, or facilitate the creation of
malicious code, malware, computer viruses or do anything else that could disable,
overburden, interfere with or impair the proper working, integrity, operation or
appearance of a website or computer system\n2. Engage in, promote, incite, facilitate,
or assist in the planning or development of activities that present a risk of death
or bodily harm to individuals, including use of Llama 3.1 related to the following:\n
\ 1. Military, warfare, nuclear industries or applications, espionage, use for
materials or activities that are subject to the International Traffic Arms Regulations
(ITAR) maintained by the United States Department of State\n 2. Guns and illegal
weapons (including weapon development)\n 3. Illegal drugs and regulated/controlled
substances\n 4. Operation of critical infrastructure, transportation technologies,
or heavy machinery\n 5. Self-harm or harm to others, including suicide, cutting,
and eating disorders\n 6. Any content intended to incite or promote violence,
abuse, or any infliction of bodily harm to an individual\n3. Intentionally deceive
or mislead others, including use of Llama 3.1 related to the following:\n 1.
Generating, promoting, or furthering fraud or the creation or promotion of disinformation\n
\ 2. Generating, promoting, or furthering defamatory content, including the creation
of defamatory statements, images, or other content\n 3. Generating, promoting,
or further distributing spam\n 4. Impersonating another individual without consent,
authorization, or legal right\n 5. Representing that the use of Llama 3.1 or
outputs are human-generated\n 6. Generating or facilitating false online engagement,
including fake reviews and other means of fake online engagement\n4. Fail to appropriately
disclose to end users any known dangers of your AI system\nPlease report any violation
of this Policy, software “bug,” or other problems that could lead to a violation
of this Policy through one of the following means:\n * Reporting issues with
the model: [https://github.com/meta-llama/llama-models/issues](https://github.com/meta-llama/llama-models/issues)\n
\ * Reporting risky content generated by the model:\n developers.facebook.com/llama_output_feedback\n
\ * Reporting bugs and security concerns: facebook.com/whitehat/info\n * Reporting
violations of the Acceptable Use Policy or unlicensed uses of Meta Llama 3: LlamaUseReport@meta.com"
language:
- en
- de
- fr
- it
- pt
- hi
- es
- th
library_name: transformers
license: llama3.1
mradermacher:
readme_rev: 1
quantized_by: mradermacher
tags:
- facebook
- unsloth
- meta
- pytorch
- llama
- llama-3
- heretic
- uncensored
- decensored
- abliterated
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
<!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S -->
<!-- ### quants_skip: -->
<!-- ### skip_mmproj: -->
weighted/imatrix quants of https://huggingface.co/p-e-w/Llama-3.1-8B-Instruct-heretic
<!-- provided-files -->
***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Llama-3.1-8B-Instruct-heretic-i1-GGUF).***
static quants are available at https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.imatrix.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-IQ1_S.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-IQ1_S.gguf) | i1-IQ1_S | 4.1 | for the desperate |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-IQ1_M.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-IQ1_M.gguf) | i1-IQ1_M | 4.4 | mostly desperate |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-IQ2_XXS.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 4.9 | |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-IQ2_XS.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-IQ2_XS.gguf) | i1-IQ2_XS | 5.3 | |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-IQ2_S.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-IQ2_S.gguf) | i1-IQ2_S | 5.6 | |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-IQ2_M.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-IQ2_M.gguf) | i1-IQ2_M | 6.0 | |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-Q2_K_S.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-Q2_K_S.gguf) | i1-Q2_K_S | 6.1 | very low quality |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-Q2_K.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-Q2_K.gguf) | i1-Q2_K | 6.5 | IQ3_XXS probably better |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-IQ3_XXS.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 6.6 | lower quality |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-IQ3_XS.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-IQ3_XS.gguf) | i1-IQ3_XS | 7.1 | |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-Q3_K_S.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-Q3_K_S.gguf) | i1-Q3_K_S | 7.4 | IQ3_XS probably better |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-IQ3_S.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-IQ3_S.gguf) | i1-IQ3_S | 7.5 | beats Q3_K* |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-IQ3_M.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-IQ3_M.gguf) | i1-IQ3_M | 7.7 | |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-Q3_K_M.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-Q3_K_M.gguf) | i1-Q3_K_M | 8.1 | IQ3_S probably better |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-Q3_K_L.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-Q3_K_L.gguf) | i1-Q3_K_L | 8.7 | IQ3_M probably better |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-IQ4_XS.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-IQ4_XS.gguf) | i1-IQ4_XS | 9.0 | |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-Q4_0.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-Q4_0.gguf) | i1-Q4_0 | 9.5 | fast, low quality |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-IQ4_NL.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-IQ4_NL.gguf) | i1-IQ4_NL | 9.5 | prefer IQ4_XS |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-Q4_K_S.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-Q4_K_S.gguf) | i1-Q4_K_S | 9.5 | optimal size/speed/quality |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-Q4_K_M.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-Q4_K_M.gguf) | i1-Q4_K_M | 9.9 | fast, recommended |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-Q4_1.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-Q4_1.gguf) | i1-Q4_1 | 10.4 | |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-Q5_K_S.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-Q5_K_S.gguf) | i1-Q5_K_S | 11.3 | |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-Q5_K_M.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-Q5_K_M.gguf) | i1-Q5_K_M | 11.6 | |
| [PART 1](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-Heretic.i1-Q6_K.gguf) [PART 2](https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-i1-GGUF/resolve/main/Llama-3.1-8B-Instruct-heretic.i1-Q6_K.gguf) | i1-Q6_K | 13.3 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->