mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-05 04:30:16 +08:00
commit f6d0fc9a98
27 changed files with 218 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
.gitattributes vendored Normal file
View File

@@ -0,0 +1,60 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
Llama3-Toxic-8B-Float16.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text

3
Llama3-Toxic-8B-Float16.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4eee97d4c621553605f4430d771425f4231b59dcffe879e15002f6be84a2a36c
size 2161972512

3
Llama3-Toxic-8B-Float16.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f1de30890bd9647bd0fd52df80e01853d0945d40b9fbbea907fa59bcda1b616d
size 2019628320

3
Llama3-Toxic-8B-Float16.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:28c2d486d502c41c61e93422610922636e7b236d618368b50cf60d390f6e6cb9
size 2948281632

3
Llama3-Toxic-8B-Float16.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:55b47d23acd082171bcecdad86bb22134eaccb7de4ec1047a96f956382eea29f
size 2758489376

3
Llama3-Toxic-8B-Float16.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:849828171a61f67117e255aeea0d1dba0eb333b80bb13c82265a4ecf7df5fb76
size 2605782304

3
Llama3-Toxic-8B-Float16.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0b08333f2e8aa7af4a0e93027041078c84d20c7ec46b483f745cc98c2cbd2659
size 2399212832

3
Llama3-Toxic-8B-Float16.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4851632a85def4156f7584a0fa26fd9b3fd89c03847c588fdfff4fb5d9937508
size 3784824096

3
Llama3-Toxic-8B-Float16.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9a362a98bb825b2bd7a62dff350e47d191d727fcb4afe07fb9540b58d0d2d578
size 3682325792

3
Llama3-Toxic-8B-Float16.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6b756d9357edc644ba96b486aa5bf04a8bb88dfbc5896ed0f31a2c485a6caa15
size 3518747936

3
Llama3-Toxic-8B-Float16.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3ebc7a630a2563abc9c246e2c5a3d47e9af911a7d00d1e8d3896b546a4111640
size 3274913056

3
Llama3-Toxic-8B-Float16.i1-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:704c0ef7f4bc88e4ea9e6536cd41764e756bb0010576789167f0b778f6e37ef2
size 4677989664

3
Llama3-Toxic-8B-Float16.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:69efc80c6b88f0e02c920a42e7282145e466439aaabf662ce6a7e3b45b34051a
size 4447663392

3
Llama3-Toxic-8B-Float16.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f352356693b9f4694e37da85af3c8fa03833779a39f570f7ac850715690d4bd9
size 3179132192

3
Llama3-Toxic-8B-Float16.i1-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c58dbd05cb5e0c9e489a96b05c61b5aafad19936af3e1abda707e0049a0ae8d2
size 2988815648

3
Llama3-Toxic-8B-Float16.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e81fdfc2c831b4e279526378a10409a8c385ad1c8a53eb73e45c2e2ec204a99d
size 4321957152

3
Llama3-Toxic-8B-Float16.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:319ca942303ff7cfe5355f0485670d90f189a962f85d403e9617aa8c708a9578
size 4018918688

3
Llama3-Toxic-8B-Float16.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3762f9897f56864949aaea7e7089cf933e444de8a8ba83e11609808b78bab2b5
size 3664500000

3
Llama3-Toxic-8B-Float16.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:61946188cababa80548536891dd458e65f328baa6e39c688698f47ac4facda42
size 4675892512

3
Llama3-Toxic-8B-Float16.i1-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6a3c549424d5a92909d5fb523d1a8d0945343c1be95dcb75e5eb09a0de393fef
size 5130253600

3
Llama3-Toxic-8B-Float16.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8a4f20fb67007b27e792fe5eb40237dde6d471b7208d0a3b6727ed7e6e250b7f
size 4920735008

3
Llama3-Toxic-8B-Float16.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e4ff57a7f2443efa673338718fb310ce14a474a22fa398e91fde35c93560d75c
size 4692669728

3
Llama3-Toxic-8B-Float16.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dc0510f2222cf6f5feaf26035887769dff2f5f31ba0a9b1bac924fa76755af27
size 5732988192

3
Llama3-Toxic-8B-Float16.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:548ac0448084ebf1611b0ab37fda8bf9a305326a62ff44a0b6913dc0bcfb43ab
size 5599294752

3
Llama3-Toxic-8B-Float16.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:312ca9010824c9e2fe5a7840a79cc9741361297b8a562974eaa0a1753579cce5
size 6596007200

83
README.md Normal file
View File

@@ -0,0 +1,83 @@
---
base_model: theminji/Llama3-Toxic-8B-Float16
language:
- en
library_name: transformers
license: apache-2.0
quantized_by: mradermacher
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- sft
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
weighted/imatrix quants of https://huggingface.co/theminji/Llama3-Toxic-8B-Float16
<!-- provided-files -->
static quants are available at https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-IQ1_S.gguf) | i1-IQ1_S | 2.1 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-IQ1_M.gguf) | i1-IQ1_M | 2.3 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.5 | |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.7 | |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-IQ2_S.gguf) | i1-IQ2_S | 2.9 | |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-IQ2_M.gguf) | i1-IQ2_M | 3.0 | |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.1 | very low quality |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-Q2_K.gguf) | i1-Q2_K | 3.3 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.4 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.6 | |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.8 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-IQ3_S.gguf) | i1-IQ3_S | 3.8 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-IQ3_M.gguf) | i1-IQ3_M | 3.9 | |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.1 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.4 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.5 | |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-Q4_0.gguf) | i1-Q4_0 | 4.8 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-IQ4_NL.gguf) | i1-IQ4_NL | 4.8 | prefer IQ4_XS |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.8 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.0 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-Q4_1.gguf) | i1-Q4_1 | 5.2 | |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.7 | |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.8 | |
| [GGUF](https://huggingface.co/mradermacher/Llama3-Toxic-8B-Float16-i1-GGUF/resolve/main/Llama3-Toxic-8B-Float16.i1-Q6_K.gguf) | i1-Q6_K | 6.7 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1cb7a906919df1b72a3bcbfa33b799cf7877c91cdc9ffb4fdadc1bbab948d82e
size 4988157