mradermacher/granite-guardian-3.2-5b-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/granite-guardian-3.2-5b-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-17 01:58:16 +08:00
commit 72e19a9ee5
14 changed files with 147 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
.gitattributes vendored Normal file
View File

@@ -0,0 +1,47 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
granite-guardian-3.2-5b.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
granite-guardian-3.2-5b.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
granite-guardian-3.2-5b.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
granite-guardian-3.2-5b.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
granite-guardian-3.2-5b.f16.gguf filter=lfs diff=lfs merge=lfs -text
granite-guardian-3.2-5b.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
granite-guardian-3.2-5b.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
granite-guardian-3.2-5b.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
granite-guardian-3.2-5b.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
granite-guardian-3.2-5b.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
granite-guardian-3.2-5b.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
granite-guardian-3.2-5b.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text

64
README.md Normal file
View File

@@ -0,0 +1,64 @@
---
base_model: ibm-granite/granite-guardian-3.2-5b
language:
- en
library_name: transformers
license: apache-2.0
quantized_by: mradermacher
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: -->
static quants of https://huggingface.co/ibm-granite/granite-guardian-3.2-5b
<!-- provided-files -->
weighted/imatrix quants are available at https://huggingface.co/mradermacher/granite-guardian-3.2-5b-i1-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/granite-guardian-3.2-5b-GGUF/resolve/main/granite-guardian-3.2-5b.Q2_K.gguf) | Q2_K | 2.3 | |
| [GGUF](https://huggingface.co/mradermacher/granite-guardian-3.2-5b-GGUF/resolve/main/granite-guardian-3.2-5b.Q3_K_S.gguf) | Q3_K_S | 2.7 | |
| [GGUF](https://huggingface.co/mradermacher/granite-guardian-3.2-5b-GGUF/resolve/main/granite-guardian-3.2-5b.Q3_K_M.gguf) | Q3_K_M | 2.9 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/granite-guardian-3.2-5b-GGUF/resolve/main/granite-guardian-3.2-5b.Q3_K_L.gguf) | Q3_K_L | 3.2 | |
| [GGUF](https://huggingface.co/mradermacher/granite-guardian-3.2-5b-GGUF/resolve/main/granite-guardian-3.2-5b.IQ4_XS.gguf) | IQ4_XS | 3.3 | |
| [GGUF](https://huggingface.co/mradermacher/granite-guardian-3.2-5b-GGUF/resolve/main/granite-guardian-3.2-5b.Q4_K_S.gguf) | Q4_K_S | 3.4 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/granite-guardian-3.2-5b-GGUF/resolve/main/granite-guardian-3.2-5b.Q4_K_M.gguf) | Q4_K_M | 3.6 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/granite-guardian-3.2-5b-GGUF/resolve/main/granite-guardian-3.2-5b.Q5_K_S.gguf) | Q5_K_S | 4.1 | |
| [GGUF](https://huggingface.co/mradermacher/granite-guardian-3.2-5b-GGUF/resolve/main/granite-guardian-3.2-5b.Q5_K_M.gguf) | Q5_K_M | 4.2 | |
| [GGUF](https://huggingface.co/mradermacher/granite-guardian-3.2-5b-GGUF/resolve/main/granite-guardian-3.2-5b.Q6_K.gguf) | Q6_K | 4.8 | very good quality |
| [GGUF](https://huggingface.co/mradermacher/granite-guardian-3.2-5b-GGUF/resolve/main/granite-guardian-3.2-5b.Q8_0.gguf) | Q8_0 | 6.2 | fast, best quality |
| [GGUF](https://huggingface.co/mradermacher/granite-guardian-3.2-5b-GGUF/resolve/main/granite-guardian-3.2-5b.f16.gguf) | f16 | 11.7 | 16 bpw, overkill |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time.
<!-- end -->

3
granite-guardian-3.2-5b.IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cc1cbde11f08614f775e8197730eb42ee1dd79ce823ae4bc6f5f7e1b55e89004
size 3174306528

3
granite-guardian-3.2-5b.Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:12b31740b21706dae80ab6b50365de7a5bb92f6124553f3e26d8bc65dfa62b2c
size 2222592736

3
granite-guardian-3.2-5b.Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3722ed0a7f702bdc5259a73ee6d7f8fa7e58123e53b984bcc4ddf51436e926c8
size 3094680288

3
granite-guardian-3.2-5b.Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:41cffa7240fb2015122bef0b3d19c8ce6fd2d1be80d616a0f3cc0006fba45775
size 2845381344

3
granite-guardian-3.2-5b.Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b4de7d48902ba47e17b624002ece79edad1e48c1056098074e8c7cdf96f2b249
size 2564821728

3
granite-guardian-3.2-5b.Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:03fea8944722f07017ed163d97837d82c8938e93cfe263c69102fe1ae9893f67
size 3510080224

3
granite-guardian-3.2-5b.Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:68929a1638295ef2318bc36880ce5ce49ccd840ddb3ccab4428c6d4978451a30
size 3327464160

3
granite-guardian-3.2-5b.Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:54dd341b681340025cb30d0c73a135f16d0eae336b1d9cbe1d9ca009518d5856
size 4108292832

3
granite-guardian-3.2-5b.Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a1864c2477ace920432c97deaa5a2277bd371a4292ae9df72691ddfedf46d2e9
size 4003009248

3
granite-guardian-3.2-5b.Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:afb533fc5f5e392b3543a46f2bde30fe5b706f81879768238b3eaffdbbd0c05c
size 4743893728

3
granite-guardian-3.2-5b.Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2d8be44e44645e68b76aa62aa45336924450a0d37fb6530bc021f514c7dc131a
size 6143680128

3
granite-guardian-3.2-5b.f16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3bfb1594abc4be0575283b200516f1fd4a9eb1d4a01f2d6727000179248cdbe3
size 11562208128