mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-10 05:46:05 +08:00
commit 102cd4ecc7
27 changed files with 220 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
.gitattributes vendored Normal file
View File

@@ -0,0 +1,60 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
FFT-IBM-Granite-4.0-micro.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text

3
FFT-IBM-Granite-4.0-micro.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c9eae7b69c02c36fc1624c6d401985adfbba26bdbe53e0563459c027fd6a0e85
size 908877920

3
FFT-IBM-Granite-4.0-micro.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e6f1db4affd33621687dab847c0b6fa94d0e6fd0ff8211629db33fa9b46a7b40
size 844980320

3
FFT-IBM-Granite-4.0-micro.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6955b403c5800a970897d3d55286be7e4ebc8e00af41b05dc66f35b9bb15ece7
size 1241473120

3
FFT-IBM-Granite-4.0-micro.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c8e03fdd4f2da37f8cfd81d8d4510a840bba21fe0c16432bf25f9fcd7456ffbe
size 1156276320

3
FFT-IBM-Granite-4.0-micro.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c9a179c2653193125291c6aaa16c940b2e3bad1c3a2956e0cacfee26c07a1d7a
size 1108762720

3
FFT-IBM-Granite-4.0-micro.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:02460ecf5d0ba764cb9424f73ad94b618cd05384f2e61571532023c3c35a1dfe
size 1015373920

3
FFT-IBM-Granite-4.0-micro.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8da51f92efd556cb220d9824417890a4e149bc8831e3e61e02b1655030f9994d
size 1622524000

3
FFT-IBM-Granite-4.0-micro.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:591eda45fb66f8ce74a8805ea0173c85651a52731591ab8a430d4c01101b0b28
size 1573781600

3
FFT-IBM-Granite-4.0-micro.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bbbcff39b545bb4077935f913a0a86d689f23d6c1c87dfad6656ee02c41a4a0b
size 1500053600

3
FFT-IBM-Granite-4.0-micro.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f991c4824a369df54f2f25aee410924bc453472eb748053c07bbb7aa58d41b0b
size 1387290720

3
FFT-IBM-Granite-4.0-micro.i1-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:655ca117d21b377beb8594e22a8287d28eb344ea8eaf09e5f53ec596f1bf9733
size 1991164000

3
FFT-IBM-Granite-4.0-micro.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4b816f46aa8740af3ec490075a954d55ac4694f716899ada5eb4bacf0beb8d3b
size 1894498400

3
FFT-IBM-Granite-4.0-micro.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dd3c76349ea59ce228ab3ae572ada57f29be54d20a7d757596b887bf94521024
size 1371439200

3
FFT-IBM-Granite-4.0-micro.i1-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fdb899aa49889f47935f030a6f602a9cd779e20e93fea4c470186b4503051adf
size 1284194400

3
FFT-IBM-Granite-4.0-micro.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:449d0b1d149b8f85d976ce60c47ba1ac7355feeeb1f6f44e27b3a4a635370b99
size 1864188000

3
FFT-IBM-Granite-4.0-micro.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:50d6ff63517544c1981cb3d5c050aaff44e40df58f98ed4f2d4359c0c28cfcfb
size 1725579360

3
FFT-IBM-Granite-4.0-micro.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2c4539eeb3a355df1139ee152f232c66d2edb30695b85ebad4a4334dfcf7438f
size 1566818400

3
FFT-IBM-Granite-4.0-micro.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5a761197f66cdde20902791b3ca4c7b0f885c9009b157cbe2e4801ce4aabdfb5
size 1991164000

3
FFT-IBM-Granite-4.0-micro.i1-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d6cc9af8def58529f2a7580df28e540129828c99bb6d43a29d571da81fc380d9
size 2181218400

3
FFT-IBM-Granite-4.0-micro.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0c368a98d38c28c8336c82f1c111f2b16e2514fd46b73eeaa536a03c86331f5c
size 2099503200

3
FFT-IBM-Granite-4.0-micro.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2f36231f245af6452b16dafa035818810f6d587b6676078b613c571676faf5e0
size 1998372960

3
FFT-IBM-Granite-4.0-micro.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:46a64ecfd576e8074c2db16e801e7b50d37828def1eaace98f6fe3e79a96a828
size 2437013600

3
FFT-IBM-Granite-4.0-micro.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9f508ef7d597eb9c2e080f96a320d14ed4872b916d08787c5925a54c1e5feb4e
size 2377826400

3
FFT-IBM-Granite-4.0-micro.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7ae2d3a11cbf2cdc8ee3644e67c2b9894a695ccd09d30c2d7bd1ed5d5c933e6a
size 2795618400

3
FFT-IBM-Granite-4.0-micro.imatrix.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d4ab7b7a90f1de539753db831bdfbf5fc095e71ad6e89055dc82be06f4032c56
size 3811392

85
README.md Normal file
View File

@@ -0,0 +1,85 @@
---
base_model: bdstar/FFT-IBM-Granite-4.0-micro
language:
- en
library_name: transformers
license: mit
mradermacher:
readme_rev: 1
quantized_by: mradermacher
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
<!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S -->
<!-- ### quants_skip: -->
<!-- ### skip_mmproj: -->
weighted/imatrix quants of https://huggingface.co/bdstar/FFT-IBM-Granite-4.0-micro
<!-- provided-files -->
***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#FFT-IBM-Granite-4.0-micro-i1-GGUF).***
static quants are available at https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-IQ1_S.gguf) | i1-IQ1_S | 0.9 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-IQ1_M.gguf) | i1-IQ1_M | 1.0 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.1 | |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.2 | |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-IQ2_S.gguf) | i1-IQ2_S | 1.3 | |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-IQ2_M.gguf) | i1-IQ2_M | 1.3 | |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.4 | very low quality |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-Q2_K.gguf) | i1-Q2_K | 1.5 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.5 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.6 | |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-Q3_K_S.gguf) | i1-Q3_K_S | 1.7 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-IQ3_S.gguf) | i1-IQ3_S | 1.7 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-IQ3_M.gguf) | i1-IQ3_M | 1.7 | |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-Q3_K_M.gguf) | i1-Q3_K_M | 1.8 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-Q3_K_L.gguf) | i1-Q3_K_L | 2.0 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-IQ4_XS.gguf) | i1-IQ4_XS | 2.0 | |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.1 | prefer IQ4_XS |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-Q4_0.gguf) | i1-Q4_0 | 2.1 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.1 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.2 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-Q4_1.gguf) | i1-Q4_1 | 2.3 | |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-Q5_K_S.gguf) | i1-Q5_K_S | 2.5 | |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-Q5_K_M.gguf) | i1-Q5_K_M | 2.5 | |
| [GGUF](https://huggingface.co/mradermacher/FFT-IBM-Granite-4.0-micro-i1-GGUF/resolve/main/FFT-IBM-Granite-4.0-micro.i1-Q6_K.gguf) | i1-Q6_K | 2.9 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->