bartowski/ibm-granite_granite-4.1-3b-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: bartowski/ibm-granite_granite-4.1-3b-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-21 02:51:15 +08:00
commit 297f2b8bca
28 changed files with 341 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

85
.gitattributes vendored Normal file
View File

@@ -0,0 +1,85 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bin.* filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zstandard filter=lfs diff=lfs merge=lfs -text
*.tfevents* filter=lfs diff=lfs merge=lfs -text
*.db* filter=lfs diff=lfs merge=lfs -text
*.ark* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ggml filter=lfs diff=lfs merge=lfs -text
*.llamafile* filter=lfs diff=lfs merge=lfs -text
*.pt2 filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q2_K_L.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q3_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q4_K_L.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q5_K_L.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-Q6_K_L.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-bf16.gguf filter=lfs diff=lfs merge=lfs -text
ibm-granite_granite-4.1-3b-imatrix.gguf filter=lfs diff=lfs merge=lfs -text

180
README.md Normal file
View File

@@ -0,0 +1,180 @@
---
quantized_by: bartowski
pipeline_tag: text-generation
base_model: ibm-granite/granite-4.1-3b
license: apache-2.0
tags:
- language
- granite-4.1
base_model_relation: quantized
---
## Llamacpp imatrix Quantizations of granite-4.1-3b by ibm-granite
Using <a href="https://github.com/ggml-org/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggml-org/llama.cpp/releases/tag/b8970">b8970</a> for quantization.
Original model: https://huggingface.co/ibm-granite/granite-4.1-3b
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/82ae9b520227f57d79ba04add13d0d0d)
Run them in your choice of tools:
- [llama.cpp](https://github.com/ggml-org/llama.cpp)
- [ramalama](https://github.com/containers/ramalama)
- [LM Studio](https://lmstudio.ai/)
- [koboldcpp](https://github.com/LostRuins/koboldcpp)
- [Jan AI](https://www.jan.ai/)
- [Text Generation Web UI](https://github.com/oobabooga/text-generation-webui)
- [LoLLMs](https://github.com/ParisNeo/lollms)
Note: if it's a newly supported model, you may need to wait for an update from the developers.
## Prompt format
```
<|start_of_role|>system<|end_of_role|>{system_prompt}<|end_of_text|>
<|start_of_role|>user<|end_of_role|>{prompt}<|end_of_text|>
<|start_of_role|>assistant<|end_of_role|>
```
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| [granite-4.1-3b-bf16.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-bf16.gguf) | bf16 | 6.81GB | false | Full BF16 weights. |
| [granite-4.1-3b-Q8_0.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q8_0.gguf) | Q8_0 | 3.62GB | false | Extremely high quality, generally unneeded but max available quant. |
| [granite-4.1-3b-Q6_K_L.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q6_K_L.gguf) | Q6_K_L | 2.97GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
| [granite-4.1-3b-Q6_K.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q6_K.gguf) | Q6_K | 2.91GB | false | Very high quality, near perfect, *recommended*. |
| [granite-4.1-3b-Q5_K_L.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q5_K_L.gguf) | Q5_K_L | 2.56GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
| [granite-4.1-3b-Q5_K_M.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q5_K_M.gguf) | Q5_K_M | 2.50GB | false | High quality, *recommended*. |
| [granite-4.1-3b-Q5_K_S.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q5_K_S.gguf) | Q5_K_S | 2.40GB | false | High quality, *recommended*. |
| [granite-4.1-3b-Q4_K_L.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q4_K_L.gguf) | Q4_K_L | 2.23GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
| [granite-4.1-3b-Q4_1.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q4_1.gguf) | Q4_1 | 2.21GB | false | Legacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon. |
| [granite-4.1-3b-Q4_K_M.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q4_K_M.gguf) | Q4_K_M | 2.17GB | false | Good quality, default size for most use cases, *recommended*. |
| [granite-4.1-3b-Q4_K_S.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q4_K_S.gguf) | Q4_K_S | 2.03GB | false | Slightly lower quality with more space savings, *recommended*. |
| [granite-4.1-3b-IQ4_NL.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-IQ4_NL.gguf) | IQ4_NL | 2.03GB | false | Similar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference. |
| [granite-4.1-3b-Q4_0.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q4_0.gguf) | Q4_0 | 2.02GB | false | Legacy format, offers online repacking for ARM and AVX CPU inference. |
| [granite-4.1-3b-IQ4_XS.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-IQ4_XS.gguf) | IQ4_XS | 1.94GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
| [granite-4.1-3b-Q3_K_XL.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q3_K_XL.gguf) | Q3_K_XL | 1.94GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
| [granite-4.1-3b-Q3_K_L.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q3_K_L.gguf) | Q3_K_L | 1.88GB | false | Lower quality but usable, good for low RAM availability. |
| [granite-4.1-3b-Q3_K_M.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q3_K_M.gguf) | Q3_K_M | 1.78GB | false | Low quality. |
| [granite-4.1-3b-IQ3_M.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-IQ3_M.gguf) | IQ3_M | 1.68GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
| [granite-4.1-3b-Q3_K_S.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q3_K_S.gguf) | Q3_K_S | 1.62GB | false | Low quality, not recommended. |
| [granite-4.1-3b-IQ3_XS.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-IQ3_XS.gguf) | IQ3_XS | 1.57GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
| [granite-4.1-3b-Q2_K_L.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q2_K_L.gguf) | Q2_K_L | 1.50GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
| [granite-4.1-3b-IQ3_XXS.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-IQ3_XXS.gguf) | IQ3_XXS | 1.46GB | false | Lower quality, new method with decent performance, comparable to Q3 quants. |
| [granite-4.1-3b-Q2_K.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-Q2_K.gguf) | Q2_K | 1.44GB | false | Very low quality but surprisingly usable. |
| [granite-4.1-3b-IQ2_M.gguf](https://huggingface.co/bartowski/ibm-granite_granite-4.1-3b-GGUF/blob/main/ibm-granite_granite-4.1-3b-IQ2_M.gguf) | IQ2_M | 1.37GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
## Embed/output weights
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
## Downloading using huggingface-cli
<details>
<summary>Click to view download instructions</summary>
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download bartowski/ibm-granite_granite-4.1-3b-GGUF --include "ibm-granite_granite-4.1-3b-Q4_K_M.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download bartowski/ibm-granite_granite-4.1-3b-GGUF --include "ibm-granite_granite-4.1-3b-Q8_0/*" --local-dir ./
```
You can either specify a new local-dir (ibm-granite_granite-4.1-3b-Q8_0) or download them all in place (./)
</details>
## ARM/AVX information
Previously, you would download Q4_0_4_4/4_8/8_8, and these would have their weights interleaved in memory in order to improve performance on ARM and AVX machines by loading up more data in one pass.
Now, however, there is something called "online repacking" for weights. details in [this PR](https://github.com/ggml-org/llama.cpp/pull/9921). If you use Q4_0 and your hardware would benefit from repacking weights, it will do it automatically on the fly.
As of llama.cpp build [b4282](https://github.com/ggml-org/llama.cpp/releases/tag/b4282) you will not be able to run the Q4_0_X_X files and will instead need to use Q4_0.
Additionally, if you want to get slightly better quality for , you can use IQ4_NL thanks to [this PR](https://github.com/ggml-org/llama.cpp/pull/10541) which will also repack the weights for ARM, though only the 4_4 for now. The loading time may be slower but it will result in an overall speed incrase.
<details>
<summary>Click to view Q4_0_X_X information (deprecated</summary>
I'm keeping this section to show the potential theoretical uplift in performance from using the Q4_0 with online repacking.
<details>
<summary>Click to view benchmarks on an AVX2 system (EPYC7702)</summary>
| model | size | params | backend | threads | test | t/s | % (vs Q4_0) |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------------: | -------------------: |-------------: |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp512 | 204.03 ± 1.03 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp1024 | 282.92 ± 0.19 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp2048 | 259.49 ± 0.44 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg128 | 39.12 ± 0.27 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg256 | 39.31 ± 0.69 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg512 | 40.52 ± 0.03 | 100% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp512 | 301.02 ± 1.74 | 147% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp1024 | 287.23 ± 0.20 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp2048 | 262.77 ± 1.81 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg128 | 18.80 ± 0.99 | 48% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg256 | 24.46 ± 3.04 | 83% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg512 | 36.32 ± 3.59 | 90% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp512 | 271.71 ± 3.53 | 133% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp1024 | 279.86 ± 45.63 | 100% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp2048 | 320.77 ± 5.00 | 124% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg128 | 43.51 ± 0.05 | 111% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg256 | 43.35 ± 0.09 | 110% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg512 | 42.60 ± 0.31 | 105% |
Q4_0_8_8 offers a nice bump to prompt processing and a small bump to text generation
</details>
</details>
## Which file should I choose?
<details>
<summary>Click here for details</summary>
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.
If you want to get more into the weeds, you can check out this extremely useful feature chart:
[llama.cpp feature matrix](https://github.com/ggml-org/llama.cpp/wiki/Feature-matrix)
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.
These I-quants can also be used on CPU, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.
</details>
## Credits
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.
Thank you ZeroWw for the inspiration to experiment with embed/output.
Thank you to LM Studio for sponsoring my work.
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}

3
ibm-granite_granite-4.1-3b-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6302ae7879bf27592b5098f2546c4df126b904f651f8c127b1c07588ca508736
size 1374878688

3
ibm-granite_granite-4.1-3b-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d9aaecf34d70d9f7ba248acabb4fb17b6cc91350c2ba2c54faa26e9bdd4b5e0f
size 1680276448

3
ibm-granite_granite-4.1-3b-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:427222a36e8d2342c33c261e3d0389246e94976463e0ea4fbbfa8a6a79b8d667
size 1565998048

3
ibm-granite_granite-4.1-3b-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:61e6c3e7714ee159dcc15b21f317e4b88325c37f0ed35ba7692bfc95bafe437f
size 1464294368

3
ibm-granite_granite-4.1-3b-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c493540614f0d88f15ff6d7729278ee8e37c5905a5dc770b58831b2b6b041237
size 2031918048

3
ibm-granite_granite-4.1-3b-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:74d1d979776834e9f70b38a4b31c4319dddea35d7ee41d7e7affbe4c35872e17
size 1939348448

3
ibm-granite_granite-4.1-3b-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e439ae7bc575481f624443a5587284d983550a8b6a784d0fd64ee2513b4bfc5c
size 1435950048

3
ibm-granite_granite-4.1-3b-Q2_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:90d5bae2e041e08a7c4f491fba5a05df2adaa7501d19f10fd9aae8dd559f8db2
size 1498168288

3
ibm-granite_granite-4.1-3b-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9a2a0e1257595dc2abe3fd42eabdc5ccbe4cd24407952c73fadd3c86950e0d33
size 1882618848

3
ibm-granite_granite-4.1-3b-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6ee47b2e8c2e52108a1e93005b44fd78ce8245679ec7ae5fbcfec1655c7f844e
size 1779727328

3
ibm-granite_granite-4.1-3b-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a11e38e6d2d39c06e217c33a9dd0fc9ab7ee0fcebf9c7aef37075b062eabcb62
size 1618017248

3
ibm-granite_granite-4.1-3b-Q3_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8d158e2d393b4dce569b99a9243750c946fce14090b198aac2ae4dbac0cef193
size 1944837088

3
ibm-granite_granite-4.1-3b-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ccfd6ac2fc8bca6cb2fa1b8cb312aaa9200f13fe65b0cd745e8316e0093013fc
size 2024954848

3
ibm-granite_granite-4.1-3b-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e21b9270252ab4c1060e5bb0a69e5ffe5c9a903cc0a5fe03718a0c5cd7b71547
size 2206817248

3
ibm-granite_granite-4.1-3b-Q4_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e3ce8ab2f27fde22f37e8cbfa0fc47ad7947df6fc527ebe2b6a15317f29518a1
size 2234833888

3
ibm-granite_granite-4.1-3b-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d5c3d3d2442cd398331548df3e2f7059225bf097dfe03d9135586e31e43f9465
size 2172615648

3
ibm-granite_granite-4.1-3b-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ae3b44a037b04efbf49635e179ee92776aa8f418905cfad68e18656f4b1b45a1
size 2034785248

3
ibm-granite_granite-4.1-3b-Q5_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e463625f45d98043478efdf7fc8fa692f192b9861cc5af9e3d6f407fdd94a4e9
size 2558417888

3
ibm-granite_granite-4.1-3b-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:49ed190d9f14d358906b833e9a4610a44065c4627c755942017f8af6510ca013
size 2496199648

3
ibm-granite_granite-4.1-3b-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9e1397c95644345202fe28fe49b1743545afd7b196fff974e9b8694b882c5db6
size 2395233248

3
ibm-granite_granite-4.1-3b-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0371b947583f27dfca124efeda87e27856df85980c14fbfb28c1435023e71718
size 2909895648

3
ibm-granite_granite-4.1-3b-Q6_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8d8c7e777fbeac672728820bf9d0ee1f283fd2baeaaa3eccfab6020a4335fc0e
size 2972113888

3
ibm-granite_granite-4.1-3b-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:69b3aef4cc576457417db283504dd67a721cdc3e3591538472011b3283ec725c
size 3619691488

3
ibm-granite_granite-4.1-3b-bf16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:020b35c0e924b482333c1f5dc08cfcfb7d0c8d810598cb4a8ebfc6ee86829c64
size 6809656000

3
ibm-granite_granite-4.1-3b-imatrix.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:66bbaf7bfe7ab17fb162336a71e5668b2f179ae27c4dfa9034513850f1a40478
size 3811424