Update README with model details

Upload granite-4.0-h-1b-DISTILL-glm-4.7-q8_0.gguf
Upload granite-4.0-h-1b-DISTILL-glm-4.7-q6_k.gguf
2025-12-23 13:01:11 +00:00 · 2025-12-23 13:01:09 +00:00 · 2025-12-23 13:00:16 +00:00 · 2025-12-23 12:59:40 +00:00 · 2025-12-23 12:59:33 +00:00 · 2025-12-23 12:59:05 +00:00
11 changed files with 124 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -39,3 +39,12 @@ granite-4.0-h-1b-DISTILL-glm-4.7-q3_k_l.gguf filter=lfs diff=lfs merge=lfs -text
 granite-4.0-h-1b-DISTILL-glm-4.7-q3_k_m.gguf filter=lfs diff=lfs merge=lfs -text
 granite-4.0-h-1b-DISTILL-glm-4.7-q3_k_s.gguf filter=lfs diff=lfs merge=lfs -text
 granite-4.0-h-1b-DISTILL-glm-4.7-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+granite-4.0-h-1b-DISTILL-glm-4.7-q4_1.gguf filter=lfs diff=lfs merge=lfs -text
+granite-4.0-h-1b-DISTILL-glm-4.7-q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text
+granite-4.0-h-1b-DISTILL-glm-4.7-q4_k_s.gguf filter=lfs diff=lfs merge=lfs -text
+granite-4.0-h-1b-DISTILL-glm-4.7-q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+granite-4.0-h-1b-DISTILL-glm-4.7-q5_1.gguf filter=lfs diff=lfs merge=lfs -text
+granite-4.0-h-1b-DISTILL-glm-4.7-q5_k_m.gguf filter=lfs diff=lfs merge=lfs -text
+granite-4.0-h-1b-DISTILL-glm-4.7-q5_k_s.gguf filter=lfs diff=lfs merge=lfs -text
+granite-4.0-h-1b-DISTILL-glm-4.7-q6_k.gguf filter=lfs diff=lfs merge=lfs -text
+granite-4.0-h-1b-DISTILL-glm-4.7-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,88 @@
+---
+base_model: glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7
+library_name: gguf
+license: apache-2.0
+language:
+- en
+- fr
+tags:
+- granite
+- gguf
+- quantized
+- llama.cpp
+- ollama
+quantized_by: llama.cpp
+pipeline_tag: text-generation
+---
+
+# granite-4.0-h-1b-DISTILL-glm-4.7-GGUF
+
+GGUF quantized versions of [granite-4.0-h-1b-DISTILL-glm-4.7](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7)
+
+## Available Formats
+
+| Filename | Size | Quant Type | Description |
+|----------|------|------------|-------------|
+| [granite-4.0-h-1b-DISTILL-glm-4.7-f16.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-f16.gguf) | 2.73 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-F16 |  |
+| [granite-4.0-h-1b-DISTILL-glm-4.7-q2_k.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-q2_k.gguf) | 0.55 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-Q2_K |  |
+| [granite-4.0-h-1b-DISTILL-glm-4.7-q3_k_l.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-q3_k_l.gguf) | 0.71 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-Q3_K_L |  |
+| [granite-4.0-h-1b-DISTILL-glm-4.7-q3_k_m.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-q3_k_m.gguf) | 0.68 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-Q3_K_M |  |
+| [granite-4.0-h-1b-DISTILL-glm-4.7-q3_k_s.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-q3_k_s.gguf) | 0.65 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-Q3_K_S |  |
+| [granite-4.0-h-1b-DISTILL-glm-4.7-q4_0.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-q4_0.gguf) | 0.81 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-Q4_0 |  |
+| [granite-4.0-h-1b-DISTILL-glm-4.7-q4_1.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-q4_1.gguf) | 0.88 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-Q4_1 |  |
+| [granite-4.0-h-1b-DISTILL-glm-4.7-q4_k_m.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-q4_k_m.gguf) | 0.84 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-Q4_K_M |  |
+| [granite-4.0-h-1b-DISTILL-glm-4.7-q4_k_s.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-q4_k_s.gguf) | 0.81 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-Q4_K_S |  |
+| [granite-4.0-h-1b-DISTILL-glm-4.7-q5_0.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-q5_0.gguf) | 0.96 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-Q5_0 |  |
+| [granite-4.0-h-1b-DISTILL-glm-4.7-q5_1.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-q5_1.gguf) | 1.04 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-Q5_1 |  |
+| [granite-4.0-h-1b-DISTILL-glm-4.7-q5_k_m.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-q5_k_m.gguf) | 0.98 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-Q5_K_M |  |
+| [granite-4.0-h-1b-DISTILL-glm-4.7-q5_k_s.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-q5_k_s.gguf) | 0.96 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-Q5_K_S |  |
+| [granite-4.0-h-1b-DISTILL-glm-4.7-q6_k.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-q6_k.gguf) | 1.12 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-Q6_K |  |
+| [granite-4.0-h-1b-DISTILL-glm-4.7-q8_0.gguf](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF/blob/main/granite-4.0-h-1b-DISTILL-glm-4.7-q8_0.gguf) | 1.45 GB | GRANITE-4.0-H-1B-DISTILL-GLM-4.7-Q8_0 |  |
+
+
+## Quick Start
+
+### Ollama
+
+```bash
+# Use Q4_K_M (recommended)
+ollama run hf.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF:Q4_K_M
+
+# Or other quantizations
+ollama run hf.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF:Q8_0
+ollama run hf.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF:Q2_K
+```
+
+### llama.cpp
+
+```bash
+# Download and run
+llama-cli --hf-repo glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF --hf-file granite-4.0-h-1b-distill-glm-4.7-q4_k_m.gguf -p "Hello, how are you?"
+
+# With server
+llama-server --hf-repo glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7-GGUF --hf-file granite-4.0-h-1b-distill-glm-4.7-q4_k_m.gguf -c 2048
+```
+
+### LM Studio / GPT4All
+
+Download the `.gguf` file of your choice and load it in your application.
+
+## Quantization Details
+
+| Type | Bits | Use Case |
+|------|------|----------|
+| Q2_K | 2 | Extreme compression, low quality |
+| Q3_K_M | 3 | Very compressed |
+| Q4_K_M | 4 | **Recommended** - Best size/quality |
+| Q5_K_M | 5 | High quality |
+| Q6_K | 6 | Very high quality |
+| Q8_0 | 8 | Near lossless |
+| F16 | 16 | Original precision |
+
+## Original Model
+
+This is the quantized version of [granite-4.0-h-1b-DISTILL-glm-4.7](https://huggingface.co/glogwa68/granite-4.0-h-1b-DISTILL-glm-4.7)
+
+- **Base Model:** ibm-granite/granite-4.0-h-1b
+- **Fine-tuning Dataset:** TeichAI/glm-4.7-2000x
+- **Training Loss:** 0.6364
--- a/granite-4.0-h-1b-DISTILL-glm-4.7-q4_1.gguf
+++ b/granite-4.0-h-1b-DISTILL-glm-4.7-q4_1.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6da9ac407edc077fd791dd9a366a8b49a1a5035e142fc758dce396a2bdd57f8
+size 949975808
--- a/granite-4.0-h-1b-DISTILL-glm-4.7-q4_k_m.gguf
+++ b/granite-4.0-h-1b-DISTILL-glm-4.7-q4_k_m.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ecf28935e21bf9c0e4054a303dd5da19eb78f3d6cb7d52b734ad9a1416720261
+size 901161728
--- a/granite-4.0-h-1b-DISTILL-glm-4.7-q4_k_s.gguf
+++ b/granite-4.0-h-1b-DISTILL-glm-4.7-q4_k_s.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c100388ff8ea6895687970292fd8a18fc1059907bf67b8698b6960a9932a91e5
+size 872641280
--- a/granite-4.0-h-1b-DISTILL-glm-4.7-q5_0.gguf
+++ b/granite-4.0-h-1b-DISTILL-glm-4.7-q5_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0db6d285de3a7bb0b25204b46f92d2c5067492348f7afd5b467fcc1cee60b6f0
+size 1031635712
--- a/granite-4.0-h-1b-DISTILL-glm-4.7-q5_1.gguf
+++ b/granite-4.0-h-1b-DISTILL-glm-4.7-q5_1.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:861a6204edd6bf3e715e4baf382f326ea37a9d578fcd586f672d17a9b1d9197a
+size 1113295616
--- a/granite-4.0-h-1b-DISTILL-glm-4.7-q5_k_m.gguf
+++ b/granite-4.0-h-1b-DISTILL-glm-4.7-q5_k_m.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ccf366b1911fed888a5f01504b9e8ab48f47f37b7b36d5cf6572627024264bfd
+size 1048556288
--- a/granite-4.0-h-1b-DISTILL-glm-4.7-q5_k_s.gguf
+++ b/granite-4.0-h-1b-DISTILL-glm-4.7-q5_k_s.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:53adc0f306e030c9d2f73bc27b22b8049095b56b6980cd88a971efdc30745ec3
+size 1031635712
--- a/granite-4.0-h-1b-DISTILL-glm-4.7-q6_k.gguf
+++ b/granite-4.0-h-1b-DISTILL-glm-4.7-q6_k.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55a41a3a68c7257b6dcda70e3852e58bf920d6c2313b300982b961b705deddd6
+size 1205163008
--- a/granite-4.0-h-1b-DISTILL-glm-4.7-q8_0.gguf
+++ b/granite-4.0-h-1b-DISTILL-glm-4.7-q8_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:05c99971220898e66ac6b249a7984a2762850cd379d96d7bb3baba4a79f4140e
+size 1558926080
Author	SHA1	Message	Date
Glogwa	9787621675	Update README with model details	2025-12-23 13:01:11 +00:00
Glogwa	9d9a1017b6	Upload granite-4.0-h-1b-DISTILL-glm-4.7-q8_0.gguf	2025-12-23 13:01:09 +00:00
Glogwa	0bcfc623c8	Upload granite-4.0-h-1b-DISTILL-glm-4.7-q6_k.gguf	2025-12-23 13:00:16 +00:00
Glogwa	b9446f9071	Upload granite-4.0-h-1b-DISTILL-glm-4.7-q5_k_s.gguf	2025-12-23 12:59:40 +00:00
Glogwa	9f6468e29e	Upload granite-4.0-h-1b-DISTILL-glm-4.7-q5_k_m.gguf	2025-12-23 12:59:33 +00:00
Glogwa	053bf7758e	Upload granite-4.0-h-1b-DISTILL-glm-4.7-q5_1.gguf	2025-12-23 12:59:05 +00:00
Glogwa	bfa8f11510	Upload granite-4.0-h-1b-DISTILL-glm-4.7-q5_0.gguf	2025-12-23 12:58:27 +00:00
Glogwa	188d5eb3ce	Upload granite-4.0-h-1b-DISTILL-glm-4.7-q4_k_s.gguf	2025-12-23 12:57:54 +00:00
Glogwa	28e10890aa	Upload granite-4.0-h-1b-DISTILL-glm-4.7-q4_k_m.gguf	2025-12-23 12:57:47 +00:00
Glogwa	99612c9a0a	Upload granite-4.0-h-1b-DISTILL-glm-4.7-q4_1.gguf	2025-12-23 12:57:20 +00:00