From e4aa52796063dd962e5214be9132b977b25f171e Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Fri, 19 Jun 2026 01:17:18 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: yasserrmd/glm5.1-distill-GGUF Source: Original Platform --- .gitattributes | 41 +++++++++++++++++++ README.md | 80 ++++++++++++++++++++++++++++++++++++++ glm5.1-distill-BF16.gguf | 3 ++ glm5.1-distill-Q4_0.gguf | 3 ++ glm5.1-distill-Q4_K_M.gguf | 3 ++ glm5.1-distill-Q5_K_M.gguf | 3 ++ glm5.1-distill-Q6_K.gguf | 3 ++ glm5.1-distill-Q8_0.gguf | 3 ++ 8 files changed, 139 insertions(+) create mode 100644 .gitattributes create mode 100644 README.md create mode 100644 glm5.1-distill-BF16.gguf create mode 100644 glm5.1-distill-Q4_0.gguf create mode 100644 glm5.1-distill-Q4_K_M.gguf create mode 100644 glm5.1-distill-Q5_K_M.gguf create mode 100644 glm5.1-distill-Q6_K.gguf create mode 100644 glm5.1-distill-Q8_0.gguf diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..4246e7a --- /dev/null +++ b/.gitattributes @@ -0,0 +1,41 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +glm5.1-distill-BF16.gguf filter=lfs diff=lfs merge=lfs -text +glm5.1-distill-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +glm5.1-distill-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +glm5.1-distill-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +glm5.1-distill-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +glm5.1-distill-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..24e946b --- /dev/null +++ b/README.md @@ -0,0 +1,80 @@ +--- +license: apache-2.0 +language: +- en +library_name: gguf +pipeline_tag: text-generation +base_model: yasserrmd/glm5.1-distill +tags: +- gguf +- llama.cpp +- lfm2 +- liquid-ai +- edge +- text-generation-inference +- conversational +--- + +# glm5.1-distill-GGUF + +GGUF quantizations of [`yasserrmd/glm5.1-distill`](https://huggingface.co/yasserrmd/glm5.1-distill), +produced with `convert_hf_to_gguf.py` and `llama-quantize` from +[`ggml-org/llama.cpp`](https://github.com/ggml-org/llama.cpp). + +The quant ladder mirrors Liquid AI's own LFM2.5-1.2B GGUF releases +(e.g. [`LiquidAI/LFM2.5-1.2B-Base-GGUF`](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Base-GGUF)). + +## Files + +| File | Quantization | Size | +|------|--------------|------| +| `glm5.1-distill-BF16.gguf` | BF16 | 2.18 GB | +| `glm5.1-distill-Q4_0.gguf` | Q4_0 | 664 MB | +| `glm5.1-distill-Q4_K_M.gguf` | Q4_K_M | 697 MB | +| `glm5.1-distill-Q5_K_M.gguf` | Q5_K_M | 804 MB | +| `glm5.1-distill-Q6_K.gguf` | Q6_K | 918 MB | +| `glm5.1-distill-Q8_0.gguf` | Q8_0 | 1.16 GB | + +## Quickstart with `llama.cpp` + +Run any quant directly from the Hub: + +```bash +llama-cli -hf yasserrmd/glm5.1-distill-GGUF:Q4_K_M --jinja --ctx-size 32768 --temp 0.1 --top-k 50 --top-p 0.1 --repeat-penalty 1.05 +``` + +Or download manually and serve via `llama-server`: + +```bash +huggingface-cli download yasserrmd/glm5.1-distill-GGUF --include "*Q4_K_M*" --local-dir ./glm5.1-distill-GGUF + +llama-server --model ./glm5.1-distill-GGUF/glm5.1-distill-Q4_K_M.gguf --alias "yasserrmd/glm5.1-distill" --threads -1 --n-gpu-layers 99 --ctx-size 32768 --port 8001 --temp 0.1 --top-k 50 --top-p 0.1 --repeat-penalty 1.05 --jinja +``` + +The recommended sampling parameters above are the official ones published by +Liquid AI for the LFM2.5 family. + +## Quickstart with Ollama + +```bash +ollama run hf.co/yasserrmd/glm5.1-distill-GGUF:Q4_K_M +``` + +## Choosing a quant + +| Use case | Recommended | +|----------|-------------| +| Maximum quality, plenty of RAM | `Q8_0` or `Q6_K` | +| Balanced default | `Q4_K_M` (matches Liquid AI's recommendation) | +| Smallest footprint, mobile / IoT | `Q4_0` | +| Lossless reference | `BF16` (only if you need it for further re-quantization) | + +> **Note**: imatrix-based quantization is currently not supported for the LFM2 +> architecture in upstream llama.cpp ([issue #14979](https://github.com/ggml-org/llama.cpp/issues/14979)). +> These files are plain k-quants, the same scheme used in Liquid AI's official +> GGUF releases. + +## Source model + +For training data, hyperparameters, evaluation, and limitations see the source +repo: [`yasserrmd/glm5.1-distill`](https://huggingface.co/yasserrmd/glm5.1-distill). diff --git a/glm5.1-distill-BF16.gguf b/glm5.1-distill-BF16.gguf new file mode 100644 index 0000000..10c5d60 --- /dev/null +++ b/glm5.1-distill-BF16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:585327b09976b302fa95bdabcfd1871dcc28ea7985e42e5ecb69d73d1e336ee3 +size 2343325600 diff --git a/glm5.1-distill-Q4_0.gguf b/glm5.1-distill-Q4_0.gguf new file mode 100644 index 0000000..c5ee490 --- /dev/null +++ b/glm5.1-distill-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6f2612854cfa24e6e06a429f4bb989db6fda0d159e5d921e6aee3638f5fc4e8c +size 695750560 diff --git a/glm5.1-distill-Q4_K_M.gguf b/glm5.1-distill-Q4_K_M.gguf new file mode 100644 index 0000000..56f2faa --- /dev/null +++ b/glm5.1-distill-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:47e149d2de40f3d187f98ee54b75a3c37ee1c52772409f84f39acd7f2cdbdf2f +size 730894240 diff --git a/glm5.1-distill-Q5_K_M.gguf b/glm5.1-distill-Q5_K_M.gguf new file mode 100644 index 0000000..af874e9 --- /dev/null +++ b/glm5.1-distill-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5f006d9e3ad786404a199b7ab536a095b78eb3c089ddc8eea2955ff7da570fe8 +size 843354016 diff --git a/glm5.1-distill-Q6_K.gguf b/glm5.1-distill-Q6_K.gguf new file mode 100644 index 0000000..0b544d0 --- /dev/null +++ b/glm5.1-distill-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b4f98d0759dec0ad9811ec8b3e5145f645289c0b3548ca1ad7a6b4c76b374a13 +size 962842528 diff --git a/glm5.1-distill-Q8_0.gguf b/glm5.1-distill-Q8_0.gguf new file mode 100644 index 0000000..49e7939 --- /dev/null +++ b/glm5.1-distill-Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:642f2dbd1bc819e09f6f88e280f818322ab313e57800d2bf3bed34cb8f194f48 +size 1246252960