From 4c47a714e04233ec23b627374a8847e57a947e4a Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Fri, 28 Aug 2026 12:20:17 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: build-small-hackathon/deku-gguf Source: Original Platform --- .gitattributes | 37 +++++++++++++++++++++++++++++++++ README.md | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++ deku-f16.gguf | 3 +++ deku-q8_0.gguf | 3 +++ gating.npz | 3 +++ 5 files changed, 102 insertions(+) create mode 100644 .gitattributes create mode 100644 README.md create mode 100644 deku-f16.gguf create mode 100644 deku-q8_0.gguf create mode 100644 gating.npz diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..11437eb --- /dev/null +++ b/.gitattributes @@ -0,0 +1,37 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +deku-f16.gguf filter=lfs diff=lfs merge=lfs -text +deku-q8_0.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..b5bd55f --- /dev/null +++ b/README.md @@ -0,0 +1,56 @@ +--- +license: apache-2.0 +base_model: build-small-hackathon/deku +tags: +- gguf +- llama.cpp +- knowledge-distillation +- qwen2 +pipeline_tag: text-generation +--- + +# Deku — GGUF (llama.cpp) + +GGUF builds of [**build-small-hackathon/deku**](https://huggingface.co/build-small-hackathon/deku), +the One for All student: a Qwen2.5-0.5B distilled from 6 teachers via gated CKA +geometry distillation. The LoRA adapter is merged into the base, then converted +with `llama.cpp`'s `convert_hf_to_gguf.py`. + +## Files + +| File | Size | Use | +|------|------|-----| +| `deku-q8_0.gguf` | ~531 MB | what the Space serves — near-lossless, CPU-friendly | +| `deku-f16.gguf` | ~994 MB | archival full-precision build | +| `gating.npz` | ~22 KB | the teacher-gating head as numpy (`weight` 6×896, `bias` 6) | + +## Run + +```bash +llama-cli -m deku-q8_0.gguf -p "Explain gradient descent in one sentence." +``` + +```python +from llama_cpp import Llama +llm = Llama(model_path="deku-q8_0.gguf", n_ctx=2048) +print(llm.create_chat_completion( + messages=[{"role": "user", "content": "Why is the sky blue?"}] +)["choices"][0]["message"]["content"]) +``` + +## Teacher gating without torch + +`gating.npz` lets you reproduce the live "teacher influence" meters from the +[Space](https://huggingface.co/spaces/build-small-hackathon/one-for-all) using +only numpy on a mean-pooled embedding from `llama.cpp`: + +```python +import numpy as np +g = np.load("gating.npz") # g["weight"] (6, 896), g["bias"] (6,) +def gate(emb): # emb: 896-dim pooled embedding + z = g["weight"] @ emb + g["bias"] + e = np.exp(z - z.max()) + return e / e.sum() # softmax over the 6 teachers +``` + +Teacher order: `qwen, smollm, phi, gemma, minicpm, nemotron`. diff --git a/deku-f16.gguf b/deku-f16.gguf new file mode 100644 index 0000000..5975d72 --- /dev/null +++ b/deku-f16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:501ce0962de4c09b0bc4aaa934e2c1f980b2d945e2acbfaad61341aef5274aec +size 994156352 diff --git a/deku-q8_0.gguf b/deku-q8_0.gguf new file mode 100644 index 0000000..b7e2927 --- /dev/null +++ b/deku-q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:df1eb37ae3f215aa2067c8c6e7d1857f38df6ae3d91e919cb55f57c9fb0f41d1 +size 531067712 diff --git a/gating.npz b/gating.npz new file mode 100644 index 0000000..3d9492d --- /dev/null +++ b/gating.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1235b920ba38c6957bf4a777edb023d8e9517bba6880db6707f92e87c6062930 +size 22034