初始化项目,由ModelHub XC社区提供模型
Model: build-small-hackathon/deku-gguf Source: Original Platform
This commit is contained in:
37
.gitattributes
vendored
Normal file
37
.gitattributes
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
deku-f16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
deku-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
56
README.md
Normal file
56
README.md
Normal file
@@ -0,0 +1,56 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
base_model: build-small-hackathon/deku
|
||||
tags:
|
||||
- gguf
|
||||
- llama.cpp
|
||||
- knowledge-distillation
|
||||
- qwen2
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# Deku — GGUF (llama.cpp)
|
||||
|
||||
GGUF builds of [**build-small-hackathon/deku**](https://huggingface.co/build-small-hackathon/deku),
|
||||
the One for All student: a Qwen2.5-0.5B distilled from 6 teachers via gated CKA
|
||||
geometry distillation. The LoRA adapter is merged into the base, then converted
|
||||
with `llama.cpp`'s `convert_hf_to_gguf.py`.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Size | Use |
|
||||
|------|------|-----|
|
||||
| `deku-q8_0.gguf` | ~531 MB | what the Space serves — near-lossless, CPU-friendly |
|
||||
| `deku-f16.gguf` | ~994 MB | archival full-precision build |
|
||||
| `gating.npz` | ~22 KB | the teacher-gating head as numpy (`weight` 6×896, `bias` 6) |
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
llama-cli -m deku-q8_0.gguf -p "Explain gradient descent in one sentence."
|
||||
```
|
||||
|
||||
```python
|
||||
from llama_cpp import Llama
|
||||
llm = Llama(model_path="deku-q8_0.gguf", n_ctx=2048)
|
||||
print(llm.create_chat_completion(
|
||||
messages=[{"role": "user", "content": "Why is the sky blue?"}]
|
||||
)["choices"][0]["message"]["content"])
|
||||
```
|
||||
|
||||
## Teacher gating without torch
|
||||
|
||||
`gating.npz` lets you reproduce the live "teacher influence" meters from the
|
||||
[Space](https://huggingface.co/spaces/build-small-hackathon/one-for-all) using
|
||||
only numpy on a mean-pooled embedding from `llama.cpp`:
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
g = np.load("gating.npz") # g["weight"] (6, 896), g["bias"] (6,)
|
||||
def gate(emb): # emb: 896-dim pooled embedding
|
||||
z = g["weight"] @ emb + g["bias"]
|
||||
e = np.exp(z - z.max())
|
||||
return e / e.sum() # softmax over the 6 teachers
|
||||
```
|
||||
|
||||
Teacher order: `qwen, smollm, phi, gemma, minicpm, nemotron`.
|
||||
3
deku-f16.gguf
Normal file
3
deku-f16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:501ce0962de4c09b0bc4aaa934e2c1f980b2d945e2acbfaad61341aef5274aec
|
||||
size 994156352
|
||||
3
deku-q8_0.gguf
Normal file
3
deku-q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:df1eb37ae3f215aa2067c8c6e7d1857f38df6ae3d91e919cb55f57c9fb0f41d1
|
||||
size 531067712
|
||||
3
gating.npz
Normal file
3
gating.npz
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1235b920ba38c6957bf4a777edb023d8e9517bba6880db6707f92e87c6062930
|
||||
size 22034
|
||||
Reference in New Issue
Block a user