初始化项目，由ModelHub XC社区提供模型

Model: prism-ml/Ternary-Bonsai-1.7B-gguf Source: Original Platform
2026-06-05 18:18:17 +08:00
commit d5a1293c70
10 changed files with 582 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,175 @@
+---
+license: apache-2.0
+library_name: gguf
+pipeline_tag: text-generation
+tags:
+- ternary
+- 1.58-bit
+- gguf
+- llama-cpp
+- q2_0
+- on-device
+- prismml
+- bonsai
+base_model:
+- prism-ml/Ternary-Bonsai-1.7B-unpacked
+---
+
+<p align="center">
+  <img src="./assets/bonsai-logo.svg" width="280" alt="Bonsai">
+</p>
+
+<p align="center">
+  <a href="https://prismml.com"><b>Prism ML Website</b></a> &nbsp;|&nbsp;
+  <a href="https://github.com/PrismML-Eng/Bonsai-demo/blob/main/ternary-bonsai-8b-whitepaper.pdf"><b>White Paper</b></a> &nbsp;|&nbsp;
+  <a href="https://github.com/PrismML-Eng/Bonsai-demo"><b>Demo &amp; Examples</b></a> &nbsp;|&nbsp;
+  <a href="https://discord.gg/prismml"><b>Discord</b></a>
+</p>
+
+# Ternary-Bonsai-1.7B-gguf
+
+Ternary (1.58-bit) language model in GGUF Q2_0 format for `llama.cpp`
+
+<p align="center">
+  <img src="./assets/frontier.svg" width="680" alt="Pareto Frontier">
+</p>
+
+## Resources
+
+- **[White Paper](https://github.com/PrismML-Eng/Bonsai-demo/blob/main/ternary-bonsai-8b-whitepaper.pdf)**
+- **[Demo repo](https://github.com/PrismML-Eng/Bonsai-demo)** — examples for serving, benchmarking, and integrating Bonsai
+- **[Discord](https://discord.gg/prismml)** — community support and updates
+- **Kernels**: Q2_0 is not yet in mainline `llama.cpp`. Use our fork at [PrismML-Eng/llama.cpp](https://github.com/PrismML-Eng/llama.cpp) (`prism` branch, default) which adds Q2_0 support for CPU (NEON/generic) and Metal. Upstream PR coming soon.
+
+## Model Overview
+
+| Item             | Specification                                                            |
+| :--------------- | :----------------------------------------------------------------------- |
+| Base model       | Qwen3-1.7B                                                               |
+| Parameters       | 1.72B                                                                    |
+| Architecture     | GQA, SwiGLU MLP, RoPE, RMSNorm                                           |
+| Context length   | 32,768 tokens                                                            |
+| Vocab size       | 151,936                                                                  |
+| Weight format    | GGUF Q2_0 g128: {-1, 0, +1} with FP16 group-wise scaling                 |
+| Packed Q2_0 size | **436 MiB** (0.46 GB)                                                    |
+| Ternary coverage | Embeddings, attention projections, MLP projections, LM head              |
+| License          | Apache 2.0                                                               |
+
+## Quantization Format: GGUF Q2_0 (g128)
+
+Each weight takes a value from {-1, 0, +1}, with one shared FP16 scale per group of 128 weights:
+
+```
+w_i = scale_g * t_i,    t_i in {-1, 0, +1}
+```
+
+Q2_0 encodes each weight as a 2-bit code `q in {0, 1, 2, 3}`, dequantized via `w = (q - 1) * scale`. One 128-element block is 34 bytes (2 bytes FP16 scale + 32 bytes of packed 2-bit codes) for an effective **2.125 bits/weight**. The fourth code point (`q = 3`, reconstructing to `+2 * scale`) is reserved for future extensions; for ternary weights it is unused.
+
+### Memory
+
+| Format            | Size         | Reduction | Ratio      |
+| :---------------- | ----------:  | --------: | ---------: |
+| FP16              | 3.44 GB      | --        | 1.0x       |
+| **GGUF Q2_0 g128**| **436 MiB** (0.46 GB) | **86.6%** | **7.5x**   |
+
+## Files in this repo
+
+| File                              | Format | Size   | Recommended |
+| :-------------------------------- | :----- | -----: | :---------- |
+| `Ternary-Bonsai-1.7B-F16.gguf`    | FP16   | 3.44 GB | baseline / re-quantization source |
+| `Ternary-Bonsai-1.7B-Q2_0.gguf`   | Q2_0 (g128) | 442 MB | **recommended** (lossless for ternary) |
+
+## Quickstart
+
+### Build from the Prism fork
+
+```bash
+git clone https://github.com/PrismML-Eng/llama.cpp
+cd llama.cpp
+cmake -B build -DGGML_METAL=ON   # or -DGGML_CUDA=ON, -DGGML_VULKAN=ON
+cmake --build build -j
+```
+
+### `llama.cpp` CLI
+
+```bash
+./build/bin/llama-cli \
+  -m Ternary-Bonsai-1.7B-Q2_0.gguf \
+  -p "Explain quantum computing in simple terms." \
+  -n 256
+```
+
+### `llama.cpp` server
+
+```bash
+./build/bin/llama-server -m Ternary-Bonsai-1.7B-Q2_0.gguf -c 4096
+```
+
+
+## Throughput (llama.cpp, Apple M4 Pro 48 GB)
+
+| Backend          | PP512 (tok/s) | TG128 (tok/s) | FP16 TG (tok/s) | Speedup    |
+| :--------------- | ------------: | ------------: | ---------------: | ---------: |
+| Metal (GPU)      | 2,088         | **229**       | 60               | **3.8x**   |
+| NEON CPU (10 t)  | 508           | **123**       | —                | —          |
+
+Flags: `-ngl 99 -fa 1` for Metal; `-ngl 0 -fa 1 -t 10` for CPU.
+
+## Fidelity (Q2_0 vs FP16 baseline)
+
+Measured on wikitext-2 (20 chunks, context 512) via `llama-perplexity --kl-divergence`:
+
+| Metric              | Value      |
+| :------------------ | ---------: |
+| Mean KL             | 0.000000   |
+| Top-1 agreement     | 100.000 %  |
+| RMS Δp              | 0.015 %    |
+| PPL ratio (Q/base)  | 1.0048     |
+
+Q2_0 is effectively lossless for ternary weights — the ternary values land exactly on three of the four 2-bit code points.
+
+## Benchmarks
+
+Evaluated with EvalScope v1.4.2 + vLLM 0.15.1 on NVIDIA H100. Full benchmark suite:
+
+| Model                       | Size         | Avg       | MMLU-R | MuSR | IFEval | GSM8K | HE+  | BFCLv3 |
+| :-------------------------- | ----------:  | --------: | -----: | ---: | -----: | ----: | ---: | -----: |
+| **Ternary Bonsai 1.7B**     | **0.44 GB**  | **58.47** | 52.9   | 50.8 | 70.1   | 74.2  | 51.8 | 51.0   |
+| *1-bit Bonsai 1.7B (prior)* | *0.24 GB*    | *49.60*   | 43.2   | 45.1 | 63.0   | 66.3  | 45.1 | 34.9   |
+| Qwen3 1.7B                  | 3.44 GB      | **66.57** | 66.8   | 50.1 | 70.3   | 83.1  | 57.3 | 71.8   |
+| Qwen3 0.6B                  | 1.19 GB      | **48.02** | 47.5   | 41.5 | 62.8   | 64.1  | 30.5 | 41.7   |
+| LFM2 1.2B                   | 2.34 GB      | **46.73** | 52.9   | 25.4 | 77.5   | 62.2  | 36.0 | 26.4   |
+| Gemma3 1B                   | 2.00 GB      | **45.53** | 43.2   | 37.0 | 61.9   | 64.4  | 40.2 | 26.5   |
+| Llama 3.2 1B                | 2.47 GB      | **39.88** | 47.2   | 29.2 | 47.7   | 49.0  | 35.4 | 30.8   |
+
+## Intelligence Density
+
+```
+density = -ln(1 - score/100) / size_GB
+```
+
+| Model                       | Size        | Intelligence Density (1/GB) |
+| :-------------------------- | ----------: | --------------------------: |
+| **Ternary Bonsai 1.7B**     | **0.44 GB** | **2.001**                   |
+| *1-bit Bonsai 1.7B (prior)* | *0.24 GB*   | *2.832*                     |
+| Qwen3 0.6B                  | 1.19 GB     | 0.549                       |
+| Qwen3 1.7B                  | 3.44 GB     | 0.318                       |
+| Gemma3 1B                   | 2.00 GB     | 0.304                       |
+| LFM2 1.2B                   | 2.34 GB     | 0.269                       |
+| Llama 3.2 1B                | 2.47 GB     | 0.206                       |
+
+## Citation
+
+```bibtex
+@techreport{ternarybonsai,
+    title   = {Ternary Bonsai: 1.58-bit Language Models at 8B, 4B, and 1.7B Scale},
+    author  = {Prism ML},
+    year    = {2026},
+    month   = {April},
+    url     = {https://prismml.com}
+}
+```
+
+## Contact
+
+For questions, feedback, or collaboration inquiries: **contact@prismml.com**