113 lines
3.7 KiB
Markdown
113 lines
3.7 KiB
Markdown
---
|
|
base_model: zed-industries/zeta-2.1
|
|
base_model_relation: quantized
|
|
quantized_by: adilkairolla
|
|
license: apache-2.0
|
|
library_name: gguf
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- gguf
|
|
- llama.cpp
|
|
- quantized
|
|
- edit-prediction
|
|
- next-edit-suggestion
|
|
- code
|
|
language:
|
|
- en
|
|
---
|
|
|
|
# Zeta 2.1 — GGUF
|
|
|
|
GGUF quantizations of [`zed-industries/zeta-2.1`](https://huggingface.co/zed-industries/zeta-2.1), a code edit prediction (next-edit suggestion) model from [Zed Industries](https://zed.dev).
|
|
|
|
These files were produced from the original BF16 safetensors using [llama.cpp `b9085`](https://github.com/ggml-org/llama.cpp/releases/tag/b9085) (`convert_hf_to_gguf.py` → `llama-quantize`). No fine-tuning or weight modification beyond format conversion and quantization.
|
|
|
|
## Files
|
|
|
|
| Quant | Size | Notes |
|
|
|-----------|-------:|-----------------------------------------------------------------------|
|
|
| `Q4_K_M` | 4.8 GB | Smallest, recommended default for CPU / 8 GB-class GPUs. |
|
|
| `Q5_K_M` | 5.5 GB | Quality / size sweet spot. |
|
|
| `Q8_0` | 8.2 GB | Near-lossless vs. the original BF16 weights. |
|
|
| `f16` | 16 GB | Reference. Useful as the source for further quantization. |
|
|
|
|
## Quickstart
|
|
|
|
### Ollama
|
|
|
|
```bash
|
|
ollama pull hf.co/adilkairolla/zeta-2.1-GGUF:Q4_K_M
|
|
```
|
|
|
|
Replace the tag with `Q5_K_M`, `Q8_0`, or `f16` for a different quant. Zeta is a code-edit-prediction model (not a chat model) — call it via `/api/generate` with the FIM prompt below, **not** via `/api/chat`.
|
|
|
|
### LM Studio
|
|
|
|
Search for `adilkairolla/zeta-2.1-GGUF` in the model browser and pick a quant. LM Studio loads it as a base completion model.
|
|
|
|
### llama.cpp
|
|
|
|
```bash
|
|
# One-shot completion (correct binary for non-chat models)
|
|
./llama-completion -m zeta-2.1-Q4_K_M.gguf -p "$(cat your-prompt.txt)" -n 256 -c 4096
|
|
```
|
|
|
|
### llama-cpp-python
|
|
|
|
```python
|
|
from llama_cpp import Llama
|
|
llm = Llama(model_path="zeta-2.1-Q4_K_M.gguf", n_ctx=4096)
|
|
out = llm(prompt, max_tokens=256, stop=["<|marker_2|>"], echo=False)
|
|
print(out["choices"][0]["text"])
|
|
```
|
|
|
|
## Prompt format
|
|
|
|
Zeta uses a Suffix-Prefix-Middle (SPM) FIM format with numbered region markers. Quoting the upstream model card:
|
|
|
|
```
|
|
<[fim-suffix]>
|
|
code after editable region
|
|
<[fim-prefix]><filename>related/file.py
|
|
related file content
|
|
|
|
<filename>edit_history
|
|
--- a/some_file.py
|
|
+++ b/some_file.py
|
|
-old
|
|
+new
|
|
|
|
<filename>path/to/target_file.py
|
|
code before editable region
|
|
<|marker_1|>
|
|
code that
|
|
needs to<|user_cursor|>
|
|
be rewritten
|
|
<|marker_2|>
|
|
<[fim-middle]>
|
|
```
|
|
|
|
Expected output:
|
|
|
|
```
|
|
<|marker_1|>
|
|
revised content for
|
|
the editable region
|
|
<|marker_2|>
|
|
```
|
|
|
|
See the upstream [`sample.prompt`](https://huggingface.co/zed-industries/zeta-2.1/blob/main/sample.prompt) and [`sample.output`](https://huggingface.co/zed-industries/zeta-2.1/blob/main/sample.output) for a real example.
|
|
|
|
## Source & lineage
|
|
|
|
- **Quantized from:** [`zed-industries/zeta-2.1`](https://huggingface.co/zed-industries/zeta-2.1) (BF16 safetensors)
|
|
- **Fine-tuned from:** [`ByteDance-Seed/Seed-Coder-8B-Base`](https://huggingface.co/ByteDance-Seed/Seed-Coder-8B-Base)
|
|
- **Architecture:** Llama (32 layers, hidden 4096, 32 heads, 8 KV heads, vocab 155136, RoPE θ 500000, ctx 32768)
|
|
- **Conversion tool:** llama.cpp `b9085`
|
|
|
|
## License
|
|
|
|
Released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0), inherited from the upstream model. The only modification relative to upstream is the conversion to GGUF and quantization to the formats listed above.
|
|
|
|
All credit for the model itself goes to Zed Industries and ByteDance-Seed. This repo is an unaffiliated quantization mirror.
|