---
base_model: zed-industries/zeta-2.1
base_model_relation: quantized
quantized_by: adilkairolla
license: apache-2.0
library_name: gguf
pipeline_tag: text-generation
tags:
  - gguf
  - llama.cpp
  - quantized
  - edit-prediction
  - next-edit-suggestion
  - code
language:
  - en
---

# Zeta 2.1 — GGUF

GGUF quantizations of [`zed-industries/zeta-2.1`](https://huggingface.co/zed-industries/zeta-2.1), a code edit prediction (next-edit suggestion) model from [Zed Industries](https://zed.dev).

These files were produced from the original BF16 safetensors using [llama.cpp `b9085`](https://github.com/ggml-org/llama.cpp/releases/tag/b9085) (`convert_hf_to_gguf.py` → `llama-quantize`). No fine-tuning or weight modification beyond format conversion and quantization.

## Files

| Quant     | Size   | Notes                                                                 |
|-----------|-------:|-----------------------------------------------------------------------|
| `Q4_K_M`  | 4.8 GB | Smallest, recommended default for CPU / 8 GB-class GPUs.              |
| `Q5_K_M`  | 5.5 GB | Quality / size sweet spot.                                            |
| `Q8_0`    | 8.2 GB | Near-lossless vs. the original BF16 weights.                          |
| `f16`     |  16 GB | Reference. Useful as the source for further quantization.             |

## Quickstart

### Ollama

```bash
ollama pull hf.co/adilkairolla/zeta-2.1-GGUF:Q4_K_M
```

Replace the tag with `Q5_K_M`, `Q8_0`, or `f16` for a different quant. Zeta is a code-edit-prediction model (not a chat model) — call it via `/api/generate` with the FIM prompt below, **not** via `/api/chat`.

### LM Studio

Search for `adilkairolla/zeta-2.1-GGUF` in the model browser and pick a quant. LM Studio loads it as a base completion model.

### llama.cpp

```bash
# One-shot completion (correct binary for non-chat models)
./llama-completion -m zeta-2.1-Q4_K_M.gguf -p "$(cat your-prompt.txt)" -n 256 -c 4096
```

### llama-cpp-python

```python
from llama_cpp import Llama
llm = Llama(model_path="zeta-2.1-Q4_K_M.gguf", n_ctx=4096)
out = llm(prompt, max_tokens=256, stop=["<|marker_2|>"], echo=False)
print(out["choices"][0]["text"])
```

## Prompt format

Zeta uses a Suffix-Prefix-Middle (SPM) FIM format with numbered region markers. Quoting the upstream model card:

```
<[fim-suffix]>
code after editable region
<[fim-prefix]><filename>related/file.py
related file content

<filename>edit_history
--- a/some_file.py
+++ b/some_file.py
-old
+new

<filename>path/to/target_file.py
code before editable region
<|marker_1|>
code that
needs to<|user_cursor|>
be rewritten
<|marker_2|>
<[fim-middle]>
```

Expected output:

```
<|marker_1|>
revised content for
the editable region
<|marker_2|>
```

See the upstream [`sample.prompt`](https://huggingface.co/zed-industries/zeta-2.1/blob/main/sample.prompt) and [`sample.output`](https://huggingface.co/zed-industries/zeta-2.1/blob/main/sample.output) for a real example.

## Source & lineage

- **Quantized from:** [`zed-industries/zeta-2.1`](https://huggingface.co/zed-industries/zeta-2.1) (BF16 safetensors)
- **Fine-tuned from:** [`ByteDance-Seed/Seed-Coder-8B-Base`](https://huggingface.co/ByteDance-Seed/Seed-Coder-8B-Base)
- **Architecture:** Llama (32 layers, hidden 4096, 32 heads, 8 KV heads, vocab 155136, RoPE θ 500000, ctx 32768)
- **Conversion tool:** llama.cpp `b9085`

## License

Released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0), inherited from the upstream model. The only modification relative to upstream is the conversion to GGUF and quantization to the formats listed above.

All credit for the model itself goes to Zed Industries and ByteDance-Seed. This repo is an unaffiliated quantization mirror.