---
license: apache-2.0
base_model: SupraLabs/Supra-50M-Instruct
library_name: llama.cpp
tags:
- gguf
- llama.cpp
- quantized
- supra
- text-generation
---

# Supra-50M-Instruct-GGUF

GGUF quantizations of [SupraLabs/Supra-50M-Instruct](https://huggingface.co/SupraLabs/Supra-50M-Instruct).

## Files

| File | Quantization | Size |
| --- | --- | ---: |
| `model.gguf` | F16 base GGUF | 100M |
| `Q8_0.gguf` | Q8_0 | 54M |
| `Q6_K.gguf` | Q6_K | 44M |
| `Q4_K_M.gguf` | Q4_K_M | 36M |
| `Q4_K_S.gguf` | Q4_K_S | 35M |
| `Q4_0.gguf` | Q4_0 | 33M |
| `Q3_K_S.gguf` | Q3_K_S | 30M |
| `Q2_K.gguf` | Q2_K | 28M |

Checksums are in `SHA256SUMS`.

## Conversion

`model.gguf` is the F16 GGUF from the source repository. The quantized files were generated with `llama.cpp` build `b9550` using `llama-quantize`.

Some K-quant outputs may contain fallback tensor types where tensor dimensions are not divisible by the required K-quant block size. This is normal `llama.cpp` behavior for this model shape.

## Usage

This model was instruction-tuned with the Alpaca prompt format. It is not a ChatML-style multi-turn chat model, so use completion mode and include the prompt template. The GGUF files intentionally do not include `tokenizer.chat_template`, because llama.cpp chat mode would otherwise try to apply the wrong template.

```text
Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
What is the capital of Japan?

### Response:
```

Example with `llama.cpp`:

```bash
cat > prompt.txt <<'EOF'
Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
What is the capital of Japan?

### Response:
EOF

llama-completion \
  -hf di-zhang-fdu/Supra-50M-Instruct-GGUF:Q4_K_M \
  -f prompt.txt \
  -n 128 \
  --temp 0.7 \
  --top-k 50 \
  --top-p 0.9 \
  --repeat-penalty 1.15 \
  -no-cnv
```

For deterministic checks, use `--temp 0 --top-k 1`.