212 lines
5.7 KiB
Markdown
212 lines
5.7 KiB
Markdown
---
|
|
license: apache-2.0
|
|
language:
|
|
- en
|
|
base_model: SupraLabs/Supra-1.5-50M-instruct-exp
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- supra
|
|
- chimera
|
|
- project-chimera
|
|
- gguf
|
|
- quantized
|
|
- instruct
|
|
- conversational
|
|
- QnA
|
|
- GPT
|
|
- CPU
|
|
- tiny
|
|
- SLM
|
|
- open
|
|
- open-source
|
|
- 50M
|
|
- llama
|
|
---
|
|
|
|
<h1 align="center">Supra-1.5 Instruct • Experimental Chat Tune — GGUF</h1>
|
|
|
|
|
|
|
|
GGUF quantizations of [SupraLabs/Supra-1.5-50M-instruct-exp](https://huggingface.co/SupraLabs/Supra-1.5-50M-instruct-exp), an experimental 50M-parameter instruction-tuned model by [SupraLabs](https://huggingface.co/SupraLabs), part of **Project Chimera**.
|
|
|
|
Run it entirely on CPU, low-VRAM GPUs, or embedded hardware. No cloud required.
|
|
|
|
> **Note:** This is an experimental model. Do not use in production.
|
|
|
|
---
|
|
|
|
## 📦 Available Quantizations
|
|
|
|
| Bits | Quantization | Size |
|
|
|:--|:--|:--|
|
|
| 1-bit | `Q1_0` | 19.6 MB |
|
|
| 1-bit | `TQ1_0` | 25.1 MB |
|
|
| 2-bit | `Q2_K` | 28.8 MB |
|
|
| 2-bit | `TQ2_0` | 26.4 MB |
|
|
| 3-bit | `IQ3_S` | 31 MB |
|
|
| 3-bit | `Q3_K_S` | 31 MB |
|
|
| 3-bit | `IQ3_M` | 31.7 MB |
|
|
| 3-bit | `Q3_K_M` | 32.7 MB |
|
|
| 3-bit | `Q3_K_L` | 33.8 MB |
|
|
| 4-bit | `IQ4_XS` | 33.8 MB |
|
|
| 4-bit | `Q4_K_S` | 35.7 MB |
|
|
| 4-bit | `IQ4_NL` | 34.7 MB |
|
|
| 4-bit | `Q4_0` | 34.5 MB |
|
|
| 4-bit | `Q4_1` | 36.8 MB |
|
|
| 4-bit | `Q4_K_M` | 37.4 MB |
|
|
| 5-bit | `Q5_K_S` | 39.5 MB |
|
|
| 5-bit | `Q5_0` | 39 MB |
|
|
| 5-bit | `Q5_1` | 41.2 MB |
|
|
| 5-bit | `Q5_K_M` | 41 MB |
|
|
| 6-bit | `Q6_K` | 45.8 MB |
|
|
| 8-bit | `Q8_0` | 56.2 MB |
|
|
| 16-bit | `BF16` | 105 MB |
|
|
| 16-bit | `F16` | 105 MB |
|
|
| 32-bit | `F32` | 208 MB |
|
|
|
|
> **`Q4_K_M`** — Usable, not recommended unless device is compute-constrained.
|
|
> **`Q8_0`** — Perfect size/performance!.
|
|
> **`Q2_K`** — ultra-constrained devices (not reccomended!).
|
|
|
|
---
|
|
|
|
## 🚀 Quick Start
|
|
|
|
### llama.cpp
|
|
|
|
```bash
|
|
# Download
|
|
huggingface-cli download SupraLabs/Supra-1.5-50M-instruct-exp-gguf \
|
|
--include "*.Q4_K_M.gguf" \
|
|
--local-dir ./
|
|
|
|
# Run
|
|
./llama-cli \
|
|
-m supra-1.5-50m-instruct-exp-Q4_K_M.gguf \
|
|
-p "### Instruction:\nWhat is machine learning?\n\n### Response:\n" \
|
|
-n 256 \
|
|
--temp 0.7 \
|
|
--repeat-penalty 1.15
|
|
```
|
|
|
|
### Ollama
|
|
|
|
```bash
|
|
ollama run hf.co/SupraLabs/Supra-1.5-50M-instruct-exp-gguf:Q4_K_M
|
|
```
|
|
|
|
### Python (llama-cpp-python)
|
|
|
|
```python
|
|
from llama_cpp import Llama
|
|
|
|
llm = Llama.from_pretrained(
|
|
repo_id="SupraLabs/Supra-1.5-50M-instruct-exp-gguf",
|
|
filename="*Q4_K_M.gguf",
|
|
n_ctx=1024,
|
|
verbose=False,
|
|
)
|
|
|
|
def chat(instruction: str, input_text: str = "") -> str:
|
|
if input_text.strip():
|
|
prompt = (
|
|
"Below is an instruction that describes a task, paired with an input "
|
|
"that provides further context. Write a response that appropriately "
|
|
"completes the request.\n\n"
|
|
f"### Instruction:\n{instruction}\n\n"
|
|
f"### Input:\n{input_text}\n\n"
|
|
"### Response:\n"
|
|
)
|
|
else:
|
|
prompt = (
|
|
"Below is an instruction that describes a task. Write a response that "
|
|
"appropriately completes the request.\n\n"
|
|
f"### Instruction:\n{instruction}\n\n"
|
|
"### Response:\n"
|
|
)
|
|
output = llm(prompt, max_tokens=256, temperature=0.7, top_k=50, top_p=0.9, repeat_penalty=1.15)
|
|
return output["choices"][0]["text"].strip()
|
|
|
|
print(chat("Explain what artificial intelligence is."))
|
|
```
|
|
|
|
---
|
|
|
|
## 💬 Prompt Format
|
|
|
|
This model uses the **Alpaca Chat Format**:
|
|
|
|
```
|
|
Below is an instruction that describes a task. Write a response that appropriately completes the request.
|
|
|
|
### Instruction:
|
|
{instruction}
|
|
|
|
### Response:
|
|
```
|
|
|
|
With optional input:
|
|
|
|
```
|
|
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
|
|
|
|
### Instruction:
|
|
{instruction}
|
|
|
|
### Input:
|
|
{input}
|
|
|
|
### Response:
|
|
```
|
|
|
|
---
|
|
|
|
## 🏆 Benchmarks
|
|
|
|
Supra-1.5-50M-instruct-exp achieves superior performance within the 50M-parameter class, with a consistent **BLiMP score of 67.4**.
|
|
|
|
Key findings from evaluation:
|
|
- **Scientific/factual tasks** perform best under raw inference (no normalization)
|
|
- **Math and logical reasoning** benefit from normalized inference
|
|
- **Top syntactic categories**: structural dependency tracking, complex clausal configurations, and subtle syntactic error detection — performing at near-flawless precision
|
|
- **Hardest categories**: advanced binding phenomena and morphological agreement edge cases, reflecting known limits of 50M-class architectures
|
|
|
|
> For full benchmark charts and BLiMP probe analysis, see the [base model card](https://huggingface.co/SupraLabs/Supra-1.5-50M-instruct-exp).
|
|
|
|
---
|
|
|
|
## 🧠 Model Architecture
|
|
|
|
| Property | Value |
|
|
|:--|:--|
|
|
| Architecture | Llama (decoder-only) |
|
|
| Parameters | ~50M |
|
|
| Vocabulary | 32,000 (custom BPE) |
|
|
| Context length | 5,120 tokens |
|
|
| Hidden size | 512 |
|
|
| Layers | 12 |
|
|
| Attention heads | 8 (GQA: 4 KV heads) |
|
|
| Base model | SupraLabs/Supra-1.5-50M-Base-exp |
|
|
| License | Apache 2.0 |
|
|
|
|
---
|
|
|
|
## 🔗 Related Models
|
|
|
|
| Model | Description |
|
|
|:--|:--|
|
|
| [Supra-1.5-50M-Base-exp](https://huggingface.co/SupraLabs/Supra-1.5-50M-Base-exp) | Pretrained base (v1.5) |
|
|
| [Supra-1.5-50M-instruct-exp](https://huggingface.co/SupraLabs/Supra-1.5-50M-instruct-exp) | fp weights |
|
|
| [Supra-50M-Base](https://huggingface.co/SupraLabs/Supra-50M-Base) | v1.0 pretrained base |
|
|
| [Supra-50M-Instruct](https://huggingface.co/SupraLabs/Supra-50M-Instruct) | v1.0 instruct model |
|
|
| [Supra-50M-Reasoning](https://huggingface.co/SupraLabs/Supra-50M-Reasoning) | Chain-of-thought reasoning variant |
|
|
|
|
---
|
|
|
|
## 📄 License
|
|
|
|
Released under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
|
|
|
|
---
|
|
|
|
*© SupraLabs 2026 — Project Chimera* |