---
base_model: OdaxAI/DANTE-Mosaic-3.5B
language:
  - en
  - it
  - es
  - fr
  - de
  - pt
  - ja
  - zh
  - ar
license: apache-2.0
tags:
  - llama-cpp
  - gguf
  - quantized
  - text-generation
  - small-language-model
  - medical
  - multilingual
  - 3b
pipeline_tag: text-generation
---

# DANTE-Mosaic-3.5B — GGUF Quantizations

GGUF quantized versions of [OdaxAI/DANTE-Mosaic-3.5B](https://huggingface.co/OdaxAI/DANTE-Mosaic-3.5B) for use with **llama.cpp**, **LM Studio**, **Ollama**, and other GGUF-compatible runtimes.

> DANTE-Mosaic-3.5B is a multilingual foundation model by [OdaxAI](https://odaxai.com), trained in **21 A100-GPU-hours** via generative cross-architecture distillation from the trillion-scale Kimi K2 teacher. **#1 on MMLU and MMLU-Pro**, **#1 on GSM8K** (tied Qwen3-4B-Base), **#1 on HellaSwag** — across the standard 3B–4B open-weight basket.

---

## Available Quantizations

| File | Quant | Size | Use case |
|------|-------|------|----------|
| `DANTE-Mosaic-3.5B-Q4_K_M.gguf` | Q4_K_M | 1.92 GB | **Recommended** — best quality/size tradeoff |
| `DANTE-Mosaic-3.5B-Q5_K_M.gguf` | Q5_K_M | 2.21 GB | Higher quality, still runs on 8 GB RAM |
| `DANTE-Mosaic-3.5B-Q8_0.gguf`   | Q8_0   | 3.28 GB | Near-lossless, for GPU inference |

---

## Quick Start

### llama.cpp

```bash
# Install (macOS)
brew install llama.cpp

# Run directly from HuggingFace
llama-cli -hf OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M

# Or start a local OpenAI-compatible server
llama-server -hf OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M
```

### Ollama

```bash
ollama run OdaxAI_00/dante-mosaic-3.5b
```

Or via HuggingFace (no account needed):

```bash
ollama run hf.co/OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M
```

Or build from the Modelfile included in this repo:

```bash
ollama create dante-mosaic -f Modelfile
ollama run dante-mosaic
```

### LM Studio

1. Open LM Studio → **Discover** tab
2. Search `OdaxAI/DANTE-Mosaic-3.5B-GGUF`
3. Download `Q4_K_M` (recommended) and start chatting

### Python (llama-cpp-python)

```python
from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="OdaxAI/DANTE-Mosaic-3.5B-GGUF",
    filename="DANTE-Mosaic-3.5B-Q4_K_M.gguf",
    n_ctx=4096,
    n_gpu_layers=-1,  # offload all layers to GPU if available
)

response = llm(
    "Solve step by step: if a train travels 120 km in 1.5 hours, what is its average speed?",
    max_tokens=256,
    temperature=0.7,
    echo=False,
)
print(response["choices"][0]["text"])
```

### Docker

```bash
docker model run hf.co/OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M
```

---

## Benchmark Results

> All scores measured on the released checkpoint with **lm-evaluation-harness v0.4.5** / **bigcode-evaluation-harness** (pinned), full datasets, 1× A100-40GB, BF16, greedy (T=0), seed 42. No subsets, no prompt engineering.

### Canonical scores (DANTE-Mosaic-3.5B)

| Benchmark | N | Setting | Score |
|-----------|---|---------|-------|
| **HellaSwag** | 10 042 | 10-shot, acc_norm | **76.73 %** |
| **GSM8K** | 1 319 | 8-shot, strict-match | **74.45 %** |
| **ARC-Challenge** | 1 172 | 25-shot, acc_norm | **62.71 %** |
| **MMLU** | 14 042 | 5-shot, acc | **59.38 %** |
| **MBPP** | 374 | pass@1, 0-shot greedy | **42.60 %** |
| **MMLU-Pro** | 4 500 | 5-shot, exact_match | **39.74 %** |
| **HumanEval** | 164 | pass@1, greedy | 6.70 % |

### Comparison vs. 3B–4B open-weight basket

| Benchmark | **DANTE-Mosaic** 3.08B | SmolLM3-3B | Qwen2.5-3B | Llama3.2-3B | Qwen3-1.7B-B | Qwen3-4B-B |
|-----------|:---:|:---:|:---:|:---:|:---:|:---:|
| HellaSwag | **76.7** | 76.2 | 74.2 | 75.5 | 60.5 | 74.4 |
| ARC-Challenge | 62.7 | **65.6** | 59.8 | 58.6 | 55.9 | 62.1 |
| MMLU★ | **59.4** | 44.1ᶜᶠ | 42.9ᶜᶠ | 41.3ᶜᶠ | 39.1ᶜᶠ | 47.7ᶜᶠ |
| MMLU-Pro | **39.7** | 32.7 | 31.3 | 25.1 | 30.4 | 41.1 |
| GSM8K | **74.5** | 67.6 | 70.1 | 25.9 | 65.9 | 74.1 |
| MBPP⁺ | 42.6 | 52.9 | 52.1 | 38.9 | 59.3 | **63.8** |
| HumanEval⁺ | 6.7 | 30.5 | 34.1 | 25.0 | 43.3 | **54.9** |

★ DANTE evaluated on canonical 5-shot MMLU; peers use harder MMLU-CF (cloze). ᶜᶠ MMLU-CF. ⁺ Harder + variants. Peer numbers from the SmolLM3 technical report (Table 1), single harness run.

**Wins vs. the basket:** #1 MMLU (+11.7 pp over next-best), #1 MMLU-Pro, #1 GSM8K (tied Qwen3-4B-Base), #1 HellaSwag. Code (HumanEval/MBPP) is the honest gap — targeted in the next iteration.

---

## About DANTE-Mosaic-3.5B

| | |
|---|---|
| **Architecture** | Dense Transformer (SmolLM3-3B base) + DANTE distillation |
| **Parameters** | ~3.08 B |
| **Teacher** | Kimi K2 (~1T params MoE, W4A16, vLLM TP=16) |
| **Training compute** | **~21 A100-GPU-hours** (~27 000× cheaper than SmolLM3-3B pretraining) |
| **Context length** | 131 072 tokens |
| **Languages** | EN, IT, ES, FR, DE, PT, JA, ZH, AR |
| **License** | Apache 2.0 |
| **Developer** | [OdaxAI](https://odaxai.com) — European foundation model company |

Full technical details, training objective (CWCE + TEA + entropy curriculum), and reproducibility scripts on the [original model card](https://huggingface.co/OdaxAI/DANTE-Mosaic-3.5B) and [OdaxAI research page](https://odaxai.com/research/dante-mosaic-3.5b).

---

*Quantizations produced with [llama.cpp](https://github.com/ggerganov/llama.cpp). Original weights: [OdaxAI/DANTE-Mosaic-3.5B](https://huggingface.co/OdaxAI/DANTE-Mosaic-3.5B).*