初始化项目，由ModelHub XC社区提供模型
Model: OdaxAI/DANTE-Mosaic-3.5B-GGUF Source: Original Platform
This commit is contained in:
ModelHub XC
164
README.md
Normal file
164
README.md
Normal file
@@ -0,0 +1,164 @@
|
||||
---
|
||||
base_model: OdaxAI/DANTE-Mosaic-3.5B
|
||||
language:
|
||||
- en
|
||||
- it
|
||||
- es
|
||||
- fr
|
||||
- de
|
||||
- pt
|
||||
- ja
|
||||
- zh
|
||||
- ar
|
||||
license: apache-2.0
|
||||
tags:
|
||||
- llama-cpp
|
||||
- gguf
|
||||
- quantized
|
||||
- text-generation
|
||||
- small-language-model
|
||||
- medical
|
||||
- multilingual
|
||||
- 3b
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# DANTE-Mosaic-3.5B — GGUF Quantizations
|
||||
|
||||
GGUF quantized versions of [OdaxAI/DANTE-Mosaic-3.5B](https://huggingface.co/OdaxAI/DANTE-Mosaic-3.5B) for use with **llama.cpp**, **LM Studio**, **Ollama**, and other GGUF-compatible runtimes.
|
||||
|
||||
> DANTE-Mosaic-3.5B is a multilingual foundation model by [OdaxAI](https://odaxai.com), trained in **21 A100-GPU-hours** via generative cross-architecture distillation from the trillion-scale Kimi K2 teacher. **#1 on MMLU and MMLU-Pro**, **#1 on GSM8K** (tied Qwen3-4B-Base), **#1 on HellaSwag** — across the standard 3B–4B open-weight basket.
|
||||
|
||||
---
|
||||
|
||||
## Available Quantizations
|
||||
|
||||
| File | Quant | Size | Use case |
|
||||
|------|-------|------|----------|
|
||||
| `DANTE-Mosaic-3.5B-Q4_K_M.gguf` | Q4_K_M | 1.92 GB | **Recommended** — best quality/size tradeoff |
|
||||
| `DANTE-Mosaic-3.5B-Q5_K_M.gguf` | Q5_K_M | 2.21 GB | Higher quality, still runs on 8 GB RAM |
|
||||
| `DANTE-Mosaic-3.5B-Q8_0.gguf` | Q8_0 | 3.28 GB | Near-lossless, for GPU inference |
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### llama.cpp
|
||||
|
||||
```bash
|
||||
# Install (macOS)
|
||||
brew install llama.cpp
|
||||
|
||||
# Run directly from HuggingFace
|
||||
llama-cli -hf OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M
|
||||
|
||||
# Or start a local OpenAI-compatible server
|
||||
llama-server -hf OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M
|
||||
```
|
||||
|
||||
### Ollama
|
||||
|
||||
```bash
|
||||
ollama run OdaxAI_00/dante-mosaic-3.5b
|
||||
```
|
||||
|
||||
Or via HuggingFace (no account needed):
|
||||
|
||||
```bash
|
||||
ollama run hf.co/OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M
|
||||
```
|
||||
|
||||
Or build from the Modelfile included in this repo:
|
||||
|
||||
```bash
|
||||
ollama create dante-mosaic -f Modelfile
|
||||
ollama run dante-mosaic
|
||||
```
|
||||
|
||||
### LM Studio
|
||||
|
||||
1. Open LM Studio → **Discover** tab
|
||||
2. Search `OdaxAI/DANTE-Mosaic-3.5B-GGUF`
|
||||
3. Download `Q4_K_M` (recommended) and start chatting
|
||||
|
||||
### Python (llama-cpp-python)
|
||||
|
||||
```python
|
||||
from llama_cpp import Llama
|
||||
|
||||
llm = Llama.from_pretrained(
|
||||
repo_id="OdaxAI/DANTE-Mosaic-3.5B-GGUF",
|
||||
filename="DANTE-Mosaic-3.5B-Q4_K_M.gguf",
|
||||
n_ctx=4096,
|
||||
n_gpu_layers=-1, # offload all layers to GPU if available
|
||||
)
|
||||
|
||||
response = llm(
|
||||
"Solve step by step: if a train travels 120 km in 1.5 hours, what is its average speed?",
|
||||
max_tokens=256,
|
||||
temperature=0.7,
|
||||
echo=False,
|
||||
)
|
||||
print(response["choices"][0]["text"])
|
||||
```
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
docker model run hf.co/OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Benchmark Results
|
||||
|
||||
> All scores measured on the released checkpoint with **lm-evaluation-harness v0.4.5** / **bigcode-evaluation-harness** (pinned), full datasets, 1× A100-40GB, BF16, greedy (T=0), seed 42. No subsets, no prompt engineering.
|
||||
|
||||
### Canonical scores (DANTE-Mosaic-3.5B)
|
||||
|
||||
| Benchmark | N | Setting | Score |
|
||||
|-----------|---|---------|-------|
|
||||
| **HellaSwag** | 10 042 | 10-shot, acc_norm | **76.73 %** |
|
||||
| **GSM8K** | 1 319 | 8-shot, strict-match | **74.45 %** |
|
||||
| **ARC-Challenge** | 1 172 | 25-shot, acc_norm | **62.71 %** |
|
||||
| **MMLU** | 14 042 | 5-shot, acc | **59.38 %** |
|
||||
| **MBPP** | 374 | pass@1, 0-shot greedy | **42.60 %** |
|
||||
| **MMLU-Pro** | 4 500 | 5-shot, exact_match | **39.74 %** |
|
||||
| **HumanEval** | 164 | pass@1, greedy | 6.70 % |
|
||||
|
||||
### Comparison vs. 3B–4B open-weight basket
|
||||
|
||||
| Benchmark | **DANTE-Mosaic** 3.08B | SmolLM3-3B | Qwen2.5-3B | Llama3.2-3B | Qwen3-1.7B-B | Qwen3-4B-B |
|
||||
|-----------|:---:|:---:|:---:|:---:|:---:|:---:|
|
||||
| HellaSwag | **76.7** | 76.2 | 74.2 | 75.5 | 60.5 | 74.4 |
|
||||
| ARC-Challenge | 62.7 | **65.6** | 59.8 | 58.6 | 55.9 | 62.1 |
|
||||
| MMLU★ | **59.4** | 44.1ᶜᶠ | 42.9ᶜᶠ | 41.3ᶜᶠ | 39.1ᶜᶠ | 47.7ᶜᶠ |
|
||||
| MMLU-Pro | **39.7** | 32.7 | 31.3 | 25.1 | 30.4 | 41.1 |
|
||||
| GSM8K | **74.5** | 67.6 | 70.1 | 25.9 | 65.9 | 74.1 |
|
||||
| MBPP⁺ | 42.6 | 52.9 | 52.1 | 38.9 | 59.3 | **63.8** |
|
||||
| HumanEval⁺ | 6.7 | 30.5 | 34.1 | 25.0 | 43.3 | **54.9** |
|
||||
|
||||
★ DANTE evaluated on canonical 5-shot MMLU; peers use harder MMLU-CF (cloze). ᶜᶠ MMLU-CF. ⁺ Harder + variants. Peer numbers from the SmolLM3 technical report (Table 1), single harness run.
|
||||
|
||||
**Wins vs. the basket:** #1 MMLU (+11.7 pp over next-best), #1 MMLU-Pro, #1 GSM8K (tied Qwen3-4B-Base), #1 HellaSwag. Code (HumanEval/MBPP) is the honest gap — targeted in the next iteration.
|
||||
|
||||
---
|
||||
|
||||
## About DANTE-Mosaic-3.5B
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **Architecture** | Dense Transformer (SmolLM3-3B base) + DANTE distillation |
|
||||
| **Parameters** | ~3.08 B |
|
||||
| **Teacher** | Kimi K2 (~1T params MoE, W4A16, vLLM TP=16) |
|
||||
| **Training compute** | **~21 A100-GPU-hours** (~27 000× cheaper than SmolLM3-3B pretraining) |
|
||||
| **Context length** | 131 072 tokens |
|
||||
| **Languages** | EN, IT, ES, FR, DE, PT, JA, ZH, AR |
|
||||
| **License** | Apache 2.0 |
|
||||
| **Developer** | [OdaxAI](https://odaxai.com) — European foundation model company |
|
||||
|
||||
Full technical details, training objective (CWCE + TEA + entropy curriculum), and reproducibility scripts on the [original model card](https://huggingface.co/OdaxAI/DANTE-Mosaic-3.5B) and [OdaxAI research page](https://odaxai.com/research/dante-mosaic-3.5b).
|
||||
|
||||
---
|
||||
|
||||
*Quantizations produced with [llama.cpp](https://github.com/ggerganov/llama.cpp). Original weights: [OdaxAI/DANTE-Mosaic-3.5B](https://huggingface.co/OdaxAI/DANTE-Mosaic-3.5B).*
|
||||
Reference in New Issue
Block a user