初始化项目，由ModelHub XC社区提供模型

Model: OdaxAI/DANTE-Mosaic-3.5B-GGUF Source: Original Platform
2026-06-14 08:22:16 +08:00
commit 46959ef2e8
6 changed files with 236 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,38 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+DANTE-Mosaic-3.5B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+DANTE-Mosaic-3.5B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+DANTE-Mosaic-3.5B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
--- a/DANTE-Mosaic-3.5B-Q4_K_M.gguf
+++ b/DANTE-Mosaic-3.5B-Q4_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bbe28f762ec1b7e86d80dd3d7dc225b1fbcdb748ee0ec580ee67500a050a4a12
+size 1915306496
--- a/DANTE-Mosaic-3.5B-Q5_K_M.gguf
+++ b/DANTE-Mosaic-3.5B-Q5_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e4535375ca7129864a96b03b3357283ecbd273ded7b0ae92afe83414483936c
+size 2213757440
--- a/DANTE-Mosaic-3.5B-Q8_0.gguf
+++ b/DANTE-Mosaic-3.5B-Q8_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b911327fec094ce8514124f2c6841ff2c77a6c57f851ccbb51a76cc0dbbc8bb
+size 3275575808
--- a/25
+++ b/25
@@ -0,0 +1,25 @@
+FROM ./DANTE-Mosaic-3.5B-Q4_K_M.gguf
+
+# Model metadata
+PARAMETER num_ctx 4096
+PARAMETER temperature 0.7
+PARAMETER top_p 0.9
+PARAMETER repeat_penalty 1.1
+PARAMETER stop "<|im_end|>"
+PARAMETER stop "<|endoftext|>"
+
+# System prompt
+SYSTEM """You are DANTE-Mosaic, a multilingual AI assistant developed by OdaxAI.
+You are helpful, accurate, and concise. You support English, Italian, Spanish, French,
+German, Portuguese, Japanese, Chinese, and Arabic.
+You are especially strong in reasoning, knowledge retrieval, and long-context tasks."""
+
+TEMPLATE """{{ if .System }}<|im_start|>system
+{{ .System }}<|im_end|>
+{{ end }}{{ if .Prompt }}<|im_start|>user
+{{ .Prompt }}<|im_end|>
+<|im_start|>assistant
+{{ end }}{{ .Response }}<|im_end|>
+"""
+
+LICENSE """Apache 2.0 — OdaxAI (https://odaxai.com)"""
--- a/README.md
+++ b/README.md
@@ -0,0 +1,164 @@
+---
+base_model: OdaxAI/DANTE-Mosaic-3.5B
+language:
+  - en
+  - it
+  - es
+  - fr
+  - de
+  - pt
+  - ja
+  - zh
+  - ar
+license: apache-2.0
+tags:
+  - llama-cpp
+  - gguf
+  - quantized
+  - text-generation
+  - small-language-model
+  - medical
+  - multilingual
+  - 3b
+pipeline_tag: text-generation
+---
+
+# DANTE-Mosaic-3.5B — GGUF Quantizations
+
+GGUF quantized versions of [OdaxAI/DANTE-Mosaic-3.5B](https://huggingface.co/OdaxAI/DANTE-Mosaic-3.5B) for use with **llama.cpp**, **LM Studio**, **Ollama**, and other GGUF-compatible runtimes.
+
+> DANTE-Mosaic-3.5B is a multilingual foundation model by [OdaxAI](https://odaxai.com), trained in **21 A100-GPU-hours** via generative cross-architecture distillation from the trillion-scale Kimi K2 teacher. **#1 on MMLU and MMLU-Pro**, **#1 on GSM8K** (tied Qwen3-4B-Base), **#1 on HellaSwag** — across the standard 3B–4B open-weight basket.
+
+---
+
+## Available Quantizations
+
+| File | Quant | Size | Use case |
+|------|-------|------|----------|
+| `DANTE-Mosaic-3.5B-Q4_K_M.gguf` | Q4_K_M | 1.92 GB | **Recommended** — best quality/size tradeoff |
+| `DANTE-Mosaic-3.5B-Q5_K_M.gguf` | Q5_K_M | 2.21 GB | Higher quality, still runs on 8 GB RAM |
+| `DANTE-Mosaic-3.5B-Q8_0.gguf`   | Q8_0   | 3.28 GB | Near-lossless, for GPU inference |
+
+---
+
+## Quick Start
+
+### llama.cpp
+
+```bash
+# Install (macOS)
+brew install llama.cpp
+
+# Run directly from HuggingFace
+llama-cli -hf OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M
+
+# Or start a local OpenAI-compatible server
+llama-server -hf OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M
+```
+
+### Ollama
+
+```bash
+ollama run OdaxAI_00/dante-mosaic-3.5b
+```
+
+Or via HuggingFace (no account needed):
+
+```bash
+ollama run hf.co/OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M
+```
+
+Or build from the Modelfile included in this repo:
+
+```bash
+ollama create dante-mosaic -f Modelfile
+ollama run dante-mosaic
+```
+
+### LM Studio
+
+1. Open LM Studio → **Discover** tab
+2. Search `OdaxAI/DANTE-Mosaic-3.5B-GGUF`
+3. Download `Q4_K_M` (recommended) and start chatting
+
+### Python (llama-cpp-python)
+
+```python
+from llama_cpp import Llama
+
+llm = Llama.from_pretrained(
+    repo_id="OdaxAI/DANTE-Mosaic-3.5B-GGUF",
+    filename="DANTE-Mosaic-3.5B-Q4_K_M.gguf",
+    n_ctx=4096,
+    n_gpu_layers=-1,  # offload all layers to GPU if available
+)
+
+response = llm(
+    "Solve step by step: if a train travels 120 km in 1.5 hours, what is its average speed?",
+    max_tokens=256,
+    temperature=0.7,
+    echo=False,
+)
+print(response["choices"][0]["text"])
+```
+
+### Docker
+
+```bash
+docker model run hf.co/OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M
+```
+
+---
+
+## Benchmark Results
+
+> All scores measured on the released checkpoint with **lm-evaluation-harness v0.4.5** / **bigcode-evaluation-harness** (pinned), full datasets, 1× A100-40GB, BF16, greedy (T=0), seed 42. No subsets, no prompt engineering.
+
+### Canonical scores (DANTE-Mosaic-3.5B)
+
+| Benchmark | N | Setting | Score |
+|-----------|---|---------|-------|
+| **HellaSwag** | 10 042 | 10-shot, acc_norm | **76.73 %** |
+| **GSM8K** | 1 319 | 8-shot, strict-match | **74.45 %** |
+| **ARC-Challenge** | 1 172 | 25-shot, acc_norm | **62.71 %** |
+| **MMLU** | 14 042 | 5-shot, acc | **59.38 %** |
+| **MBPP** | 374 | pass@1, 0-shot greedy | **42.60 %** |
+| **MMLU-Pro** | 4 500 | 5-shot, exact_match | **39.74 %** |
+| **HumanEval** | 164 | pass@1, greedy | 6.70 % |
+
+### Comparison vs. 3B–4B open-weight basket
+
+| Benchmark | **DANTE-Mosaic** 3.08B | SmolLM3-3B | Qwen2.5-3B | Llama3.2-3B | Qwen3-1.7B-B | Qwen3-4B-B |
+|-----------|:---:|:---:|:---:|:---:|:---:|:---:|
+| HellaSwag | **76.7** | 76.2 | 74.2 | 75.5 | 60.5 | 74.4 |
+| ARC-Challenge | 62.7 | **65.6** | 59.8 | 58.6 | 55.9 | 62.1 |
+| MMLU★ | **59.4** | 44.1ᶜᶠ | 42.9ᶜᶠ | 41.3ᶜᶠ | 39.1ᶜᶠ | 47.7ᶜᶠ |
+| MMLU-Pro | **39.7** | 32.7 | 31.3 | 25.1 | 30.4 | 41.1 |
+| GSM8K | **74.5** | 67.6 | 70.1 | 25.9 | 65.9 | 74.1 |
+| MBPP⁺ | 42.6 | 52.9 | 52.1 | 38.9 | 59.3 | **63.8** |
+| HumanEval⁺ | 6.7 | 30.5 | 34.1 | 25.0 | 43.3 | **54.9** |
+
+★ DANTE evaluated on canonical 5-shot MMLU; peers use harder MMLU-CF (cloze). ᶜᶠ MMLU-CF. ⁺ Harder + variants. Peer numbers from the SmolLM3 technical report (Table 1), single harness run.
+
+**Wins vs. the basket:** #1 MMLU (+11.7 pp over next-best), #1 MMLU-Pro, #1 GSM8K (tied Qwen3-4B-Base), #1 HellaSwag. Code (HumanEval/MBPP) is the honest gap — targeted in the next iteration.
+
+---
+
+## About DANTE-Mosaic-3.5B
+
+| | |
+|---|---|
+| **Architecture** | Dense Transformer (SmolLM3-3B base) + DANTE distillation |
+| **Parameters** | ~3.08 B |
+| **Teacher** | Kimi K2 (~1T params MoE, W4A16, vLLM TP=16) |
+| **Training compute** | **~21 A100-GPU-hours** (~27 000× cheaper than SmolLM3-3B pretraining) |
+| **Context length** | 131 072 tokens |
+| **Languages** | EN, IT, ES, FR, DE, PT, JA, ZH, AR |
+| **License** | Apache 2.0 |
+| **Developer** | [OdaxAI](https://odaxai.com) — European foundation model company |
+
+Full technical details, training objective (CWCE + TEA + entropy curriculum), and reproducibility scripts on the [original model card](https://huggingface.co/OdaxAI/DANTE-Mosaic-3.5B) and [OdaxAI research page](https://odaxai.com/research/dante-mosaic-3.5b).
+
+---
+
+*Quantizations produced with [llama.cpp](https://github.com/ggerganov/llama.cpp). Original weights: [OdaxAI/DANTE-Mosaic-3.5B](https://huggingface.co/OdaxAI/DANTE-Mosaic-3.5B).*