commit 46959ef2e8e3e3d229cd334aa4dca9e3f32d743b Author: ModelHub XC Date: Sun Jun 14 08:22:16 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: OdaxAI/DANTE-Mosaic-3.5B-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..88dbda1 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,38 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +DANTE-Mosaic-3.5B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +DANTE-Mosaic-3.5B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +DANTE-Mosaic-3.5B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/DANTE-Mosaic-3.5B-Q4_K_M.gguf b/DANTE-Mosaic-3.5B-Q4_K_M.gguf new file mode 100644 index 0000000..dab447a --- /dev/null +++ b/DANTE-Mosaic-3.5B-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bbe28f762ec1b7e86d80dd3d7dc225b1fbcdb748ee0ec580ee67500a050a4a12 +size 1915306496 diff --git a/DANTE-Mosaic-3.5B-Q5_K_M.gguf b/DANTE-Mosaic-3.5B-Q5_K_M.gguf new file mode 100644 index 0000000..ba391d1 --- /dev/null +++ b/DANTE-Mosaic-3.5B-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6e4535375ca7129864a96b03b3357283ecbd273ded7b0ae92afe83414483936c +size 2213757440 diff --git a/DANTE-Mosaic-3.5B-Q8_0.gguf b/DANTE-Mosaic-3.5B-Q8_0.gguf new file mode 100644 index 0000000..7f21ba7 --- /dev/null +++ b/DANTE-Mosaic-3.5B-Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7b911327fec094ce8514124f2c6841ff2c77a6c57f851ccbb51a76cc0dbbc8bb +size 3275575808 diff --git a/Modelfile b/Modelfile new file mode 100644 index 0000000..d7acfc0 --- /dev/null +++ b/Modelfile @@ -0,0 +1,25 @@ +FROM ./DANTE-Mosaic-3.5B-Q4_K_M.gguf + +# Model metadata +PARAMETER num_ctx 4096 +PARAMETER temperature 0.7 +PARAMETER top_p 0.9 +PARAMETER repeat_penalty 1.1 +PARAMETER stop "<|im_end|>" +PARAMETER stop "<|endoftext|>" + +# System prompt +SYSTEM """You are DANTE-Mosaic, a multilingual AI assistant developed by OdaxAI. +You are helpful, accurate, and concise. You support English, Italian, Spanish, French, +German, Portuguese, Japanese, Chinese, and Arabic. +You are especially strong in reasoning, knowledge retrieval, and long-context tasks.""" + +TEMPLATE """{{ if .System }}<|im_start|>system +{{ .System }}<|im_end|> +{{ end }}{{ if .Prompt }}<|im_start|>user +{{ .Prompt }}<|im_end|> +<|im_start|>assistant +{{ end }}{{ .Response }}<|im_end|> +""" + +LICENSE """Apache 2.0 — OdaxAI (https://odaxai.com)""" diff --git a/README.md b/README.md new file mode 100644 index 0000000..4a089d5 --- /dev/null +++ b/README.md @@ -0,0 +1,164 @@ +--- +base_model: OdaxAI/DANTE-Mosaic-3.5B +language: + - en + - it + - es + - fr + - de + - pt + - ja + - zh + - ar +license: apache-2.0 +tags: + - llama-cpp + - gguf + - quantized + - text-generation + - small-language-model + - medical + - multilingual + - 3b +pipeline_tag: text-generation +--- + +# DANTE-Mosaic-3.5B — GGUF Quantizations + +GGUF quantized versions of [OdaxAI/DANTE-Mosaic-3.5B](https://huggingface.co/OdaxAI/DANTE-Mosaic-3.5B) for use with **llama.cpp**, **LM Studio**, **Ollama**, and other GGUF-compatible runtimes. + +> DANTE-Mosaic-3.5B is a multilingual foundation model by [OdaxAI](https://odaxai.com), trained in **21 A100-GPU-hours** via generative cross-architecture distillation from the trillion-scale Kimi K2 teacher. **#1 on MMLU and MMLU-Pro**, **#1 on GSM8K** (tied Qwen3-4B-Base), **#1 on HellaSwag** — across the standard 3B–4B open-weight basket. + +--- + +## Available Quantizations + +| File | Quant | Size | Use case | +|------|-------|------|----------| +| `DANTE-Mosaic-3.5B-Q4_K_M.gguf` | Q4_K_M | 1.92 GB | **Recommended** — best quality/size tradeoff | +| `DANTE-Mosaic-3.5B-Q5_K_M.gguf` | Q5_K_M | 2.21 GB | Higher quality, still runs on 8 GB RAM | +| `DANTE-Mosaic-3.5B-Q8_0.gguf` | Q8_0 | 3.28 GB | Near-lossless, for GPU inference | + +--- + +## Quick Start + +### llama.cpp + +```bash +# Install (macOS) +brew install llama.cpp + +# Run directly from HuggingFace +llama-cli -hf OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M + +# Or start a local OpenAI-compatible server +llama-server -hf OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M +``` + +### Ollama + +```bash +ollama run OdaxAI_00/dante-mosaic-3.5b +``` + +Or via HuggingFace (no account needed): + +```bash +ollama run hf.co/OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M +``` + +Or build from the Modelfile included in this repo: + +```bash +ollama create dante-mosaic -f Modelfile +ollama run dante-mosaic +``` + +### LM Studio + +1. Open LM Studio → **Discover** tab +2. Search `OdaxAI/DANTE-Mosaic-3.5B-GGUF` +3. Download `Q4_K_M` (recommended) and start chatting + +### Python (llama-cpp-python) + +```python +from llama_cpp import Llama + +llm = Llama.from_pretrained( + repo_id="OdaxAI/DANTE-Mosaic-3.5B-GGUF", + filename="DANTE-Mosaic-3.5B-Q4_K_M.gguf", + n_ctx=4096, + n_gpu_layers=-1, # offload all layers to GPU if available +) + +response = llm( + "Solve step by step: if a train travels 120 km in 1.5 hours, what is its average speed?", + max_tokens=256, + temperature=0.7, + echo=False, +) +print(response["choices"][0]["text"]) +``` + +### Docker + +```bash +docker model run hf.co/OdaxAI/DANTE-Mosaic-3.5B-GGUF:Q4_K_M +``` + +--- + +## Benchmark Results + +> All scores measured on the released checkpoint with **lm-evaluation-harness v0.4.5** / **bigcode-evaluation-harness** (pinned), full datasets, 1× A100-40GB, BF16, greedy (T=0), seed 42. No subsets, no prompt engineering. + +### Canonical scores (DANTE-Mosaic-3.5B) + +| Benchmark | N | Setting | Score | +|-----------|---|---------|-------| +| **HellaSwag** | 10 042 | 10-shot, acc_norm | **76.73 %** | +| **GSM8K** | 1 319 | 8-shot, strict-match | **74.45 %** | +| **ARC-Challenge** | 1 172 | 25-shot, acc_norm | **62.71 %** | +| **MMLU** | 14 042 | 5-shot, acc | **59.38 %** | +| **MBPP** | 374 | pass@1, 0-shot greedy | **42.60 %** | +| **MMLU-Pro** | 4 500 | 5-shot, exact_match | **39.74 %** | +| **HumanEval** | 164 | pass@1, greedy | 6.70 % | + +### Comparison vs. 3B–4B open-weight basket + +| Benchmark | **DANTE-Mosaic** 3.08B | SmolLM3-3B | Qwen2.5-3B | Llama3.2-3B | Qwen3-1.7B-B | Qwen3-4B-B | +|-----------|:---:|:---:|:---:|:---:|:---:|:---:| +| HellaSwag | **76.7** | 76.2 | 74.2 | 75.5 | 60.5 | 74.4 | +| ARC-Challenge | 62.7 | **65.6** | 59.8 | 58.6 | 55.9 | 62.1 | +| MMLU★ | **59.4** | 44.1ᶜᶠ | 42.9ᶜᶠ | 41.3ᶜᶠ | 39.1ᶜᶠ | 47.7ᶜᶠ | +| MMLU-Pro | **39.7** | 32.7 | 31.3 | 25.1 | 30.4 | 41.1 | +| GSM8K | **74.5** | 67.6 | 70.1 | 25.9 | 65.9 | 74.1 | +| MBPP⁺ | 42.6 | 52.9 | 52.1 | 38.9 | 59.3 | **63.8** | +| HumanEval⁺ | 6.7 | 30.5 | 34.1 | 25.0 | 43.3 | **54.9** | + +★ DANTE evaluated on canonical 5-shot MMLU; peers use harder MMLU-CF (cloze). ᶜᶠ MMLU-CF. ⁺ Harder + variants. Peer numbers from the SmolLM3 technical report (Table 1), single harness run. + +**Wins vs. the basket:** #1 MMLU (+11.7 pp over next-best), #1 MMLU-Pro, #1 GSM8K (tied Qwen3-4B-Base), #1 HellaSwag. Code (HumanEval/MBPP) is the honest gap — targeted in the next iteration. + +--- + +## About DANTE-Mosaic-3.5B + +| | | +|---|---| +| **Architecture** | Dense Transformer (SmolLM3-3B base) + DANTE distillation | +| **Parameters** | ~3.08 B | +| **Teacher** | Kimi K2 (~1T params MoE, W4A16, vLLM TP=16) | +| **Training compute** | **~21 A100-GPU-hours** (~27 000× cheaper than SmolLM3-3B pretraining) | +| **Context length** | 131 072 tokens | +| **Languages** | EN, IT, ES, FR, DE, PT, JA, ZH, AR | +| **License** | Apache 2.0 | +| **Developer** | [OdaxAI](https://odaxai.com) — European foundation model company | + +Full technical details, training objective (CWCE + TEA + entropy curriculum), and reproducibility scripts on the [original model card](https://huggingface.co/OdaxAI/DANTE-Mosaic-3.5B) and [OdaxAI research page](https://odaxai.com/research/dante-mosaic-3.5b). + +--- + +*Quantizations produced with [llama.cpp](https://github.com/ggerganov/llama.cpp). Original weights: [OdaxAI/DANTE-Mosaic-3.5B](https://huggingface.co/OdaxAI/DANTE-Mosaic-3.5B).*