---
license: apache-2.0
language:
- it
base_model: mii-llm/zagreus-0.4B-ita
pipeline_tag: text-generation
datasets:
- idealab-cs2/italic-softkd-pool
tags:
- italian
- small-language-model
- knowledge-distillation
- italic
---

# zagreus-0.4B-italic-softkd

Submission for the [mii-llm Italian Post-Training Challenge](https://huggingface.co/spaces/mii-llm/Post-Training-Challenge). This is [mii-llm/zagreus-0.4B-ita](https://huggingface.co/mii-llm/zagreus-0.4B-ita) (437M, Llama architecture) post-trained for the [ITALIC benchmark](https://github.com/Crisp-Unimib/ITALIC), 10,000 multiple-choice questions on Italian culture and language.

| Model | ITALIC full 10K (5-shot fast, temp 0, official harness) |
|---|---|
| zagreus-0.4B-ita (base) | 0.2802 |
| public leaderboard best at submission time | 0.372 |
| **this model** | **0.4787** (confirmed 0.4784 on an independent rerun) |

**Update (2026-08-05):** superseded by [idealab-cs2/zagreus-0.4B-italic-extkd](https://huggingface.co/idealab-cs2/zagreus-0.4B-italic-extkd) — same recipe extended with two more softKD stages (culture pool, then external in-distribution pool), 0.4921/0.4929/0.4932 full-10K.

## Training

One soft knowledge-distillation run from the base model, within the challenge budget of 10 H100 GPU hours (measured replay: ~4.9 H100h including data generation, teacher labeling and evaluation).

For every training question a 3-teacher committee (Mistral-Small-3.2-24B, Gemma-3-27B, Qwen3-32B, each fitting a single H100 in bf16) provides its probability distribution over the answer letters at the answer token. The student is trained in fp32 with a KL term on that distribution (temperature 2.0) plus cross-entropy on the gold letter (weight 0.5). The training pool has 21,606 items: pinocchio-derived knowledge MCQs, committee-labeled grammar MCQs, and generated orthography/morphology MCQs for the categories where public data is scarce. Every pool is decontaminated against the ITALIC test set (exact match plus semantic similarity at cosine 0.80; 0 exact leaks, max similarity 0.7999).

Training data is published on the Hub: [italic-softkd-pool](https://huggingface.co/datasets/idealab-cs2/italic-softkd-pool) (the exact training file) (the exact training file; provenance splits knowledge/grammar/orthomorph included as named splits). Training code, the decontamination manifest and the complete report (including what did not work) are in the submission repository: https://github.com/ruggsea/italic-challenge-submission

## Evaluation

Official [ITALIC harness](https://github.com/Crisp-Unimib/ITALIC), unmodified: vLLM server, fast mode (letter-only answers), 5-shot, temperature 0, accuracy over all 10,000 questions. The chat template is embedded in the tokenizer config; serve the model with its own template.

```bash
vllm serve idealab-cs2/zagreus-0.4B-italic-softkd --chat-template-content-format auto
```

Per-category accuracy: civic education .56, lexicon .52, tourism .50, geography .47, art history .45, history .45, synonyms .47, syntax .42, orthography .39, morphology .36.

## Acknowledgments

Training, data curation and evaluation by [ruggsea](https://huggingface.co/ruggsea), using compute resources of the Complex Social & Computational Systems (CS²) group, IDea_Lab, University of Graz.

## Intended use

This model is specialized for Italian multiple-choice question answering in the ITALIC format. It is a 0.4B model: outside this format expect base-model-level capabilities.