初始化项目，由ModelHub XC社区提供模型

Model: GitMarco27/zagreus-0.4b-italic-kl Source: Original Platform
2026-07-25 11:26:09 +08:00
commit 30e7dd7c9b
14 changed files with 514 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,161 @@
+---
+language:
+- it
+- en
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
+base_model: mii-llm/zagreus-0.4B-ita
+datasets:
+- efederici/pinocchio
+- mik3ml/italian-dictionary
+- fraug-library/synonyms_dictionnaries
+tags:
+- italian
+- multiple-choice
+- knowledge-distillation
+- italic-benchmark
+model-index:
+- name: Zagreus 0.4B ITALIC KL
+  results:
+  - task:
+      type: text-generation
+      name: ITALIC multiple-choice generation
+    dataset:
+      name: ITALIC
+      type: Crisp-Unimib/ITALIC
+    metrics:
+    - type: accuracy
+      value: 0.3381
+      name: Official zero-shot fast accuracy
+---
+
+# Zagreus 0.4B ITALIC KL
+
+This model is a full BF16 fine-tune of
+[`mii-llm/zagreus-0.4B-ita`](https://huggingface.co/mii-llm/zagreus-0.4B-ita) for the
+[Italian Post-Training Challenge 2026](https://huggingface.co/spaces/mii-llm/Post-Training-Challenge).
+It is trained to express the base model's answer-text probabilities through the first generated
+multiple-choice letter.
+
+- Training code: https://github.com/GitMarco27/zagreus-italic-kl
+- Training data and KL targets: https://huggingface.co/datasets/GitMarco27/zagreus-0.4b-italic-kl-data
+- Complete evaluation: [`evaluation/official_results.json`](evaluation/official_results.json)
+
+## Results
+
+| Evaluation | Setting | Questions | Accuracy |
+|---|---|---:|---:|
+| ITALIC official `run_eval.py` | zero-shot, `fast=true`, temperature 0 | 10,000 | **0.3381** |
+| Public domain development set | zero-shot, greedy | 480 | 0.5813 |
+| KL held-out set | first-token letter accuracy | 924 records after filtering | 0.6190 |
+
+The official evaluator is pinned to commit `92df420ff686babeea54e217b9f90f8471374916`. Zero-shot evaluation is
+configured by setting `few_shot_file: null` in the official ITALIC configuration. This differs from the
+repository's five-shot default and was accepted for this challenge submission.
+
+## Method
+
+For each answer option, the base model produces a domain-conditional cloze-PMI score:
+
+```text
+s_i = log P(option_i | question) - log P(option_i | domain prompt)
+q   = softmax(s / tau)
+```
+
+Two balanced option permutations are created for each question. At the first generation position, the model is
+trained with the following loss:
+
+```text
+L = KL(q || softmax(z_ABCD))
+  + lambda * CE(z_ABCD, gold)
+  + mu * (-log P(first token is A, B, C, or D))
+```
+
+The final hyperparameters are `tau=1`, `lambda=1`, `mu=0.1`, learning rate `1e-4`, batch size 16, five epochs,
+weight decay 0.01, and seed 42. The teacher and student both start from the mandatory base model. No external
+teacher checkpoint or generation API is used.
+
+## Training data
+
+The 21,847-question source pool contains:
+
+- 9,847 culture and civics questions from `efederici/pinocchio`;
+- 6,000 definition questions derived from `mik3ml/italian-dictionary`;
+- 6,000 synonym questions derived from `fraug-library/synonyms_dictionnaries`.
+
+The KL build selects 5,000 training questions and 500 question-disjoint held-out questions, then creates two
+permutations per question. The 256-token prompt filter leaves 9,294 training records and 924 held-out records.
+
+The source pool is deduplicated and checked against all ITALIC questions with exact normalized matching,
+MinHash LSH, and RapidFuzz confirmation. The final audit found no question overlap. ITALIC labels are not used
+for training.
+
+The dataset repository records the original terms for each source. The Hugging Face card for the synonym mirror
+does not declare a license, so users should review its upstream terms before redistributing derived data.
+
+## Usage
+
+This checkpoint is specialized for four-option multiple-choice questions. The following example uses the same
+prompt style as the official fast evaluation:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "GitMarco27/zagreus-0.4b-italic-kl"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
+
+messages = [
+    {"role": "system", "content": "Sei un assistente utile."},
+    {
+        "role": "user",
+        "content": (
+            "Rispondi alla seguente domanda a scelta multipla. "
+            "Scrivi solo la lettera corretta.\n\n"
+            "Qual è la capitale d'Italia?\n\n"
+            "A) Milano\nB) Roma\nC) Torino\nD) Napoli\n\nRisposta:"
+        ),
+    },
+]
+
+inputs = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt",
+    return_dict=True,
+).to(model.device)
+output = model.generate(**inputs, do_sample=False, max_new_tokens=1)
+print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
+```
+
+## Final run
+
+The final run used one NVIDIA A100 80GB PCIe GPU. The allocation lasted 0.7531 hours, including setup, target
+generation, training, evaluation, transfer, and verification. Based on the recorded provider price ratio, this
+equals 0.3622 H100-equivalent hours.
+
+The immutable base revision is `f463a5beabf054809ee3cc702c711215a5fde690`. The model weights have SHA-256
+`5e0284467f727f22000b897a3c40f3e5c24b519de463f71a4212149fe1913c6e`.
+
+## Limitations
+
+This is a benchmark-specific model, not a general instruction model. Training directly supervises the first
+generated token, so later tokens may repeat. For normal multiple-choice inference, use greedy decoding with
+`max_new_tokens=1`.
+
+The model has about 438 million parameters. Its training data covers lexical, cultural, and civic questions
+better than syntax, orthography, and morphology.
+
+## Development tools
+
+Development of this project relied on AI assistants: Claude Fable 5, GPT-5.6-sol, Claude Opus, and
+Qwen3.6 27B. They supported code writing, experiment analysis, and documentation. The assistants were not
+used to generate training data: all KL targets were produced with the mandatory base model itself, as
+described above, and no external API was used for synthetic generation.
+
+## References
+
+- Seveso et al. (2025), [ITALIC: An Italian Culture-Aware Natural Language Benchmark](https://aclanthology.org/2025.naacl-long.68/).
+- Holtzman et al. (2021), [Surface Form Competition: Why the Highest Probability Answer Isn't Always Right](https://aclanthology.org/2021.emnlp-main.564/).
+- Hinton, Vinyals, and Dean (2015), [Distilling the Knowledge in a Neural Network](https://arxiv.org/abs/1503.02531).