59 lines
2.9 KiB
Markdown
59 lines
2.9 KiB
Markdown
---
|
||
base_model: mii-llm/nesso-0.4B-agentic
|
||
language:
|
||
- it
|
||
pipeline_tag: text-generation
|
||
tags:
|
||
- distillation
|
||
- on-policy-distillation
|
||
- italic
|
||
- italian
|
||
- mcqa
|
||
---
|
||
|
||
# zagreus_0.4_competition
|
||
|
||
**nesso-0.4B-agentic improved on the [ITALIC](https://github.com/Crisp-Unimib/ITALIC) benchmark by on-policy distillation from Coloss/nesso-3B.**
|
||
|
||
Official ITALIC harness (`run_eval.py`, full 10,000 questions, 5-shot):
|
||
|
||
| Model | fast | slow (CoT) |
|
||
|---|---:|---:|
|
||
| mii-llm/nesso-0.4B-agentic (baseline) | 33.1% | 0.0%* |
|
||
| **this model** | **37.2%** | **33.2%** |
|
||
| Coloss/nesso-3B (teacher) | 50.7% | 2.5%* |
|
||
|
||
\* baseline/teacher answer with a bare letter, which the official slow-mode extractor cannot parse; this model answers parseably in both modes.
|
||
|
||
## Method
|
||
|
||
On-policy distillation ([tinker-cookbook recipe](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/distillation/on_policy_distillation.py), reimplemented in [mii-llm/palingenesis](https://github.com/mii-llm/palingenesis) branch `odp`): every step the student samples completions with its current weights and the loss is the full-distribution reverse KL to the teacher over exactly those tokens. Student (ChatML) and teacher (Llama-3 template) share the Llama-3 base vocabulary; prompts are rendered per-model and the student's `<|im_end|>` mass is merged into the teacher's `<|eot_id|>` slot, so stopping behavior is distilled too.
|
||
|
||
The winning run (`opd_v3`, checkpoint step 550 of 600):
|
||
|
||
- pool of 49,643 ITALIC-format MCQA prompts (pinocchio-raw + MMLU-ita sources, deduped against the ITALIC 10k test set by normalized-question hash), **filtered to rows the teacher answers correctly** (`pgs distill-score` — pure KL would distill the teacher's errors too)
|
||
- Italian-language (vocabulary/grammar/comprehension) rows upweighted ×4
|
||
- 80% of prompts rendered with ITALIC's official 5 shots, `max_new_tokens 8` (terse supervision)
|
||
- 600 steps, batch 32 prompts, lr 1e-5 cosine, full reverse KL, ~40 min on one A100 80GB
|
||
|
||
Training/eval code: [giux78/zagreus_0.4_competition](https://github.com/giux78/zagreus_0.4_competition) (experiment) + [mii-llm/palingenesis](https://github.com/mii-llm/palingenesis) `odp` branch (library, `pgs distill`).
|
||
|
||
## Usage
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
tok = AutoTokenizer.from_pretrained("giux78/zagreus_0.4_competition")
|
||
model = AutoModelForCausalLM.from_pretrained("giux78/zagreus_0.4_competition", dtype="bfloat16")
|
||
|
||
messages = [
|
||
{"role": "system", "content": "Sei un assistente utile."},
|
||
{"role": "user", "content": "Rispondi alla seguente domanda a scelta multipla..."},
|
||
]
|
||
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
|
||
out = model.generate(ids, max_new_tokens=8, do_sample=False)
|
||
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
|
||
```
|
||
|
||
For ITALIC-style evaluation use `max_new_tokens=8` and greedy decoding (the model answers with the bare option letter).
|