115 lines
2.9 KiB
Markdown
115 lines
2.9 KiB
Markdown
---
|
|
license: openrail
|
|
language:
|
|
- it
|
|
tags:
|
|
- gguf
|
|
- italian
|
|
- text-generation
|
|
- emma-5
|
|
base_model: eldavoo/emma-5
|
|
---
|
|
|
|
# Emma-5 GGUF
|
|
|
|
Unofficial GGUF conversion of [eldavoo/emma-5](https://huggingface.co/eldavoo/emma-5), archived from the original Egomnia Emma-5 release.
|
|
|
|
The conversion was validated by reconstructing the ONNX attention graph in PyTorch and comparing logits against the original ONNX Runtime output before exporting to GGUF.
|
|
|
|
The original ONNX model remains the reference runtime. This GGUF is intended for raw llama.cpp completion use, not chat-template frontends.
|
|
|
|
## Files
|
|
|
|
| File | Description |
|
|
|------|-------------|
|
|
| `emma-5-f16.gguf` | FP16 GGUF, ~1.2 GB |
|
|
|
|
## Conversion Details
|
|
|
|
The original model is distributed as ONNX only. Direct conversion to GGUF required matching the exported attention graph exactly.
|
|
|
|
Pipeline summary:
|
|
|
|
1. ONNX weights and rotary constants were loaded.
|
|
2. The exported attention graph was matched in PyTorch for validation.
|
|
3. Logit parity was checked against ONNX Runtime.
|
|
4. Q/K rows were permuted from split-half RoPE layout to llama.cpp interleaved layout.
|
|
5. The model was exported as FP16 GGUF with `rms_norm_eps=1e-6`.
|
|
|
|
Validation result:
|
|
|
|
```text
|
|
max absolute logit error: ~4e-5
|
|
```
|
|
|
|
## Architecture
|
|
|
|
| Parameter | Value |
|
|
|-----------|-------|
|
|
| Parameters | ~550M |
|
|
| Layers | 28 |
|
|
| Hidden size | 1280 |
|
|
| Q heads | 16 |
|
|
| KV heads | 4 |
|
|
| Head dim | 80 |
|
|
| GQA ratio | 4:1 |
|
|
| FFN | SwiGLU |
|
|
| Context | 2048 |
|
|
| Vocab | 50000 |
|
|
| RoPE theta | 10000 |
|
|
| Embeddings | Tied |
|
|
|
|
## Prompt Format
|
|
|
|
Use a single-line Italian instruction format:
|
|
|
|
```text
|
|
### Istruzione: Qual e la capitale d Italia? ### Risposta:
|
|
```
|
|
|
|
Use raw completion mode only.
|
|
|
|
Do **not** use ChatML, Llama 3 chat templates, OpenAI-style messages, or frontend "conversation" wrappers. In particular, prompts containing markers like `<|im_start|>` or `<|im_end|>` are not supported and may cause repeated marker output.
|
|
|
|
Avoid multiline templates in llama.cpp. The original ONNX SentencePiece path normalizes whitespace differently, while llama.cpp may preserve newline tokens.
|
|
|
|
For the most faithful behavior, compare against or use the original ONNX Runtime model.
|
|
|
|
## Example
|
|
|
|
```bash
|
|
./llama-cli \
|
|
-m emma-5-f16.gguf \
|
|
--no-conversation \
|
|
-p "### Istruzione: Qual e la capitale d Italia? ### Risposta:" \
|
|
-n 64 \
|
|
--temp 0.7 \
|
|
--repeat-penalty 1.2
|
|
```
|
|
|
|
Expected style:
|
|
|
|
```text
|
|
La capitale d'Italia è Roma...
|
|
```
|
|
|
|
## Quantization
|
|
|
|
This repo provides the FP16 GGUF. Other quantizations can be generated with llama.cpp:
|
|
|
|
```bash
|
|
./llama-quantize emma-5-f16.gguf emma-5-Q4_K_M.gguf Q4_K_M
|
|
```
|
|
|
|
## Compatibility
|
|
|
|
Converted and smoke-tested with a recent llama.cpp build.
|
|
|
|
## Credits
|
|
|
|
- Original model by [Egomnia S.p.A.](https://www.egomnia.com)
|
|
- Archived by [eldavoo](https://huggingface.co/eldavoo/emma-5)
|
|
- GGUF conversion by [Spadav](https://github.com/Spadav)
|
|
|
|
This is an unofficial conversion.
|