AMALIA-9B-0626-DPO-GGUF/README.md

license, language, base_model, base_model_relation, pipeline_tag, library_name, tags

license	language	base_model	base_model_relation	pipeline_tag	library_name	tags
apache-2.0	pt	amalia-llm/AMALIA-9B-0626-DPO	quantized	text-generation	llama.cpp	gguf llama.cpp quantized portuguese european-portuguese amalia

AMALIA-9B-0626-DPO — GGUF

GGUF quantizations of amalia-llm/AMALIA-9B-0626-DPO for use with llama.cpp and compatible runtimes (LM Studio, Ollama, Jan, etc.).

AMALIA is an open-source language model targeting European Portuguese, developed by a consortium of Portuguese universities and research centres and funded by the Government of Portugal. This is the DPO (preference-tuned) variant. See the original model card and the technical report for full details.

Files

File	Quant	Size	Notes
AMALIA-9B-0626-DPO-Q4_K_S.gguf	Q4_K_S	~5.4 GB	4-bit small, quantized directly from bf16
AMALIA-9B-0626-DPO-Q4_K_M.gguf	Q4_K_M	~5.6 GB	4-bit medium, recommended balance of quality and size
AMALIA-9B-0626-DPO-Q8_0.gguf	Q8_0	~9.7 GB	8-bit, near-lossless
AMALIA-9B-0626-DPO-BF16.gguf	BF16	~18.3 GB	full-precision original weights in GGUF

Converted with convert_hf_to_gguf.py (llama.cpp b8470) from the original bf16 weights. BF16 and Q4_K_S come directly from the original weights; Q8_0 was converted directly from bf16 and Q4_K_M was requantized from that Q8_0 (Q8_0 is near-lossless relative to bf16). The chat template (ChatML with the AMALIA Portuguese system prompt) is embedded in the GGUF metadata.

Usage

# llama.cpp CLI (chat template is picked up automatically)
llama-cli -hf layerx-labs/AMALIA-9B-0626-DPO-GGUF:Q4_K_M

# or with a local file
llama-cli -m AMALIA-9B-0626-DPO-Q4_K_M.gguf -cnv

# OpenAI-compatible server
llama-server -m AMALIA-9B-0626-DPO-Q4_K_M.gguf -c 8192

License

Apache 2.0, same as the original model. All credit for the model goes to the AMALIA project.