Apertus-v1.1-4B-GGUF/README.md

base_model, extra_gated_button_content, extra_gated_fields, extra_gated_prompt, language, library_name, license, mradermacher, quantized_by, tags

base_model	extra_gated_button_content	extra_gated_fields	extra_gated_prompt	language	library_name	license	mradermacher	quantized_by	tags
swiss-ai/Apertus-v1.1-4B	Submit	Affiliation By clicking Submit below I accept the terms of use Country Your Name geo text checkbox country text ip_location	### Apertus LLM Acceptable Use Policy (1.0 | September 1, 2025) "Agreement" The Swiss National AI Institute (SNAI) is a partnership between the two Swiss Federal Institutes of Technology, ETH Zurich and EPFL. By using the Apertus LLM you agree to indemnify, defend, and hold harmless ETH Zurich and EPFL against any third-party claims arising from your use of Apertus LLM. The training data and the Apertus LLM may contain or generate information that directly or indirectly refers to an identifiable individual (Personal Data). You process Personal Data as independent controller in accordance with applicable data protection law. SNAI will regularly provide a file with hash values for download which you can apply as an output filter to your use of our Apertus LLM. The file reflects data protection deletion requests which have been addressed to SNAI as the developer of the Apertus LLM. It allows you to remove Personal Data contained in the model output. We strongly advise downloading and applying this output filter from SNAI every six months following the release of the model.	en	transformers	apache-2.0	readme_rev 1	mradermacher	multilingual compliant swiss-ai apertus

About

static quants of https://huggingface.co/swiss-ai/Apertus-v1.1-4B

For a convenient overview and download list, visit our model page for this model.

weighted/imatrix quants are available at https://huggingface.co/mradermacher/Apertus-v1.1-4B-i1-GGUF

Usage

If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files.

Provided Quants

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Link	Type	Size/GB	Notes
GGUF	Q2_K	1.7
GGUF	Q3_K_S	1.9
GGUF	Q3_K_M	2.1	lower quality
GGUF	IQ4_XS	2.3
GGUF	Q3_K_L	2.3
GGUF	Q4_K_S	2.4	fast, recommended
GGUF	Q4_K_M	2.5	fast, recommended
GGUF	Q5_K_S	2.8
GGUF	Q5_K_M	2.9
GGUF	Q6_K	3.2	very good quality
GGUF	Q8_0	4.2	fast, best quality
GGUF	f16	7.8	16 bpw, overkill

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9

FAQ / Model Request

See https://huggingface.co/mradermacher/model_requests for some answers to questions you might have and/or if you want some other model quantized.

Thanks

I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.