commit ca0c05d6920375ef9da738a2dbd3894c0146bf22 Author: ModelHub XC Date: Sat Sep 26 04:35:22 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/AFM-4.5B-Base-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..e4f8bb9 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +AFM-4.5B-Base.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/AFM-4.5B-Base.i1-IQ1_M.gguf b/AFM-4.5B-Base.i1-IQ1_M.gguf new file mode 100644 index 0000000..02dbee2 --- /dev/null +++ b/AFM-4.5B-Base.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:31b5f5ea4f9fb6d9d4be14ef74016c8f10c84fcd9045a8d11c4092ed29ea9bf4 +size 1254505600 diff --git a/AFM-4.5B-Base.i1-IQ1_S.gguf b/AFM-4.5B-Base.i1-IQ1_S.gguf new file mode 100644 index 0000000..097fe1e --- /dev/null +++ b/AFM-4.5B-Base.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1702f9d6c517852be70cc269568787db84ed3542c4c096eb6562f4767fe282cc +size 1172667520 diff --git a/AFM-4.5B-Base.i1-IQ2_M.gguf b/AFM-4.5B-Base.i1-IQ2_M.gguf new file mode 100644 index 0000000..e9d9d5f --- /dev/null +++ b/AFM-4.5B-Base.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3bf4b316715284c3b61cfa8208d2a6cfa080d64461f3b9bc4e2c9784aed76dc3 +size 1702138016 diff --git a/AFM-4.5B-Base.i1-IQ2_S.gguf b/AFM-4.5B-Base.i1-IQ2_S.gguf new file mode 100644 index 0000000..99d4a03 --- /dev/null +++ b/AFM-4.5B-Base.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3fa36f123b53dec3d43ae21aab17f1bfabfa3a2ec3236abe5f05ad4e09d01cb9 +size 1593020576 diff --git a/AFM-4.5B-Base.i1-IQ2_XS.gguf b/AFM-4.5B-Base.i1-IQ2_XS.gguf new file mode 100644 index 0000000..a934bbd --- /dev/null +++ b/AFM-4.5B-Base.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:785de8a0c44f6b96bdddc17dddd41a08bbdc52cd03293230063e2db789e8a473 +size 1507392640 diff --git a/AFM-4.5B-Base.i1-IQ2_XXS.gguf b/AFM-4.5B-Base.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..b8bdc91 --- /dev/null +++ b/AFM-4.5B-Base.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2d54c93749b32726bfd456d9117da248c7b3e9ed283b0cc44af4cb70a9784042 +size 1390902400 diff --git a/AFM-4.5B-Base.i1-IQ3_M.gguf b/AFM-4.5B-Base.i1-IQ3_M.gguf new file mode 100644 index 0000000..3bcb069 --- /dev/null +++ b/AFM-4.5B-Base.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:706667c398db1c898aeb5724c01a03eb6c05a5373c584bb963bf0114ec870dfb +size 2183972352 diff --git a/AFM-4.5B-Base.i1-IQ3_S.gguf b/AFM-4.5B-Base.i1-IQ3_S.gguf new file mode 100644 index 0000000..0fdb430 --- /dev/null +++ b/AFM-4.5B-Base.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4542f1198518f4b0353e79aad9f0a177c5d8fd629a75598174e76aab0a263837 +size 2127570432 diff --git a/AFM-4.5B-Base.i1-IQ3_XS.gguf b/AFM-4.5B-Base.i1-IQ3_XS.gguf new file mode 100644 index 0000000..9f0ddc2 --- /dev/null +++ b/AFM-4.5B-Base.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e593b3f0dae4c0de4c37e581892bebdcf338c342b09148f3e079611104763531 +size 2054579712 diff --git a/AFM-4.5B-Base.i1-IQ3_XXS.gguf b/AFM-4.5B-Base.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..9ed25f1 --- /dev/null +++ b/AFM-4.5B-Base.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:26fcb38772b34e2bf148a158a826c4c98391c4f568eb7d3f75f70a289a225d2e +size 1893830816 diff --git a/AFM-4.5B-Base.i1-IQ4_NL.gguf b/AFM-4.5B-Base.i1-IQ4_NL.gguf new file mode 100644 index 0000000..839d61f --- /dev/null +++ b/AFM-4.5B-Base.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a88b965c79addf00a38920d96917e889b1857f0f83dda17ce0d13f4c6cbbce9d +size 2697141056 diff --git a/AFM-4.5B-Base.i1-IQ4_XS.gguf b/AFM-4.5B-Base.i1-IQ4_XS.gguf new file mode 100644 index 0000000..20c41ee --- /dev/null +++ b/AFM-4.5B-Base.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a5a6f68e2ed7871ba7674cb60d29b64f99e56bd28f2a7e5cb5544a31187cacaf +size 2564512256 diff --git a/AFM-4.5B-Base.i1-Q2_K.gguf b/AFM-4.5B-Base.i1-Q2_K.gguf new file mode 100644 index 0000000..74c29a1 --- /dev/null +++ b/AFM-4.5B-Base.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:872d9647bd9f27af14b1d3f1c1c590a43cecce27cd7d655ada9de6ec3d72b417 +size 1893011936 diff --git a/AFM-4.5B-Base.i1-Q2_K_S.gguf b/AFM-4.5B-Base.i1-Q2_K_S.gguf new file mode 100644 index 0000000..77e0657 --- /dev/null +++ b/AFM-4.5B-Base.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dda7fd61a8f48c460be322ced6546dcf10917ecfd6ce0b958c86d5dbd26207a9 +size 1740763616 diff --git a/AFM-4.5B-Base.i1-Q3_K_L.gguf b/AFM-4.5B-Base.i1-Q3_K_L.gguf new file mode 100644 index 0000000..14faf63 --- /dev/null +++ b/AFM-4.5B-Base.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4efd21baf00db929f5023ab98078f60bb2db2d432cb459811ea06c2e5c3664bc +size 2632238592 diff --git a/AFM-4.5B-Base.i1-Q3_K_M.gguf b/AFM-4.5B-Base.i1-Q3_K_M.gguf new file mode 100644 index 0000000..523596e --- /dev/null +++ b/AFM-4.5B-Base.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a8ab4460301147c1f1a2821818c2c78b41398041d6e9ffef0b9253e1bb1a491f +size 2396636672 diff --git a/AFM-4.5B-Base.i1-Q3_K_S.gguf b/AFM-4.5B-Base.i1-Q3_K_S.gguf new file mode 100644 index 0000000..6fee695 --- /dev/null +++ b/AFM-4.5B-Base.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3078994c547e637edd0c7f1521093b403e8621b3b8d6deb99dbfe380eeedf472 +size 2121303552 diff --git a/AFM-4.5B-Base.i1-Q4_0.gguf b/AFM-4.5B-Base.i1-Q4_0.gguf new file mode 100644 index 0000000..e9c7dbf --- /dev/null +++ b/AFM-4.5B-Base.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bdc8cfb8f121fd3c9a7b058dd5b1c849deca29b75c5fd29489ade8fd078c85e9 +size 2703039296 diff --git a/AFM-4.5B-Base.i1-Q4_1.gguf b/AFM-4.5B-Base.i1-Q4_1.gguf new file mode 100644 index 0000000..121ae29 --- /dev/null +++ b/AFM-4.5B-Base.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0fa9194b96b117483085ef308a8f7297f299507a2b0b664af42c5e41067b35fa +size 2959449536 diff --git a/AFM-4.5B-Base.i1-Q4_K_M.gguf b/AFM-4.5B-Base.i1-Q4_K_M.gguf new file mode 100644 index 0000000..4d4c17c --- /dev/null +++ b/AFM-4.5B-Base.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5c34eb8f5a11fe1382f6c8c07f0c79e75672588a8f211944c8b4063265a958a0 +size 2916297536 diff --git a/AFM-4.5B-Base.i1-Q4_K_S.gguf b/AFM-4.5B-Base.i1-Q4_K_S.gguf new file mode 100644 index 0000000..6cfffed --- /dev/null +++ b/AFM-4.5B-Base.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dbc3fdace54a566cdfc7636fd9d30d19b88743d3781f1a5df29ea669e3eb94b7 +size 2715491136 diff --git a/AFM-4.5B-Base.i1-Q5_K_M.gguf b/AFM-4.5B-Base.i1-Q5_K_M.gguf new file mode 100644 index 0000000..5096ccb --- /dev/null +++ b/AFM-4.5B-Base.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:82a6087e6287fc581df1afde98de2ca454d47e57c9bc6eb75fdc9feb8b5e1095 +size 3343593536 diff --git a/AFM-4.5B-Base.i1-Q5_K_S.gguf b/AFM-4.5B-Base.i1-Q5_K_S.gguf new file mode 100644 index 0000000..81243d4 --- /dev/null +++ b/AFM-4.5B-Base.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8fe49b82842acfa3c025b1acea6fe1d38c4d19a513a1a80ab79df2749555d27f +size 3227656256 diff --git a/AFM-4.5B-Base.i1-Q6_K.gguf b/AFM-4.5B-Base.i1-Q6_K.gguf new file mode 100644 index 0000000..bc3afd9 --- /dev/null +++ b/AFM-4.5B-Base.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a5258bdb0fea469012cda3f441762d4d5c3e9dd2202ee483368febab80e2abb6 +size 3797595552 diff --git a/AFM-4.5B-Base.imatrix.gguf b/AFM-4.5B-Base.imatrix.gguf new file mode 100644 index 0000000..fa4469e --- /dev/null +++ b/AFM-4.5B-Base.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0f23b2dade6291ec1b0b087eb8f516046894c9272d449197ae4043b2254f6a0f +size 4530624 diff --git a/README.md b/README.md new file mode 100644 index 0000000..c9b7d69 --- /dev/null +++ b/README.md @@ -0,0 +1,97 @@ +--- +base_model: arcee-ai/AFM-4.5B-Base +extra_gated_prompt: Company name is optional, please put NA if you would prefer not + to share it. +language: +- en +- es +- fr +- de +- it +- pt +- ru +- ar +- hi +- ko +- zh +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/arcee-ai/AFM-4.5B-Base + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#AFM-4.5B-Base-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/AFM-4.5B-Base-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-IQ1_S.gguf) | i1-IQ1_S | 1.3 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-IQ1_M.gguf) | i1-IQ1_M | 1.4 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-IQ2_S.gguf) | i1-IQ2_S | 1.7 | | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-IQ2_M.gguf) | i1-IQ2_M | 1.8 | | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.8 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-Q2_K.gguf) | i1-Q2_K | 2.0 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 2.0 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-IQ3_XS.gguf) | i1-IQ3_XS | 2.2 | | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-Q3_K_S.gguf) | i1-Q3_K_S | 2.2 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-IQ3_S.gguf) | i1-IQ3_S | 2.2 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-IQ3_M.gguf) | i1-IQ3_M | 2.3 | | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-Q3_K_M.gguf) | i1-Q3_K_M | 2.5 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-IQ4_XS.gguf) | i1-IQ4_XS | 2.7 | | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-Q3_K_L.gguf) | i1-Q3_K_L | 2.7 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.8 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-Q4_0.gguf) | i1-Q4_0 | 2.8 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.8 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-Q4_K_M.gguf) | i1-Q4_K_M | 3.0 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-Q4_1.gguf) | i1-Q4_1 | 3.1 | | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-Q5_K_S.gguf) | i1-Q5_K_S | 3.3 | | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-Q5_K_M.gguf) | i1-Q5_K_M | 3.4 | | +| [GGUF](https://huggingface.co/mradermacher/AFM-4.5B-Base-i1-GGUF/resolve/main/AFM-4.5B-Base.i1-Q6_K.gguf) | i1-Q6_K | 3.9 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +