From 6afe2e923f8be926763a2340589738c730f6fce9 Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Tue, 19 May 2026 02:57:54 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF Source: Original Platform --- .gitattributes | 57 +++++++++++++ ...3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ1_M.gguf | 3 + ...3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ1_S.gguf | 3 + ...3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_M.gguf | 3 + ...3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_S.gguf | 3 + ....1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_XS.gguf | 3 + ...1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_XXS.gguf | 3 + ...3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_M.gguf | 3 + ...3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_S.gguf | 3 + ....1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_XS.gguf | 3 + ...1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_XXS.gguf | 3 + ....1-8B-Magpie-Align-SFT-v0.1.i1-IQ4_XS.gguf | 3 + ...-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q2_K.gguf | 3 + ....1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_L.gguf | 3 + ....1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_M.gguf | 3 + ....1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_S.gguf | 3 + ...-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_0.gguf | 3 + ....1-8B-Magpie-Align-SFT-v0.1.i1-Q4_K_M.gguf | 3 + ....1-8B-Magpie-Align-SFT-v0.1.i1-Q4_K_S.gguf | 3 + ....1-8B-Magpie-Align-SFT-v0.1.i1-Q5_K_M.gguf | 3 + ....1-8B-Magpie-Align-SFT-v0.1.i1-Q5_K_S.gguf | 3 + ...-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q6_K.gguf | 3 + README.md | 79 +++++++++++++++++++ imatrix.dat | 3 + 24 files changed, 202 insertions(+) create mode 100644 .gitattributes create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ1_M.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ1_S.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_M.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_S.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_XS.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_XXS.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_M.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_S.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_XS.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_XXS.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ4_XS.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q2_K.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_L.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_M.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_S.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_0.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_K_M.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_K_S.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q5_K_M.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q5_K_S.gguf create mode 100644 Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q6_K.gguf create mode 100644 README.md create mode 100644 imatrix.dat diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..d73621e --- /dev/null +++ b/.gitattributes @@ -0,0 +1,57 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ1_M.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ1_M.gguf new file mode 100644 index 0000000..a7a71ae --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2328d4c1b6c81bde363613ae0a7f78c19513b9d982d71339e26ab2b17b48d4d8 +size 2161985792 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ1_S.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ1_S.gguf new file mode 100644 index 0000000..519d2c3 --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ea50be2f1a37e0879bc56c1ba6a8939859ba3f4ed27d250074e97a398abecdd8 +size 2019641600 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_M.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_M.gguf new file mode 100644 index 0000000..933a355 --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ec5bbc957c632f037204fa28cea18921dc9bbff9f94f608e9b87bdd18a378fae +size 2948296160 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_S.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_S.gguf new file mode 100644 index 0000000..58c8d89 --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd034cd7948c8eacb54167a04dafd78b86adeea6b558350f85c6a3d235a9cda3 +size 2758503904 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_XS.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_XS.gguf new file mode 100644 index 0000000..e650cda --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d310d5bc7bbfdfa31c212e6a425f5b5c783963b6a27db284ffec0e7145088a30 +size 2605795584 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_XXS.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..32917fa --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e1f6ab43a0dfc7fab3f2a04b3551aacb06f7547864ae7c42e000342961d05000 +size 2399226112 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_M.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_M.gguf new file mode 100644 index 0000000..cdb8ce5 --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0c77247012e87e10278a61a7d508a821051462a9921098d4eac26cf9178bdee9 +size 3784840256 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_S.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_S.gguf new file mode 100644 index 0000000..3e6aa6b --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:efab3725434789b9fe431582ab309cd5f98155d9c9f2336d12bb61a447bd6506 +size 3682341952 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_XS.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_XS.gguf new file mode 100644 index 0000000..bff4feb --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:68206670bbb1d2f03ce9d0bcf2c0e881fc7761fb930f8eeb753bfa55a9ef0adc +size 3518764096 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_XXS.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..b47091b --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9d9091569e7108c2c66f60604e5d626d8bb7c8569496131bbfbe9432ffccb972 +size 3274927584 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ4_XS.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ4_XS.gguf new file mode 100644 index 0000000..50304b6 --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2c1887ebaf4516d2cf02a2b62884f8aa8e9400aa04b8d2341cd916fabb03b1a +size 4447680800 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q2_K.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q2_K.gguf new file mode 100644 index 0000000..67a05be --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d2fb228800758214461fe8ec7475b540d6e1c39ff1b9d3cc535cea7bddb34d20 +size 3179147104 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_L.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_L.gguf new file mode 100644 index 0000000..5b76db9 --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5401c960e1d3f3b36d983a9aee77c5515bb251c8214b74bffd3a9f6f4a8bba5 +size 4321973312 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_M.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_M.gguf new file mode 100644 index 0000000..8fa058e --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ba5030d2ba5eb3e7aab05710d0e0dc87f48b5882f19a5c6ae6ef51f4cd402394 +size 4018934848 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_S.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_S.gguf new file mode 100644 index 0000000..154e915 --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c29c4e5c9c25ba79df729c8d609e3a805e730901a3aed684df809e9e2c20dfad +size 3664516160 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_0.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_0.gguf new file mode 100644 index 0000000..e470170 --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ed268494ff7f2ea4d238273f0efd698d6bd52210671cf6a1bb00b537643925e +size 4675910304 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_K_M.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_K_M.gguf new file mode 100644 index 0000000..fce812f --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ed96b63b2d96f411840783ac7ebd8d79711d0b4b46297596b29eeb8307f3b62 +size 4920752800 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_K_S.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_K_S.gguf new file mode 100644 index 0000000..df40735 --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:acba5bf6c747bdf53fa0337ab8e20dc531e6ceaa3ee8614d811fbc76aa081abf +size 4692687520 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q5_K_M.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q5_K_M.gguf new file mode 100644 index 0000000..4b958df --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3cbb1655f759153b7265df76ae317f149e91ac6aeac8fcaaa3b7537ae93023af +size 5733007520 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q5_K_S.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q5_K_S.gguf new file mode 100644 index 0000000..e0834f8 --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d659c7ad8fa8bf07fcfe4eac79aba4c468bf91f0221a39663aaac3b48e9b9f07 +size 5599314080 diff --git a/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q6_K.gguf b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q6_K.gguf new file mode 100644 index 0000000..7423311 --- /dev/null +++ b/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ebc0ecd6dcd6aedc4862749ebc1a2cabc00927d692aa5a61d23701e22ecb4d6f +size 6596028160 diff --git a/README.md b/README.md new file mode 100644 index 0000000..db3910d --- /dev/null +++ b/README.md @@ -0,0 +1,79 @@ +--- +base_model: Magpie-Align/Llama-3.1-8B-Magpie-Align-SFT-v0.1 +datasets: +- Magpie-Align/Magpie-Reasoning-150K +- Magpie-Align/Magpie-Pro-MT-300K-v0.1 +language: +- en +library_name: transformers +license: llama3.1 +quantized_by: mradermacher +tags: +- axolotl +- generated_from_trainer +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/Magpie-Align/Llama-3.1-8B-Magpie-Align-SFT-v0.1 + + +static quants are available at https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ1_S.gguf) | i1-IQ1_S | 2.1 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ1_M.gguf) | i1-IQ1_M | 2.3 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.5 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.7 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_S.gguf) | i1-IQ2_S | 2.9 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ2_M.gguf) | i1-IQ2_M | 3.0 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q2_K.gguf) | i1-Q2_K | 3.3 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.4 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.6 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.8 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_S.gguf) | i1-IQ3_S | 3.8 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ3_M.gguf) | i1-IQ3_M | 3.9 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.1 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.4 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.5 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_0.gguf) | i1-Q4_0 | 4.8 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.8 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.0 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.7 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.8 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-Magpie-Align-SFT-v0.1-i1-GGUF/resolve/main/Llama-3.1-8B-Magpie-Align-SFT-v0.1.i1-Q6_K.gguf) | i1-Q6_K | 6.7 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..ac291ee --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ea61f07fb4753b16fdfb46832287009976321a793d8bbe1730bcecdf30274c2b +size 4988157