From 6c97a8e2808713a53e0d06431c50120414aafd23 Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Sat, 11 Apr 2026 11:12:14 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF Source: Original Platform --- .gitattributes | 60 +++++++++++++++++ Llama3.2-ColdBrew-4B-RP.i1-IQ1_M.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-IQ1_S.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-IQ2_M.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-IQ2_S.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-IQ2_XS.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-IQ2_XXS.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-IQ3_M.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-IQ3_S.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-IQ3_XS.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-IQ3_XXS.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-IQ4_NL.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-IQ4_XS.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-Q2_K.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-Q2_K_S.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-Q3_K_L.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-Q3_K_M.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-Q3_K_S.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-Q4_0.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-Q4_1.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-Q4_K_M.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-Q4_K_S.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-Q5_K_M.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-Q5_K_S.gguf | 3 + Llama3.2-ColdBrew-4B-RP.i1-Q6_K.gguf | 3 + Llama3.2-ColdBrew-4B-RP.imatrix.gguf | 3 + README.md | 90 +++++++++++++++++++++++++ 27 files changed, 225 insertions(+) create mode 100644 .gitattributes create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-IQ1_M.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-IQ1_S.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-IQ2_M.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-IQ2_S.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-IQ2_XS.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-IQ2_XXS.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-IQ3_M.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-IQ3_S.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-IQ3_XS.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-IQ3_XXS.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-IQ4_NL.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-IQ4_XS.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-Q2_K.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-Q2_K_S.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-Q3_K_L.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-Q3_K_M.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-Q3_K_S.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-Q4_0.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-Q4_1.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-Q4_K_M.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-Q4_K_S.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-Q5_K_M.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-Q5_K_S.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.i1-Q6_K.gguf create mode 100644 Llama3.2-ColdBrew-4B-RP.imatrix.gguf create mode 100644 README.md diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..269d402 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.2-ColdBrew-4B-RP.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Llama3.2-ColdBrew-4B-RP.i1-IQ1_M.gguf b/Llama3.2-ColdBrew-4B-RP.i1-IQ1_M.gguf new file mode 100644 index 0000000..32dca0c --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:42a3eb4d3185767b5f7c051f08894ea12ab10e5dd2cf21e3e5fe207835be391a +size 1053474944 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-IQ1_S.gguf b/Llama3.2-ColdBrew-4B-RP.i1-IQ1_S.gguf new file mode 100644 index 0000000..f97326d --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:385a17a836958f78bd4b3ce12a9752190749a1bcd96c753a9da143a7e3e5ead8 +size 997441664 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-IQ2_M.gguf b/Llama3.2-ColdBrew-4B-RP.i1-IQ2_M.gguf new file mode 100644 index 0000000..5bfaac9 --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:069e64661b4edf015a29dec8ab03f19ad1cae2d7a34ca466f48690efbe26cadf +size 1398331520 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-IQ2_S.gguf b/Llama3.2-ColdBrew-4B-RP.i1-IQ2_S.gguf new file mode 100644 index 0000000..7413480 --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5e5c21ecc33b8f51aade7504e9e6eb7507b84089b8e4f1618c9488cf9e6674a1 +size 1323620480 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-IQ2_XS.gguf b/Llama3.2-ColdBrew-4B-RP.i1-IQ2_XS.gguf new file mode 100644 index 0000000..62c0a17 --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6b0dd5fefb4b880802dc7aded6cbca6e22804ae74633fae6199c02d8ba4f6160 +size 1229832320 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-IQ2_XXS.gguf b/Llama3.2-ColdBrew-4B-RP.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..b6df7cf --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f5cd0fe2f5ccd8fc6d3acf73ff375a0474ed9517d13405122cb1e3fb8025bbc7 +size 1146863744 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-IQ3_M.gguf b/Llama3.2-ColdBrew-4B-RP.i1-IQ3_M.gguf new file mode 100644 index 0000000..1063fce --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:321f2cb0e43f2bac25167bed47d68b3b97d53fdaf31c5c4fc669854055435452 +size 1768968320 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-IQ3_S.gguf b/Llama3.2-ColdBrew-4B-RP.i1-IQ3_S.gguf new file mode 100644 index 0000000..987effb --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:00ab23d5dbf236c1e41ea53b143d78e2ecd4d3be48b24a21992efe5cdda46f91 +size 1712148608 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-IQ3_XS.gguf b/Llama3.2-ColdBrew-4B-RP.i1-IQ3_XS.gguf new file mode 100644 index 0000000..d65f7e7 --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:12a28cc1e4c4ffc06113b565001b7dcc871ef0f4bd7a2d257947ed1b9b230f6b +size 1646088320 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-IQ3_XXS.gguf b/Llama3.2-ColdBrew-4B-RP.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..a8e64d4 --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1ef02bb5eda1d2865e515049b288a5df15465dce20f674023c238f8a7aa05223 +size 1518065792 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-IQ4_NL.gguf b/Llama3.2-ColdBrew-4B-RP.i1-IQ4_NL.gguf new file mode 100644 index 0000000..aab691b --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de6100df29fa0f3ddebd62e50d7c167a77d28a539955f6ef2d19633c2d3a6126 +size 2138818688 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-IQ4_XS.gguf b/Llama3.2-ColdBrew-4B-RP.i1-IQ4_XS.gguf new file mode 100644 index 0000000..c427af3 --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4ed79c8ea1377052cebed2d1a5ff68984556d738b843d6bb7b76720b95e86c49 +size 2038425728 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-Q2_K.gguf b/Llama3.2-ColdBrew-4B-RP.i1-Q2_K.gguf new file mode 100644 index 0000000..203a0cc --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:86942c344658b74f9054d886a28826b3ac07df9eb9d2028a82fb5e62787e11b6 +size 1493219456 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-Q2_K_S.gguf b/Llama3.2-ColdBrew-4B-RP.i1-Q2_K_S.gguf new file mode 100644 index 0000000..421637c --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ef3c9ff8177f6fab6425c86383509534f41c83b8c52d3eddc18232db082858b0 +size 1403566208 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-Q3_K_L.gguf b/Llama3.2-ColdBrew-4B-RP.i1-Q3_K_L.gguf new file mode 100644 index 0000000..91564cc --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d441b4c8544e6e7c9a17e71e87fdf1330952c8e7bc6bf857c21c13b6e1f61cdc +size 1984647296 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-Q3_K_M.gguf b/Llama3.2-ColdBrew-4B-RP.i1-Q3_K_M.gguf new file mode 100644 index 0000000..c54ffe6 --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a4026f4cc6a1b81e178b7579bab63d6244f21aef04490aad048efdf18d67452d +size 1856458880 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-Q3_K_S.gguf b/Llama3.2-ColdBrew-4B-RP.i1-Q3_K_S.gguf new file mode 100644 index 0000000..b8e4b72 --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bc7ac4d448b2a04f1ce1f28a32deb2e5dc918f9aee567881dfadbb16ce499109 +size 1712148608 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-Q4_0.gguf b/Llama3.2-ColdBrew-4B-RP.i1-Q4_0.gguf new file mode 100644 index 0000000..fa20eb3 --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ba0982be1a31fad7885ed73b708b2c39742e45c38fce325e4356819dc7756358 +size 2143537280 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-Q4_1.gguf b/Llama3.2-ColdBrew-4B-RP.i1-Q4_1.gguf new file mode 100644 index 0000000..ae086c1 --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7cbe0d7dcd91842628c96abc38d522ed5943558093093b320b58e7cb92951393 +size 2339604608 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-Q4_K_M.gguf b/Llama3.2-ColdBrew-4B-RP.i1-Q4_K_M.gguf new file mode 100644 index 0000000..6e7762c --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:005fa2ed642999814bef24a82b6492e81cd9f30c5e733265ab82fb2c3e71cd40 +size 2241005696 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-Q4_K_S.gguf b/Llama3.2-ColdBrew-4B-RP.i1-Q4_K_S.gguf new file mode 100644 index 0000000..b758ea6 --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:64f572f0ae0e126ce73e64f8675c199c0ead522abc2b693eeb0be05505dfd83b +size 2149828736 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-Q5_K_M.gguf b/Llama3.2-ColdBrew-4B-RP.i1-Q5_K_M.gguf new file mode 100644 index 0000000..dd6b7ec --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3f887a35387403c2e9036f66fb01c33361e9338b3c650ca7e51e13b93e524d85 +size 2593032320 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-Q5_K_S.gguf b/Llama3.2-ColdBrew-4B-RP.i1-Q5_K_S.gguf new file mode 100644 index 0000000..ed84306 --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:190c1917268b708052a21bb4576ecdfa51b613046314f664c4f80fd59c4a13a6 +size 2540390528 diff --git a/Llama3.2-ColdBrew-4B-RP.i1-Q6_K.gguf b/Llama3.2-ColdBrew-4B-RP.i1-Q6_K.gguf new file mode 100644 index 0000000..2c7d84e --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6fe2b2505729fb0659c18024e718a390ed3fe23e97e1d3c120fcbf6aa2fc81ef +size 2967060608 diff --git a/Llama3.2-ColdBrew-4B-RP.imatrix.gguf b/Llama3.2-ColdBrew-4B-RP.imatrix.gguf new file mode 100644 index 0000000..ba06155 --- /dev/null +++ b/Llama3.2-ColdBrew-4B-RP.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:636a286a12b867c5facf01d92f9a4fb2674d24807ac18acf041d7cae65d67670 +size 3012064 diff --git a/README.md b/README.md new file mode 100644 index 0000000..9ebe314 --- /dev/null +++ b/README.md @@ -0,0 +1,90 @@ +--- +base_model: SvalTek/Llama3.2-ColdBrew-4B-RP +language: +- en +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- text-generation-inference +- transformers +- unsloth +- llama +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/SvalTek/Llama3.2-ColdBrew-4B-RP + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Llama3.2-ColdBrew-4B-RP-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-IQ1_S.gguf) | i1-IQ1_S | 1.1 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-IQ1_M.gguf) | i1-IQ1_M | 1.2 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.2 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.3 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-IQ2_S.gguf) | i1-IQ2_S | 1.4 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-IQ2_M.gguf) | i1-IQ2_M | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.5 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-Q2_K.gguf) | i1-Q2_K | 1.6 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.6 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.7 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-IQ3_S.gguf) | i1-IQ3_S | 1.8 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-Q3_K_S.gguf) | i1-Q3_K_S | 1.8 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-IQ3_M.gguf) | i1-IQ3_M | 1.9 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-Q3_K_M.gguf) | i1-Q3_K_M | 2.0 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-Q3_K_L.gguf) | i1-Q3_K_L | 2.1 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-IQ4_XS.gguf) | i1-IQ4_XS | 2.1 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.2 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-Q4_0.gguf) | i1-Q4_0 | 2.2 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.2 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.3 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-Q4_1.gguf) | i1-Q4_1 | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-Q5_K_S.gguf) | i1-Q5_K_S | 2.6 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-Q5_K_M.gguf) | i1-Q5_K_M | 2.7 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.2-ColdBrew-4B-RP-i1-GGUF/resolve/main/Llama3.2-ColdBrew-4B-RP.i1-Q6_K.gguf) | i1-Q6_K | 3.1 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +