From 36c448086beb804d572df966ec9fcbe594530b99 Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Sun, 31 May 2026 02:57:16 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: mradermacher/Phi-4Garu-telugu-14b-i1-GGUF Source: Original Platform --- .gitattributes | 60 ++++++++++++++++++++ Phi-4Garu-telugu-14b.i1-IQ1_M.gguf | 3 + Phi-4Garu-telugu-14b.i1-IQ1_S.gguf | 3 + Phi-4Garu-telugu-14b.i1-IQ2_M.gguf | 3 + Phi-4Garu-telugu-14b.i1-IQ2_S.gguf | 3 + Phi-4Garu-telugu-14b.i1-IQ2_XS.gguf | 3 + Phi-4Garu-telugu-14b.i1-IQ2_XXS.gguf | 3 + Phi-4Garu-telugu-14b.i1-IQ3_M.gguf | 3 + Phi-4Garu-telugu-14b.i1-IQ3_S.gguf | 3 + Phi-4Garu-telugu-14b.i1-IQ3_XS.gguf | 3 + Phi-4Garu-telugu-14b.i1-IQ3_XXS.gguf | 3 + Phi-4Garu-telugu-14b.i1-IQ4_NL.gguf | 3 + Phi-4Garu-telugu-14b.i1-IQ4_XS.gguf | 3 + Phi-4Garu-telugu-14b.i1-Q2_K.gguf | 3 + Phi-4Garu-telugu-14b.i1-Q2_K_S.gguf | 3 + Phi-4Garu-telugu-14b.i1-Q3_K_L.gguf | 3 + Phi-4Garu-telugu-14b.i1-Q3_K_M.gguf | 3 + Phi-4Garu-telugu-14b.i1-Q3_K_S.gguf | 3 + Phi-4Garu-telugu-14b.i1-Q4_0.gguf | 3 + Phi-4Garu-telugu-14b.i1-Q4_1.gguf | 3 + Phi-4Garu-telugu-14b.i1-Q4_K_M.gguf | 3 + Phi-4Garu-telugu-14b.i1-Q4_K_S.gguf | 3 + Phi-4Garu-telugu-14b.i1-Q5_K_M.gguf | 3 + Phi-4Garu-telugu-14b.i1-Q5_K_S.gguf | 3 + Phi-4Garu-telugu-14b.i1-Q6_K.gguf | 3 + README.md | 83 ++++++++++++++++++++++++++++ imatrix.dat | 3 + 27 files changed, 218 insertions(+) create mode 100644 .gitattributes create mode 100644 Phi-4Garu-telugu-14b.i1-IQ1_M.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-IQ1_S.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-IQ2_M.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-IQ2_S.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-IQ2_XS.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-IQ2_XXS.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-IQ3_M.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-IQ3_S.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-IQ3_XS.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-IQ3_XXS.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-IQ4_NL.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-IQ4_XS.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-Q2_K.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-Q2_K_S.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-Q3_K_L.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-Q3_K_M.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-Q3_K_S.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-Q4_0.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-Q4_1.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-Q4_K_M.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-Q4_K_S.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-Q5_K_M.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-Q5_K_S.gguf create mode 100644 Phi-4Garu-telugu-14b.i1-Q6_K.gguf create mode 100644 README.md create mode 100644 imatrix.dat diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..4649403 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Phi-4Garu-telugu-14b.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Phi-4Garu-telugu-14b.i1-IQ1_M.gguf b/Phi-4Garu-telugu-14b.i1-IQ1_M.gguf new file mode 100644 index 0000000..a309cda --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0bc678578b04bba3e1e8b43ce6818c67cc08eaee14f800e1e027d1a0c4c0ea9f +size 3690189248 diff --git a/Phi-4Garu-telugu-14b.i1-IQ1_S.gguf b/Phi-4Garu-telugu-14b.i1-IQ1_S.gguf new file mode 100644 index 0000000..fead7f2 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5ef278d48e04c177a906be881b66d160136891ef650cf46f5e37359cd58103c7 +size 3412173248 diff --git a/Phi-4Garu-telugu-14b.i1-IQ2_M.gguf b/Phi-4Garu-telugu-14b.i1-IQ2_M.gguf new file mode 100644 index 0000000..bbddca9 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:afb2aa100e8d3b0d77f3fe0e1d4cf78de9b400cb0739a77e65b42ea5a9a10f79 +size 5173924288 diff --git a/Phi-4Garu-telugu-14b.i1-IQ2_S.gguf b/Phi-4Garu-telugu-14b.i1-IQ2_S.gguf new file mode 100644 index 0000000..1a32ac1 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0115ddf991c47ff40a85cd5cc7c59eb67651c5f14a7853fc95f69a89bf3c00c1 +size 4803236288 diff --git a/Phi-4Garu-telugu-14b.i1-IQ2_XS.gguf b/Phi-4Garu-telugu-14b.i1-IQ2_XS.gguf new file mode 100644 index 0000000..ec16a92 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a02bff71586fa80be08949b749492e8d63fc6ca227708fbdbb9c082f7197ea6d +size 4557005248 diff --git a/Phi-4Garu-telugu-14b.i1-IQ2_XXS.gguf b/Phi-4Garu-telugu-14b.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..fd667a9 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4a1703f6af6b4f77260f8e0f327a27c7dc09da9dafd06730e2c4eb06b11cc963 +size 4153549248 diff --git a/Phi-4Garu-telugu-14b.i1-IQ3_M.gguf b/Phi-4Garu-telugu-14b.i1-IQ3_M.gguf new file mode 100644 index 0000000..15d44f2 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:103ed4a10be0533a34eb1c5af81711948e72b02290fb7203d5430df5b31e701e +size 6739763648 diff --git a/Phi-4Garu-telugu-14b.i1-IQ3_S.gguf b/Phi-4Garu-telugu-14b.i1-IQ3_S.gguf new file mode 100644 index 0000000..023383c --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ea8ef908ea00410290407bc17152aca70d06ac6248232dd95ea8a1c82b773988 +size 6539571648 diff --git a/Phi-4Garu-telugu-14b.i1-IQ3_XS.gguf b/Phi-4Garu-telugu-14b.i1-IQ3_XS.gguf new file mode 100644 index 0000000..3586729 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9a800ad3f7393be4f988c3fe343a2c86ff6b20b7ae0ba2d4715a0f483c847527 +size 6220083648 diff --git a/Phi-4Garu-telugu-14b.i1-IQ3_XXS.gguf b/Phi-4Garu-telugu-14b.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..2786950 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d9366db836fb13d9b0b460bff2d9dc173fbc72c6eeae205c83b9dee3c5ff0d64 +size 5811876288 diff --git a/Phi-4Garu-telugu-14b.i1-IQ4_NL.gguf b/Phi-4Garu-telugu-14b.i1-IQ4_NL.gguf new file mode 100644 index 0000000..30a32e4 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7982561ba9cdf7947025ff9acb77610798d9fae9e79759f7c1dfdd0f7ac7e09c +size 8416195008 diff --git a/Phi-4Garu-telugu-14b.i1-IQ4_XS.gguf b/Phi-4Garu-telugu-14b.i1-IQ4_XS.gguf new file mode 100644 index 0000000..fdc8f72 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8db87bd406aa0d742fcbaf2ec22c11ff2bb93f98e3e2f6a019e7b9a353b8164b +size 7982346688 diff --git a/Phi-4Garu-telugu-14b.i1-Q2_K.gguf b/Phi-4Garu-telugu-14b.i1-Q2_K.gguf new file mode 100644 index 0000000..53a9b84 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d371f321295adc4e93a1b59b8ce3ff35872df097198e048013662f6b1c10c8f9 +size 5608796608 diff --git a/Phi-4Garu-telugu-14b.i1-Q2_K_S.gguf b/Phi-4Garu-telugu-14b.i1-Q2_K_S.gguf new file mode 100644 index 0000000..4396db5 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:751799342e18349f5776b281844e58428b65a0c1516593e05cdec42bcf68436f +size 5237084608 diff --git a/Phi-4Garu-telugu-14b.i1-Q3_K_L.gguf b/Phi-4Garu-telugu-14b.i1-Q3_K_L.gguf new file mode 100644 index 0000000..8007ec5 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4b843d82c08c20504a882bfb73a0fa2e1c25570b1ccfe5d4b746b0322d4c8b6a +size 7788851648 diff --git a/Phi-4Garu-telugu-14b.i1-Q3_K_M.gguf b/Phi-4Garu-telugu-14b.i1-Q3_K_M.gguf new file mode 100644 index 0000000..6eb9154 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:383dafc08b29f714ebcff1f7025aafaf455c3acd28e26d23478db0d54f8b52a5 +size 7190835648 diff --git a/Phi-4Garu-telugu-14b.i1-Q3_K_S.gguf b/Phi-4Garu-telugu-14b.i1-Q3_K_S.gguf new file mode 100644 index 0000000..ee51862 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:41d7b41b969d3afa5bed4e5418cf0918a5ab2efb5b781105f9a83d419418e745 +size 6504755648 diff --git a/Phi-4Garu-telugu-14b.i1-Q4_0.gguf b/Phi-4Garu-telugu-14b.i1-Q4_0.gguf new file mode 100644 index 0000000..b7facf9 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7d152e24819ad77f61c52b9cb1f6caac2ca5039339f55138b7cf886930031927 +size 8412099008 diff --git a/Phi-4Garu-telugu-14b.i1-Q4_1.gguf b/Phi-4Garu-telugu-14b.i1-Q4_1.gguf new file mode 100644 index 0000000..8229de1 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0bd1d0837db07c95db0454085eaee1d0fb24d3a7a49ecefdd948b45d45e37f3c +size 9267507648 diff --git a/Phi-4Garu-telugu-14b.i1-Q4_K_M.gguf b/Phi-4Garu-telugu-14b.i1-Q4_K_M.gguf new file mode 100644 index 0000000..a93ce73 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6df47adfdf1de606fb2aa1d3104914ada9e9c4a330304628d816bce4ce390fc1 +size 8890307008 diff --git a/Phi-4Garu-telugu-14b.i1-Q4_K_S.gguf b/Phi-4Garu-telugu-14b.i1-Q4_K_S.gguf new file mode 100644 index 0000000..fec44fa --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b63c131b77b5a45cc65b0420c83ba229f546cb5e4a7d1af42c430ecdebb0ab52 +size 8444047808 diff --git a/Phi-4Garu-telugu-14b.i1-Q5_K_M.gguf b/Phi-4Garu-telugu-14b.i1-Q5_K_M.gguf new file mode 100644 index 0000000..ba04211 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b736d6ff6ff67c6757b2427dfb49b24b2e4e8b03c8d4b92b9589fcc89a046445 +size 10412708288 diff --git a/Phi-4Garu-telugu-14b.i1-Q5_K_S.gguf b/Phi-4Garu-telugu-14b.i1-Q5_K_S.gguf new file mode 100644 index 0000000..877a0ab --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4f12128c3368e526eadc92a866b042a539c9468506d343753386878d7cd25abb +size 10151588288 diff --git a/Phi-4Garu-telugu-14b.i1-Q6_K.gguf b/Phi-4Garu-telugu-14b.i1-Q6_K.gguf new file mode 100644 index 0000000..dc4cdd6 --- /dev/null +++ b/Phi-4Garu-telugu-14b.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a55078aaf4631763529e3d3c4e7cd55aec0a0c48c0ea218ccc47b7369c7c5f39 +size 12030259648 diff --git a/README.md b/README.md new file mode 100644 index 0000000..c012add --- /dev/null +++ b/README.md @@ -0,0 +1,83 @@ +--- +base_model: dumbequation/Phi-4Garu-telugu-14b +datasets: +- NLPT/Telugu_Dilaog_Dataset +language: +- en +- te +library_name: transformers +license: apache-2.0 +quantized_by: mradermacher +tags: +- text-generation-inference +- transformers +- trl +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/dumbequation/Phi-4Garu-telugu-14b + + +static quants are available at https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-IQ1_S.gguf) | i1-IQ1_S | 3.5 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-IQ1_M.gguf) | i1-IQ1_M | 3.8 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 4.3 | | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.7 | | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-IQ2_S.gguf) | i1-IQ2_S | 4.9 | | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-IQ2_M.gguf) | i1-IQ2_M | 5.3 | | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-Q2_K_S.gguf) | i1-Q2_K_S | 5.3 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-Q2_K.gguf) | i1-Q2_K | 5.7 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 5.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-IQ3_XS.gguf) | i1-IQ3_XS | 6.3 | | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-Q3_K_S.gguf) | i1-Q3_K_S | 6.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-IQ3_S.gguf) | i1-IQ3_S | 6.6 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-IQ3_M.gguf) | i1-IQ3_M | 6.8 | | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-Q3_K_M.gguf) | i1-Q3_K_M | 7.3 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-Q3_K_L.gguf) | i1-Q3_K_L | 7.9 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-IQ4_XS.gguf) | i1-IQ4_XS | 8.1 | | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-Q4_0.gguf) | i1-Q4_0 | 8.5 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-IQ4_NL.gguf) | i1-IQ4_NL | 8.5 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-Q4_K_S.gguf) | i1-Q4_K_S | 8.5 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-Q4_K_M.gguf) | i1-Q4_K_M | 9.0 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-Q4_1.gguf) | i1-Q4_1 | 9.4 | | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-Q5_K_S.gguf) | i1-Q5_K_S | 10.3 | | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-Q5_K_M.gguf) | i1-Q5_K_M | 10.5 | | +| [GGUF](https://huggingface.co/mradermacher/Phi-4Garu-telugu-14b-i1-GGUF/resolve/main/Phi-4Garu-telugu-14b.i1-Q6_K.gguf) | i1-Q6_K | 12.1 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..9305021 --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d9f239d8834f1f5100d85a3bcdfcf71395e7ea2e02d015c94e395d22f16b9c31 +size 7791685