commit 6927ed24328b95472f9b56fcec1e6e30fcdcdbb4 Author: ModelHub XC Date: Mon May 4 21:37:23 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..0d3dd46 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,81 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-instruct.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_8.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text +Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Athene-Phi-3.5-instruct.i1-IQ1_M.gguf b/Athene-Phi-3.5-instruct.i1-IQ1_M.gguf new file mode 100644 index 0000000..c9b701b --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a7c70297b5ed4d1c8ece4f20651a6a857f8019aa6574ec943697770199bd1fe6 +size 950143488 diff --git a/Athene-Phi-3.5-instruct.i1-IQ1_S.gguf b/Athene-Phi-3.5-instruct.i1-IQ1_S.gguf new file mode 100644 index 0000000..2352841 --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e555fddaf7c8ffb74ded9af791c31310fd1529657a5ea3e75179f4e95b0cd834 +size 881723904 diff --git a/Athene-Phi-3.5-instruct.i1-IQ2_M.gguf b/Athene-Phi-3.5-instruct.i1-IQ2_M.gguf new file mode 100644 index 0000000..c212bc9 --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:56ecc32df37ca9a8421dfff5608892058f47282b58e39e77e68cd603b3df78d9 +size 1349431296 diff --git a/Athene-Phi-3.5-instruct.i1-IQ2_S.gguf b/Athene-Phi-3.5-instruct.i1-IQ2_S.gguf new file mode 100644 index 0000000..aa0300a --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1c431e17fb4cc5c5eea2bc97d11e32b681fa3b6f1cf1983d0de1a329484d7955 +size 1258205184 diff --git a/Athene-Phi-3.5-instruct.i1-IQ2_XS.gguf b/Athene-Phi-3.5-instruct.i1-IQ2_XS.gguf new file mode 100644 index 0000000..f8af9d1 --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:05b1b1743f2213b7bc571c7ff42959c726ed2d06441a5a15b4f95cbbf607c00c +size 1164839424 diff --git a/Athene-Phi-3.5-instruct.i1-IQ2_XXS.gguf b/Athene-Phi-3.5-instruct.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..70964b7 --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b784e43d470e3dd96715b6165294c4f0caa98350e8cc8a656d5db2bd7d692db5 +size 1064176128 diff --git a/Athene-Phi-3.5-instruct.i1-IQ3_M.gguf b/Athene-Phi-3.5-instruct.i1-IQ3_M.gguf new file mode 100644 index 0000000..5b70308 --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d6bebe1e2331e8d461c077063bb691be5ee394847fcfc856d8f61a2ebfb56baa +size 1775390208 diff --git a/Athene-Phi-3.5-instruct.i1-IQ3_S.gguf b/Athene-Phi-3.5-instruct.i1-IQ3_S.gguf new file mode 100644 index 0000000..d19a05f --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6965d896742ecb9d8bb970ff1c53326e97da6158451ac618f28ae79d978bfd3b +size 1681804800 diff --git a/Athene-Phi-3.5-instruct.i1-IQ3_XS.gguf b/Athene-Phi-3.5-instruct.i1-IQ3_XS.gguf new file mode 100644 index 0000000..b345676 --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b3e47566c9533a2f835a5a9f79435eee2b443dc7795dd0384ef75df908eb0292 +size 1596870144 diff --git a/Athene-Phi-3.5-instruct.i1-IQ3_XXS.gguf b/Athene-Phi-3.5-instruct.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..2e92cac --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1d1996bfbac6f15669c6568ad5dc1969c5b973ba730b30a74213279829b7cc38 +size 1475260416 diff --git a/Athene-Phi-3.5-instruct.i1-IQ4_XS.gguf b/Athene-Phi-3.5-instruct.i1-IQ4_XS.gguf new file mode 100644 index 0000000..b01560d --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fea2b77a8d6d2c6dcac8f1362c72bdc7e5834d5f104857b4d7f455d92edac310 +size 2059858944 diff --git a/Athene-Phi-3.5-instruct.i1-Q2_K.gguf b/Athene-Phi-3.5-instruct.i1-Q2_K.gguf new file mode 100644 index 0000000..fcf503a --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6b3a00f875aab335f9c979d7434bead39524fe7b75ff75f6bb09bb562a59bdd0 +size 1446881280 diff --git a/Athene-Phi-3.5-instruct.i1-Q3_K_L.gguf b/Athene-Phi-3.5-instruct.i1-Q3_K_L.gguf new file mode 100644 index 0000000..0342960 --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37d76782a5823896d1edc81afa2ed24fa30f2649963fde9f2864ae7b0e6c1d3f +size 2045136384 diff --git a/Athene-Phi-3.5-instruct.i1-Q3_K_M.gguf b/Athene-Phi-3.5-instruct.i1-Q3_K_M.gguf new file mode 100644 index 0000000..fc982f4 --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3b045fa2525b77df5ca3744d48f06326707b8520164dc8f9ff986904744909db +size 1877626368 diff --git a/Athene-Phi-3.5-instruct.i1-Q3_K_S.gguf b/Athene-Phi-3.5-instruct.i1-Q3_K_S.gguf new file mode 100644 index 0000000..e685bec --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fd7aeb5279648b65a34e7fa001e8b79704ea86e550f0e108268f9212616fd597 +size 1681804800 diff --git a/Athene-Phi-3.5-instruct.i1-Q4_0.gguf b/Athene-Phi-3.5-instruct.i1-Q4_0.gguf new file mode 100644 index 0000000..e07997a --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4f33606110ee8bde4b26028f39703b6c23b25b1cc88b690f5ab6610d28ed493b +size 2182474752 diff --git a/Athene-Phi-3.5-instruct.i1-Q4_K_M.gguf b/Athene-Phi-3.5-instruct.i1-Q4_K_M.gguf new file mode 100644 index 0000000..b0671ec --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a8349f262abc5619125222353492ff073e174c0caa91b7eb9ebd47bd493e6fea +size 2318920704 diff --git a/Athene-Phi-3.5-instruct.i1-Q4_K_S.gguf b/Athene-Phi-3.5-instruct.i1-Q4_K_S.gguf new file mode 100644 index 0000000..0d50be4 --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8f63f1e2cffff0f42c461e7573b56df574863a8743a52833c221186a667d6c8e +size 2193484800 diff --git a/Athene-Phi-3.5-instruct.i1-Q5_K_M.gguf b/Athene-Phi-3.5-instruct.i1-Q5_K_M.gguf new file mode 100644 index 0000000..94c15f0 --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7bbf29ea869d8e98ff7e35429c640f3580b2b61d0da9aa73da6bab2f1424a9c9 +size 2715012096 diff --git a/Athene-Phi-3.5-instruct.i1-Q5_K_S.gguf b/Athene-Phi-3.5-instruct.i1-Q5_K_S.gguf new file mode 100644 index 0000000..d766c9f --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e98e9b0d14c9bfd11fc698dec1b5240912bb495e7514d4efb0a148fbd82fc9c9 +size 2641480704 diff --git a/Athene-Phi-3.5-instruct.i1-Q6_K.gguf b/Athene-Phi-3.5-instruct.i1-Q6_K.gguf new file mode 100644 index 0000000..d34565b --- /dev/null +++ b/Athene-Phi-3.5-instruct.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7c42c5d9fada3d0fdc7957468e5a869cb9cc30546b64eb9244ddb52ebc943855 +size 3135859200 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_M.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_M.gguf new file mode 100644 index 0000000..e3623cd --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f89e7c751d73efe968e5361e091dca432bd2f8eb0ff11aefb28c3abce959b75c +size 950143552 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_S.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_S.gguf new file mode 100644 index 0000000..819bdee --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3fe491bbb24ee5a7e4f957e8c1f70dc232c29a60722fdb9ec2db3f139cd56f58 +size 881723968 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_M.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_M.gguf new file mode 100644 index 0000000..670756d --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c8d1be92c7071c1211950e56b65469f324dcf9ecfb527c02ee04876521b84818 +size 1349431360 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_S.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_S.gguf new file mode 100644 index 0000000..6e0d183 --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3f0e5eacdbb48dec528a2f94bf3851082218630fd572b0dae33728dfd76846c +size 1258205248 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XS.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XS.gguf new file mode 100644 index 0000000..8da8433 --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:98227a8f2292c88ecd3d4f4f5f2a1f33cb746631ef892a7554716ca77403c599 +size 1164839488 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XXS.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..b84d378 --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0949ebd4d5299d7202cd193269e630e450c7c0599257e8ece024e5ac1385db2e +size 1064176192 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_M.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_M.gguf new file mode 100644 index 0000000..da0b085 --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:136a1e899ad2af1d265435a001f07cf8e616d521259f0c9bd1a3716c893b8148 +size 1775390272 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_S.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_S.gguf new file mode 100644 index 0000000..8692f2e --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8f15197de0db488d95fc843d8ad060b98e0c47f27577aeea0882229c65321e40 +size 1681804864 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XS.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XS.gguf new file mode 100644 index 0000000..26db968 --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2d2637cc7b7d2ceb9d98619de2c60c1eef721d0ce6ca50b308eaff4d84798e83 +size 1596870208 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XXS.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..2d2fbaa --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:485a9a2f5861b1aa3e5886bda665a1a195ec1d1cb945c1c7926a53ba9a0f9ed3 +size 1475260480 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ4_XS.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ4_XS.gguf new file mode 100644 index 0000000..6d3486f --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2aee88cedcfc5e36b09dfeedd9de1a3b8bd09facf9bdc75cf55ded611b7ef7be +size 2059859008 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-Q2_K.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q2_K.gguf new file mode 100644 index 0000000..1bb51ef --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6d315bc5ec740ccda4b60e43e83e2ede7d24e28b736f97535d667289e0166cbc +size 1446881344 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_L.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_L.gguf new file mode 100644 index 0000000..5316103 --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a1aec550e50da0ef2e971638801491d6a384df467a52b5c401b88026130f6ddd +size 2045136448 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_M.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_M.gguf new file mode 100644 index 0000000..35b05a8 --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e50dd461f726962dd324f1981fca5ff62f10437268ac74e18ab8d0c3c84b8631 +size 1877626432 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_S.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_S.gguf new file mode 100644 index 0000000..b69ffa2 --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:95ab2dd44512808df70fbfb463b86af7925cd5464673da5e415b6cbf8fa37b45 +size 1681804864 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0.gguf new file mode 100644 index 0000000..209faec --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6e5dfbb8c63674ffe09a3749a0829ed90e40261964a9c2c118c275d8d9109230 +size 2182474816 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_4.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_4.gguf new file mode 100644 index 0000000..39091dc --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_4.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4a75f15bd4c102c860a6fe27fc4e42fd638220861826df5e47d1cd4b48b4a3de +size 2176183360 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_8.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_8.gguf new file mode 100644 index 0000000..4bdd401 --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_8.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:53264bf2ea454aae6e3258216d34b50613bdca62254b669a5ec0d31945d6453c +size 2176183360 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_8_8.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_8_8.gguf new file mode 100644 index 0000000..91a613b --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_8_8.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ded7e9e1bceb3a5a8a26f38b30974aec73144d437112012e78d4fbf8eec8b266 +size 2176183360 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_M.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_M.gguf new file mode 100644 index 0000000..cb7e1ba --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:412ccd9a8331124e877a45c3c944175b7614a90a7d375ba14afb691781c50758 +size 2318920768 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_S.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_S.gguf new file mode 100644 index 0000000..291cbc9 --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7fee233b7a9897cf85278304bc6c2cf1148e6f17ed713321abe3a2b14943d95f +size 2193484864 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_M.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_M.gguf new file mode 100644 index 0000000..74fb624 --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:baef4624e11afc1b1558caaf2cb3c6a26852f1e4ac7ae2839ff920a7a5750e37 +size 2715012160 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_S.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_S.gguf new file mode 100644 index 0000000..d27f85b --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6cd1319f0b1293a203834d68f1dbad5e7c436ea4c1d7d29a9deb89fe9c71234a +size 2641480768 diff --git a/Athene-Phi-3.5-mini-instruct-orpo.i1-Q6_K.gguf b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q6_K.gguf new file mode 100644 index 0000000..c03e9a6 --- /dev/null +++ b/Athene-Phi-3.5-mini-instruct-orpo.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8fff247d8a75ca1e5853eaaf5af5438d0794ae8ab6f4dadef9bb4a59d54d4c62 +size 3135859264 diff --git a/README.md b/README.md new file mode 100644 index 0000000..8ae5f9a --- /dev/null +++ b/README.md @@ -0,0 +1,82 @@ +--- +base_model: EpistemeAI/Athene-Phi-3.5-mini-instruct-orpo +language: +- en +library_name: transformers +license: apache-2.0 +quantized_by: mradermacher +tags: +- text-generation-inference +- transformers +- unsloth +- llama +- trl +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/EpistemeAI/Athene-Phi-3.5-mini-instruct-orpo + + +static quants are available at https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ1_S.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_S.gguf) | i1-IQ1_S | 1.9 | for the desperate | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ1_M.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_M.gguf) | i1-IQ1_M | 2.0 | mostly desperate | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ2_XXS.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.2 | | +| [GGUF](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_4.gguf) | i1-Q4_0_4_4 | 2.3 | fast on arm, low quality | +| [GGUF](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_8.gguf) | i1-Q4_0_4_8 | 2.3 | fast on arm+i8mm, low quality | +| [GGUF](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_8_8.gguf) | i1-Q4_0_8_8 | 2.3 | fast on arm+sve, low quality | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ2_XS.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.4 | | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ2_S.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_S.gguf) | i1-IQ2_S | 2.6 | | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ2_M.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_M.gguf) | i1-IQ2_M | 2.8 | | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q2_K.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q2_K.gguf) | i1-Q2_K | 3.0 | IQ3_XXS probably better | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ3_XXS.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.1 | lower quality | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ3_XS.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.3 | | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ3_S.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_S.gguf) | i1-IQ3_S | 3.5 | beats Q3_K* | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q3_K_S.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.5 | IQ3_XS probably better | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ3_M.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_M.gguf) | i1-IQ3_M | 3.7 | | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q3_K_M.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_M.gguf) | i1-Q3_K_M | 3.9 | IQ3_S probably better | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q3_K_L.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.2 | IQ3_M probably better | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ4_XS.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.2 | | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q4_0.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0.gguf) | i1-Q4_0 | 4.5 | fast, low quality | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q4_K_S.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.5 | optimal size/speed/quality | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q4_K_M.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_M.gguf) | i1-Q4_K_M | 4.7 | fast, recommended | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q5_K_S.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.4 | | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q5_K_M.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.5 | | +| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q6_K.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q6_K.gguf) | i1-Q6_K | 6.4 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..af35695 --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e134916da118213559b3a2c446de64cc4b075786e70c10b5fc3e6dc3b575e388 +size 3415293