mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-04 21:37:23 +08:00
commit 6927ed2432
48 changed files with 301 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

81
.gitattributes vendored Normal file
View File

@@ -0,0 +1,81 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-instruct.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_8.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text

3
Athene-Phi-3.5-instruct.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a7c70297b5ed4d1c8ece4f20651a6a857f8019aa6574ec943697770199bd1fe6
size 950143488

3
Athene-Phi-3.5-instruct.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e555fddaf7c8ffb74ded9af791c31310fd1529657a5ea3e75179f4e95b0cd834
size 881723904

3
Athene-Phi-3.5-instruct.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:56ecc32df37ca9a8421dfff5608892058f47282b58e39e77e68cd603b3df78d9
size 1349431296

3
Athene-Phi-3.5-instruct.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1c431e17fb4cc5c5eea2bc97d11e32b681fa3b6f1cf1983d0de1a329484d7955
size 1258205184

3
Athene-Phi-3.5-instruct.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:05b1b1743f2213b7bc571c7ff42959c726ed2d06441a5a15b4f95cbbf607c00c
size 1164839424

3
Athene-Phi-3.5-instruct.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b784e43d470e3dd96715b6165294c4f0caa98350e8cc8a656d5db2bd7d692db5
size 1064176128

3
Athene-Phi-3.5-instruct.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d6bebe1e2331e8d461c077063bb691be5ee394847fcfc856d8f61a2ebfb56baa
size 1775390208

3
Athene-Phi-3.5-instruct.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6965d896742ecb9d8bb970ff1c53326e97da6158451ac618f28ae79d978bfd3b
size 1681804800

3
Athene-Phi-3.5-instruct.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b3e47566c9533a2f835a5a9f79435eee2b443dc7795dd0384ef75df908eb0292
size 1596870144

3
Athene-Phi-3.5-instruct.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1d1996bfbac6f15669c6568ad5dc1969c5b973ba730b30a74213279829b7cc38
size 1475260416

3
Athene-Phi-3.5-instruct.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fea2b77a8d6d2c6dcac8f1362c72bdc7e5834d5f104857b4d7f455d92edac310
size 2059858944

3
Athene-Phi-3.5-instruct.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6b3a00f875aab335f9c979d7434bead39524fe7b75ff75f6bb09bb562a59bdd0
size 1446881280

3
Athene-Phi-3.5-instruct.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:37d76782a5823896d1edc81afa2ed24fa30f2649963fde9f2864ae7b0e6c1d3f
size 2045136384

3
Athene-Phi-3.5-instruct.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3b045fa2525b77df5ca3744d48f06326707b8520164dc8f9ff986904744909db
size 1877626368

3
Athene-Phi-3.5-instruct.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fd7aeb5279648b65a34e7fa001e8b79704ea86e550f0e108268f9212616fd597
size 1681804800

3
Athene-Phi-3.5-instruct.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4f33606110ee8bde4b26028f39703b6c23b25b1cc88b690f5ab6610d28ed493b
size 2182474752

3
Athene-Phi-3.5-instruct.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a8349f262abc5619125222353492ff073e174c0caa91b7eb9ebd47bd493e6fea
size 2318920704

3
Athene-Phi-3.5-instruct.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8f63f1e2cffff0f42c461e7573b56df574863a8743a52833c221186a667d6c8e
size 2193484800

3
Athene-Phi-3.5-instruct.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7bbf29ea869d8e98ff7e35429c640f3580b2b61d0da9aa73da6bab2f1424a9c9
size 2715012096

3
Athene-Phi-3.5-instruct.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e98e9b0d14c9bfd11fc698dec1b5240912bb495e7514d4efb0a148fbd82fc9c9
size 2641480704

3
Athene-Phi-3.5-instruct.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7c42c5d9fada3d0fdc7957468e5a869cb9cc30546b64eb9244ddb52ebc943855
size 3135859200

3
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f89e7c751d73efe968e5361e091dca432bd2f8eb0ff11aefb28c3abce959b75c
size 950143552

3
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3fe491bbb24ee5a7e4f957e8c1f70dc232c29a60722fdb9ec2db3f139cd56f58
size 881723968

3
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c8d1be92c7071c1211950e56b65469f324dcf9ecfb527c02ee04876521b84818
size 1349431360

3
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f3f0e5eacdbb48dec528a2f94bf3851082218630fd572b0dae33728dfd76846c
size 1258205248

3
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:98227a8f2292c88ecd3d4f4f5f2a1f33cb746631ef892a7554716ca77403c599
size 1164839488

3
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0949ebd4d5299d7202cd193269e630e450c7c0599257e8ece024e5ac1385db2e
size 1064176192

3
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:136a1e899ad2af1d265435a001f07cf8e616d521259f0c9bd1a3716c893b8148
size 1775390272

3
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8f15197de0db488d95fc843d8ad060b98e0c47f27577aeea0882229c65321e40
size 1681804864

3
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2d2637cc7b7d2ceb9d98619de2c60c1eef721d0ce6ca50b308eaff4d84798e83
size 1596870208

3
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:485a9a2f5861b1aa3e5886bda665a1a195ec1d1cb945c1c7926a53ba9a0f9ed3
size 1475260480

3
Athene-Phi-3.5-mini-instruct-orpo.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2aee88cedcfc5e36b09dfeedd9de1a3b8bd09facf9bdc75cf55ded611b7ef7be
size 2059859008

3
Athene-Phi-3.5-mini-instruct-orpo.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6d315bc5ec740ccda4b60e43e83e2ede7d24e28b736f97535d667289e0166cbc
size 1446881344

3
Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a1aec550e50da0ef2e971638801491d6a384df467a52b5c401b88026130f6ddd
size 2045136448

3
Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e50dd461f726962dd324f1981fca5ff62f10437268ac74e18ab8d0c3c84b8631
size 1877626432

3
Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:95ab2dd44512808df70fbfb463b86af7925cd5464673da5e415b6cbf8fa37b45
size 1681804864

3
Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6e5dfbb8c63674ffe09a3749a0829ed90e40261964a9c2c118c275d8d9109230
size 2182474816

3
Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_4.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4a75f15bd4c102c860a6fe27fc4e42fd638220861826df5e47d1cd4b48b4a3de
size 2176183360

3
Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_8.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:53264bf2ea454aae6e3258216d34b50613bdca62254b669a5ec0d31945d6453c
size 2176183360

3
Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_8_8.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ded7e9e1bceb3a5a8a26f38b30974aec73144d437112012e78d4fbf8eec8b266
size 2176183360

3
Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:412ccd9a8331124e877a45c3c944175b7614a90a7d375ba14afb691781c50758
size 2318920768

3
Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7fee233b7a9897cf85278304bc6c2cf1148e6f17ed713321abe3a2b14943d95f
size 2193484864

3
Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:baef4624e11afc1b1558caaf2cb3c6a26852f1e4ac7ae2839ff920a7a5750e37
size 2715012160

3
Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6cd1319f0b1293a203834d68f1dbad5e7c436ea4c1d7d29a9deb89fe9c71234a
size 2641480768

3
Athene-Phi-3.5-mini-instruct-orpo.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8fff247d8a75ca1e5853eaaf5af5438d0794ae8ab6f4dadef9bb4a59d54d4c62
size 3135859264

82
README.md Normal file
View File

@@ -0,0 +1,82 @@
---
base_model: EpistemeAI/Athene-Phi-3.5-mini-instruct-orpo
language:
- en
library_name: transformers
license: apache-2.0
quantized_by: mradermacher
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
weighted/imatrix quants of https://huggingface.co/EpistemeAI/Athene-Phi-3.5-mini-instruct-orpo
<!-- provided-files -->
static quants are available at https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ1_S.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_S.gguf) | i1-IQ1_S | 1.9 | for the desperate |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ1_M.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ1_M.gguf) | i1-IQ1_M | 2.0 | mostly desperate |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ2_XXS.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.2 | |
| [GGUF](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_4.gguf) | i1-Q4_0_4_4 | 2.3 | fast on arm, low quality |
| [GGUF](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_4_8.gguf) | i1-Q4_0_4_8 | 2.3 | fast on arm+i8mm, low quality |
| [GGUF](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0_8_8.gguf) | i1-Q4_0_8_8 | 2.3 | fast on arm+sve, low quality |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ2_XS.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.4 | |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ2_S.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_S.gguf) | i1-IQ2_S | 2.6 | |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ2_M.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ2_M.gguf) | i1-IQ2_M | 2.8 | |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q2_K.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q2_K.gguf) | i1-Q2_K | 3.0 | IQ3_XXS probably better |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ3_XXS.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.1 | lower quality |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ3_XS.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.3 | |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ3_S.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_S.gguf) | i1-IQ3_S | 3.5 | beats Q3_K* |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q3_K_S.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.5 | IQ3_XS probably better |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ3_M.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ3_M.gguf) | i1-IQ3_M | 3.7 | |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q3_K_M.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_M.gguf) | i1-Q3_K_M | 3.9 | IQ3_S probably better |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q3_K_L.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.2 | IQ3_M probably better |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-IQ4_XS.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.2 | |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q4_0.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_0.gguf) | i1-Q4_0 | 4.5 | fast, low quality |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q4_K_S.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.5 | optimal size/speed/quality |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q4_K_M.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q4_K_M.gguf) | i1-Q4_K_M | 4.7 | fast, recommended |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q5_K_S.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.4 | |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q5_K_M.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.5 | |
| [PART 1](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-instruct.i1-Q6_K.gguf) [PART 2](https://huggingface.co/mradermacher/Athene-Phi-3.5-mini-instruct-orpo-i1-GGUF/resolve/main/Athene-Phi-3.5-mini-instruct-orpo.i1-Q6_K.gguf) | i1-Q6_K | 6.4 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e134916da118213559b3a2c446de64cc4b075786e70c10b5fc3e6dc3b575e388
size 3415293