Generate Perplexity, KLD, ARC, HellaSwag, MMLU, Truthful QA and WinoGrande scores

scores/Dolphin-Mistral-24B-Venice-Edition-iq3_m.ppx

@@ -0,0 +1,37 @@
+====== Perplexity statistics ======
+Mean PPL(Q)                   :  20.379006 ±   0.160275
+Mean PPL(base)                :   6.180577 ±   0.041038
+Cor(ln(PPL(Q)), ln(PPL(base))):  73.93%
+Mean ln(PPL(Q)/PPL(base))     :   1.193094 ±   0.005360
+Mean PPL(Q)/PPL(base)         :   3.297266 ±   0.017673
+Mean PPL(Q)-PPL(base)         :  14.198428 ±   0.132841
+
+====== KL divergence statistics ======
+Mean    KLD:   1.290608 ±   0.004304
+Maximum KLD:  27.217335
+99.9%   KLD:  13.970652
+99.0%   KLD:   8.700209
+99.0%   KLD:   8.700209
+Median  KLD:   0.800781
+10.0%   KLD:   0.078073
+ 5.0%   KLD:   0.028142
+ 1.0%   KLD:   0.004895
+Minimum KLD:   0.000072
+
+====== Token probability statistics ======
+Mean    Δp: -18.226 ± 0.085 %
+Maximum Δp: 95.351%
+99.9%   Δp: 73.378%
+99.0%   Δp: 48.957%
+95.0%   Δp: 22.742%
+90.0%   Δp:  9.547%
+75.0%   Δp:  0.003%
+Median  Δp: -5.000%
+25.0%   Δp: -33.511%
+10.0%   Δp: -75.041%
+ 5.0%   Δp: -91.958%
+ 1.0%   Δp: -99.782%
+ 0.1%   Δp: -99.968%
+Minimum Δp: -99.998%
+RMS Δp    : 37.928 ± 0.088 %
+Same top p: 62.059 ± 0.125 %