Generate Perplexity, KLD, ARC, HellaSwag, MMLU, Truthful QA and WinoGrande scores

scores/Dolphin-Mistral-24B-Venice-Edition-iq4_nl.ppx

@@ -0,0 +1,37 @@
+====== Perplexity statistics ======
+Mean PPL(Q)                   :  18.783744 ±   0.146959
+Mean PPL(base)                :   6.180577 ±   0.041038
+Cor(ln(PPL(Q)), ln(PPL(base))):  74.79%
+Mean ln(PPL(Q)/PPL(base))     :   1.111580 ±   0.005253
+Mean PPL(Q)/PPL(base)         :   3.039157 ±   0.015966
+Mean PPL(Q)-PPL(base)         :  12.603167 ±   0.119417
+
+====== KL divergence statistics ======
+Mean    KLD:   1.199318 ±   0.004258
+Maximum KLD:  26.543749
+99.9%   KLD:  14.340773
+99.0%   KLD:   8.742259
+99.0%   KLD:   8.742259
+Median  KLD:   0.715601
+10.0%   KLD:   0.071172
+ 5.0%   KLD:   0.025864
+ 1.0%   KLD:   0.004589
+Minimum KLD:   0.000142
+
+====== Token probability statistics ======
+Mean    Δp: -17.171 ± 0.083 %
+Maximum Δp: 92.904%
+99.9%   Δp: 72.496%
+99.0%   Δp: 48.216%
+95.0%   Δp: 22.569%
+90.0%   Δp: 10.000%
+75.0%   Δp:  0.011%
+Median  Δp: -4.438%
+25.0%   Δp: -30.853%
+10.0%   Δp: -71.739%
+ 5.0%   Δp: -90.613%
+ 1.0%   Δp: -99.767%
+ 0.1%   Δp: -99.967%
+Minimum Δp: -99.998%
+RMS Δp    : 36.745 ± 0.088 %
+Same top p: 64.223 ± 0.123 %