Generate Perplexity, KLD, ARC, HellaSwag, MMLU, Truthful QA and WinoGrande scores

scores/Dolphin-Mistral-24B-Venice-Edition-iq3_s.ppx

@@ -0,0 +1,37 @@
+====== Perplexity statistics ======
+Mean PPL(Q)                   :  21.165413 ±   0.164512
+Mean PPL(base)                :   6.180577 ±   0.041038
+Cor(ln(PPL(Q)), ln(PPL(base))):  73.80%
+Mean ln(PPL(Q)/PPL(base))     :   1.230957 ±   0.005322
+Mean PPL(Q)/PPL(base)         :   3.424504 ±   0.018226
+Mean PPL(Q)-PPL(base)         :  14.984836 ±   0.137053
+
+====== KL divergence statistics ======
+Mean    KLD:   1.340446 ±   0.004301
+Maximum KLD:  26.031479
+99.9%   KLD:  13.794025
+99.0%   KLD:   8.598367
+99.0%   KLD:   8.598367
+Median  KLD:   0.843369
+10.0%   KLD:   0.087419
+ 5.0%   KLD:   0.032046
+ 1.0%   KLD:   0.005851
+Minimum KLD:   0.000204
+
+====== Token probability statistics ======
+Mean    Δp: -19.454 ± 0.086 %
+Maximum Δp: 95.317%
+99.9%   Δp: 71.779%
+99.0%   Δp: 46.392%
+95.0%   Δp: 20.287%
+90.0%   Δp:  7.866%
+75.0%   Δp: -0.004%
+Median  Δp: -5.901%
+25.0%   Δp: -35.629%
+10.0%   Δp: -76.898%
+ 5.0%   Δp: -92.591%
+ 1.0%   Δp: -99.784%
+ 0.1%   Δp: -99.968%
+Minimum Δp: -99.998%
+RMS Δp    : 38.586 ± 0.088 %
+Same top p: 61.705 ± 0.125 %