38 lines
1.0 KiB
Plaintext
38 lines
1.0 KiB
Plaintext
====== Perplexity statistics ======
|
|
Mean PPL(Q) : 21.165413 ± 0.164512
|
|
Mean PPL(base) : 6.180577 ± 0.041038
|
|
Cor(ln(PPL(Q)), ln(PPL(base))): 73.80%
|
|
Mean ln(PPL(Q)/PPL(base)) : 1.230957 ± 0.005322
|
|
Mean PPL(Q)/PPL(base) : 3.424504 ± 0.018226
|
|
Mean PPL(Q)-PPL(base) : 14.984836 ± 0.137053
|
|
|
|
====== KL divergence statistics ======
|
|
Mean KLD: 1.340446 ± 0.004301
|
|
Maximum KLD: 26.031479
|
|
99.9% KLD: 13.794025
|
|
99.0% KLD: 8.598367
|
|
99.0% KLD: 8.598367
|
|
Median KLD: 0.843369
|
|
10.0% KLD: 0.087419
|
|
5.0% KLD: 0.032046
|
|
1.0% KLD: 0.005851
|
|
Minimum KLD: 0.000204
|
|
|
|
====== Token probability statistics ======
|
|
Mean Δp: -19.454 ± 0.086 %
|
|
Maximum Δp: 95.317%
|
|
99.9% Δp: 71.779%
|
|
99.0% Δp: 46.392%
|
|
95.0% Δp: 20.287%
|
|
90.0% Δp: 7.866%
|
|
75.0% Δp: -0.004%
|
|
Median Δp: -5.901%
|
|
25.0% Δp: -35.629%
|
|
10.0% Δp: -76.898%
|
|
5.0% Δp: -92.591%
|
|
1.0% Δp: -99.784%
|
|
0.1% Δp: -99.968%
|
|
Minimum Δp: -99.998%
|
|
RMS Δp : 38.586 ± 0.088 %
|
|
Same top p: 61.705 ± 0.125 %
|