333 lines
38 KiB
Plaintext
333 lines
38 KiB
Plaintext
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 124610 MiB):
|
||
Device 0: NVIDIA GB10, compute capability 12.1, VMM: yes, VRAM: 124610 MiB
|
||
|
||
Computing statistics for imatrix/imatrix-granite-4.1-8b-medium.gguf (280 tensors)
|
||
|
||
Layer Tensor Σ(Act²) Min Max μ σ % Active N Entropy E (norm) ZD CosSim
|
||
=========================================================================================================================================================================
|
||
39 attn_k 174643.00 2.0243 18914.7031 42.64 497.38 100.00% 4096 8.8904 74.09% 0.37% 0.8865
|
||
38 attn_k 166087.69 0.0000 20050.7402 40.55 448.81 99.98% 4096 8.6133 71.78% 0.46% 0.9835
|
||
37 attn_k 146614.73 0.7595 20756.5176 35.79 423.56 100.00% 4096 8.3834 69.86% 0.44% 0.9842
|
||
36 attn_k 134513.23 0.5200 19003.3398 32.84 386.52 100.00% 4096 8.2079 68.40% 0.51% 0.9501
|
||
35 attn_k 124035.38 1.3198 11533.5068 30.28 305.84 100.00% 4096 8.2914 69.09% 0.71% 0.9665
|
||
34 attn_k 123896.47 0.5387 17763.3496 30.25 472.60 100.00% 4096 6.6833 55.69% 0.44% 0.9800
|
||
32 attn_k 111128.92 0.1590 25305.4961 27.13 504.76 100.00% 4096 6.2867 52.39% 0.37% 0.9931
|
||
30 attn_k 109705.93 0.0000 33505.8359 26.78 617.29 99.98% 4096 5.4138 45.11% 0.27% 0.9759
|
||
33 attn_k 109045.48 0.9963 20537.8262 26.62 448.68 100.00% 4096 6.6424 55.35% 0.37% 0.9914
|
||
29 attn_k 87316.21 0.0000 20308.4004 21.32 398.84 99.95% 4096 6.1991 51.66% 0.37% 0.9936
|
||
31 attn_k 87185.37 0.0000 22471.5254 21.29 421.05 99.98% 4096 6.2130 51.78% 0.34% 0.9963
|
||
28 attn_k 82016.98 0.0000 23024.2656 20.02 430.64 99.95% 4096 5.6830 47.36% 0.29% 0.9976
|
||
27 attn_k 76821.73 0.0000 17911.5156 18.76 348.43 99.98% 4096 6.3444 52.87% 0.34% 0.9791
|
||
16 attn_k 76133.66 0.0000 11891.1016 18.59 311.73 99.93% 4096 6.1208 51.01% 0.42% 0.9643
|
||
23 attn_k 74987.37 0.0000 18502.6172 18.31 356.15 99.95% 4096 6.1167 50.97% 0.39% 0.9980
|
||
22 attn_k 71226.52 0.0000 18282.5273 17.39 343.00 99.98% 4096 6.2861 52.38% 0.32% 0.9901
|
||
21 attn_k 69479.59 0.0000 14531.5439 16.96 297.70 99.95% 4096 6.6280 55.23% 0.39% 0.9882
|
||
20 attn_k 68286.74 0.0000 18747.7070 16.67 345.57 99.95% 4096 6.0506 50.42% 0.34% 0.9917
|
||
25 attn_k 68221.13 0.0000 14562.2783 16.66 294.34 99.98% 4096 6.3805 53.17% 0.39% 0.9829
|
||
7 attn_k 68092.71 0.0000 17005.3926 16.62 348.43 99.90% 4096 5.9826 49.85% 0.32% 0.9814
|
||
18 attn_k 66295.24 0.0000 12255.0391 16.19 262.31 99.95% 4096 6.9157 57.63% 0.42% 0.9978
|
||
26 attn_k 65291.02 0.0000 12545.3467 15.94 268.54 99.98% 4096 6.6067 55.06% 0.39% 0.9920
|
||
13 attn_k 59896.29 0.0000 8712.0469 14.62 232.75 99.90% 4096 6.9162 57.63% 0.32% 0.9786
|
||
24 attn_k 56921.47 0.0000 9241.3613 13.90 213.49 99.98% 4096 7.0602 58.84% 0.54% 0.9757
|
||
17 attn_k 56760.55 0.0000 10113.2490 13.86 223.49 99.95% 4096 6.7278 56.06% 0.44% 0.9569
|
||
15 attn_k 55194.11 0.0000 9697.7783 13.48 229.94 99.93% 4096 5.9460 49.55% 0.37% 0.9288
|
||
14 attn_k 54794.45 0.0000 8291.7021 13.38 212.09 99.90% 4096 7.0371 58.64% 0.37% 0.9885
|
||
6 attn_k 53044.36 0.0000 16721.8047 12.95 321.79 99.88% 4096 4.5291 37.74% 0.32% 0.9479
|
||
19 attn_k 52733.22 0.0000 11920.8223 12.87 233.97 99.95% 4096 6.5100 54.25% 0.39% 0.9902
|
||
8 attn_k 51986.04 0.0000 10429.2197 12.69 233.14 99.90% 4096 6.5500 54.58% 0.29% 0.9699
|
||
9 attn_k 49641.15 0.0000 10131.9502 12.12 241.91 99.90% 4096 5.8877 49.06% 0.34% 0.9950
|
||
12 attn_k 48607.36 0.0000 7781.5098 11.87 191.06 99.93% 4096 7.0521 58.77% 0.37% 0.9965
|
||
4 attn_k 46316.68 0.0000 16862.5684 11.31 309.35 99.85% 4096 5.2333 43.61% 0.22% 0.7166
|
||
5 attn_k 45620.26 0.0000 16766.0762 11.14 317.06 99.88% 4096 4.5011 37.51% 0.22% 0.9947
|
||
11 attn_k 45317.79 0.0000 6714.2949 11.06 172.61 99.93% 4096 7.2487 60.41% 0.37% 0.9866
|
||
10 attn_k 43706.77 0.0000 6960.2920 10.67 180.78 99.93% 4096 6.7438 56.20% 0.34% 0.9923
|
||
2 attn_k 19532.52 0.0000 6018.6182 4.77 99.81 99.90% 4096 7.6636 63.86% 0.20% 0.3212
|
||
3 attn_k 17324.65 0.0000 5779.3477 4.23 110.43 99.93% 4096 5.5684 46.40% 0.22% 0.9580
|
||
1 attn_k 15856.24 0.0000 5183.1831 3.87 86.07 98.66% 4096 6.6712 55.59% 0.22% 0.3483
|
||
0 attn_k 3049.93 0.0000 247.0233 0.74 7.67 97.14% 4096 6.8129 56.77% 1.12% 0.0000
|
||
39 attn_output 34614.42 1.0529 359.3260 8.45 11.78 100.00% 4096 11.3365 94.47% 7.59% 0.3770
|
||
38 attn_output 13420.36 0.2976 103.0346 3.28 4.13 100.00% 4096 11.3248 94.37% 8.98% 0.2804
|
||
37 attn_output 8831.00 0.2297 108.9257 2.16 3.37 100.00% 4096 11.2679 93.90% 5.49% 0.2283
|
||
36 attn_output 8013.26 0.1413 95.6638 1.96 3.93 100.00% 4096 11.0147 91.79% 4.32% 0.2457
|
||
35 attn_output 3675.13 0.1878 48.8232 0.90 1.65 100.00% 4096 11.2243 93.54% 4.52% 0.2357
|
||
34 attn_output 3483.17 0.0606 33.0821 0.85 1.59 100.00% 4096 11.1613 93.01% 3.96% 0.1202
|
||
33 attn_output 3199.42 0.0368 114.1278 0.78 2.50 100.00% 4096 10.6861 89.05% 3.47% 0.1774
|
||
31 attn_output 2234.35 0.0523 41.9673 0.55 1.30 100.00% 4096 10.8825 90.69% 5.00% 0.2217
|
||
1 attn_output 1997.94 0.0106 18.5997 0.49 0.98 100.00% 4096 10.5931 88.28% 10.03% 0.0200
|
||
2 attn_output 1738.62 0.0199 8.3539 0.42 0.55 100.00% 4096 11.1150 92.63% 16.77% 0.3314
|
||
32 attn_output 1524.28 0.0124 13.4450 0.37 0.70 100.00% 4096 11.1955 93.30% 3.03% 0.1768
|
||
30 attn_output 1314.30 0.0875 9.0597 0.32 0.51 100.00% 4096 11.3404 94.50% 3.64% 0.2404
|
||
28 attn_output 1114.76 0.0367 12.3692 0.27 0.65 100.00% 4096 10.8974 90.81% 3.59% 0.1806
|
||
3 attn_output 1080.53 0.0156 5.2009 0.26 0.36 100.00% 4096 11.1964 93.30% 10.28% 0.2966
|
||
24 attn_output 914.61 0.0400 6.5156 0.22 0.32 100.00% 4096 11.3443 94.54% 6.69% 0.1864
|
||
23 attn_output 900.86 0.0061 16.3921 0.22 0.55 100.00% 4096 10.9512 91.26% 3.27% 0.1833
|
||
29 attn_output 868.15 0.0168 8.5233 0.21 0.45 100.00% 4096 11.0238 91.87% 4.83% 0.1848
|
||
27 attn_output 855.07 0.0081 8.6770 0.21 0.41 100.00% 4096 11.1371 92.81% 3.39% 0.2051
|
||
22 attn_output 812.40 0.0144 7.8408 0.20 0.37 100.00% 4096 11.1740 93.12% 3.88% 0.2016
|
||
17 attn_output 711.99 0.0085 4.7093 0.17 0.28 100.00% 4096 11.1963 93.30% 4.81% 0.2343
|
||
21 attn_output 709.29 0.0000 10.5542 0.17 0.37 100.00% 4096 10.9892 91.58% 4.57% 0.1817
|
||
19 attn_output 707.10 0.0186 6.2341 0.17 0.23 100.00% 4096 11.4981 95.82% 3.22% 0.2283
|
||
20 attn_output 703.72 0.0083 5.4485 0.17 0.28 100.00% 4096 11.2945 94.12% 3.93% 0.2992
|
||
25 attn_output 656.68 0.0052 10.7580 0.16 0.38 100.00% 4096 11.0354 91.96% 2.08% 0.2214
|
||
15 attn_output 646.34 0.0052 8.5272 0.16 0.29 100.00% 4096 11.0779 92.32% 6.45% 0.2046
|
||
16 attn_output 610.13 0.0001 6.2649 0.15 0.35 100.00% 4096 10.7437 89.53% 3.83% 0.1555
|
||
26 attn_output 609.93 0.0232 5.7634 0.15 0.28 100.00% 4096 11.2354 93.63% 3.52% 0.1915
|
||
13 attn_output 548.35 0.0033 4.0648 0.13 0.21 100.00% 4096 10.8651 90.54% 12.60% 0.4666
|
||
18 attn_output 516.94 0.0000 10.2987 0.13 0.28 99.76% 4096 11.0928 92.44% 2.69% 0.2541
|
||
4 attn_output 453.05 0.0051 3.9059 0.11 0.15 100.00% 4096 11.3247 94.37% 8.91% 0.3225
|
||
0 attn_output 447.61 0.0008 91.7077 0.11 1.72 100.00% 4096 6.9468 57.89% 0.73% 0.0000
|
||
10 attn_output 439.24 0.0036 3.4716 0.11 0.26 100.00% 4096 10.4123 86.77% 7.06% 0.2021
|
||
11 attn_output 401.22 0.0051 1.9410 0.10 0.15 100.00% 4096 11.0397 92.00% 10.16% 0.1149
|
||
14 attn_output 350.10 0.0024 3.9673 0.09 0.14 100.00% 4096 11.0800 92.33% 6.81% 0.2386
|
||
5 attn_output 296.64 0.0028 3.5316 0.07 0.11 100.00% 4096 11.1694 93.08% 6.15% 0.2990
|
||
7 attn_output 240.15 0.0033 3.3802 0.06 0.11 100.00% 4096 11.1050 92.54% 5.96% 0.3540
|
||
6 attn_output 218.34 0.0015 1.1016 0.05 0.07 100.00% 4096 11.2492 93.74% 10.18% 0.2733
|
||
8 attn_output 208.74 0.0015 5.2777 0.05 0.12 100.00% 4096 10.9654 91.38% 5.47% 0.1650
|
||
12 attn_output 208.53 0.0032 2.3583 0.05 0.07 100.00% 4096 11.4232 95.19% 9.01% 0.3528
|
||
9 attn_output 151.65 0.0025 1.4596 0.04 0.05 100.00% 4096 11.3319 94.43% 6.32% 0.2434
|
||
39 attn_q 174643.00 2.0243 18914.7031 42.64 497.38 100.00% 4096 8.8904 74.09% 0.37% 0.8865
|
||
38 attn_q 166087.69 0.0000 20050.7402 40.55 448.81 99.98% 4096 8.6133 71.78% 0.46% 0.9835
|
||
37 attn_q 146614.73 0.7595 20756.5176 35.79 423.56 100.00% 4096 8.3834 69.86% 0.44% 0.9842
|
||
36 attn_q 134513.23 0.5200 19003.3398 32.84 386.52 100.00% 4096 8.2079 68.40% 0.51% 0.9501
|
||
35 attn_q 124035.38 1.3198 11533.5068 30.28 305.84 100.00% 4096 8.2914 69.09% 0.71% 0.9665
|
||
34 attn_q 123896.47 0.5387 17763.3496 30.25 472.60 100.00% 4096 6.6833 55.69% 0.44% 0.9800
|
||
32 attn_q 111128.92 0.1590 25305.4961 27.13 504.76 100.00% 4096 6.2867 52.39% 0.37% 0.9931
|
||
30 attn_q 109705.93 0.0000 33505.8359 26.78 617.29 99.98% 4096 5.4138 45.11% 0.27% 0.9759
|
||
33 attn_q 109045.48 0.9963 20537.8262 26.62 448.68 100.00% 4096 6.6424 55.35% 0.37% 0.9914
|
||
29 attn_q 87316.21 0.0000 20308.4004 21.32 398.84 99.95% 4096 6.1991 51.66% 0.37% 0.9936
|
||
31 attn_q 87185.37 0.0000 22471.5254 21.29 421.05 99.98% 4096 6.2130 51.78% 0.34% 0.9963
|
||
28 attn_q 82016.98 0.0000 23024.2656 20.02 430.64 99.95% 4096 5.6830 47.36% 0.29% 0.9976
|
||
27 attn_q 76821.73 0.0000 17911.5156 18.76 348.43 99.98% 4096 6.3444 52.87% 0.34% 0.9791
|
||
16 attn_q 76133.66 0.0000 11891.1016 18.59 311.73 99.93% 4096 6.1208 51.01% 0.42% 0.9643
|
||
23 attn_q 74987.37 0.0000 18502.6172 18.31 356.15 99.95% 4096 6.1167 50.97% 0.39% 0.9980
|
||
22 attn_q 71226.52 0.0000 18282.5273 17.39 343.00 99.98% 4096 6.2861 52.38% 0.32% 0.9901
|
||
21 attn_q 69479.59 0.0000 14531.5439 16.96 297.70 99.95% 4096 6.6280 55.23% 0.39% 0.9882
|
||
20 attn_q 68286.74 0.0000 18747.7070 16.67 345.57 99.95% 4096 6.0506 50.42% 0.34% 0.9917
|
||
25 attn_q 68221.13 0.0000 14562.2783 16.66 294.34 99.98% 4096 6.3805 53.17% 0.39% 0.9829
|
||
7 attn_q 68092.71 0.0000 17005.3926 16.62 348.43 99.90% 4096 5.9826 49.85% 0.32% 0.9814
|
||
18 attn_q 66295.24 0.0000 12255.0391 16.19 262.31 99.95% 4096 6.9157 57.63% 0.42% 0.9978
|
||
26 attn_q 65291.02 0.0000 12545.3467 15.94 268.54 99.98% 4096 6.6067 55.06% 0.39% 0.9920
|
||
13 attn_q 59896.29 0.0000 8712.0469 14.62 232.75 99.90% 4096 6.9162 57.63% 0.32% 0.9786
|
||
24 attn_q 56921.47 0.0000 9241.3613 13.90 213.49 99.98% 4096 7.0602 58.84% 0.54% 0.9757
|
||
17 attn_q 56760.55 0.0000 10113.2490 13.86 223.49 99.95% 4096 6.7278 56.06% 0.44% 0.9569
|
||
15 attn_q 55194.11 0.0000 9697.7783 13.48 229.94 99.93% 4096 5.9460 49.55% 0.37% 0.9288
|
||
14 attn_q 54794.45 0.0000 8291.7021 13.38 212.09 99.90% 4096 7.0371 58.64% 0.37% 0.9885
|
||
6 attn_q 53044.36 0.0000 16721.8047 12.95 321.79 99.88% 4096 4.5291 37.74% 0.32% 0.9479
|
||
19 attn_q 52733.22 0.0000 11920.8223 12.87 233.97 99.95% 4096 6.5100 54.25% 0.39% 0.9902
|
||
8 attn_q 51986.04 0.0000 10429.2197 12.69 233.14 99.90% 4096 6.5500 54.58% 0.29% 0.9699
|
||
9 attn_q 49641.15 0.0000 10131.9502 12.12 241.91 99.90% 4096 5.8877 49.06% 0.34% 0.9950
|
||
12 attn_q 48607.36 0.0000 7781.5098 11.87 191.06 99.93% 4096 7.0521 58.77% 0.37% 0.9965
|
||
4 attn_q 46316.68 0.0000 16862.5684 11.31 309.35 99.85% 4096 5.2333 43.61% 0.22% 0.7166
|
||
5 attn_q 45620.26 0.0000 16766.0762 11.14 317.06 99.88% 4096 4.5011 37.51% 0.22% 0.9947
|
||
11 attn_q 45317.79 0.0000 6714.2949 11.06 172.61 99.93% 4096 7.2487 60.41% 0.37% 0.9866
|
||
10 attn_q 43706.77 0.0000 6960.2920 10.67 180.78 99.93% 4096 6.7438 56.20% 0.34% 0.9923
|
||
2 attn_q 19532.52 0.0000 6018.6182 4.77 99.81 99.90% 4096 7.6636 63.86% 0.20% 0.3212
|
||
3 attn_q 17324.65 0.0000 5779.3477 4.23 110.43 99.93% 4096 5.5684 46.40% 0.22% 0.9580
|
||
1 attn_q 15856.24 0.0000 5183.1831 3.87 86.07 98.66% 4096 6.6712 55.59% 0.22% 0.3483
|
||
0 attn_q 3049.93 0.0000 247.0233 0.74 7.67 97.14% 4096 6.8129 56.77% 1.12% 0.0000
|
||
39 attn_v 174643.00 2.0243 18914.7031 42.64 497.38 100.00% 4096 8.8904 74.09% 0.37% 0.8865
|
||
38 attn_v 166087.69 0.0000 20050.7402 40.55 448.81 99.98% 4096 8.6133 71.78% 0.46% 0.9835
|
||
37 attn_v 146614.73 0.7595 20756.5176 35.79 423.56 100.00% 4096 8.3834 69.86% 0.44% 0.9842
|
||
36 attn_v 134513.23 0.5200 19003.3398 32.84 386.52 100.00% 4096 8.2079 68.40% 0.51% 0.9501
|
||
35 attn_v 124035.38 1.3198 11533.5068 30.28 305.84 100.00% 4096 8.2914 69.09% 0.71% 0.9665
|
||
34 attn_v 123896.47 0.5387 17763.3496 30.25 472.60 100.00% 4096 6.6833 55.69% 0.44% 0.9800
|
||
32 attn_v 111128.92 0.1590 25305.4961 27.13 504.76 100.00% 4096 6.2867 52.39% 0.37% 0.9931
|
||
30 attn_v 109705.93 0.0000 33505.8359 26.78 617.29 99.98% 4096 5.4138 45.11% 0.27% 0.9759
|
||
33 attn_v 109045.48 0.9963 20537.8262 26.62 448.68 100.00% 4096 6.6424 55.35% 0.37% 0.9914
|
||
29 attn_v 87316.21 0.0000 20308.4004 21.32 398.84 99.95% 4096 6.1991 51.66% 0.37% 0.9936
|
||
31 attn_v 87185.37 0.0000 22471.5254 21.29 421.05 99.98% 4096 6.2130 51.78% 0.34% 0.9963
|
||
28 attn_v 82016.98 0.0000 23024.2656 20.02 430.64 99.95% 4096 5.6830 47.36% 0.29% 0.9976
|
||
27 attn_v 76821.73 0.0000 17911.5156 18.76 348.43 99.98% 4096 6.3444 52.87% 0.34% 0.9791
|
||
16 attn_v 76133.66 0.0000 11891.1016 18.59 311.73 99.93% 4096 6.1208 51.01% 0.42% 0.9643
|
||
23 attn_v 74987.37 0.0000 18502.6172 18.31 356.15 99.95% 4096 6.1167 50.97% 0.39% 0.9980
|
||
22 attn_v 71226.52 0.0000 18282.5273 17.39 343.00 99.98% 4096 6.2861 52.38% 0.32% 0.9901
|
||
21 attn_v 69479.59 0.0000 14531.5439 16.96 297.70 99.95% 4096 6.6280 55.23% 0.39% 0.9882
|
||
20 attn_v 68286.74 0.0000 18747.7070 16.67 345.57 99.95% 4096 6.0506 50.42% 0.34% 0.9917
|
||
25 attn_v 68221.13 0.0000 14562.2783 16.66 294.34 99.98% 4096 6.3805 53.17% 0.39% 0.9829
|
||
7 attn_v 68092.71 0.0000 17005.3926 16.62 348.43 99.90% 4096 5.9826 49.85% 0.32% 0.9814
|
||
18 attn_v 66295.24 0.0000 12255.0391 16.19 262.31 99.95% 4096 6.9157 57.63% 0.42% 0.9978
|
||
26 attn_v 65291.02 0.0000 12545.3467 15.94 268.54 99.98% 4096 6.6067 55.06% 0.39% 0.9920
|
||
13 attn_v 59896.29 0.0000 8712.0469 14.62 232.75 99.90% 4096 6.9162 57.63% 0.32% 0.9786
|
||
24 attn_v 56921.47 0.0000 9241.3613 13.90 213.49 99.98% 4096 7.0602 58.84% 0.54% 0.9757
|
||
17 attn_v 56760.55 0.0000 10113.2490 13.86 223.49 99.95% 4096 6.7278 56.06% 0.44% 0.9569
|
||
15 attn_v 55194.11 0.0000 9697.7783 13.48 229.94 99.93% 4096 5.9460 49.55% 0.37% 0.9288
|
||
14 attn_v 54794.45 0.0000 8291.7021 13.38 212.09 99.90% 4096 7.0371 58.64% 0.37% 0.9885
|
||
6 attn_v 53044.36 0.0000 16721.8047 12.95 321.79 99.88% 4096 4.5291 37.74% 0.32% 0.9479
|
||
19 attn_v 52733.22 0.0000 11920.8223 12.87 233.97 99.95% 4096 6.5100 54.25% 0.39% 0.9902
|
||
8 attn_v 51986.04 0.0000 10429.2197 12.69 233.14 99.90% 4096 6.5500 54.58% 0.29% 0.9699
|
||
9 attn_v 49641.15 0.0000 10131.9502 12.12 241.91 99.90% 4096 5.8877 49.06% 0.34% 0.9950
|
||
12 attn_v 48607.36 0.0000 7781.5098 11.87 191.06 99.93% 4096 7.0521 58.77% 0.37% 0.9965
|
||
4 attn_v 46316.68 0.0000 16862.5684 11.31 309.35 99.85% 4096 5.2333 43.61% 0.22% 0.7166
|
||
5 attn_v 45620.26 0.0000 16766.0762 11.14 317.06 99.88% 4096 4.5011 37.51% 0.22% 0.9947
|
||
11 attn_v 45317.79 0.0000 6714.2949 11.06 172.61 99.93% 4096 7.2487 60.41% 0.37% 0.9866
|
||
10 attn_v 43706.77 0.0000 6960.2920 10.67 180.78 99.93% 4096 6.7438 56.20% 0.34% 0.9923
|
||
2 attn_v 19532.52 0.0000 6018.6182 4.77 99.81 99.90% 4096 7.6636 63.86% 0.20% 0.3212
|
||
3 attn_v 17324.65 0.0000 5779.3477 4.23 110.43 99.93% 4096 5.5684 46.40% 0.22% 0.9580
|
||
1 attn_v 15856.24 0.0000 5183.1831 3.87 86.07 98.66% 4096 6.6712 55.59% 0.22% 0.3483
|
||
0 attn_v 3049.93 0.0000 247.0233 0.74 7.67 97.14% 4096 6.8129 56.77% 1.12% 0.0000
|
||
39 ffn_down 1645556.38 0.0689 81837.1094 128.56 1422.92 100.00% 12800 9.8110 71.91% 1.00% 0.0173
|
||
5 ffn_down 381550.03 0.0094 210713.1406 29.81 2109.55 100.00% 12800 1.6182 11.86% 0.03% 0.0011
|
||
38 ffn_down 242713.44 0.2437 7073.1206 18.96 84.75 100.00% 12800 12.6848 92.97% 0.70% 0.0476
|
||
3 ffn_down 188951.92 0.0109 147793.5938 14.76 1332.36 100.00% 12800 1.2131 8.89% 0.03% 0.0026
|
||
37 ffn_down 142317.03 0.0870 5205.8745 11.12 52.73 100.00% 12800 12.6322 92.59% 0.67% 0.0329
|
||
36 ffn_down 83945.62 0.0217 2698.7646 6.56 37.12 100.00% 12800 11.9912 87.89% 1.06% 0.0443
|
||
35 ffn_down 42474.45 0.0859 952.5922 3.32 13.19 100.00% 12800 12.1797 89.27% 2.48% 0.0830
|
||
34 ffn_down 26102.06 0.0438 229.3722 2.04 5.52 100.00% 12800 12.2821 90.02% 4.28% 0.1049
|
||
33 ffn_down 12264.68 0.0564 152.2174 0.96 2.91 100.00% 12800 12.2756 89.97% 3.31% 0.0943
|
||
0 ffn_down 6042.44 0.0000 2911.0620 0.47 25.98 100.00% 12800 5.6061 41.09% 0.12% 0.0000
|
||
32 ffn_down 5920.96 0.0421 99.5476 0.46 1.42 100.00% 12800 12.4740 91.43% 3.20% 0.0558
|
||
31 ffn_down 3874.26 0.0193 185.5614 0.30 1.72 100.00% 12800 12.4092 90.95% 1.28% 0.0903
|
||
30 ffn_down 3486.52 0.0333 71.5827 0.27 0.87 100.00% 12800 12.5027 91.64% 2.80% 0.0724
|
||
29 ffn_down 2260.60 0.0105 59.7711 0.18 0.70 100.00% 12800 12.4988 91.61% 1.99% 0.0480
|
||
2 ffn_down 1941.90 0.0007 217.4457 0.15 1.95 100.00% 12800 11.9799 87.80% 0.16% 0.0440
|
||
28 ffn_down 1810.21 0.0343 70.6592 0.14 0.70 100.00% 12800 12.6709 92.87% 0.97% 0.0639
|
||
1 ffn_down 1559.32 0.0002 17.3429 0.12 0.21 100.00% 12800 13.0362 95.55% 5.21% 0.0058
|
||
27 ffn_down 1505.68 0.0334 16.9273 0.12 0.33 100.00% 12800 12.8432 94.13% 1.88% 0.0300
|
||
23 ffn_down 1423.83 0.0149 23.6004 0.11 0.42 100.00% 12800 12.5741 92.16% 1.63% 0.0554
|
||
24 ffn_down 1422.49 0.0148 57.0210 0.11 0.56 100.00% 12800 12.6908 93.01% 0.84% 0.0463
|
||
26 ffn_down 1401.04 0.0277 123.4429 0.11 1.10 100.00% 12800 12.4077 90.94% 0.29% 0.0151
|
||
25 ffn_down 1375.32 0.0282 80.1291 0.11 0.74 100.00% 12800 12.6175 92.48% 0.48% 0.0268
|
||
22 ffn_down 1156.34 0.0120 34.4371 0.09 0.38 100.00% 12800 12.6976 93.06% 1.30% 0.0673
|
||
21 ffn_down 1138.10 0.0147 24.3792 0.09 0.28 100.00% 12800 12.7839 93.70% 2.28% 0.0947
|
||
20 ffn_down 1136.30 0.0127 21.3931 0.09 0.26 100.00% 12800 12.8313 94.04% 2.33% 0.1177
|
||
18 ffn_down 1119.47 0.0176 4.8772 0.09 0.14 100.00% 12800 12.9995 95.28% 5.52% 0.2902
|
||
19 ffn_down 1116.88 0.0151 20.6434 0.09 0.23 100.00% 12800 12.9391 94.83% 2.98% 0.1868
|
||
16 ffn_down 1105.64 0.0134 17.2072 0.09 0.19 100.00% 12800 13.0591 95.71% 3.31% 0.0832
|
||
17 ffn_down 1041.27 0.0162 9.5109 0.08 0.13 100.00% 12800 13.1452 96.34% 4.94% 0.2264
|
||
15 ffn_down 1024.12 0.0230 43.3770 0.08 0.40 100.00% 12800 12.7413 93.39% 1.24% 0.0284
|
||
6 ffn_down 1022.13 0.0078 352.7156 0.08 3.12 100.00% 12800 9.2787 68.01% 0.05% 0.0001
|
||
14 ffn_down 951.70 0.0253 49.8595 0.07 0.50 100.00% 12800 12.6100 92.42% 0.50% 0.0720
|
||
11 ffn_down 926.64 0.0218 12.4872 0.07 0.13 100.00% 12800 13.2980 97.47% 1.80% 0.2713
|
||
4 ffn_down 923.82 0.0080 22.2460 0.07 0.37 100.00% 12800 12.2201 89.57% 1.02% 0.0012
|
||
12 ffn_down 888.91 0.0240 7.9189 0.07 0.12 100.00% 12800 13.2283 96.95% 2.80% 0.2412
|
||
10 ffn_down 857.79 0.0211 5.1418 0.07 0.10 100.00% 12800 13.1941 96.70% 3.75% 0.1709
|
||
13 ffn_down 845.87 0.0263 8.8175 0.07 0.11 100.00% 12800 13.2368 97.02% 2.59% 0.2583
|
||
9 ffn_down 829.72 0.0127 11.8182 0.06 0.21 100.00% 12800 12.6312 92.58% 1.71% 0.1047
|
||
7 ffn_down 761.20 0.0088 17.7730 0.06 0.18 100.00% 12800 13.0089 95.35% 1.20% 0.0092
|
||
8 ffn_down 714.58 0.0115 9.3180 0.06 0.15 100.00% 12800 12.9415 94.85% 1.95% 0.1090
|
||
39 ffn_gate 73855.20 0.0000 27529.4883 18.03 432.89 99.98% 4096 7.6055 63.38% 0.27% 0.9793
|
||
38 ffn_gate 29146.70 0.0000 3154.5896 7.12 50.92 99.98% 4096 10.6959 89.13% 0.63% 0.9972
|
||
37 ffn_gate 23469.74 0.0000 1999.8401 5.73 32.33 100.00% 4096 11.0207 91.84% 0.66% 0.9944
|
||
36 ffn_gate 18879.52 0.0000 1587.3755 4.61 26.11 99.98% 4096 10.9542 91.28% 0.68% 0.9535
|
||
35 ffn_gate 12609.52 0.0000 691.8613 3.08 13.18 99.95% 4096 10.9728 91.44% 1.10% 0.7708
|
||
34 ffn_gate 9369.36 0.0000 369.9386 2.29 8.82 99.98% 4096 10.8088 90.07% 1.39% 0.9703
|
||
33 ffn_gate 6623.49 0.0000 206.4922 1.62 5.04 99.98% 4096 11.0320 91.93% 1.56% 0.9570
|
||
32 ffn_gate 4946.19 0.2047 142.2780 1.21 3.44 100.00% 4096 11.1646 93.04% 1.49% 0.8361
|
||
0 ffn_gate 4647.99 0.0000 530.0613 1.13 10.15 68.41% 4096 8.6122 71.77% 1.15% 0.0000
|
||
31 ffn_gate 4116.51 0.0000 180.2167 1.01 3.74 99.98% 4096 11.0407 92.01% 1.32% 0.5779
|
||
1 ffn_gate 3766.61 0.0000 642.9065 0.92 11.02 99.22% 4096 9.7890 81.57% 0.12% 0.0923
|
||
30 ffn_gate 3736.23 0.0000 93.6410 0.91 2.60 100.00% 4096 11.2482 93.73% 1.29% 0.9941
|
||
29 ffn_gate 3591.15 0.0000 88.1915 0.88 2.41 99.98% 4096 11.3407 94.51% 1.10% 0.9684
|
||
25 ffn_gate 3420.11 0.0000 73.2920 0.83 2.34 99.98% 4096 11.3321 94.43% 0.85% 0.9818
|
||
12 ffn_gate 3417.91 0.0000 98.0158 0.83 1.89 99.95% 4096 11.6038 96.70% 0.49% 0.9870
|
||
26 ffn_gate 3406.55 0.0000 75.7852 0.83 2.13 99.98% 4096 11.4214 95.18% 0.88% 0.9927
|
||
13 ffn_gate 3312.10 0.0000 90.1448 0.81 1.61 99.98% 4096 11.6814 97.34% 0.51% 0.9759
|
||
11 ffn_gate 3301.55 0.0000 105.7369 0.81 2.04 99.95% 4096 11.5318 96.10% 0.44% 0.9904
|
||
28 ffn_gate 3275.83 0.0000 68.6310 0.80 1.93 99.98% 4096 11.4178 95.15% 1.22% 0.9578
|
||
27 ffn_gate 3236.11 0.0000 72.2997 0.79 1.87 99.98% 4096 11.4812 95.68% 1.03% 0.9950
|
||
14 ffn_gate 3080.71 0.0000 65.3723 0.75 1.33 99.98% 4096 11.6985 97.49% 0.71% 0.9506
|
||
16 ffn_gate 3016.78 0.0000 55.2134 0.74 1.58 99.98% 4096 11.5581 96.32% 0.76% 0.9874
|
||
24 ffn_gate 3010.61 0.0000 73.8905 0.74 2.21 100.00% 4096 11.2557 93.80% 0.93% 0.9690
|
||
22 ffn_gate 2879.76 0.0001 77.3697 0.70 2.02 100.00% 4096 11.3401 94.50% 0.81% 0.9957
|
||
17 ffn_gate 2870.70 0.0000 51.7692 0.70 1.48 99.98% 4096 11.5662 96.39% 0.73% 0.9959
|
||
23 ffn_gate 2843.65 0.0001 89.6189 0.69 2.21 100.00% 4096 11.2398 93.66% 0.95% 0.9937
|
||
10 ffn_gate 2833.02 0.0000 85.4598 0.69 1.74 99.90% 4096 11.5267 96.06% 0.46% 0.8381
|
||
19 ffn_gate 2776.20 0.0000 70.3380 0.68 1.86 100.00% 4096 11.3917 94.93% 0.71% 0.9944
|
||
15 ffn_gate 2760.72 0.0000 57.4514 0.67 1.45 99.95% 4096 11.5659 96.38% 0.83% 0.9038
|
||
18 ffn_gate 2754.00 0.0001 61.1560 0.67 1.55 100.00% 4096 11.5243 96.04% 0.76% 0.9902
|
||
21 ffn_gate 2737.92 0.0000 70.9311 0.67 1.86 99.98% 4096 11.3741 94.78% 0.81% 0.9902
|
||
20 ffn_gate 2699.39 0.0000 65.5763 0.66 1.79 99.98% 4096 11.3889 94.91% 0.83% 0.9932
|
||
8 ffn_gate 2469.90 0.0000 103.2819 0.60 1.84 99.93% 4096 11.4803 95.67% 0.56% 0.9061
|
||
2 ffn_gate 2319.60 0.0000 630.8475 0.57 10.17 99.80% 4096 8.7421 72.85% 0.12% 0.0200
|
||
9 ffn_gate 2312.98 0.0000 70.3176 0.56 1.47 99.93% 4096 11.4845 95.70% 0.76% 0.9660
|
||
7 ffn_gate 2253.67 0.0000 58.9012 0.55 1.44 99.93% 4096 11.5263 96.05% 0.54% 0.9532
|
||
5 ffn_gate 2213.64 0.0000 330.8649 0.54 5.46 99.93% 4096 10.0780 83.98% 0.24% 0.2463
|
||
6 ffn_gate 1791.24 0.0000 42.9261 0.44 1.12 99.90% 4096 11.4775 95.65% 0.66% 0.2691
|
||
4 ffn_gate 1789.65 0.0000 112.6252 0.44 2.11 99.93% 4096 11.0733 92.28% 0.29% 0.9765
|
||
3 ffn_gate 1716.85 0.0000 175.6357 0.42 3.39 99.90% 4096 10.3538 86.28% 0.22% 0.8606
|
||
39 ffn_up 73855.20 0.0000 27529.4883 18.03 432.89 99.98% 4096 7.6055 63.38% 0.27% 0.9793
|
||
38 ffn_up 29146.70 0.0000 3154.5896 7.12 50.92 99.98% 4096 10.6959 89.13% 0.63% 0.9972
|
||
37 ffn_up 23469.74 0.0000 1999.8401 5.73 32.33 100.00% 4096 11.0207 91.84% 0.66% 0.9944
|
||
36 ffn_up 18879.52 0.0000 1587.3755 4.61 26.11 99.98% 4096 10.9542 91.28% 0.68% 0.9535
|
||
35 ffn_up 12609.52 0.0000 691.8613 3.08 13.18 99.95% 4096 10.9728 91.44% 1.10% 0.7708
|
||
34 ffn_up 9369.36 0.0000 369.9386 2.29 8.82 99.98% 4096 10.8088 90.07% 1.39% 0.9703
|
||
33 ffn_up 6623.49 0.0000 206.4922 1.62 5.04 99.98% 4096 11.0320 91.93% 1.56% 0.9570
|
||
32 ffn_up 4946.19 0.2047 142.2780 1.21 3.44 100.00% 4096 11.1646 93.04% 1.49% 0.8361
|
||
0 ffn_up 4647.99 0.0000 530.0613 1.13 10.15 68.41% 4096 8.6122 71.77% 1.15% 0.0000
|
||
31 ffn_up 4116.51 0.0000 180.2167 1.01 3.74 99.98% 4096 11.0407 92.01% 1.32% 0.5779
|
||
1 ffn_up 3766.61 0.0000 642.9065 0.92 11.02 99.22% 4096 9.7890 81.57% 0.12% 0.0923
|
||
30 ffn_up 3736.23 0.0000 93.6410 0.91 2.60 100.00% 4096 11.2482 93.73% 1.29% 0.9941
|
||
29 ffn_up 3591.15 0.0000 88.1915 0.88 2.41 99.98% 4096 11.3407 94.51% 1.10% 0.9684
|
||
25 ffn_up 3420.11 0.0000 73.2920 0.83 2.34 99.98% 4096 11.3321 94.43% 0.85% 0.9818
|
||
12 ffn_up 3417.91 0.0000 98.0158 0.83 1.89 99.95% 4096 11.6038 96.70% 0.49% 0.9870
|
||
26 ffn_up 3406.55 0.0000 75.7852 0.83 2.13 99.98% 4096 11.4214 95.18% 0.88% 0.9927
|
||
13 ffn_up 3312.10 0.0000 90.1448 0.81 1.61 99.98% 4096 11.6814 97.34% 0.51% 0.9759
|
||
11 ffn_up 3301.55 0.0000 105.7369 0.81 2.04 99.95% 4096 11.5318 96.10% 0.44% 0.9904
|
||
28 ffn_up 3275.83 0.0000 68.6310 0.80 1.93 99.98% 4096 11.4178 95.15% 1.22% 0.9578
|
||
27 ffn_up 3236.11 0.0000 72.2997 0.79 1.87 99.98% 4096 11.4812 95.68% 1.03% 0.9950
|
||
14 ffn_up 3080.71 0.0000 65.3723 0.75 1.33 99.98% 4096 11.6985 97.49% 0.71% 0.9506
|
||
16 ffn_up 3016.78 0.0000 55.2134 0.74 1.58 99.98% 4096 11.5581 96.32% 0.76% 0.9874
|
||
24 ffn_up 3010.61 0.0000 73.8905 0.74 2.21 100.00% 4096 11.2557 93.80% 0.93% 0.9690
|
||
22 ffn_up 2879.76 0.0001 77.3697 0.70 2.02 100.00% 4096 11.3401 94.50% 0.81% 0.9957
|
||
17 ffn_up 2870.70 0.0000 51.7692 0.70 1.48 99.98% 4096 11.5662 96.39% 0.73% 0.9959
|
||
23 ffn_up 2843.65 0.0001 89.6189 0.69 2.21 100.00% 4096 11.2398 93.66% 0.95% 0.9937
|
||
10 ffn_up 2833.02 0.0000 85.4598 0.69 1.74 99.90% 4096 11.5267 96.06% 0.46% 0.8381
|
||
19 ffn_up 2776.20 0.0000 70.3380 0.68 1.86 100.00% 4096 11.3917 94.93% 0.71% 0.9944
|
||
15 ffn_up 2760.72 0.0000 57.4514 0.67 1.45 99.95% 4096 11.5659 96.38% 0.83% 0.9038
|
||
18 ffn_up 2754.00 0.0001 61.1560 0.67 1.55 100.00% 4096 11.5243 96.04% 0.76% 0.9902
|
||
21 ffn_up 2737.92 0.0000 70.9311 0.67 1.86 99.98% 4096 11.3741 94.78% 0.81% 0.9902
|
||
20 ffn_up 2699.39 0.0000 65.5763 0.66 1.79 99.98% 4096 11.3889 94.91% 0.83% 0.9932
|
||
8 ffn_up 2469.90 0.0000 103.2819 0.60 1.84 99.93% 4096 11.4803 95.67% 0.56% 0.9061
|
||
2 ffn_up 2319.60 0.0000 630.8475 0.57 10.17 99.80% 4096 8.7421 72.85% 0.12% 0.0200
|
||
9 ffn_up 2312.98 0.0000 70.3176 0.56 1.47 99.93% 4096 11.4845 95.70% 0.76% 0.9660
|
||
7 ffn_up 2253.67 0.0000 58.9012 0.55 1.44 99.93% 4096 11.5263 96.05% 0.54% 0.9532
|
||
5 ffn_up 2213.64 0.0000 330.8649 0.54 5.46 99.93% 4096 10.0780 83.98% 0.24% 0.2463
|
||
6 ffn_up 1791.24 0.0000 42.9261 0.44 1.12 99.90% 4096 11.4775 95.65% 0.66% 0.2691
|
||
4 ffn_up 1789.65 0.0000 112.6252 0.44 2.11 99.93% 4096 11.0733 92.28% 0.29% 0.9765
|
||
3 ffn_up 1716.85 0.0000 175.6357 0.42 3.39 99.90% 4096 10.3538 86.28% 0.22% 0.8606
|
||
|
||
Computing weighted average statistics per layer (40 layers)
|
||
|
||
Layer μΣ(Act²) μZD μCosSim
|
||
================================================
|
||
0 4139.83 0.7411% 0.0000
|
||
1 6791.54 2.9832% 0.1389
|
||
2 7785.62 1.9852% 0.1614
|
||
3 70900.05 1.2575% 0.5370
|
||
4 15985.68 1.4608% 0.4854
|
||
5 146183.94 0.8107% 0.4142
|
||
6 18205.81 1.3806% 0.4006
|
||
7 23167.60 1.2842% 0.5735
|
||
8 17900.25 1.4849% 0.5729
|
||
9 17128.10 1.5571% 0.6014
|
||
10 15332.18 2.2715% 0.5906
|
||
11 15983.94 1.9451% 0.6469
|
||
12 17056.90 2.1752% 0.6652
|
||
13 20767.65 2.4856% 0.6752
|
||
14 19054.13 1.1933% 0.5841
|
||
15 19172.65 1.4341% 0.5356
|
||
16 26136.96 1.8568% 0.5790
|
||
17 19724.82 2.5230% 0.6361
|
||
18 22839.35 2.4882% 0.6723
|
||
19 18405.42 1.6588% 0.6325
|
||
20 23508.35 1.5224% 0.6168
|
||
21 23910.19 1.5866% 0.5942
|
||
22 24533.16 1.1531% 0.5889
|
||
23 25862.99 1.2548% 0.5850
|
||
24 19961.15 1.3993% 0.5695
|
||
25 23721.45 0.7063% 0.5718
|
||
26 22758.83 0.8053% 0.5699
|
||
27 26575.09 1.3538% 0.5727
|
||
28 28424.57 1.0889% 0.5796
|
||
29 30363.12 1.5732% 0.5756
|
||
30 38224.65 1.7311% 0.5899
|
||
31 31137.59 1.3886% 0.5094
|
||
32 39814.40 1.8755% 0.5482
|
||
33 41853.15 1.9772% 0.5874
|
||
34 52107.42 2.3491% 0.5840
|
||
35 58491.29 1.8193% 0.5410
|
||
36 77988.16 1.1558% 0.5635
|
||
37 103052.64 1.1210% 0.5778
|
||
38 145584.28 1.5143% 0.5889
|
||
39 640944.31 1.3538% 0.5533
|