AlphaGaO/Qwen3-4B-GPTQ
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: AlphaGaO/Qwen3-4B-GPTQ
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-11 07:41:18 +08:00
commit a5e8bf8139
13 changed files with 152422 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

253
quant_log.csv Normal file
View File

@@ -0,0 +1,253 @@
layer,module,loss,samples,damp,time
0,self_attn.k_proj,0.00260956,0.01000,0.829
0,self_attn.v_proj,0.00233771,0.01000,0.591
0,self_attn.q_proj,0.00945148,0.01000,0.609
0,self_attn.o_proj,0.00626256,0.01000,0.988
0,mlp.up_proj,0.49306521,0.01000,0.624
0,mlp.gate_proj,0.56335938,0.01000,0.567
0,mlp.down_proj,0.05435039,0.01000,2.701
1,self_attn.k_proj,0.00402028,0.01000,0.648
1,self_attn.v_proj,0.00413055,0.01000,0.553
1,self_attn.q_proj,0.01495032,0.01000,0.557
1,self_attn.o_proj,0.00761302,0.01000,0.994
1,mlp.up_proj,6.14568138,0.01000,0.673
1,mlp.gate_proj,13.63503647,0.01000,0.573
1,mlp.down_proj,0.06105804,0.01000,2.646
2,self_attn.k_proj,0.01301107,0.01000,0.640
2,self_attn.v_proj,0.01256006,0.01000,0.552
2,self_attn.q_proj,0.04551908,0.01000,0.557
2,self_attn.o_proj,0.00959794,0.01000,1.076
2,mlp.up_proj,15.95272827,0.01000,0.671
2,mlp.gate_proj,20.05839157,0.01000,0.615
2,mlp.down_proj,0.05727252,0.01000,2.898
3,self_attn.k_proj,0.02194048,0.01000,0.648
3,self_attn.v_proj,0.02231835,0.01000,0.556
3,self_attn.q_proj,0.08355284,0.01000,0.557
3,self_attn.o_proj,0.01671291,0.01000,1.074
3,mlp.up_proj,12.27301884,0.01000,0.705
3,mlp.gate_proj,23.91833115,0.01000,0.619
3,mlp.down_proj,0.16981319,0.01000,2.797
4,self_attn.k_proj,0.04863824,0.01000,0.635
4,self_attn.v_proj,0.04902033,0.01000,0.553
4,self_attn.q_proj,0.17482278,0.01000,0.557
4,self_attn.o_proj,0.02858547,0.01000,0.979
4,mlp.up_proj,10.48973846,0.01000,0.676
4,mlp.gate_proj,26.65770721,0.01000,0.563
4,mlp.down_proj,0.28997350,0.01000,2.733
5,self_attn.k_proj,0.04801822,0.01000,0.634
5,self_attn.v_proj,0.05179255,0.01000,0.554
5,self_attn.q_proj,0.18255764,0.01000,0.556
5,self_attn.o_proj,0.04555316,0.01000,0.979
5,mlp.up_proj,5.48301125,0.01000,0.633
5,mlp.gate_proj,10.99185562,0.01000,0.565
5,mlp.down_proj,0.45506358,0.01000,2.682
6,self_attn.k_proj,0.09959402,0.01000,0.627
6,self_attn.v_proj,0.11477924,0.01000,0.555
6,self_attn.q_proj,0.41121966,0.01000,0.557
6,self_attn.o_proj,0.13184954,0.01000,0.966
6,mlp.up_proj,7.32487488,0.01000,0.630
6,mlp.gate_proj,12.34899235,0.01000,0.565
6,mlp.down_proj,137.00463867,0.01000,2.872
7,self_attn.k_proj,0.21236567,0.01000,0.624
7,self_attn.v_proj,0.21460515,0.01000,0.555
7,self_attn.q_proj,0.78465915,0.01000,0.556
7,self_attn.o_proj,0.13444883,0.01000,0.989
7,mlp.up_proj,8.85077858,0.01000,0.629
7,mlp.gate_proj,14.10528946,0.01000,0.567
7,mlp.down_proj,0.73926550,0.01000,2.479
8,self_attn.k_proj,0.34496000,0.01000,0.620
8,self_attn.v_proj,0.37624916,0.01000,0.552
8,self_attn.q_proj,1.29985523,0.01000,0.556
8,self_attn.o_proj,0.22197893,0.01000,0.986
8,mlp.up_proj,9.42709351,0.01000,0.624
8,mlp.gate_proj,11.46964645,0.01000,0.565
8,mlp.down_proj,1.18484640,0.01000,2.471
9,self_attn.k_proj,0.47145227,0.01000,0.641
9,self_attn.v_proj,0.46269092,0.01000,0.552
9,self_attn.q_proj,1.65912795,0.01000,0.554
9,self_attn.o_proj,0.26991138,0.01000,0.960
9,mlp.up_proj,12.27880478,0.01000,0.625
9,mlp.gate_proj,18.87297821,0.01000,0.565
9,mlp.down_proj,1.32681251,0.01000,2.463
10,self_attn.k_proj,0.72835219,0.01000,0.620
10,self_attn.v_proj,0.85435569,0.01000,0.552
10,self_attn.q_proj,2.74349403,0.01000,0.554
10,self_attn.o_proj,0.36292076,0.01000,0.956
10,mlp.up_proj,11.62601280,0.01000,0.622
10,mlp.gate_proj,16.43843842,0.01000,0.566
10,mlp.down_proj,1.36114752,0.01000,2.477
11,self_attn.k_proj,0.39711615,0.01000,0.660
11,self_attn.v_proj,0.42185795,0.01000,0.594
11,self_attn.q_proj,1.41280699,0.01000,0.598
11,self_attn.o_proj,0.27962011,0.01000,0.957
11,mlp.up_proj,11.56846046,0.01000,0.629
11,mlp.gate_proj,14.54604149,0.01000,0.571
11,mlp.down_proj,1.35389423,0.01000,2.474
12,self_attn.k_proj,0.46813077,0.01000,0.613
12,self_attn.v_proj,0.50627100,0.01000,0.551
12,self_attn.q_proj,1.68932188,0.01000,0.554
12,self_attn.o_proj,0.32685736,0.01000,0.954
12,mlp.up_proj,11.97426701,0.01000,0.666
12,mlp.gate_proj,13.71073341,0.01000,0.606
12,mlp.down_proj,1.34886861,0.01000,2.631
13,self_attn.k_proj,0.34903091,0.01000,0.648
13,self_attn.v_proj,0.35327876,0.01000,0.588
13,self_attn.q_proj,1.34817195,0.01000,0.581
13,self_attn.o_proj,0.22018546,0.01000,0.977
13,mlp.up_proj,12.59172344,0.01000,0.665
13,mlp.gate_proj,13.15475464,0.01000,0.583
13,mlp.down_proj,1.50714171,0.01000,2.509
14,self_attn.k_proj,0.61326528,0.01000,0.655
14,self_attn.v_proj,0.64901257,0.01000,0.588
14,self_attn.q_proj,2.28437233,0.01000,0.598
14,self_attn.o_proj,0.45074007,0.01000,1.035
14,mlp.up_proj,12.52815437,0.01000,0.663
14,mlp.gate_proj,13.09080696,0.01000,0.601
14,mlp.down_proj,1.38611388,0.01000,2.572
15,self_attn.k_proj,0.60994160,0.01000,0.638
15,self_attn.v_proj,0.59522879,0.01000,0.553
15,self_attn.q_proj,2.29438472,0.01000,0.556
15,self_attn.o_proj,0.35033396,0.01000,0.986
15,mlp.up_proj,11.78823280,0.01000,0.640
15,mlp.gate_proj,11.89937019,0.01000,0.567
15,mlp.down_proj,1.51105714,0.01000,2.595
16,self_attn.k_proj,1.04074728,0.01000,0.646
16,self_attn.v_proj,1.19501519,0.01000,0.560
16,self_attn.q_proj,4.24412441,0.01000,0.563
16,self_attn.o_proj,0.47474825,0.01000,1.000
16,mlp.up_proj,13.40410614,0.01000,0.654
16,mlp.gate_proj,14.28797817,0.01000,0.578
16,mlp.down_proj,5.78086710,0.01000,2.659
17,self_attn.k_proj,0.93727171,0.01000,0.632
17,self_attn.v_proj,1.08398855,0.01000,0.552
17,self_attn.q_proj,4.18204689,0.01000,0.556
17,self_attn.o_proj,0.54972744,0.01000,0.985
17,mlp.up_proj,13.20138741,0.01000,0.647
17,mlp.gate_proj,13.71052074,0.01000,0.566
17,mlp.down_proj,2.41512918,0.01000,2.736
18,self_attn.k_proj,1.14480364,0.01000,0.650
18,self_attn.v_proj,1.27249444,0.01000,0.555
18,self_attn.q_proj,4.76111126,0.01000,0.558
18,self_attn.o_proj,0.54431963,0.01000,0.999
18,mlp.up_proj,15.39452362,0.01000,0.646
18,mlp.gate_proj,15.82460308,0.01000,0.567
18,mlp.down_proj,2.41237640,0.01000,2.696
19,self_attn.k_proj,2.23619556,0.01000,0.651
19,self_attn.v_proj,2.33040428,0.01000,0.555
19,self_attn.q_proj,9.37097549,0.01000,0.556
19,self_attn.o_proj,0.89986920,0.01000,0.987
19,mlp.up_proj,17.55361557,0.01000,0.651
19,mlp.gate_proj,17.96214294,0.01000,0.563
19,mlp.down_proj,2.75978541,0.01000,2.821
20,self_attn.k_proj,1.84364629,0.01000,0.692
20,self_attn.v_proj,2.03417015,0.01000,0.602
20,self_attn.q_proj,8.70787430,0.01000,0.605
20,self_attn.o_proj,0.82709742,0.01000,0.994
20,mlp.up_proj,19.15014076,0.01000,0.695
20,mlp.gate_proj,19.27664566,0.01000,0.609
20,mlp.down_proj,2.91169357,0.01000,2.637
21,self_attn.k_proj,2.38239908,0.01000,0.698
21,self_attn.v_proj,2.78411126,0.01000,0.601
21,self_attn.q_proj,10.69113255,0.01000,0.606
21,self_attn.o_proj,0.86217725,0.01000,1.063
21,mlp.up_proj,22.03649902,0.01000,0.657
21,mlp.gate_proj,22.27753639,0.01000,0.568
21,mlp.down_proj,3.55271626,0.01000,2.668
22,self_attn.k_proj,5.02607822,0.01000,0.646
22,self_attn.v_proj,5.24235153,0.01000,0.553
22,self_attn.q_proj,20.75082779,0.01000,0.559
22,self_attn.o_proj,1.57138991,0.01000,0.989
22,mlp.up_proj,24.25587654,0.01000,0.652
22,mlp.gate_proj,25.06272507,0.01000,0.567
22,mlp.down_proj,5.23081446,0.01000,2.644
23,self_attn.k_proj,4.54250908,0.01000,0.654
23,self_attn.v_proj,5.57414436,0.01000,0.558
23,self_attn.q_proj,20.68773270,0.01000,0.561
23,self_attn.o_proj,2.26957107,0.01000,0.998
23,mlp.up_proj,27.23978996,0.01000,0.660
23,mlp.gate_proj,29.70862198,0.01000,0.570
23,mlp.down_proj,6.30178261,0.01000,2.689
24,self_attn.k_proj,7.05984497,0.01000,0.643
24,self_attn.v_proj,8.92666817,0.01000,0.584
24,self_attn.q_proj,31.14116287,0.01000,0.586
24,self_attn.o_proj,2.77548218,0.01000,1.071
24,mlp.up_proj,29.65335464,0.01000,0.649
24,mlp.gate_proj,33.00664520,0.01000,0.565
24,mlp.down_proj,7.27749395,0.01000,2.815
25,self_attn.k_proj,5.49420643,0.01000,0.700
25,self_attn.v_proj,6.28153324,0.01000,0.608
25,self_attn.q_proj,22.71930885,0.01000,0.612
25,self_attn.o_proj,1.31197178,0.01000,1.056
25,mlp.up_proj,32.39283371,0.01000,0.647
25,mlp.gate_proj,36.58316803,0.01000,0.561
25,mlp.down_proj,8.68100739,0.01000,2.829
26,self_attn.k_proj,8.18059158,0.01000,0.703
26,self_attn.v_proj,9.73805428,0.01000,0.606
26,self_attn.q_proj,35.29013062,0.01000,0.609
26,self_attn.o_proj,1.64416206,0.01000,1.029
26,mlp.up_proj,37.90194702,0.01000,0.655
26,mlp.gate_proj,41.78019714,0.01000,0.570
26,mlp.down_proj,9.65947247,0.01000,2.646
27,self_attn.k_proj,9.37205124,0.01000,0.644
27,self_attn.v_proj,12.03858185,0.01000,0.552
27,self_attn.q_proj,41.76227951,0.01000,0.554
27,self_attn.o_proj,1.72142029,0.01000,0.998
27,mlp.up_proj,42.76330566,0.01000,0.647
27,mlp.gate_proj,45.72332764,0.01000,0.566
27,mlp.down_proj,13.29294109,0.01000,2.657
28,self_attn.k_proj,11.01844501,0.01000,0.636
28,self_attn.v_proj,12.33791637,0.01000,0.550
28,self_attn.q_proj,44.51463318,0.01000,0.554
28,self_attn.o_proj,2.65718555,0.01000,1.000
28,mlp.up_proj,49.45624161,0.01000,0.653
28,mlp.gate_proj,50.92063141,0.01000,0.563
28,mlp.down_proj,18.92489243,0.01000,2.806
29,self_attn.k_proj,23.87117004,0.01000,0.650
29,self_attn.v_proj,33.11904907,0.01000,0.551
29,self_attn.q_proj,99.74765778,0.01000,0.552
29,self_attn.o_proj,3.10340786,0.01000,0.981
29,mlp.up_proj,59.07789612,0.01000,0.652
29,mlp.gate_proj,58.26216125,0.01000,0.562
29,mlp.down_proj,20.03937149,0.01000,2.770
30,self_attn.k_proj,30.15834808,0.01000,0.636
30,self_attn.v_proj,37.56179428,0.01000,0.550
30,self_attn.q_proj,120.38098145,0.01000,0.552
30,self_attn.o_proj,4.69311810,0.01000,0.974
30,mlp.up_proj,64.21419525,0.01000,0.655
30,mlp.gate_proj,61.41302109,0.01000,0.566
30,mlp.down_proj,26.27643013,0.01000,2.646
31,self_attn.k_proj,39.68087006,0.01000,0.644
31,self_attn.v_proj,55.66039276,0.01000,0.550
31,self_attn.q_proj,148.81207275,0.01000,0.554
31,self_attn.o_proj,5.62770224,0.01000,0.992
31,mlp.up_proj,67.36556244,0.01000,0.656
31,mlp.gate_proj,61.42463684,0.01000,0.564
31,mlp.down_proj,33.80282211,0.01000,2.631
32,self_attn.k_proj,52.13278198,0.01000,0.647
32,self_attn.v_proj,80.29966736,0.01000,0.550
32,self_attn.q_proj,210.71531677,0.01000,0.547
32,self_attn.o_proj,7.62773275,0.01000,0.983
32,mlp.up_proj,70.70653534,0.01000,0.653
32,mlp.gate_proj,63.02287674,0.01000,0.565
32,mlp.down_proj,44.33407593,0.01000,2.744
33,self_attn.k_proj,91.17549133,0.01000,0.645
33,self_attn.v_proj,181.99508667,0.01000,0.548
33,self_attn.q_proj,422.93328857,0.01000,0.555
33,self_attn.o_proj,16.37356377,0.01000,0.993
33,mlp.up_proj,76.78594971,0.01000,0.654
33,mlp.gate_proj,67.32464600,0.01000,0.564
33,mlp.down_proj,56.63261795,0.01000,2.873
34,self_attn.k_proj,80.26489258,0.01000,0.644
34,self_attn.v_proj,133.13087463,0.01000,0.546
34,self_attn.q_proj,352.78515625,0.01000,0.548
34,self_attn.o_proj,23.87093353,0.01000,0.984
34,mlp.up_proj,84.84834290,0.01000,0.647
34,mlp.gate_proj,79.85603333,0.01000,0.565
34,mlp.down_proj,74.65843201,0.01000,2.654
35,self_attn.k_proj,48.68472672,0.01000,0.644
35,self_attn.v_proj,63.22115326,0.01000,0.549
35,self_attn.q_proj,187.34757996,0.01000,0.551
35,self_attn.o_proj,21.02386856,0.01000,0.983
35,mlp.up_proj,132.50225830,0.01000,0.646
35,mlp.gate_proj,127.69544983,0.01000,0.560
35,mlp.down_proj,131.40872192,0.01000,2.672
1 layer,module,loss,samples,damp,time
2 0,self_attn.k_proj,0.00260956,0.01000,0.829
3 0,self_attn.v_proj,0.00233771,0.01000,0.591
4 0,self_attn.q_proj,0.00945148,0.01000,0.609
5 0,self_attn.o_proj,0.00626256,0.01000,0.988
6 0,mlp.up_proj,0.49306521,0.01000,0.624
7 0,mlp.gate_proj,0.56335938,0.01000,0.567
8 0,mlp.down_proj,0.05435039,0.01000,2.701
9 1,self_attn.k_proj,0.00402028,0.01000,0.648
10 1,self_attn.v_proj,0.00413055,0.01000,0.553
11 1,self_attn.q_proj,0.01495032,0.01000,0.557
12 1,self_attn.o_proj,0.00761302,0.01000,0.994
13 1,mlp.up_proj,6.14568138,0.01000,0.673
14 1,mlp.gate_proj,13.63503647,0.01000,0.573
15 1,mlp.down_proj,0.06105804,0.01000,2.646
16 2,self_attn.k_proj,0.01301107,0.01000,0.640
17 2,self_attn.v_proj,0.01256006,0.01000,0.552
18 2,self_attn.q_proj,0.04551908,0.01000,0.557
19 2,self_attn.o_proj,0.00959794,0.01000,1.076
20 2,mlp.up_proj,15.95272827,0.01000,0.671
21 2,mlp.gate_proj,20.05839157,0.01000,0.615
22 2,mlp.down_proj,0.05727252,0.01000,2.898
23 3,self_attn.k_proj,0.02194048,0.01000,0.648
24 3,self_attn.v_proj,0.02231835,0.01000,0.556
25 3,self_attn.q_proj,0.08355284,0.01000,0.557
26 3,self_attn.o_proj,0.01671291,0.01000,1.074
27 3,mlp.up_proj,12.27301884,0.01000,0.705
28 3,mlp.gate_proj,23.91833115,0.01000,0.619
29 3,mlp.down_proj,0.16981319,0.01000,2.797
30 4,self_attn.k_proj,0.04863824,0.01000,0.635
31 4,self_attn.v_proj,0.04902033,0.01000,0.553
32 4,self_attn.q_proj,0.17482278,0.01000,0.557
33 4,self_attn.o_proj,0.02858547,0.01000,0.979
34 4,mlp.up_proj,10.48973846,0.01000,0.676
35 4,mlp.gate_proj,26.65770721,0.01000,0.563
36 4,mlp.down_proj,0.28997350,0.01000,2.733
37 5,self_attn.k_proj,0.04801822,0.01000,0.634
38 5,self_attn.v_proj,0.05179255,0.01000,0.554
39 5,self_attn.q_proj,0.18255764,0.01000,0.556
40 5,self_attn.o_proj,0.04555316,0.01000,0.979
41 5,mlp.up_proj,5.48301125,0.01000,0.633
42 5,mlp.gate_proj,10.99185562,0.01000,0.565
43 5,mlp.down_proj,0.45506358,0.01000,2.682
44 6,self_attn.k_proj,0.09959402,0.01000,0.627
45 6,self_attn.v_proj,0.11477924,0.01000,0.555
46 6,self_attn.q_proj,0.41121966,0.01000,0.557
47 6,self_attn.o_proj,0.13184954,0.01000,0.966
48 6,mlp.up_proj,7.32487488,0.01000,0.630
49 6,mlp.gate_proj,12.34899235,0.01000,0.565
50 6,mlp.down_proj,137.00463867,0.01000,2.872
51 7,self_attn.k_proj,0.21236567,0.01000,0.624
52 7,self_attn.v_proj,0.21460515,0.01000,0.555
53 7,self_attn.q_proj,0.78465915,0.01000,0.556
54 7,self_attn.o_proj,0.13444883,0.01000,0.989
55 7,mlp.up_proj,8.85077858,0.01000,0.629
56 7,mlp.gate_proj,14.10528946,0.01000,0.567
57 7,mlp.down_proj,0.73926550,0.01000,2.479
58 8,self_attn.k_proj,0.34496000,0.01000,0.620
59 8,self_attn.v_proj,0.37624916,0.01000,0.552
60 8,self_attn.q_proj,1.29985523,0.01000,0.556
61 8,self_attn.o_proj,0.22197893,0.01000,0.986
62 8,mlp.up_proj,9.42709351,0.01000,0.624
63 8,mlp.gate_proj,11.46964645,0.01000,0.565
64 8,mlp.down_proj,1.18484640,0.01000,2.471
65 9,self_attn.k_proj,0.47145227,0.01000,0.641
66 9,self_attn.v_proj,0.46269092,0.01000,0.552
67 9,self_attn.q_proj,1.65912795,0.01000,0.554
68 9,self_attn.o_proj,0.26991138,0.01000,0.960
69 9,mlp.up_proj,12.27880478,0.01000,0.625
70 9,mlp.gate_proj,18.87297821,0.01000,0.565
71 9,mlp.down_proj,1.32681251,0.01000,2.463
72 10,self_attn.k_proj,0.72835219,0.01000,0.620
73 10,self_attn.v_proj,0.85435569,0.01000,0.552
74 10,self_attn.q_proj,2.74349403,0.01000,0.554
75 10,self_attn.o_proj,0.36292076,0.01000,0.956
76 10,mlp.up_proj,11.62601280,0.01000,0.622
77 10,mlp.gate_proj,16.43843842,0.01000,0.566
78 10,mlp.down_proj,1.36114752,0.01000,2.477
79 11,self_attn.k_proj,0.39711615,0.01000,0.660
80 11,self_attn.v_proj,0.42185795,0.01000,0.594
81 11,self_attn.q_proj,1.41280699,0.01000,0.598
82 11,self_attn.o_proj,0.27962011,0.01000,0.957
83 11,mlp.up_proj,11.56846046,0.01000,0.629
84 11,mlp.gate_proj,14.54604149,0.01000,0.571
85 11,mlp.down_proj,1.35389423,0.01000,2.474
86 12,self_attn.k_proj,0.46813077,0.01000,0.613
87 12,self_attn.v_proj,0.50627100,0.01000,0.551
88 12,self_attn.q_proj,1.68932188,0.01000,0.554
89 12,self_attn.o_proj,0.32685736,0.01000,0.954
90 12,mlp.up_proj,11.97426701,0.01000,0.666
91 12,mlp.gate_proj,13.71073341,0.01000,0.606
92 12,mlp.down_proj,1.34886861,0.01000,2.631
93 13,self_attn.k_proj,0.34903091,0.01000,0.648
94 13,self_attn.v_proj,0.35327876,0.01000,0.588
95 13,self_attn.q_proj,1.34817195,0.01000,0.581
96 13,self_attn.o_proj,0.22018546,0.01000,0.977
97 13,mlp.up_proj,12.59172344,0.01000,0.665
98 13,mlp.gate_proj,13.15475464,0.01000,0.583
99 13,mlp.down_proj,1.50714171,0.01000,2.509
100 14,self_attn.k_proj,0.61326528,0.01000,0.655
101 14,self_attn.v_proj,0.64901257,0.01000,0.588
102 14,self_attn.q_proj,2.28437233,0.01000,0.598
103 14,self_attn.o_proj,0.45074007,0.01000,1.035
104 14,mlp.up_proj,12.52815437,0.01000,0.663
105 14,mlp.gate_proj,13.09080696,0.01000,0.601
106 14,mlp.down_proj,1.38611388,0.01000,2.572
107 15,self_attn.k_proj,0.60994160,0.01000,0.638
108 15,self_attn.v_proj,0.59522879,0.01000,0.553
109 15,self_attn.q_proj,2.29438472,0.01000,0.556
110 15,self_attn.o_proj,0.35033396,0.01000,0.986
111 15,mlp.up_proj,11.78823280,0.01000,0.640
112 15,mlp.gate_proj,11.89937019,0.01000,0.567
113 15,mlp.down_proj,1.51105714,0.01000,2.595
114 16,self_attn.k_proj,1.04074728,0.01000,0.646
115 16,self_attn.v_proj,1.19501519,0.01000,0.560
116 16,self_attn.q_proj,4.24412441,0.01000,0.563
117 16,self_attn.o_proj,0.47474825,0.01000,1.000
118 16,mlp.up_proj,13.40410614,0.01000,0.654
119 16,mlp.gate_proj,14.28797817,0.01000,0.578
120 16,mlp.down_proj,5.78086710,0.01000,2.659
121 17,self_attn.k_proj,0.93727171,0.01000,0.632
122 17,self_attn.v_proj,1.08398855,0.01000,0.552
123 17,self_attn.q_proj,4.18204689,0.01000,0.556
124 17,self_attn.o_proj,0.54972744,0.01000,0.985
125 17,mlp.up_proj,13.20138741,0.01000,0.647
126 17,mlp.gate_proj,13.71052074,0.01000,0.566
127 17,mlp.down_proj,2.41512918,0.01000,2.736
128 18,self_attn.k_proj,1.14480364,0.01000,0.650
129 18,self_attn.v_proj,1.27249444,0.01000,0.555
130 18,self_attn.q_proj,4.76111126,0.01000,0.558
131 18,self_attn.o_proj,0.54431963,0.01000,0.999
132 18,mlp.up_proj,15.39452362,0.01000,0.646
133 18,mlp.gate_proj,15.82460308,0.01000,0.567
134 18,mlp.down_proj,2.41237640,0.01000,2.696
135 19,self_attn.k_proj,2.23619556,0.01000,0.651
136 19,self_attn.v_proj,2.33040428,0.01000,0.555
137 19,self_attn.q_proj,9.37097549,0.01000,0.556
138 19,self_attn.o_proj,0.89986920,0.01000,0.987
139 19,mlp.up_proj,17.55361557,0.01000,0.651
140 19,mlp.gate_proj,17.96214294,0.01000,0.563
141 19,mlp.down_proj,2.75978541,0.01000,2.821
142 20,self_attn.k_proj,1.84364629,0.01000,0.692
143 20,self_attn.v_proj,2.03417015,0.01000,0.602
144 20,self_attn.q_proj,8.70787430,0.01000,0.605
145 20,self_attn.o_proj,0.82709742,0.01000,0.994
146 20,mlp.up_proj,19.15014076,0.01000,0.695
147 20,mlp.gate_proj,19.27664566,0.01000,0.609
148 20,mlp.down_proj,2.91169357,0.01000,2.637
149 21,self_attn.k_proj,2.38239908,0.01000,0.698
150 21,self_attn.v_proj,2.78411126,0.01000,0.601
151 21,self_attn.q_proj,10.69113255,0.01000,0.606
152 21,self_attn.o_proj,0.86217725,0.01000,1.063
153 21,mlp.up_proj,22.03649902,0.01000,0.657
154 21,mlp.gate_proj,22.27753639,0.01000,0.568
155 21,mlp.down_proj,3.55271626,0.01000,2.668
156 22,self_attn.k_proj,5.02607822,0.01000,0.646
157 22,self_attn.v_proj,5.24235153,0.01000,0.553
158 22,self_attn.q_proj,20.75082779,0.01000,0.559
159 22,self_attn.o_proj,1.57138991,0.01000,0.989
160 22,mlp.up_proj,24.25587654,0.01000,0.652
161 22,mlp.gate_proj,25.06272507,0.01000,0.567
162 22,mlp.down_proj,5.23081446,0.01000,2.644
163 23,self_attn.k_proj,4.54250908,0.01000,0.654
164 23,self_attn.v_proj,5.57414436,0.01000,0.558
165 23,self_attn.q_proj,20.68773270,0.01000,0.561
166 23,self_attn.o_proj,2.26957107,0.01000,0.998
167 23,mlp.up_proj,27.23978996,0.01000,0.660
168 23,mlp.gate_proj,29.70862198,0.01000,0.570
169 23,mlp.down_proj,6.30178261,0.01000,2.689
170 24,self_attn.k_proj,7.05984497,0.01000,0.643
171 24,self_attn.v_proj,8.92666817,0.01000,0.584
172 24,self_attn.q_proj,31.14116287,0.01000,0.586
173 24,self_attn.o_proj,2.77548218,0.01000,1.071
174 24,mlp.up_proj,29.65335464,0.01000,0.649
175 24,mlp.gate_proj,33.00664520,0.01000,0.565
176 24,mlp.down_proj,7.27749395,0.01000,2.815
177 25,self_attn.k_proj,5.49420643,0.01000,0.700
178 25,self_attn.v_proj,6.28153324,0.01000,0.608
179 25,self_attn.q_proj,22.71930885,0.01000,0.612
180 25,self_attn.o_proj,1.31197178,0.01000,1.056
181 25,mlp.up_proj,32.39283371,0.01000,0.647
182 25,mlp.gate_proj,36.58316803,0.01000,0.561
183 25,mlp.down_proj,8.68100739,0.01000,2.829
184 26,self_attn.k_proj,8.18059158,0.01000,0.703
185 26,self_attn.v_proj,9.73805428,0.01000,0.606
186 26,self_attn.q_proj,35.29013062,0.01000,0.609
187 26,self_attn.o_proj,1.64416206,0.01000,1.029
188 26,mlp.up_proj,37.90194702,0.01000,0.655
189 26,mlp.gate_proj,41.78019714,0.01000,0.570
190 26,mlp.down_proj,9.65947247,0.01000,2.646
191 27,self_attn.k_proj,9.37205124,0.01000,0.644
192 27,self_attn.v_proj,12.03858185,0.01000,0.552
193 27,self_attn.q_proj,41.76227951,0.01000,0.554
194 27,self_attn.o_proj,1.72142029,0.01000,0.998
195 27,mlp.up_proj,42.76330566,0.01000,0.647
196 27,mlp.gate_proj,45.72332764,0.01000,0.566
197 27,mlp.down_proj,13.29294109,0.01000,2.657
198 28,self_attn.k_proj,11.01844501,0.01000,0.636
199 28,self_attn.v_proj,12.33791637,0.01000,0.550
200 28,self_attn.q_proj,44.51463318,0.01000,0.554
201 28,self_attn.o_proj,2.65718555,0.01000,1.000
202 28,mlp.up_proj,49.45624161,0.01000,0.653
203 28,mlp.gate_proj,50.92063141,0.01000,0.563
204 28,mlp.down_proj,18.92489243,0.01000,2.806
205 29,self_attn.k_proj,23.87117004,0.01000,0.650
206 29,self_attn.v_proj,33.11904907,0.01000,0.551
207 29,self_attn.q_proj,99.74765778,0.01000,0.552
208 29,self_attn.o_proj,3.10340786,0.01000,0.981
209 29,mlp.up_proj,59.07789612,0.01000,0.652
210 29,mlp.gate_proj,58.26216125,0.01000,0.562
211 29,mlp.down_proj,20.03937149,0.01000,2.770
212 30,self_attn.k_proj,30.15834808,0.01000,0.636
213 30,self_attn.v_proj,37.56179428,0.01000,0.550
214 30,self_attn.q_proj,120.38098145,0.01000,0.552
215 30,self_attn.o_proj,4.69311810,0.01000,0.974
216 30,mlp.up_proj,64.21419525,0.01000,0.655
217 30,mlp.gate_proj,61.41302109,0.01000,0.566
218 30,mlp.down_proj,26.27643013,0.01000,2.646
219 31,self_attn.k_proj,39.68087006,0.01000,0.644
220 31,self_attn.v_proj,55.66039276,0.01000,0.550
221 31,self_attn.q_proj,148.81207275,0.01000,0.554
222 31,self_attn.o_proj,5.62770224,0.01000,0.992
223 31,mlp.up_proj,67.36556244,0.01000,0.656
224 31,mlp.gate_proj,61.42463684,0.01000,0.564
225 31,mlp.down_proj,33.80282211,0.01000,2.631
226 32,self_attn.k_proj,52.13278198,0.01000,0.647
227 32,self_attn.v_proj,80.29966736,0.01000,0.550
228 32,self_attn.q_proj,210.71531677,0.01000,0.547
229 32,self_attn.o_proj,7.62773275,0.01000,0.983
230 32,mlp.up_proj,70.70653534,0.01000,0.653
231 32,mlp.gate_proj,63.02287674,0.01000,0.565
232 32,mlp.down_proj,44.33407593,0.01000,2.744
233 33,self_attn.k_proj,91.17549133,0.01000,0.645
234 33,self_attn.v_proj,181.99508667,0.01000,0.548
235 33,self_attn.q_proj,422.93328857,0.01000,0.555
236 33,self_attn.o_proj,16.37356377,0.01000,0.993
237 33,mlp.up_proj,76.78594971,0.01000,0.654
238 33,mlp.gate_proj,67.32464600,0.01000,0.564
239 33,mlp.down_proj,56.63261795,0.01000,2.873
240 34,self_attn.k_proj,80.26489258,0.01000,0.644
241 34,self_attn.v_proj,133.13087463,0.01000,0.546
242 34,self_attn.q_proj,352.78515625,0.01000,0.548
243 34,self_attn.o_proj,23.87093353,0.01000,0.984
244 34,mlp.up_proj,84.84834290,0.01000,0.647
245 34,mlp.gate_proj,79.85603333,0.01000,0.565
246 34,mlp.down_proj,74.65843201,0.01000,2.654
247 35,self_attn.k_proj,48.68472672,0.01000,0.644
248 35,self_attn.v_proj,63.22115326,0.01000,0.549
249 35,self_attn.q_proj,187.34757996,0.01000,0.551
250 35,self_attn.o_proj,21.02386856,0.01000,0.983
251 35,mlp.up_proj,132.50225830,0.01000,0.646
252 35,mlp.gate_proj,127.69544983,0.01000,0.560
253 35,mlp.down_proj,131.40872192,0.01000,2.672