gemma-3-12b-it-pruned-GGUF/scores/gemma-3-12b-it-F16.md

gemma-3-12b-it-F16.gguf - GGUF Internal File Dump

• Endian: LITTLE endian

Key Value Metadata Store

There are 43 key-value pairs in this file

POS	TYPE	Count	Key	Value
1	UINT32	1	GGUF.version	3
2	UINT64	1	GGUF.tensor_count	626
3	UINT64	1	GGUF.kv_count	40
4	STRING	1	general.architecture	gemma3
5	STRING	1	general.type	model
6	STRING	1	general.name	Gemma 3 12b It
7	STRING	1	general.finetune	it
8	STRING	1	general.basename	gemma-3
9	STRING	1	general.size_label	12B
10	STRING	1	general.license	gemma
11	UINT32	1	general.base_model.count	1
12	STRING	1	general.base_model.0.name	Gemma 3 12b Pt
13	STRING	1	general.base_model.0.organization	Google
14	STRING	1	general.base_model.0.repo_url	https://huggingface.co/google/gemma-3-12b-pt
15	[STRING]	1	general.tags	[ image-text-to-text ]
16	UINT32	1	gemma3.context_length	131072
17	UINT32	1	gemma3.embedding_length	3840
18	UINT32	1	gemma3.block_count	48
19	UINT32	1	gemma3.feed_forward_length	15360
20	UINT32	1	gemma3.attention.head_count	16
21	FLOAT32	1	gemma3.attention.layer_norm_rms_epsilon	1e-06
22	UINT32	1	gemma3.attention.key_length	256
23	UINT32	1	gemma3.attention.value_length	256
24	UINT32	1	general.file_type	1
25	FLOAT32	1	gemma3.rope.freq_base	1000000.0
26	UINT32	1	gemma3.attention.sliding_window	1024
27	UINT32	1	gemma3.attention.head_count_kv	8
28	STRING	1	gemma3.rope.scaling.type	linear
29	FLOAT32	1	gemma3.rope.scaling.factor	8.0
30	UINT32	1	general.quantization_version	2
31	STRING	1	tokenizer.ggml.model	llama
32	STRING	1	tokenizer.ggml.pre	default
33	[STRING]	262144	tokenizer.ggml.tokens	[ <pad>, <eos>, <bos>, <unk>, <mask>, ... ]
34	[FLOAT32]	262144	tokenizer.ggml.scores	[ -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, ... ]
35	[INT32]	262144	tokenizer.ggml.token_type	[ 3, 3, 3, 3, 3, 4, 3, ... ]
36	UINT32	1	tokenizer.ggml.bos_token_id	2
37	UINT32	1	tokenizer.ggml.eos_token_id	1
38	UINT32	1	tokenizer.ggml.unknown_token_id	3
39	UINT32	1	tokenizer.ggml.padding_token_id	0
40	BOOL	1	tokenizer.ggml.add_bos_token	True
41	BOOL	1	tokenizer.ggml.add_eos_token	False
42	STRING	1	tokenizer.chat_template	{{ bos_token }}{%- if message..._turn>model'}}{%- endif -%}
43	BOOL	1	tokenizer.ggml.add_space_prefix	False

Tensors Overview ~12B Elements

Total number of elements in all tensors: 11765788416 Elements

• gemma-3-12b-it-F16.gguf - GGUF Internal File Dump
  • Key Value Metadata Store
  • Tensors Overview ~12B Elements
    • Tensor Data Offset
    • Base Tensor Group : ~1B Elements
    • Block 0 Tensor Group : ~224M Elements
    • Block 1 Tensor Group : ~224M Elements
    • Block 2 Tensor Group : ~224M Elements
    • Block 3 Tensor Group : ~224M Elements
    • Block 4 Tensor Group : ~224M Elements
    • Block 10 Tensor Group : ~224M Elements
    • Block 11 Tensor Group : ~224M Elements
    • Block 12 Tensor Group : ~224M Elements
    • Block 13 Tensor Group : ~224M Elements
    • Block 14 Tensor Group : ~224M Elements
    • Block 15 Tensor Group : ~224M Elements
    • Block 5 Tensor Group : ~224M Elements
    • Block 6 Tensor Group : ~224M Elements
    • Block 7 Tensor Group : ~224M Elements
    • Block 8 Tensor Group : ~224M Elements
    • Block 9 Tensor Group : ~224M Elements
    • Block 16 Tensor Group : ~224M Elements
    • Block 17 Tensor Group : ~224M Elements
    • Block 18 Tensor Group : ~224M Elements
    • Block 19 Tensor Group : ~224M Elements
    • Block 20 Tensor Group : ~224M Elements
    • Block 21 Tensor Group : ~224M Elements
    • Block 22 Tensor Group : ~224M Elements
    • Block 23 Tensor Group : ~224M Elements
    • Block 24 Tensor Group : ~224M Elements
    • Block 25 Tensor Group : ~224M Elements
    • Block 26 Tensor Group : ~224M Elements
    • Block 27 Tensor Group : ~224M Elements
    • Block 28 Tensor Group : ~224M Elements
    • Block 29 Tensor Group : ~224M Elements
    • Block 30 Tensor Group : ~224M Elements
    • Block 31 Tensor Group : ~224M Elements
    • Block 32 Tensor Group : ~224M Elements
    • Block 33 Tensor Group : ~224M Elements
    • Block 34 Tensor Group : ~224M Elements
    • Block 35 Tensor Group : ~224M Elements
    • Block 36 Tensor Group : ~224M Elements
    • Block 37 Tensor Group : ~224M Elements
    • Block 38 Tensor Group : ~224M Elements
    • Block 39 Tensor Group : ~224M Elements
    • Block 40 Tensor Group : ~224M Elements
    • Block 41 Tensor Group : ~224M Elements
    • Block 42 Tensor Group : ~224M Elements
    • Block 43 Tensor Group : ~224M Elements
    • Block 44 Tensor Group : ~224M Elements
    • Block 45 Tensor Group : ~224M Elements
    • Block 46 Tensor Group : ~224M Elements
    • Block 47 Tensor Group : ~224M Elements

Tensor Data Offset

This table contains the offset and data segment relative to start of file

T_ID	Tensor Layer Name	Data Offset (B)	Data Size (B)
0	token_embd.weight	0x63f300	0x78000000
1	blk.0.attn_norm.weight	0x7863f300	0x3c00
2	blk.0.ffn_down.weight	0x78642f00	0x7080000
3	blk.0.ffn_gate.weight	0x7f6c2f00	0x7080000
4	blk.0.ffn_up.weight	0x86742f00	0x7080000
5	blk.0.post_attention_norm.weight	0x8d7c2f00	0x3c00
6	blk.0.post_ffw_norm.weight	0x8d7c6b00	0x3c00
7	blk.0.ffn_norm.weight	0x8d7ca700	0x3c00
8	blk.0.attn_k_norm.weight	0x8d7ce300	0x400
9	blk.0.attn_k.weight	0x8d7ce700	0xf00000
10	blk.0.attn_output.weight	0x8e6ce700	0x1e00000
11	blk.0.attn_q_norm.weight	0x904ce700	0x400
12	blk.0.attn_q.weight	0x904ceb00	0x1e00000
13	blk.0.attn_v.weight	0x922ceb00	0xf00000
14	blk.1.attn_norm.weight	0x931ceb00	0x3c00
15	blk.1.ffn_down.weight	0x931d2700	0x7080000
16	blk.1.ffn_gate.weight	0x9a252700	0x7080000
17	blk.1.ffn_up.weight	0xa12d2700	0x7080000
18	blk.1.post_attention_norm.weight	0xa8352700	0x3c00
19	blk.1.post_ffw_norm.weight	0xa8356300	0x3c00
20	blk.1.ffn_norm.weight	0xa8359f00	0x3c00
21	blk.1.attn_k_norm.weight	0xa835db00	0x400
22	blk.1.attn_k.weight	0xa835df00	0xf00000
23	blk.1.attn_output.weight	0xa925df00	0x1e00000
24	blk.1.attn_q_norm.weight	0xab05df00	0x400
25	blk.1.attn_q.weight	0xab05e300	0x1e00000
26	blk.1.attn_v.weight	0xace5e300	0xf00000
27	blk.2.attn_norm.weight	0xadd5e300	0x3c00
28	blk.2.ffn_down.weight	0xadd61f00	0x7080000
29	blk.2.ffn_gate.weight	0xb4de1f00	0x7080000
30	blk.2.ffn_up.weight	0xbbe61f00	0x7080000
31	blk.2.post_attention_norm.weight	0xc2ee1f00	0x3c00
32	blk.2.post_ffw_norm.weight	0xc2ee5b00	0x3c00
33	blk.2.ffn_norm.weight	0xc2ee9700	0x3c00
34	blk.2.attn_k_norm.weight	0xc2eed300	0x400
35	blk.2.attn_k.weight	0xc2eed700	0xf00000
36	blk.2.attn_output.weight	0xc3ded700	0x1e00000
37	blk.2.attn_q_norm.weight	0xc5bed700	0x400
38	blk.2.attn_q.weight	0xc5bedb00	0x1e00000
39	blk.2.attn_v.weight	0xc79edb00	0xf00000
40	blk.3.attn_norm.weight	0xc88edb00	0x3c00
41	blk.3.ffn_down.weight	0xc88f1700	0x7080000
42	blk.3.ffn_gate.weight	0xcf971700	0x7080000
43	blk.3.ffn_up.weight	0xd69f1700	0x7080000
44	blk.3.post_attention_norm.weight	0xdda71700	0x3c00
45	blk.3.post_ffw_norm.weight	0xdda75300	0x3c00
46	blk.3.ffn_norm.weight	0xdda78f00	0x3c00
47	blk.3.attn_k_norm.weight	0xdda7cb00	0x400
48	blk.3.attn_k.weight	0xdda7cf00	0xf00000
49	blk.3.attn_output.weight	0xde97cf00	0x1e00000
50	blk.3.attn_q_norm.weight	0xe077cf00	0x400
51	blk.3.attn_q.weight	0xe077d300	0x1e00000
52	blk.3.attn_v.weight	0xe257d300	0xf00000
53	blk.4.ffn_gate.weight	0xe347d300	0x7080000
54	blk.4.ffn_up.weight	0xea4fd300	0x7080000
55	blk.4.attn_k_norm.weight	0xf157d300	0x400
56	blk.4.attn_k.weight	0xf157d700	0xf00000
57	blk.4.attn_output.weight	0xf247d700	0x1e00000
58	blk.4.attn_q_norm.weight	0xf427d700	0x400
59	blk.4.attn_q.weight	0xf427db00	0x1e00000
60	blk.4.attn_v.weight	0xf607db00	0xf00000
61	blk.10.attn_norm.weight	0xf6f7db00	0x3c00
62	blk.10.ffn_down.weight	0xf6f81700	0x7080000
63	blk.10.ffn_gate.weight	0xfe001700	0x7080000
64	blk.10.ffn_up.weight	0x105081700	0x7080000
65	blk.10.post_attention_norm.weight	0x10c101700	0x3c00
66	blk.10.post_ffw_norm.weight	0x10c105300	0x3c00
67	blk.10.ffn_norm.weight	0x10c108f00	0x3c00
68	blk.10.attn_k_norm.weight	0x10c10cb00	0x400
69	blk.10.attn_k.weight	0x10c10cf00	0xf00000
70	blk.10.attn_output.weight	0x10d00cf00	0x1e00000
71	blk.10.attn_q_norm.weight	0x10ee0cf00	0x400
72	blk.10.attn_q.weight	0x10ee0d300	0x1e00000
73	blk.10.attn_v.weight	0x110c0d300	0xf00000
74	blk.11.attn_norm.weight	0x111b0d300	0x3c00
75	blk.11.ffn_down.weight	0x111b10f00	0x7080000
76	blk.11.ffn_gate.weight	0x118b90f00	0x7080000
77	blk.11.ffn_up.weight	0x11fc10f00	0x7080000
78	blk.11.post_attention_norm.weight	0x126c90f00	0x3c00
79	blk.11.post_ffw_norm.weight	0x126c94b00	0x3c00
80	blk.11.ffn_norm.weight	0x126c98700	0x3c00
81	blk.11.attn_k_norm.weight	0x126c9c300	0x400
82	blk.11.attn_k.weight	0x126c9c700	0xf00000
83	blk.11.attn_output.weight	0x127b9c700	0x1e00000
84	blk.11.attn_q_norm.weight	0x12999c700	0x400
85	blk.11.attn_q.weight	0x12999cb00	0x1e00000
86	blk.11.attn_v.weight	0x12b79cb00	0xf00000
87	blk.12.attn_norm.weight	0x12c69cb00	0x3c00
88	blk.12.ffn_down.weight	0x12c6a0700	0x7080000
89	blk.12.ffn_gate.weight	0x133720700	0x7080000
90	blk.12.ffn_up.weight	0x13a7a0700	0x7080000
91	blk.12.post_attention_norm.weight	0x141820700	0x3c00
92	blk.12.post_ffw_norm.weight	0x141824300	0x3c00
93	blk.12.ffn_norm.weight	0x141827f00	0x3c00
94	blk.12.attn_k_norm.weight	0x14182bb00	0x400
95	blk.12.attn_k.weight	0x14182bf00	0xf00000
96	blk.12.attn_output.weight	0x14272bf00	0x1e00000
97	blk.12.attn_q_norm.weight	0x14452bf00	0x400
98	blk.12.attn_q.weight	0x14452c300	0x1e00000
99	blk.12.attn_v.weight	0x14632c300	0xf00000
100	blk.13.attn_norm.weight	0x14722c300	0x3c00
101	blk.13.ffn_down.weight	0x14722ff00	0x7080000
102	blk.13.ffn_gate.weight	0x14e2aff00	0x7080000
103	blk.13.ffn_up.weight	0x15532ff00	0x7080000
104	blk.13.post_attention_norm.weight	0x15c3aff00	0x3c00
105	blk.13.post_ffw_norm.weight	0x15c3b3b00	0x3c00
106	blk.13.ffn_norm.weight	0x15c3b7700	0x3c00
107	blk.13.attn_k_norm.weight	0x15c3bb300	0x400
108	blk.13.attn_k.weight	0x15c3bb700	0xf00000
109	blk.13.attn_output.weight	0x15d2bb700	0x1e00000
110	blk.13.attn_q_norm.weight	0x15f0bb700	0x400
111	blk.13.attn_q.weight	0x15f0bbb00	0x1e00000
112	blk.13.attn_v.weight	0x160ebbb00	0xf00000
113	blk.14.attn_norm.weight	0x161dbbb00	0x3c00
114	blk.14.ffn_down.weight	0x161dbf700	0x7080000
115	blk.14.ffn_gate.weight	0x168e3f700	0x7080000
116	blk.14.ffn_up.weight	0x16febf700	0x7080000
117	blk.14.post_attention_norm.weight	0x176f3f700	0x3c00
118	blk.14.post_ffw_norm.weight	0x176f43300	0x3c00
119	blk.14.ffn_norm.weight	0x176f46f00	0x3c00
120	blk.14.attn_k_norm.weight	0x176f4ab00	0x400
121	blk.14.attn_k.weight	0x176f4af00	0xf00000
122	blk.14.attn_output.weight	0x177e4af00	0x1e00000
123	blk.14.attn_q_norm.weight	0x179c4af00	0x400
124	blk.14.attn_q.weight	0x179c4b300	0x1e00000
125	blk.14.attn_v.weight	0x17ba4b300	0xf00000
126	blk.15.ffn_gate.weight	0x17c94b300	0x7080000
127	blk.15.ffn_up.weight	0x1839cb300	0x7080000
128	blk.15.attn_k_norm.weight	0x18aa4b300	0x400
129	blk.15.attn_k.weight	0x18aa4b700	0xf00000
130	blk.15.attn_output.weight	0x18b94b700	0x1e00000
131	blk.15.attn_q_norm.weight	0x18d74b700	0x400
132	blk.15.attn_q.weight	0x18d74bb00	0x1e00000
133	blk.15.attn_v.weight	0x18f54bb00	0xf00000
134	blk.4.attn_norm.weight	0x19044bb00	0x3c00
135	blk.4.ffn_down.weight	0x19044f700	0x7080000
136	blk.4.post_attention_norm.weight	0x1974cf700	0x3c00
137	blk.4.post_ffw_norm.weight	0x1974d3300	0x3c00
138	blk.4.ffn_norm.weight	0x1974d6f00	0x3c00
139	blk.5.attn_norm.weight	0x1974dab00	0x3c00
140	blk.5.ffn_down.weight	0x1974de700	0x7080000
141	blk.5.ffn_gate.weight	0x19e55e700	0x7080000
142	blk.5.ffn_up.weight	0x1a55de700	0x7080000
143	blk.5.post_attention_norm.weight	0x1ac65e700	0x3c00
144	blk.5.post_ffw_norm.weight	0x1ac662300	0x3c00
145	blk.5.ffn_norm.weight	0x1ac665f00	0x3c00
146	blk.5.attn_k_norm.weight	0x1ac669b00	0x400
147	blk.5.attn_k.weight	0x1ac669f00	0xf00000
148	blk.5.attn_output.weight	0x1ad569f00	0x1e00000
149	blk.5.attn_q_norm.weight	0x1af369f00	0x400
150	blk.5.attn_q.weight	0x1af36a300	0x1e00000
151	blk.5.attn_v.weight	0x1b116a300	0xf00000
152	blk.6.attn_norm.weight	0x1b206a300	0x3c00
153	blk.6.ffn_down.weight	0x1b206df00	0x7080000
154	blk.6.ffn_gate.weight	0x1b90edf00	0x7080000
155	blk.6.ffn_up.weight	0x1c016df00	0x7080000
156	blk.6.post_attention_norm.weight	0x1c71edf00	0x3c00
157	blk.6.post_ffw_norm.weight	0x1c71f1b00	0x3c00
158	blk.6.ffn_norm.weight	0x1c71f5700	0x3c00
159	blk.6.attn_k_norm.weight	0x1c71f9300	0x400
160	blk.6.attn_k.weight	0x1c71f9700	0xf00000
161	blk.6.attn_output.weight	0x1c80f9700	0x1e00000
162	blk.6.attn_q_norm.weight	0x1c9ef9700	0x400
163	blk.6.attn_q.weight	0x1c9ef9b00	0x1e00000
164	blk.6.attn_v.weight	0x1cbcf9b00	0xf00000
165	blk.7.attn_norm.weight	0x1ccbf9b00	0x3c00
166	blk.7.ffn_down.weight	0x1ccbfd700	0x7080000
167	blk.7.ffn_gate.weight	0x1d3c7d700	0x7080000
168	blk.7.ffn_up.weight	0x1dacfd700	0x7080000
169	blk.7.post_attention_norm.weight	0x1e1d7d700	0x3c00
170	blk.7.post_ffw_norm.weight	0x1e1d81300	0x3c00
171	blk.7.ffn_norm.weight	0x1e1d84f00	0x3c00
172	blk.7.attn_k_norm.weight	0x1e1d88b00	0x400
173	blk.7.attn_k.weight	0x1e1d88f00	0xf00000
174	blk.7.attn_output.weight	0x1e2c88f00	0x1e00000
175	blk.7.attn_q_norm.weight	0x1e4a88f00	0x400
176	blk.7.attn_q.weight	0x1e4a89300	0x1e00000
177	blk.7.attn_v.weight	0x1e6889300	0xf00000
178	blk.8.attn_norm.weight	0x1e7789300	0x3c00
179	blk.8.ffn_down.weight	0x1e778cf00	0x7080000
180	blk.8.ffn_gate.weight	0x1ee80cf00	0x7080000
181	blk.8.ffn_up.weight	0x1f588cf00	0x7080000
182	blk.8.post_attention_norm.weight	0x1fc90cf00	0x3c00
183	blk.8.post_ffw_norm.weight	0x1fc910b00	0x3c00
184	blk.8.ffn_norm.weight	0x1fc914700	0x3c00
185	blk.8.attn_k_norm.weight	0x1fc918300	0x400
186	blk.8.attn_k.weight	0x1fc918700	0xf00000
187	blk.8.attn_output.weight	0x1fd818700	0x1e00000
188	blk.8.attn_q_norm.weight	0x1ff618700	0x400
189	blk.8.attn_q.weight	0x1ff618b00	0x1e00000
190	blk.8.attn_v.weight	0x201418b00	0xf00000
191	blk.9.attn_norm.weight	0x202318b00	0x3c00
192	blk.9.ffn_down.weight	0x20231c700	0x7080000
193	blk.9.ffn_gate.weight	0x20939c700	0x7080000
194	blk.9.ffn_up.weight	0x21041c700	0x7080000
195	blk.9.post_attention_norm.weight	0x21749c700	0x3c00
196	blk.9.post_ffw_norm.weight	0x2174a0300	0x3c00
197	blk.9.ffn_norm.weight	0x2174a3f00	0x3c00
198	blk.9.attn_k_norm.weight	0x2174a7b00	0x400
199	blk.9.attn_k.weight	0x2174a7f00	0xf00000
200	blk.9.attn_output.weight	0x2183a7f00	0x1e00000
201	blk.9.attn_q_norm.weight	0x21a1a7f00	0x400
202	blk.9.attn_q.weight	0x21a1a8300	0x1e00000
203	blk.9.attn_v.weight	0x21bfa8300	0xf00000
204	blk.15.attn_norm.weight	0x21cea8300	0x3c00
205	blk.15.ffn_down.weight	0x21ceabf00	0x7080000
206	blk.15.post_attention_norm.weight	0x223f2bf00	0x3c00
207	blk.15.post_ffw_norm.weight	0x223f2fb00	0x3c00
208	blk.15.ffn_norm.weight	0x223f33700	0x3c00
209	blk.16.attn_norm.weight	0x223f37300	0x3c00
210	blk.16.ffn_down.weight	0x223f3af00	0x7080000
211	blk.16.ffn_gate.weight	0x22afbaf00	0x7080000
212	blk.16.ffn_up.weight	0x23203af00	0x7080000
213	blk.16.post_attention_norm.weight	0x2390baf00	0x3c00
214	blk.16.post_ffw_norm.weight	0x2390beb00	0x3c00
215	blk.16.ffn_norm.weight	0x2390c2700	0x3c00
216	blk.16.attn_k_norm.weight	0x2390c6300	0x400
217	blk.16.attn_k.weight	0x2390c6700	0xf00000
218	blk.16.attn_output.weight	0x239fc6700	0x1e00000
219	blk.16.attn_q_norm.weight	0x23bdc6700	0x400
220	blk.16.attn_q.weight	0x23bdc6b00	0x1e00000
221	blk.16.attn_v.weight	0x23dbc6b00	0xf00000
222	blk.17.attn_norm.weight	0x23eac6b00	0x3c00
223	blk.17.ffn_down.weight	0x23eaca700	0x7080000
224	blk.17.ffn_gate.weight	0x245b4a700	0x7080000
225	blk.17.ffn_up.weight	0x24cbca700	0x7080000
226	blk.17.post_attention_norm.weight	0x253c4a700	0x3c00
227	blk.17.post_ffw_norm.weight	0x253c4e300	0x3c00
228	blk.17.ffn_norm.weight	0x253c51f00	0x3c00
229	blk.17.attn_k_norm.weight	0x253c55b00	0x400
230	blk.17.attn_k.weight	0x253c55f00	0xf00000
231	blk.17.attn_output.weight	0x254b55f00	0x1e00000
232	blk.17.attn_q_norm.weight	0x256955f00	0x400
233	blk.17.attn_q.weight	0x256956300	0x1e00000
234	blk.17.attn_v.weight	0x258756300	0xf00000
235	blk.18.attn_norm.weight	0x259656300	0x3c00
236	blk.18.ffn_down.weight	0x259659f00	0x7080000
237	blk.18.ffn_gate.weight	0x2606d9f00	0x7080000
238	blk.18.ffn_up.weight	0x267759f00	0x7080000
239	blk.18.post_attention_norm.weight	0x26e7d9f00	0x3c00
240	blk.18.post_ffw_norm.weight	0x26e7ddb00	0x3c00
241	blk.18.ffn_norm.weight	0x26e7e1700	0x3c00
242	blk.18.attn_k_norm.weight	0x26e7e5300	0x400
243	blk.18.attn_k.weight	0x26e7e5700	0xf00000
244	blk.18.attn_output.weight	0x26f6e5700	0x1e00000
245	blk.18.attn_q_norm.weight	0x2714e5700	0x400
246	blk.18.attn_q.weight	0x2714e5b00	0x1e00000
247	blk.18.attn_v.weight	0x2732e5b00	0xf00000
248	blk.19.attn_norm.weight	0x2741e5b00	0x3c00
249	blk.19.ffn_down.weight	0x2741e9700	0x7080000
250	blk.19.ffn_gate.weight	0x27b269700	0x7080000
251	blk.19.ffn_up.weight	0x2822e9700	0x7080000
252	blk.19.post_attention_norm.weight	0x289369700	0x3c00
253	blk.19.post_ffw_norm.weight	0x28936d300	0x3c00
254	blk.19.ffn_norm.weight	0x289370f00	0x3c00
255	blk.19.attn_k_norm.weight	0x289374b00	0x400
256	blk.19.attn_k.weight	0x289374f00	0xf00000
257	blk.19.attn_output.weight	0x28a274f00	0x1e00000
258	blk.19.attn_q_norm.weight	0x28c074f00	0x400
259	blk.19.attn_q.weight	0x28c075300	0x1e00000
260	blk.19.attn_v.weight	0x28de75300	0xf00000
261	blk.20.attn_norm.weight	0x28ed75300	0x3c00
262	blk.20.ffn_down.weight	0x28ed78f00	0x7080000
263	blk.20.ffn_gate.weight	0x295df8f00	0x7080000
264	blk.20.ffn_up.weight	0x29ce78f00	0x7080000
265	blk.20.post_attention_norm.weight	0x2a3ef8f00	0x3c00
266	blk.20.post_ffw_norm.weight	0x2a3efcb00	0x3c00
267	blk.20.ffn_norm.weight	0x2a3f00700	0x3c00
268	blk.20.attn_k_norm.weight	0x2a3f04300	0x400
269	blk.20.attn_k.weight	0x2a3f04700	0xf00000
270	blk.20.attn_output.weight	0x2a4e04700	0x1e00000
271	blk.20.attn_q_norm.weight	0x2a6c04700	0x400
272	blk.20.attn_q.weight	0x2a6c04b00	0x1e00000
273	blk.20.attn_v.weight	0x2a8a04b00	0xf00000
274	blk.21.attn_norm.weight	0x2a9904b00	0x3c00
275	blk.21.ffn_down.weight	0x2a9908700	0x7080000
276	blk.21.ffn_gate.weight	0x2b0988700	0x7080000
277	blk.21.ffn_up.weight	0x2b7a08700	0x7080000
278	blk.21.post_attention_norm.weight	0x2bea88700	0x3c00
279	blk.21.post_ffw_norm.weight	0x2bea8c300	0x3c00
280	blk.21.ffn_norm.weight	0x2bea8ff00	0x3c00
281	blk.21.attn_k_norm.weight	0x2bea93b00	0x400
282	blk.21.attn_k.weight	0x2bea93f00	0xf00000
283	blk.21.attn_output.weight	0x2bf993f00	0x1e00000
284	blk.21.attn_q_norm.weight	0x2c1793f00	0x400
285	blk.21.attn_q.weight	0x2c1794300	0x1e00000
286	blk.21.attn_v.weight	0x2c3594300	0xf00000
287	blk.22.attn_norm.weight	0x2c4494300	0x3c00
288	blk.22.ffn_down.weight	0x2c4497f00	0x7080000
289	blk.22.ffn_gate.weight	0x2cb517f00	0x7080000
290	blk.22.ffn_up.weight	0x2d2597f00	0x7080000
291	blk.22.post_attention_norm.weight	0x2d9617f00	0x3c00
292	blk.22.post_ffw_norm.weight	0x2d961bb00	0x3c00
293	blk.22.ffn_norm.weight	0x2d961f700	0x3c00
294	blk.22.attn_k_norm.weight	0x2d9623300	0x400
295	blk.22.attn_k.weight	0x2d9623700	0xf00000
296	blk.22.attn_output.weight	0x2da523700	0x1e00000
297	blk.22.attn_q_norm.weight	0x2dc323700	0x400
298	blk.22.attn_q.weight	0x2dc323b00	0x1e00000
299	blk.22.attn_v.weight	0x2de123b00	0xf00000
300	blk.23.attn_norm.weight	0x2df023b00	0x3c00
301	blk.23.ffn_down.weight	0x2df027700	0x7080000
302	blk.23.ffn_gate.weight	0x2e60a7700	0x7080000
303	blk.23.ffn_up.weight	0x2ed127700	0x7080000
304	blk.23.post_attention_norm.weight	0x2f41a7700	0x3c00
305	blk.23.post_ffw_norm.weight	0x2f41ab300	0x3c00
306	blk.23.ffn_norm.weight	0x2f41aef00	0x3c00
307	blk.23.attn_k_norm.weight	0x2f41b2b00	0x400
308	blk.23.attn_k.weight	0x2f41b2f00	0xf00000
309	blk.23.attn_output.weight	0x2f50b2f00	0x1e00000
310	blk.23.attn_q_norm.weight	0x2f6eb2f00	0x400
311	blk.23.attn_q.weight	0x2f6eb3300	0x1e00000
312	blk.23.attn_v.weight	0x2f8cb3300	0xf00000
313	blk.24.attn_norm.weight	0x2f9bb3300	0x3c00
314	blk.24.ffn_down.weight	0x2f9bb6f00	0x7080000
315	blk.24.ffn_gate.weight	0x300c36f00	0x7080000
316	blk.24.ffn_up.weight	0x307cb6f00	0x7080000
317	blk.24.post_attention_norm.weight	0x30ed36f00	0x3c00
318	blk.24.post_ffw_norm.weight	0x30ed3ab00	0x3c00
319	blk.24.ffn_norm.weight	0x30ed3e700	0x3c00
320	blk.24.attn_k_norm.weight	0x30ed42300	0x400
321	blk.24.attn_k.weight	0x30ed42700	0xf00000
322	blk.24.attn_output.weight	0x30fc42700	0x1e00000
323	blk.24.attn_q_norm.weight	0x311a42700	0x400
324	blk.24.attn_q.weight	0x311a42b00	0x1e00000
325	blk.24.attn_v.weight	0x313842b00	0xf00000
326	blk.25.attn_norm.weight	0x314742b00	0x3c00
327	blk.25.ffn_down.weight	0x314746700	0x7080000
328	blk.25.ffn_gate.weight	0x31b7c6700	0x7080000
329	blk.25.ffn_up.weight	0x322846700	0x7080000
330	blk.25.post_attention_norm.weight	0x3298c6700	0x3c00
331	blk.25.post_ffw_norm.weight	0x3298ca300	0x3c00
332	blk.25.ffn_norm.weight	0x3298cdf00	0x3c00
333	blk.25.attn_k_norm.weight	0x3298d1b00	0x400
334	blk.25.attn_k.weight	0x3298d1f00	0xf00000
335	blk.25.attn_output.weight	0x32a7d1f00	0x1e00000
336	blk.25.attn_q_norm.weight	0x32c5d1f00	0x400
337	blk.25.attn_q.weight	0x32c5d2300	0x1e00000
338	blk.25.attn_v.weight	0x32e3d2300	0xf00000
339	blk.26.ffn_gate.weight	0x32f2d2300	0x7080000
340	blk.26.ffn_up.weight	0x336352300	0x7080000
341	blk.26.attn_k_norm.weight	0x33d3d2300	0x400
342	blk.26.attn_k.weight	0x33d3d2700	0xf00000
343	blk.26.attn_output.weight	0x33e2d2700	0x1e00000
344	blk.26.attn_q_norm.weight	0x3400d2700	0x400
345	blk.26.attn_q.weight	0x3400d2b00	0x1e00000
346	blk.26.attn_v.weight	0x341ed2b00	0xf00000
347	blk.26.attn_norm.weight	0x342dd2b00	0x3c00
348	blk.26.ffn_down.weight	0x342dd6700	0x7080000
349	blk.26.post_attention_norm.weight	0x349e56700	0x3c00
350	blk.26.post_ffw_norm.weight	0x349e5a300	0x3c00
351	blk.26.ffn_norm.weight	0x349e5df00	0x3c00
352	blk.27.attn_norm.weight	0x349e61b00	0x3c00
353	blk.27.ffn_down.weight	0x349e65700	0x7080000
354	blk.27.ffn_gate.weight	0x350ee5700	0x7080000
355	blk.27.ffn_up.weight	0x357f65700	0x7080000
356	blk.27.post_attention_norm.weight	0x35efe5700	0x3c00
357	blk.27.post_ffw_norm.weight	0x35efe9300	0x3c00
358	blk.27.ffn_norm.weight	0x35efecf00	0x3c00
359	blk.27.attn_k_norm.weight	0x35eff0b00	0x400
360	blk.27.attn_k.weight	0x35eff0f00	0xf00000
361	blk.27.attn_output.weight	0x35fef0f00	0x1e00000
362	blk.27.attn_q_norm.weight	0x361cf0f00	0x400
363	blk.27.attn_q.weight	0x361cf1300	0x1e00000
364	blk.27.attn_v.weight	0x363af1300	0xf00000
365	blk.28.attn_norm.weight	0x3649f1300	0x3c00
366	blk.28.ffn_down.weight	0x3649f4f00	0x7080000
367	blk.28.ffn_gate.weight	0x36ba74f00	0x7080000
368	blk.28.ffn_up.weight	0x372af4f00	0x7080000
369	blk.28.post_attention_norm.weight	0x379b74f00	0x3c00
370	blk.28.post_ffw_norm.weight	0x379b78b00	0x3c00
371	blk.28.ffn_norm.weight	0x379b7c700	0x3c00
372	blk.28.attn_k_norm.weight	0x379b80300	0x400
373	blk.28.attn_k.weight	0x379b80700	0xf00000
374	blk.28.attn_output.weight	0x37aa80700	0x1e00000
375	blk.28.attn_q_norm.weight	0x37c880700	0x400
376	blk.28.attn_q.weight	0x37c880b00	0x1e00000
377	blk.28.attn_v.weight	0x37e680b00	0xf00000
378	blk.29.attn_norm.weight	0x37f580b00	0x3c00
379	blk.29.ffn_down.weight	0x37f584700	0x7080000
380	blk.29.ffn_gate.weight	0x386604700	0x7080000
381	blk.29.ffn_up.weight	0x38d684700	0x7080000
382	blk.29.post_attention_norm.weight	0x394704700	0x3c00
383	blk.29.post_ffw_norm.weight	0x394708300	0x3c00
384	blk.29.ffn_norm.weight	0x39470bf00	0x3c00
385	blk.29.attn_k_norm.weight	0x39470fb00	0x400
386	blk.29.attn_k.weight	0x39470ff00	0xf00000
387	blk.29.attn_output.weight	0x39560ff00	0x1e00000
388	blk.29.attn_q_norm.weight	0x39740ff00	0x400
389	blk.29.attn_q.weight	0x397410300	0x1e00000
390	blk.29.attn_v.weight	0x399210300	0xf00000
391	blk.30.attn_norm.weight	0x39a110300	0x3c00
392	blk.30.ffn_down.weight	0x39a113f00	0x7080000
393	blk.30.ffn_gate.weight	0x3a1193f00	0x7080000
394	blk.30.ffn_up.weight	0x3a8213f00	0x7080000
395	blk.30.post_attention_norm.weight	0x3af293f00	0x3c00
396	blk.30.post_ffw_norm.weight	0x3af297b00	0x3c00
397	blk.30.ffn_norm.weight	0x3af29b700	0x3c00
398	blk.30.attn_k_norm.weight	0x3af29f300	0x400
399	blk.30.attn_k.weight	0x3af29f700	0xf00000
400	blk.30.attn_output.weight	0x3b019f700	0x1e00000
401	blk.30.attn_q_norm.weight	0x3b1f9f700	0x400
402	blk.30.attn_q.weight	0x3b1f9fb00	0x1e00000
403	blk.30.attn_v.weight	0x3b3d9fb00	0xf00000
404	blk.31.attn_norm.weight	0x3b4c9fb00	0x3c00
405	blk.31.ffn_down.weight	0x3b4ca3700	0x7080000
406	blk.31.ffn_gate.weight	0x3bbd23700	0x7080000
407	blk.31.ffn_up.weight	0x3c2da3700	0x7080000
408	blk.31.post_attention_norm.weight	0x3c9e23700	0x3c00
409	blk.31.post_ffw_norm.weight	0x3c9e27300	0x3c00
410	blk.31.ffn_norm.weight	0x3c9e2af00	0x3c00
411	blk.31.attn_k_norm.weight	0x3c9e2eb00	0x400
412	blk.31.attn_k.weight	0x3c9e2ef00	0xf00000
413	blk.31.attn_output.weight	0x3cad2ef00	0x1e00000
414	blk.31.attn_q_norm.weight	0x3ccb2ef00	0x400
415	blk.31.attn_q.weight	0x3ccb2f300	0x1e00000
416	blk.31.attn_v.weight	0x3ce92f300	0xf00000
417	blk.32.attn_norm.weight	0x3cf82f300	0x3c00
418	blk.32.ffn_down.weight	0x3cf832f00	0x7080000
419	blk.32.ffn_gate.weight	0x3d68b2f00	0x7080000
420	blk.32.ffn_up.weight	0x3dd932f00	0x7080000
421	blk.32.post_attention_norm.weight	0x3e49b2f00	0x3c00
422	blk.32.post_ffw_norm.weight	0x3e49b6b00	0x3c00
423	blk.32.ffn_norm.weight	0x3e49ba700	0x3c00
424	blk.32.attn_k_norm.weight	0x3e49be300	0x400
425	blk.32.attn_k.weight	0x3e49be700	0xf00000
426	blk.32.attn_output.weight	0x3e58be700	0x1e00000
427	blk.32.attn_q_norm.weight	0x3e76be700	0x400
428	blk.32.attn_q.weight	0x3e76beb00	0x1e00000
429	blk.32.attn_v.weight	0x3e94beb00	0xf00000
430	blk.33.attn_norm.weight	0x3ea3beb00	0x3c00
431	blk.33.ffn_down.weight	0x3ea3c2700	0x7080000
432	blk.33.ffn_gate.weight	0x3f1442700	0x7080000
433	blk.33.ffn_up.weight	0x3f84c2700	0x7080000
434	blk.33.post_attention_norm.weight	0x3ff542700	0x3c00
435	blk.33.post_ffw_norm.weight	0x3ff546300	0x3c00
436	blk.33.ffn_norm.weight	0x3ff549f00	0x3c00
437	blk.33.attn_k_norm.weight	0x3ff54db00	0x400
438	blk.33.attn_k.weight	0x3ff54df00	0xf00000
439	blk.33.attn_output.weight	0x40044df00	0x1e00000
440	blk.33.attn_q_norm.weight	0x40224df00	0x400
441	blk.33.attn_q.weight	0x40224e300	0x1e00000
442	blk.33.attn_v.weight	0x40404e300	0xf00000
443	blk.34.attn_norm.weight	0x404f4e300	0x3c00
444	blk.34.ffn_down.weight	0x404f51f00	0x7080000
445	blk.34.ffn_gate.weight	0x40bfd1f00	0x7080000
446	blk.34.ffn_up.weight	0x413051f00	0x7080000
447	blk.34.post_attention_norm.weight	0x41a0d1f00	0x3c00
448	blk.34.post_ffw_norm.weight	0x41a0d5b00	0x3c00
449	blk.34.ffn_norm.weight	0x41a0d9700	0x3c00
450	blk.34.attn_k_norm.weight	0x41a0dd300	0x400
451	blk.34.attn_k.weight	0x41a0dd700	0xf00000
452	blk.34.attn_output.weight	0x41afdd700	0x1e00000
453	blk.34.attn_q_norm.weight	0x41cddd700	0x400
454	blk.34.attn_q.weight	0x41cdddb00	0x1e00000
455	blk.34.attn_v.weight	0x41ebddb00	0xf00000
456	blk.35.attn_norm.weight	0x41faddb00	0x3c00
457	blk.35.ffn_down.weight	0x41fae1700	0x7080000
458	blk.35.ffn_gate.weight	0x426b61700	0x7080000
459	blk.35.ffn_up.weight	0x42dbe1700	0x7080000
460	blk.35.post_attention_norm.weight	0x434c61700	0x3c00
461	blk.35.post_ffw_norm.weight	0x434c65300	0x3c00
462	blk.35.ffn_norm.weight	0x434c68f00	0x3c00
463	blk.35.attn_k_norm.weight	0x434c6cb00	0x400
464	blk.35.attn_k.weight	0x434c6cf00	0xf00000
465	blk.35.attn_output.weight	0x435b6cf00	0x1e00000
466	blk.35.attn_q_norm.weight	0x43796cf00	0x400
467	blk.35.attn_q.weight	0x43796d300	0x1e00000
468	blk.35.attn_v.weight	0x43976d300	0xf00000
469	blk.36.attn_norm.weight	0x43a66d300	0x3c00
470	blk.36.ffn_down.weight	0x43a670f00	0x7080000
471	blk.36.ffn_gate.weight	0x4416f0f00	0x7080000
472	blk.36.ffn_up.weight	0x448770f00	0x7080000
473	blk.36.post_attention_norm.weight	0x44f7f0f00	0x3c00
474	blk.36.post_ffw_norm.weight	0x44f7f4b00	0x3c00
475	blk.36.ffn_norm.weight	0x44f7f8700	0x3c00
476	blk.36.attn_k_norm.weight	0x44f7fc300	0x400
477	blk.36.attn_k.weight	0x44f7fc700	0xf00000
478	blk.36.attn_output.weight	0x4506fc700	0x1e00000
479	blk.36.attn_q_norm.weight	0x4524fc700	0x400
480	blk.36.attn_q.weight	0x4524fcb00	0x1e00000
481	blk.36.attn_v.weight	0x4542fcb00	0xf00000
482	blk.37.ffn_gate.weight	0x4551fcb00	0x7080000
483	blk.37.ffn_up.weight	0x45c27cb00	0x7080000
484	blk.37.attn_k_norm.weight	0x4632fcb00	0x400
485	blk.37.attn_k.weight	0x4632fcf00	0xf00000
486	blk.37.attn_output.weight	0x4641fcf00	0x1e00000
487	blk.37.attn_q_norm.weight	0x465ffcf00	0x400
488	blk.37.attn_q.weight	0x465ffd300	0x1e00000
489	blk.37.attn_v.weight	0x467dfd300	0xf00000
490	blk.37.attn_norm.weight	0x468cfd300	0x3c00
491	blk.37.ffn_down.weight	0x468d00f00	0x7080000
492	blk.37.post_attention_norm.weight	0x46fd80f00	0x3c00
493	blk.37.post_ffw_norm.weight	0x46fd84b00	0x3c00
494	blk.37.ffn_norm.weight	0x46fd88700	0x3c00
495	blk.38.attn_norm.weight	0x46fd8c300	0x3c00
496	blk.38.ffn_down.weight	0x46fd8ff00	0x7080000
497	blk.38.ffn_gate.weight	0x476e0ff00	0x7080000
498	blk.38.ffn_up.weight	0x47de8ff00	0x7080000
499	blk.38.post_attention_norm.weight	0x484f0ff00	0x3c00
500	blk.38.post_ffw_norm.weight	0x484f13b00	0x3c00
501	blk.38.ffn_norm.weight	0x484f17700	0x3c00
502	blk.38.attn_k_norm.weight	0x484f1b300	0x400
503	blk.38.attn_k.weight	0x484f1b700	0xf00000
504	blk.38.attn_output.weight	0x485e1b700	0x1e00000
505	blk.38.attn_q_norm.weight	0x487c1b700	0x400
506	blk.38.attn_q.weight	0x487c1bb00	0x1e00000
507	blk.38.attn_v.weight	0x489a1bb00	0xf00000
508	blk.39.attn_norm.weight	0x48a91bb00	0x3c00
509	blk.39.ffn_down.weight	0x48a91f700	0x7080000
510	blk.39.ffn_gate.weight	0x49199f700	0x7080000
511	blk.39.ffn_up.weight	0x498a1f700	0x7080000
512	blk.39.post_attention_norm.weight	0x49fa9f700	0x3c00
513	blk.39.post_ffw_norm.weight	0x49faa3300	0x3c00
514	blk.39.ffn_norm.weight	0x49faa6f00	0x3c00
515	blk.39.attn_k_norm.weight	0x49faaab00	0x400
516	blk.39.attn_k.weight	0x49faaaf00	0xf00000
517	blk.39.attn_output.weight	0x4a09aaf00	0x1e00000
518	blk.39.attn_q_norm.weight	0x4a27aaf00	0x400
519	blk.39.attn_q.weight	0x4a27ab300	0x1e00000
520	blk.39.attn_v.weight	0x4a45ab300	0xf00000
521	blk.40.attn_norm.weight	0x4a54ab300	0x3c00
522	blk.40.ffn_down.weight	0x4a54aef00	0x7080000
523	blk.40.ffn_gate.weight	0x4ac52ef00	0x7080000
524	blk.40.ffn_up.weight	0x4b35aef00	0x7080000
525	blk.40.post_attention_norm.weight	0x4ba62ef00	0x3c00
526	blk.40.post_ffw_norm.weight	0x4ba632b00	0x3c00
527	blk.40.ffn_norm.weight	0x4ba636700	0x3c00
528	blk.40.attn_k_norm.weight	0x4ba63a300	0x400
529	blk.40.attn_k.weight	0x4ba63a700	0xf00000
530	blk.40.attn_output.weight	0x4bb53a700	0x1e00000
531	blk.40.attn_q_norm.weight	0x4bd33a700	0x400
532	blk.40.attn_q.weight	0x4bd33ab00	0x1e00000
533	blk.40.attn_v.weight	0x4bf13ab00	0xf00000
534	blk.41.attn_norm.weight	0x4c003ab00	0x3c00
535	blk.41.ffn_down.weight	0x4c003e700	0x7080000
536	blk.41.ffn_gate.weight	0x4c70be700	0x7080000
537	blk.41.ffn_up.weight	0x4ce13e700	0x7080000
538	blk.41.post_attention_norm.weight	0x4d51be700	0x3c00
539	blk.41.post_ffw_norm.weight	0x4d51c2300	0x3c00
540	blk.41.ffn_norm.weight	0x4d51c5f00	0x3c00
541	blk.41.attn_k_norm.weight	0x4d51c9b00	0x400
542	blk.41.attn_k.weight	0x4d51c9f00	0xf00000
543	blk.41.attn_output.weight	0x4d60c9f00	0x1e00000
544	blk.41.attn_q_norm.weight	0x4d7ec9f00	0x400
545	blk.41.attn_q.weight	0x4d7eca300	0x1e00000
546	blk.41.attn_v.weight	0x4d9cca300	0xf00000
547	blk.42.attn_norm.weight	0x4dabca300	0x3c00
548	blk.42.ffn_down.weight	0x4dabcdf00	0x7080000
549	blk.42.ffn_gate.weight	0x4e1c4df00	0x7080000
550	blk.42.ffn_up.weight	0x4e8ccdf00	0x7080000
551	blk.42.post_attention_norm.weight	0x4efd4df00	0x3c00
552	blk.42.post_ffw_norm.weight	0x4efd51b00	0x3c00
553	blk.42.ffn_norm.weight	0x4efd55700	0x3c00
554	blk.42.attn_k_norm.weight	0x4efd59300	0x400
555	blk.42.attn_k.weight	0x4efd59700	0xf00000
556	blk.42.attn_output.weight	0x4f0c59700	0x1e00000
557	blk.42.attn_q_norm.weight	0x4f2a59700	0x400
558	blk.42.attn_q.weight	0x4f2a59b00	0x1e00000
559	blk.42.attn_v.weight	0x4f4859b00	0xf00000
560	blk.43.attn_norm.weight	0x4f5759b00	0x3c00
561	blk.43.ffn_down.weight	0x4f575d700	0x7080000
562	blk.43.ffn_gate.weight	0x4fc7dd700	0x7080000
563	blk.43.ffn_up.weight	0x50385d700	0x7080000
564	blk.43.post_attention_norm.weight	0x50a8dd700	0x3c00
565	blk.43.post_ffw_norm.weight	0x50a8e1300	0x3c00
566	blk.43.ffn_norm.weight	0x50a8e4f00	0x3c00
567	blk.43.attn_k_norm.weight	0x50a8e8b00	0x400
568	blk.43.attn_k.weight	0x50a8e8f00	0xf00000
569	blk.43.attn_output.weight	0x50b7e8f00	0x1e00000
570	blk.43.attn_q_norm.weight	0x50d5e8f00	0x400
571	blk.43.attn_q.weight	0x50d5e9300	0x1e00000
572	blk.43.attn_v.weight	0x50f3e9300	0xf00000
573	blk.44.attn_norm.weight	0x5102e9300	0x3c00
574	blk.44.ffn_down.weight	0x5102ecf00	0x7080000
575	blk.44.ffn_gate.weight	0x51736cf00	0x7080000
576	blk.44.ffn_up.weight	0x51e3ecf00	0x7080000
577	blk.44.post_attention_norm.weight	0x52546cf00	0x3c00
578	blk.44.post_ffw_norm.weight	0x525470b00	0x3c00
579	blk.44.ffn_norm.weight	0x525474700	0x3c00
580	blk.44.attn_k_norm.weight	0x525478300	0x400
581	blk.44.attn_k.weight	0x525478700	0xf00000
582	blk.44.attn_output.weight	0x526378700	0x1e00000
583	blk.44.attn_q_norm.weight	0x528178700	0x400
584	blk.44.attn_q.weight	0x528178b00	0x1e00000
585	blk.44.attn_v.weight	0x529f78b00	0xf00000
586	blk.45.attn_norm.weight	0x52ae78b00	0x3c00
587	blk.45.ffn_down.weight	0x52ae7c700	0x7080000
588	blk.45.ffn_gate.weight	0x531efc700	0x7080000
589	blk.45.ffn_up.weight	0x538f7c700	0x7080000
590	blk.45.post_attention_norm.weight	0x53fffc700	0x3c00
591	blk.45.post_ffw_norm.weight	0x540000300	0x3c00
592	blk.45.ffn_norm.weight	0x540003f00	0x3c00
593	blk.45.attn_k_norm.weight	0x540007b00	0x400
594	blk.45.attn_k.weight	0x540007f00	0xf00000
595	blk.45.attn_output.weight	0x540f07f00	0x1e00000
596	blk.45.attn_q_norm.weight	0x542d07f00	0x400
597	blk.45.attn_q.weight	0x542d08300	0x1e00000
598	blk.45.attn_v.weight	0x544b08300	0xf00000
599	blk.46.attn_norm.weight	0x545a08300	0x3c00
600	blk.46.ffn_down.weight	0x545a0bf00	0x7080000
601	blk.46.ffn_gate.weight	0x54ca8bf00	0x7080000
602	blk.46.ffn_up.weight	0x553b0bf00	0x7080000
603	blk.46.post_attention_norm.weight	0x55ab8bf00	0x3c00
604	blk.46.post_ffw_norm.weight	0x55ab8fb00	0x3c00
605	blk.46.ffn_norm.weight	0x55ab93700	0x3c00
606	blk.46.attn_k_norm.weight	0x55ab97300	0x400
607	blk.46.attn_k.weight	0x55ab97700	0xf00000
608	blk.46.attn_output.weight	0x55ba97700	0x1e00000
609	blk.46.attn_q_norm.weight	0x55d897700	0x400
610	blk.46.attn_q.weight	0x55d897b00	0x1e00000
611	blk.46.attn_v.weight	0x55f697b00	0xf00000
612	blk.47.attn_norm.weight	0x560597b00	0x3c00
613	blk.47.ffn_down.weight	0x56059b700	0x7080000
614	blk.47.ffn_gate.weight	0x56761b700	0x7080000
615	blk.47.ffn_up.weight	0x56e69b700	0x7080000
616	blk.47.post_attention_norm.weight	0x57571b700	0x3c00
617	blk.47.post_ffw_norm.weight	0x57571f300	0x3c00
618	blk.47.ffn_norm.weight	0x575722f00	0x3c00
619	blk.47.attn_k_norm.weight	0x575726b00	0x400
620	blk.47.attn_k.weight	0x575726f00	0xf00000
621	blk.47.attn_output.weight	0x576626f00	0x1e00000
622	blk.47.attn_q_norm.weight	0x578426f00	0x400
623	blk.47.attn_q.weight	0x578427300	0x1e00000
624	blk.47.attn_v.weight	0x57a227300	0xf00000
625	output_norm.weight	0x57b127300	0x3c00

Base Tensor Group : ~1B Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
0	token_embd.weight	Token Embedding (W)	(~1B) 1006632960	3840 x 262144 x 1 x 1	F16
625	output_norm.weight	Output Normalization (W)	(~4K) 3840	3840 x 1 x 1 x 1	F32

• Total elements in base: ( ~1B) 1006636800
• Percentage of total elements: 8.56%

Block 0 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
1	blk.0.attn_norm.weight	Block 0 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
2	blk.0.ffn_down.weight	Block 0 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
3	blk.0.ffn_gate.weight	Block 0 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
4	blk.0.ffn_up.weight	Block 0 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
5	blk.0.post_attention_norm.weight	Block 0 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
6	blk.0.post_ffw_norm.weight	Block 0 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
7	blk.0.ffn_norm.weight	Block 0 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
8	blk.0.attn_k_norm.weight	Block 0 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
9	blk.0.attn_k.weight	Block 0 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
10	blk.0.attn_output.weight	Block 0 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
11	blk.0.attn_q_norm.weight	Block 0 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
12	blk.0.attn_q.weight	Block 0 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
13	blk.0.attn_v.weight	Block 0 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.0: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 1 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
14	blk.1.attn_norm.weight	Block 1 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
15	blk.1.ffn_down.weight	Block 1 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
16	blk.1.ffn_gate.weight	Block 1 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
17	blk.1.ffn_up.weight	Block 1 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
18	blk.1.post_attention_norm.weight	Block 1 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
19	blk.1.post_ffw_norm.weight	Block 1 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
20	blk.1.ffn_norm.weight	Block 1 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
21	blk.1.attn_k_norm.weight	Block 1 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
22	blk.1.attn_k.weight	Block 1 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
23	blk.1.attn_output.weight	Block 1 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
24	blk.1.attn_q_norm.weight	Block 1 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
25	blk.1.attn_q.weight	Block 1 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
26	blk.1.attn_v.weight	Block 1 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.1: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 2 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
27	blk.2.attn_norm.weight	Block 2 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
28	blk.2.ffn_down.weight	Block 2 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
29	blk.2.ffn_gate.weight	Block 2 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
30	blk.2.ffn_up.weight	Block 2 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
31	blk.2.post_attention_norm.weight	Block 2 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
32	blk.2.post_ffw_norm.weight	Block 2 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
33	blk.2.ffn_norm.weight	Block 2 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
34	blk.2.attn_k_norm.weight	Block 2 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
35	blk.2.attn_k.weight	Block 2 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
36	blk.2.attn_output.weight	Block 2 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
37	blk.2.attn_q_norm.weight	Block 2 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
38	blk.2.attn_q.weight	Block 2 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
39	blk.2.attn_v.weight	Block 2 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.2: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 3 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
40	blk.3.attn_norm.weight	Block 3 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
41	blk.3.ffn_down.weight	Block 3 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
42	blk.3.ffn_gate.weight	Block 3 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
43	blk.3.ffn_up.weight	Block 3 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
44	blk.3.post_attention_norm.weight	Block 3 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
45	blk.3.post_ffw_norm.weight	Block 3 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
46	blk.3.ffn_norm.weight	Block 3 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
47	blk.3.attn_k_norm.weight	Block 3 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
48	blk.3.attn_k.weight	Block 3 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
49	blk.3.attn_output.weight	Block 3 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
50	blk.3.attn_q_norm.weight	Block 3 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
51	blk.3.attn_q.weight	Block 3 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
52	blk.3.attn_v.weight	Block 3 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.3: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 4 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
53	blk.4.ffn_gate.weight	Block 4 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
54	blk.4.ffn_up.weight	Block 4 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
55	blk.4.attn_k_norm.weight	Block 4 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
56	blk.4.attn_k.weight	Block 4 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
57	blk.4.attn_output.weight	Block 4 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
58	blk.4.attn_q_norm.weight	Block 4 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
59	blk.4.attn_q.weight	Block 4 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
60	blk.4.attn_v.weight	Block 4 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
134	blk.4.attn_norm.weight	Block 4 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
135	blk.4.ffn_down.weight	Block 4 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
136	blk.4.post_attention_norm.weight	Block 4 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
137	blk.4.post_ffw_norm.weight	Block 4 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
138	blk.4.ffn_norm.weight	Block 4 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32

• Total elements in blk.4: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 10 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
61	blk.10.attn_norm.weight	Block 10 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
62	blk.10.ffn_down.weight	Block 10 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
63	blk.10.ffn_gate.weight	Block 10 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
64	blk.10.ffn_up.weight	Block 10 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
65	blk.10.post_attention_norm.weight	Block 10 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
66	blk.10.post_ffw_norm.weight	Block 10 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
67	blk.10.ffn_norm.weight	Block 10 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
68	blk.10.attn_k_norm.weight	Block 10 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
69	blk.10.attn_k.weight	Block 10 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
70	blk.10.attn_output.weight	Block 10 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
71	blk.10.attn_q_norm.weight	Block 10 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
72	blk.10.attn_q.weight	Block 10 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
73	blk.10.attn_v.weight	Block 10 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.10: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 11 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
74	blk.11.attn_norm.weight	Block 11 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
75	blk.11.ffn_down.weight	Block 11 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
76	blk.11.ffn_gate.weight	Block 11 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
77	blk.11.ffn_up.weight	Block 11 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
78	blk.11.post_attention_norm.weight	Block 11 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
79	blk.11.post_ffw_norm.weight	Block 11 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
80	blk.11.ffn_norm.weight	Block 11 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
81	blk.11.attn_k_norm.weight	Block 11 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
82	blk.11.attn_k.weight	Block 11 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
83	blk.11.attn_output.weight	Block 11 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
84	blk.11.attn_q_norm.weight	Block 11 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
85	blk.11.attn_q.weight	Block 11 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
86	blk.11.attn_v.weight	Block 11 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.11: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 12 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
87	blk.12.attn_norm.weight	Block 12 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
88	blk.12.ffn_down.weight	Block 12 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
89	blk.12.ffn_gate.weight	Block 12 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
90	blk.12.ffn_up.weight	Block 12 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
91	blk.12.post_attention_norm.weight	Block 12 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
92	blk.12.post_ffw_norm.weight	Block 12 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
93	blk.12.ffn_norm.weight	Block 12 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
94	blk.12.attn_k_norm.weight	Block 12 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
95	blk.12.attn_k.weight	Block 12 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
96	blk.12.attn_output.weight	Block 12 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
97	blk.12.attn_q_norm.weight	Block 12 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
98	blk.12.attn_q.weight	Block 12 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
99	blk.12.attn_v.weight	Block 12 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.12: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 13 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
100	blk.13.attn_norm.weight	Block 13 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
101	blk.13.ffn_down.weight	Block 13 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
102	blk.13.ffn_gate.weight	Block 13 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
103	blk.13.ffn_up.weight	Block 13 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
104	blk.13.post_attention_norm.weight	Block 13 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
105	blk.13.post_ffw_norm.weight	Block 13 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
106	blk.13.ffn_norm.weight	Block 13 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
107	blk.13.attn_k_norm.weight	Block 13 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
108	blk.13.attn_k.weight	Block 13 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
109	blk.13.attn_output.weight	Block 13 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
110	blk.13.attn_q_norm.weight	Block 13 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
111	blk.13.attn_q.weight	Block 13 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
112	blk.13.attn_v.weight	Block 13 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.13: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 14 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
113	blk.14.attn_norm.weight	Block 14 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
114	blk.14.ffn_down.weight	Block 14 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
115	blk.14.ffn_gate.weight	Block 14 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
116	blk.14.ffn_up.weight	Block 14 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
117	blk.14.post_attention_norm.weight	Block 14 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
118	blk.14.post_ffw_norm.weight	Block 14 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
119	blk.14.ffn_norm.weight	Block 14 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
120	blk.14.attn_k_norm.weight	Block 14 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
121	blk.14.attn_k.weight	Block 14 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
122	blk.14.attn_output.weight	Block 14 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
123	blk.14.attn_q_norm.weight	Block 14 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
124	blk.14.attn_q.weight	Block 14 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
125	blk.14.attn_v.weight	Block 14 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.14: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 15 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
126	blk.15.ffn_gate.weight	Block 15 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
127	blk.15.ffn_up.weight	Block 15 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
128	blk.15.attn_k_norm.weight	Block 15 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
129	blk.15.attn_k.weight	Block 15 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
130	blk.15.attn_output.weight	Block 15 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
131	blk.15.attn_q_norm.weight	Block 15 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
132	blk.15.attn_q.weight	Block 15 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
133	blk.15.attn_v.weight	Block 15 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
204	blk.15.attn_norm.weight	Block 15 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
205	blk.15.ffn_down.weight	Block 15 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
206	blk.15.post_attention_norm.weight	Block 15 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
207	blk.15.post_ffw_norm.weight	Block 15 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
208	blk.15.ffn_norm.weight	Block 15 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32

• Total elements in blk.15: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 5 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
139	blk.5.attn_norm.weight	Block 5 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
140	blk.5.ffn_down.weight	Block 5 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
141	blk.5.ffn_gate.weight	Block 5 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
142	blk.5.ffn_up.weight	Block 5 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
143	blk.5.post_attention_norm.weight	Block 5 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
144	blk.5.post_ffw_norm.weight	Block 5 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
145	blk.5.ffn_norm.weight	Block 5 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
146	blk.5.attn_k_norm.weight	Block 5 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
147	blk.5.attn_k.weight	Block 5 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
148	blk.5.attn_output.weight	Block 5 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
149	blk.5.attn_q_norm.weight	Block 5 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
150	blk.5.attn_q.weight	Block 5 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
151	blk.5.attn_v.weight	Block 5 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.5: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 6 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
152	blk.6.attn_norm.weight	Block 6 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
153	blk.6.ffn_down.weight	Block 6 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
154	blk.6.ffn_gate.weight	Block 6 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
155	blk.6.ffn_up.weight	Block 6 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
156	blk.6.post_attention_norm.weight	Block 6 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
157	blk.6.post_ffw_norm.weight	Block 6 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
158	blk.6.ffn_norm.weight	Block 6 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
159	blk.6.attn_k_norm.weight	Block 6 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
160	blk.6.attn_k.weight	Block 6 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
161	blk.6.attn_output.weight	Block 6 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
162	blk.6.attn_q_norm.weight	Block 6 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
163	blk.6.attn_q.weight	Block 6 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
164	blk.6.attn_v.weight	Block 6 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.6: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 7 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
165	blk.7.attn_norm.weight	Block 7 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
166	blk.7.ffn_down.weight	Block 7 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
167	blk.7.ffn_gate.weight	Block 7 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
168	blk.7.ffn_up.weight	Block 7 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
169	blk.7.post_attention_norm.weight	Block 7 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
170	blk.7.post_ffw_norm.weight	Block 7 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
171	blk.7.ffn_norm.weight	Block 7 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
172	blk.7.attn_k_norm.weight	Block 7 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
173	blk.7.attn_k.weight	Block 7 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
174	blk.7.attn_output.weight	Block 7 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
175	blk.7.attn_q_norm.weight	Block 7 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
176	blk.7.attn_q.weight	Block 7 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
177	blk.7.attn_v.weight	Block 7 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.7: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 8 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
178	blk.8.attn_norm.weight	Block 8 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
179	blk.8.ffn_down.weight	Block 8 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
180	blk.8.ffn_gate.weight	Block 8 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
181	blk.8.ffn_up.weight	Block 8 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
182	blk.8.post_attention_norm.weight	Block 8 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
183	blk.8.post_ffw_norm.weight	Block 8 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
184	blk.8.ffn_norm.weight	Block 8 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
185	blk.8.attn_k_norm.weight	Block 8 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
186	blk.8.attn_k.weight	Block 8 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
187	blk.8.attn_output.weight	Block 8 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
188	blk.8.attn_q_norm.weight	Block 8 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
189	blk.8.attn_q.weight	Block 8 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
190	blk.8.attn_v.weight	Block 8 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.8: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 9 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
191	blk.9.attn_norm.weight	Block 9 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
192	blk.9.ffn_down.weight	Block 9 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
193	blk.9.ffn_gate.weight	Block 9 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
194	blk.9.ffn_up.weight	Block 9 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
195	blk.9.post_attention_norm.weight	Block 9 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
196	blk.9.post_ffw_norm.weight	Block 9 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
197	blk.9.ffn_norm.weight	Block 9 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
198	blk.9.attn_k_norm.weight	Block 9 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
199	blk.9.attn_k.weight	Block 9 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
200	blk.9.attn_output.weight	Block 9 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
201	blk.9.attn_q_norm.weight	Block 9 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
202	blk.9.attn_q.weight	Block 9 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
203	blk.9.attn_v.weight	Block 9 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.9: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 16 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
209	blk.16.attn_norm.weight	Block 16 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
210	blk.16.ffn_down.weight	Block 16 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
211	blk.16.ffn_gate.weight	Block 16 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
212	blk.16.ffn_up.weight	Block 16 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
213	blk.16.post_attention_norm.weight	Block 16 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
214	blk.16.post_ffw_norm.weight	Block 16 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
215	blk.16.ffn_norm.weight	Block 16 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
216	blk.16.attn_k_norm.weight	Block 16 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
217	blk.16.attn_k.weight	Block 16 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
218	blk.16.attn_output.weight	Block 16 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
219	blk.16.attn_q_norm.weight	Block 16 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
220	blk.16.attn_q.weight	Block 16 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
221	blk.16.attn_v.weight	Block 16 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.16: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 17 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
222	blk.17.attn_norm.weight	Block 17 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
223	blk.17.ffn_down.weight	Block 17 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
224	blk.17.ffn_gate.weight	Block 17 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
225	blk.17.ffn_up.weight	Block 17 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
226	blk.17.post_attention_norm.weight	Block 17 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
227	blk.17.post_ffw_norm.weight	Block 17 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
228	blk.17.ffn_norm.weight	Block 17 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
229	blk.17.attn_k_norm.weight	Block 17 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
230	blk.17.attn_k.weight	Block 17 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
231	blk.17.attn_output.weight	Block 17 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
232	blk.17.attn_q_norm.weight	Block 17 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
233	blk.17.attn_q.weight	Block 17 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
234	blk.17.attn_v.weight	Block 17 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.17: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 18 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
235	blk.18.attn_norm.weight	Block 18 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
236	blk.18.ffn_down.weight	Block 18 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
237	blk.18.ffn_gate.weight	Block 18 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
238	blk.18.ffn_up.weight	Block 18 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
239	blk.18.post_attention_norm.weight	Block 18 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
240	blk.18.post_ffw_norm.weight	Block 18 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
241	blk.18.ffn_norm.weight	Block 18 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
242	blk.18.attn_k_norm.weight	Block 18 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
243	blk.18.attn_k.weight	Block 18 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
244	blk.18.attn_output.weight	Block 18 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
245	blk.18.attn_q_norm.weight	Block 18 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
246	blk.18.attn_q.weight	Block 18 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
247	blk.18.attn_v.weight	Block 18 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.18: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 19 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
248	blk.19.attn_norm.weight	Block 19 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
249	blk.19.ffn_down.weight	Block 19 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
250	blk.19.ffn_gate.weight	Block 19 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
251	blk.19.ffn_up.weight	Block 19 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
252	blk.19.post_attention_norm.weight	Block 19 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
253	blk.19.post_ffw_norm.weight	Block 19 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
254	blk.19.ffn_norm.weight	Block 19 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
255	blk.19.attn_k_norm.weight	Block 19 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
256	blk.19.attn_k.weight	Block 19 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
257	blk.19.attn_output.weight	Block 19 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
258	blk.19.attn_q_norm.weight	Block 19 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
259	blk.19.attn_q.weight	Block 19 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
260	blk.19.attn_v.weight	Block 19 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.19: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 20 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
261	blk.20.attn_norm.weight	Block 20 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
262	blk.20.ffn_down.weight	Block 20 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
263	blk.20.ffn_gate.weight	Block 20 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
264	blk.20.ffn_up.weight	Block 20 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
265	blk.20.post_attention_norm.weight	Block 20 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
266	blk.20.post_ffw_norm.weight	Block 20 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
267	blk.20.ffn_norm.weight	Block 20 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
268	blk.20.attn_k_norm.weight	Block 20 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
269	blk.20.attn_k.weight	Block 20 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
270	blk.20.attn_output.weight	Block 20 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
271	blk.20.attn_q_norm.weight	Block 20 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
272	blk.20.attn_q.weight	Block 20 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
273	blk.20.attn_v.weight	Block 20 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.20: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 21 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
274	blk.21.attn_norm.weight	Block 21 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
275	blk.21.ffn_down.weight	Block 21 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
276	blk.21.ffn_gate.weight	Block 21 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
277	blk.21.ffn_up.weight	Block 21 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
278	blk.21.post_attention_norm.weight	Block 21 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
279	blk.21.post_ffw_norm.weight	Block 21 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
280	blk.21.ffn_norm.weight	Block 21 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
281	blk.21.attn_k_norm.weight	Block 21 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
282	blk.21.attn_k.weight	Block 21 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
283	blk.21.attn_output.weight	Block 21 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
284	blk.21.attn_q_norm.weight	Block 21 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
285	blk.21.attn_q.weight	Block 21 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
286	blk.21.attn_v.weight	Block 21 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.21: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 22 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
287	blk.22.attn_norm.weight	Block 22 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
288	blk.22.ffn_down.weight	Block 22 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
289	blk.22.ffn_gate.weight	Block 22 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
290	blk.22.ffn_up.weight	Block 22 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
291	blk.22.post_attention_norm.weight	Block 22 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
292	blk.22.post_ffw_norm.weight	Block 22 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
293	blk.22.ffn_norm.weight	Block 22 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
294	blk.22.attn_k_norm.weight	Block 22 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
295	blk.22.attn_k.weight	Block 22 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
296	blk.22.attn_output.weight	Block 22 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
297	blk.22.attn_q_norm.weight	Block 22 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
298	blk.22.attn_q.weight	Block 22 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
299	blk.22.attn_v.weight	Block 22 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.22: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 23 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
300	blk.23.attn_norm.weight	Block 23 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
301	blk.23.ffn_down.weight	Block 23 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
302	blk.23.ffn_gate.weight	Block 23 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
303	blk.23.ffn_up.weight	Block 23 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
304	blk.23.post_attention_norm.weight	Block 23 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
305	blk.23.post_ffw_norm.weight	Block 23 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
306	blk.23.ffn_norm.weight	Block 23 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
307	blk.23.attn_k_norm.weight	Block 23 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
308	blk.23.attn_k.weight	Block 23 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
309	blk.23.attn_output.weight	Block 23 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
310	blk.23.attn_q_norm.weight	Block 23 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
311	blk.23.attn_q.weight	Block 23 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
312	blk.23.attn_v.weight	Block 23 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.23: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 24 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
313	blk.24.attn_norm.weight	Block 24 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
314	blk.24.ffn_down.weight	Block 24 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
315	blk.24.ffn_gate.weight	Block 24 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
316	blk.24.ffn_up.weight	Block 24 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
317	blk.24.post_attention_norm.weight	Block 24 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
318	blk.24.post_ffw_norm.weight	Block 24 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
319	blk.24.ffn_norm.weight	Block 24 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
320	blk.24.attn_k_norm.weight	Block 24 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
321	blk.24.attn_k.weight	Block 24 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
322	blk.24.attn_output.weight	Block 24 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
323	blk.24.attn_q_norm.weight	Block 24 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
324	blk.24.attn_q.weight	Block 24 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
325	blk.24.attn_v.weight	Block 24 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.24: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 25 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
326	blk.25.attn_norm.weight	Block 25 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
327	blk.25.ffn_down.weight	Block 25 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
328	blk.25.ffn_gate.weight	Block 25 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
329	blk.25.ffn_up.weight	Block 25 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
330	blk.25.post_attention_norm.weight	Block 25 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
331	blk.25.post_ffw_norm.weight	Block 25 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
332	blk.25.ffn_norm.weight	Block 25 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
333	blk.25.attn_k_norm.weight	Block 25 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
334	blk.25.attn_k.weight	Block 25 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
335	blk.25.attn_output.weight	Block 25 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
336	blk.25.attn_q_norm.weight	Block 25 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
337	blk.25.attn_q.weight	Block 25 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
338	blk.25.attn_v.weight	Block 25 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.25: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 26 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
339	blk.26.ffn_gate.weight	Block 26 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
340	blk.26.ffn_up.weight	Block 26 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
341	blk.26.attn_k_norm.weight	Block 26 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
342	blk.26.attn_k.weight	Block 26 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
343	blk.26.attn_output.weight	Block 26 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
344	blk.26.attn_q_norm.weight	Block 26 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
345	blk.26.attn_q.weight	Block 26 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
346	blk.26.attn_v.weight	Block 26 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
347	blk.26.attn_norm.weight	Block 26 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
348	blk.26.ffn_down.weight	Block 26 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
349	blk.26.post_attention_norm.weight	Block 26 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
350	blk.26.post_ffw_norm.weight	Block 26 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
351	blk.26.ffn_norm.weight	Block 26 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32

• Total elements in blk.26: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 27 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
352	blk.27.attn_norm.weight	Block 27 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
353	blk.27.ffn_down.weight	Block 27 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
354	blk.27.ffn_gate.weight	Block 27 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
355	blk.27.ffn_up.weight	Block 27 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
356	blk.27.post_attention_norm.weight	Block 27 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
357	blk.27.post_ffw_norm.weight	Block 27 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
358	blk.27.ffn_norm.weight	Block 27 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
359	blk.27.attn_k_norm.weight	Block 27 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
360	blk.27.attn_k.weight	Block 27 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
361	blk.27.attn_output.weight	Block 27 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
362	blk.27.attn_q_norm.weight	Block 27 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
363	blk.27.attn_q.weight	Block 27 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
364	blk.27.attn_v.weight	Block 27 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.27: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 28 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
365	blk.28.attn_norm.weight	Block 28 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
366	blk.28.ffn_down.weight	Block 28 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
367	blk.28.ffn_gate.weight	Block 28 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
368	blk.28.ffn_up.weight	Block 28 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
369	blk.28.post_attention_norm.weight	Block 28 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
370	blk.28.post_ffw_norm.weight	Block 28 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
371	blk.28.ffn_norm.weight	Block 28 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
372	blk.28.attn_k_norm.weight	Block 28 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
373	blk.28.attn_k.weight	Block 28 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
374	blk.28.attn_output.weight	Block 28 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
375	blk.28.attn_q_norm.weight	Block 28 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
376	blk.28.attn_q.weight	Block 28 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
377	blk.28.attn_v.weight	Block 28 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.28: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 29 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
378	blk.29.attn_norm.weight	Block 29 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
379	blk.29.ffn_down.weight	Block 29 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
380	blk.29.ffn_gate.weight	Block 29 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
381	blk.29.ffn_up.weight	Block 29 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
382	blk.29.post_attention_norm.weight	Block 29 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
383	blk.29.post_ffw_norm.weight	Block 29 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
384	blk.29.ffn_norm.weight	Block 29 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
385	blk.29.attn_k_norm.weight	Block 29 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
386	blk.29.attn_k.weight	Block 29 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
387	blk.29.attn_output.weight	Block 29 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
388	blk.29.attn_q_norm.weight	Block 29 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
389	blk.29.attn_q.weight	Block 29 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
390	blk.29.attn_v.weight	Block 29 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.29: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 30 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
391	blk.30.attn_norm.weight	Block 30 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
392	blk.30.ffn_down.weight	Block 30 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
393	blk.30.ffn_gate.weight	Block 30 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
394	blk.30.ffn_up.weight	Block 30 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
395	blk.30.post_attention_norm.weight	Block 30 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
396	blk.30.post_ffw_norm.weight	Block 30 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
397	blk.30.ffn_norm.weight	Block 30 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
398	blk.30.attn_k_norm.weight	Block 30 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
399	blk.30.attn_k.weight	Block 30 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
400	blk.30.attn_output.weight	Block 30 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
401	blk.30.attn_q_norm.weight	Block 30 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
402	blk.30.attn_q.weight	Block 30 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
403	blk.30.attn_v.weight	Block 30 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.30: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 31 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
404	blk.31.attn_norm.weight	Block 31 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
405	blk.31.ffn_down.weight	Block 31 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
406	blk.31.ffn_gate.weight	Block 31 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
407	blk.31.ffn_up.weight	Block 31 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
408	blk.31.post_attention_norm.weight	Block 31 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
409	blk.31.post_ffw_norm.weight	Block 31 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
410	blk.31.ffn_norm.weight	Block 31 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
411	blk.31.attn_k_norm.weight	Block 31 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
412	blk.31.attn_k.weight	Block 31 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
413	blk.31.attn_output.weight	Block 31 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
414	blk.31.attn_q_norm.weight	Block 31 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
415	blk.31.attn_q.weight	Block 31 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
416	blk.31.attn_v.weight	Block 31 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.31: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 32 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
417	blk.32.attn_norm.weight	Block 32 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
418	blk.32.ffn_down.weight	Block 32 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
419	blk.32.ffn_gate.weight	Block 32 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
420	blk.32.ffn_up.weight	Block 32 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
421	blk.32.post_attention_norm.weight	Block 32 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
422	blk.32.post_ffw_norm.weight	Block 32 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
423	blk.32.ffn_norm.weight	Block 32 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
424	blk.32.attn_k_norm.weight	Block 32 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
425	blk.32.attn_k.weight	Block 32 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
426	blk.32.attn_output.weight	Block 32 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
427	blk.32.attn_q_norm.weight	Block 32 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
428	blk.32.attn_q.weight	Block 32 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
429	blk.32.attn_v.weight	Block 32 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.32: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 33 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
430	blk.33.attn_norm.weight	Block 33 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
431	blk.33.ffn_down.weight	Block 33 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
432	blk.33.ffn_gate.weight	Block 33 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
433	blk.33.ffn_up.weight	Block 33 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
434	blk.33.post_attention_norm.weight	Block 33 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
435	blk.33.post_ffw_norm.weight	Block 33 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
436	blk.33.ffn_norm.weight	Block 33 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
437	blk.33.attn_k_norm.weight	Block 33 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
438	blk.33.attn_k.weight	Block 33 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
439	blk.33.attn_output.weight	Block 33 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
440	blk.33.attn_q_norm.weight	Block 33 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
441	blk.33.attn_q.weight	Block 33 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
442	blk.33.attn_v.weight	Block 33 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.33: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 34 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
443	blk.34.attn_norm.weight	Block 34 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
444	blk.34.ffn_down.weight	Block 34 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
445	blk.34.ffn_gate.weight	Block 34 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
446	blk.34.ffn_up.weight	Block 34 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
447	blk.34.post_attention_norm.weight	Block 34 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
448	blk.34.post_ffw_norm.weight	Block 34 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
449	blk.34.ffn_norm.weight	Block 34 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
450	blk.34.attn_k_norm.weight	Block 34 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
451	blk.34.attn_k.weight	Block 34 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
452	blk.34.attn_output.weight	Block 34 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
453	blk.34.attn_q_norm.weight	Block 34 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
454	blk.34.attn_q.weight	Block 34 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
455	blk.34.attn_v.weight	Block 34 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.34: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 35 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
456	blk.35.attn_norm.weight	Block 35 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
457	blk.35.ffn_down.weight	Block 35 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
458	blk.35.ffn_gate.weight	Block 35 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
459	blk.35.ffn_up.weight	Block 35 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
460	blk.35.post_attention_norm.weight	Block 35 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
461	blk.35.post_ffw_norm.weight	Block 35 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
462	blk.35.ffn_norm.weight	Block 35 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
463	blk.35.attn_k_norm.weight	Block 35 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
464	blk.35.attn_k.weight	Block 35 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
465	blk.35.attn_output.weight	Block 35 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
466	blk.35.attn_q_norm.weight	Block 35 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
467	blk.35.attn_q.weight	Block 35 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
468	blk.35.attn_v.weight	Block 35 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.35: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 36 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
469	blk.36.attn_norm.weight	Block 36 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
470	blk.36.ffn_down.weight	Block 36 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
471	blk.36.ffn_gate.weight	Block 36 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
472	blk.36.ffn_up.weight	Block 36 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
473	blk.36.post_attention_norm.weight	Block 36 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
474	blk.36.post_ffw_norm.weight	Block 36 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
475	blk.36.ffn_norm.weight	Block 36 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
476	blk.36.attn_k_norm.weight	Block 36 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
477	blk.36.attn_k.weight	Block 36 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
478	blk.36.attn_output.weight	Block 36 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
479	blk.36.attn_q_norm.weight	Block 36 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
480	blk.36.attn_q.weight	Block 36 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
481	blk.36.attn_v.weight	Block 36 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.36: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 37 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
482	blk.37.ffn_gate.weight	Block 37 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
483	blk.37.ffn_up.weight	Block 37 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
484	blk.37.attn_k_norm.weight	Block 37 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
485	blk.37.attn_k.weight	Block 37 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
486	blk.37.attn_output.weight	Block 37 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
487	blk.37.attn_q_norm.weight	Block 37 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
488	blk.37.attn_q.weight	Block 37 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
489	blk.37.attn_v.weight	Block 37 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
490	blk.37.attn_norm.weight	Block 37 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
491	blk.37.ffn_down.weight	Block 37 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
492	blk.37.post_attention_norm.weight	Block 37 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
493	blk.37.post_ffw_norm.weight	Block 37 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
494	blk.37.ffn_norm.weight	Block 37 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32

• Total elements in blk.37: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 38 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
495	blk.38.attn_norm.weight	Block 38 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
496	blk.38.ffn_down.weight	Block 38 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
497	blk.38.ffn_gate.weight	Block 38 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
498	blk.38.ffn_up.weight	Block 38 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
499	blk.38.post_attention_norm.weight	Block 38 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
500	blk.38.post_ffw_norm.weight	Block 38 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
501	blk.38.ffn_norm.weight	Block 38 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
502	blk.38.attn_k_norm.weight	Block 38 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
503	blk.38.attn_k.weight	Block 38 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
504	blk.38.attn_output.weight	Block 38 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
505	blk.38.attn_q_norm.weight	Block 38 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
506	blk.38.attn_q.weight	Block 38 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
507	blk.38.attn_v.weight	Block 38 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.38: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 39 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
508	blk.39.attn_norm.weight	Block 39 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
509	blk.39.ffn_down.weight	Block 39 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
510	blk.39.ffn_gate.weight	Block 39 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
511	blk.39.ffn_up.weight	Block 39 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
512	blk.39.post_attention_norm.weight	Block 39 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
513	blk.39.post_ffw_norm.weight	Block 39 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
514	blk.39.ffn_norm.weight	Block 39 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
515	blk.39.attn_k_norm.weight	Block 39 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
516	blk.39.attn_k.weight	Block 39 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
517	blk.39.attn_output.weight	Block 39 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
518	blk.39.attn_q_norm.weight	Block 39 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
519	blk.39.attn_q.weight	Block 39 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
520	blk.39.attn_v.weight	Block 39 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.39: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 40 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
521	blk.40.attn_norm.weight	Block 40 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
522	blk.40.ffn_down.weight	Block 40 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
523	blk.40.ffn_gate.weight	Block 40 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
524	blk.40.ffn_up.weight	Block 40 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
525	blk.40.post_attention_norm.weight	Block 40 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
526	blk.40.post_ffw_norm.weight	Block 40 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
527	blk.40.ffn_norm.weight	Block 40 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
528	blk.40.attn_k_norm.weight	Block 40 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
529	blk.40.attn_k.weight	Block 40 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
530	blk.40.attn_output.weight	Block 40 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
531	blk.40.attn_q_norm.weight	Block 40 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
532	blk.40.attn_q.weight	Block 40 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
533	blk.40.attn_v.weight	Block 40 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.40: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 41 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
534	blk.41.attn_norm.weight	Block 41 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
535	blk.41.ffn_down.weight	Block 41 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
536	blk.41.ffn_gate.weight	Block 41 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
537	blk.41.ffn_up.weight	Block 41 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
538	blk.41.post_attention_norm.weight	Block 41 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
539	blk.41.post_ffw_norm.weight	Block 41 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
540	blk.41.ffn_norm.weight	Block 41 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
541	blk.41.attn_k_norm.weight	Block 41 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
542	blk.41.attn_k.weight	Block 41 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
543	blk.41.attn_output.weight	Block 41 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
544	blk.41.attn_q_norm.weight	Block 41 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
545	blk.41.attn_q.weight	Block 41 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
546	blk.41.attn_v.weight	Block 41 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.41: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 42 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
547	blk.42.attn_norm.weight	Block 42 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
548	blk.42.ffn_down.weight	Block 42 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
549	blk.42.ffn_gate.weight	Block 42 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
550	blk.42.ffn_up.weight	Block 42 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
551	blk.42.post_attention_norm.weight	Block 42 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
552	blk.42.post_ffw_norm.weight	Block 42 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
553	blk.42.ffn_norm.weight	Block 42 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
554	blk.42.attn_k_norm.weight	Block 42 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
555	blk.42.attn_k.weight	Block 42 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
556	blk.42.attn_output.weight	Block 42 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
557	blk.42.attn_q_norm.weight	Block 42 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
558	blk.42.attn_q.weight	Block 42 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
559	blk.42.attn_v.weight	Block 42 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.42: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 43 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
560	blk.43.attn_norm.weight	Block 43 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
561	blk.43.ffn_down.weight	Block 43 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
562	blk.43.ffn_gate.weight	Block 43 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
563	blk.43.ffn_up.weight	Block 43 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
564	blk.43.post_attention_norm.weight	Block 43 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
565	blk.43.post_ffw_norm.weight	Block 43 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
566	blk.43.ffn_norm.weight	Block 43 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
567	blk.43.attn_k_norm.weight	Block 43 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
568	blk.43.attn_k.weight	Block 43 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
569	blk.43.attn_output.weight	Block 43 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
570	blk.43.attn_q_norm.weight	Block 43 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
571	blk.43.attn_q.weight	Block 43 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
572	blk.43.attn_v.weight	Block 43 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.43: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 44 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
573	blk.44.attn_norm.weight	Block 44 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
574	blk.44.ffn_down.weight	Block 44 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
575	blk.44.ffn_gate.weight	Block 44 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
576	blk.44.ffn_up.weight	Block 44 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
577	blk.44.post_attention_norm.weight	Block 44 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
578	blk.44.post_ffw_norm.weight	Block 44 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
579	blk.44.ffn_norm.weight	Block 44 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
580	blk.44.attn_k_norm.weight	Block 44 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
581	blk.44.attn_k.weight	Block 44 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
582	blk.44.attn_output.weight	Block 44 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
583	blk.44.attn_q_norm.weight	Block 44 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
584	blk.44.attn_q.weight	Block 44 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
585	blk.44.attn_v.weight	Block 44 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.44: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 45 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
586	blk.45.attn_norm.weight	Block 45 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
587	blk.45.ffn_down.weight	Block 45 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
588	blk.45.ffn_gate.weight	Block 45 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
589	blk.45.ffn_up.weight	Block 45 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
590	blk.45.post_attention_norm.weight	Block 45 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
591	blk.45.post_ffw_norm.weight	Block 45 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
592	blk.45.ffn_norm.weight	Block 45 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
593	blk.45.attn_k_norm.weight	Block 45 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
594	blk.45.attn_k.weight	Block 45 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
595	blk.45.attn_output.weight	Block 45 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
596	blk.45.attn_q_norm.weight	Block 45 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
597	blk.45.attn_q.weight	Block 45 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
598	blk.45.attn_v.weight	Block 45 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.45: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 46 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
599	blk.46.attn_norm.weight	Block 46 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
600	blk.46.ffn_down.weight	Block 46 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
601	blk.46.ffn_gate.weight	Block 46 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
602	blk.46.ffn_up.weight	Block 46 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
603	blk.46.post_attention_norm.weight	Block 46 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
604	blk.46.post_ffw_norm.weight	Block 46 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
605	blk.46.ffn_norm.weight	Block 46 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
606	blk.46.attn_k_norm.weight	Block 46 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
607	blk.46.attn_k.weight	Block 46 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
608	blk.46.attn_output.weight	Block 46 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
609	blk.46.attn_q_norm.weight	Block 46 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
610	blk.46.attn_q.weight	Block 46 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
611	blk.46.attn_v.weight	Block 46 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.46: (~224M) 224148992
• Percentage of total elements: 1.91%

Block 47 Tensor Group : ~224M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
612	blk.47.attn_norm.weight	Block 47 Attention Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
613	blk.47.ffn_down.weight	Block 47 Feed-Forward Network "Down" (W)	(~59M) 58982400	15360 x 3840 x 1 x 1	F16
614	blk.47.ffn_gate.weight	Block 47 Feed-Forward Network "Gate" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
615	blk.47.ffn_up.weight	Block 47 Feed-Forward Network "Up" (W)	(~59M) 58982400	3840 x 15360 x 1 x 1	F16
616	blk.47.post_attention_norm.weight	Block 47 Post_Attention_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
617	blk.47.post_ffw_norm.weight	Block 47 Post_Ffw_Norm (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
618	blk.47.ffn_norm.weight	Block 47 Feed-Forward Network Normalization (W)	( ~4K) 3840	3840 x 1 x 1 x 1	F32
619	blk.47.attn_k_norm.weight	Block 47 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
620	blk.47.attn_k.weight	Block 47 Attention Key (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16
621	blk.47.attn_output.weight	Block 47 Attention Output (W)	(~16M) 15728640	4096 x 3840 x 1 x 1	F16
622	blk.47.attn_q_norm.weight	Block 47 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32
623	blk.47.attn_q.weight	Block 47 Attention Query (W)	(~16M) 15728640	3840 x 4096 x 1 x 1	F16
624	blk.47.attn_v.weight	Block 47 Attention Value (W)	( ~8M) 7864320	3840 x 2048 x 1 x 1	F16

• Total elements in blk.47: (~224M) 224148992
• Percentage of total elements: 1.91%