eaddario/Dolphin-Mistral-24B-Venice-Edition-pruned-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
c2d890814358976ea97a2f4dcd165c2ddb1df55f
Dolphin-Mistral-24B-Venice-…/scores/Dolphin-Mistral-24B-Venice-Edition-F16.md
Ed Addario 35b1741d2b Add GGUF internal file structure
2025-07-01 08:19:46 +01:00

102 KiB
Raw Blame History

Dolphin-Mistral-24B-Venice-Edition-F16.gguf - GGUF Internal File Dump

  • Endian: LITTLE endian

Key Value Metadata Store

There are 42 key-value pairs in this file

POS TYPE Count Key Value
1 UINT32 1 GGUF.version 3
2 UINT64 1 GGUF.tensor_count 363
3 UINT64 1 GGUF.kv_count 39
4 STRING 1 general.architecture llama
5 STRING 1 general.type model
6 STRING 1 general.name Dolphin Mistral 24B Venice Edition
7 STRING 1 general.finetune Venice-Edition
8 STRING 1 general.basename Dolphin-Mistral
9 STRING 1 general.size_label 24B
10 STRING 1 general.license apache-2.0
11 UINT32 1 general.base_model.count 1
12 STRING 1 general.base_model.0.name Mistral Small 24B Instruct 2501
13 STRING 1 general.base_model.0.version 2501
14 STRING 1 general.base_model.0.organization Mistralai
15 STRING 1 general.base_model.0.repo_url https://huggingface.co/mistral...istral-Small-24B-Instruct-2501
16 UINT32 1 llama.block_count 40
17 UINT32 1 llama.context_length 32768
18 UINT32 1 llama.embedding_length 5120
19 UINT32 1 llama.feed_forward_length 32768
20 UINT32 1 llama.attention.head_count 32
21 UINT32 1 llama.attention.head_count_kv 8
22 FLOAT32 1 llama.rope.freq_base 100000000.0
23 FLOAT32 1 llama.attention.layer_norm_rms_epsilon 1e-05
24 UINT32 1 llama.attention.key_length 128
25 UINT32 1 llama.attention.value_length 128
26 UINT32 1 general.file_type 1
27 UINT32 1 llama.vocab_size 131072
28 UINT32 1 llama.rope.dimension_count 128
29 UINT32 1 general.quantization_version 2
30 STRING 1 tokenizer.ggml.model gpt2
31 STRING 1 tokenizer.ggml.pre tekken
32 [STRING] 131072 tokenizer.ggml.tokens [ <unk>, <s>, </s>, [INST], [/INST], ... ]
33 [INT32] 131072 tokenizer.ggml.token_type [ 3, 3, 3, 3, 3, 3, 3, ... ]
34 [STRING] 269443 tokenizer.ggml.merges [ Ġ Ġ, Ġ t, e r, i n, Ġ ĠĠĠ, ... ]
35 UINT32 1 tokenizer.ggml.bos_token_id 1
36 UINT32 1 tokenizer.ggml.eos_token_id 2
37 UINT32 1 tokenizer.ggml.unknown_token_id 0
38 UINT32 1 tokenizer.ggml.padding_token_id 11
39 BOOL 1 tokenizer.ggml.add_bos_token True
40 BOOL 1 tokenizer.ggml.add_eos_token False
41 STRING 1 tokenizer.chat_template {%- set today = strftime_now("... {%- endif %}{%- endfor %}
42 BOOL 1 tokenizer.ggml.add_space_prefix False

Tensors Overview ~24B Elements

Total number of elements in all tensors: 23572403200 Elements

Tensor Data Offset

This table contains the offset and data segment relative to start of file

T_ID Tensor Layer Name Data Offset (B) Data Size (B)
0 output.weight 0x784800 0x50000000
1 token_embd.weight 0x50784800 0x50000000
2 blk.0.attn_norm.weight 0xa0784800 0x5000
3 blk.0.ffn_down.weight 0xa0789800 0x14000000
4 blk.0.ffn_gate.weight 0xb4789800 0x14000000
5 blk.0.ffn_up.weight 0xc8789800 0x14000000
6 blk.0.ffn_norm.weight 0xdc789800 0x5000
7 blk.0.attn_k.weight 0xdc78e800 0xa00000
8 blk.0.attn_output.weight 0xdd18e800 0x2800000
9 blk.0.attn_q.weight 0xdf98e800 0x2800000
10 blk.0.attn_v.weight 0xe218e800 0xa00000
11 blk.1.attn_norm.weight 0xe2b8e800 0x5000
12 blk.1.ffn_down.weight 0xe2b93800 0x14000000
13 blk.1.ffn_gate.weight 0xf6b93800 0x14000000
14 blk.1.ffn_up.weight 0x10ab93800 0x14000000
15 blk.1.ffn_norm.weight 0x11eb93800 0x5000
16 blk.1.attn_k.weight 0x11eb98800 0xa00000
17 blk.1.attn_output.weight 0x11f598800 0x2800000
18 blk.1.attn_q.weight 0x121d98800 0x2800000
19 blk.1.attn_v.weight 0x124598800 0xa00000
20 blk.10.attn_norm.weight 0x124f98800 0x5000
21 blk.10.ffn_down.weight 0x124f9d800 0x14000000
22 blk.10.ffn_gate.weight 0x138f9d800 0x14000000
23 blk.10.ffn_up.weight 0x14cf9d800 0x14000000
24 blk.10.ffn_norm.weight 0x160f9d800 0x5000
25 blk.10.attn_k.weight 0x160fa2800 0xa00000
26 blk.10.attn_output.weight 0x1619a2800 0x2800000
27 blk.10.attn_q.weight 0x1641a2800 0x2800000
28 blk.10.attn_v.weight 0x1669a2800 0xa00000
29 blk.11.attn_norm.weight 0x1673a2800 0x5000
30 blk.11.ffn_down.weight 0x1673a7800 0x14000000
31 blk.11.ffn_gate.weight 0x17b3a7800 0x14000000
32 blk.11.ffn_up.weight 0x18f3a7800 0x14000000
33 blk.11.ffn_norm.weight 0x1a33a7800 0x5000
34 blk.11.attn_k.weight 0x1a33ac800 0xa00000
35 blk.11.attn_output.weight 0x1a3dac800 0x2800000
36 blk.11.attn_q.weight 0x1a65ac800 0x2800000
37 blk.11.attn_v.weight 0x1a8dac800 0xa00000
38 blk.12.attn_norm.weight 0x1a97ac800 0x5000
39 blk.12.ffn_down.weight 0x1a97b1800 0x14000000
40 blk.12.ffn_gate.weight 0x1bd7b1800 0x14000000
41 blk.12.ffn_up.weight 0x1d17b1800 0x14000000
42 blk.12.ffn_norm.weight 0x1e57b1800 0x5000
43 blk.12.attn_k.weight 0x1e57b6800 0xa00000
44 blk.12.attn_output.weight 0x1e61b6800 0x2800000
45 blk.12.attn_q.weight 0x1e89b6800 0x2800000
46 blk.12.attn_v.weight 0x1eb1b6800 0xa00000
47 blk.13.attn_norm.weight 0x1ebbb6800 0x5000
48 blk.13.ffn_down.weight 0x1ebbbb800 0x14000000
49 blk.13.ffn_gate.weight 0x1ffbbb800 0x14000000
50 blk.13.ffn_up.weight 0x213bbb800 0x14000000
51 blk.13.ffn_norm.weight 0x227bbb800 0x5000
52 blk.13.attn_k.weight 0x227bc0800 0xa00000
53 blk.13.attn_output.weight 0x2285c0800 0x2800000
54 blk.13.attn_q.weight 0x22adc0800 0x2800000
55 blk.13.attn_v.weight 0x22d5c0800 0xa00000
56 blk.14.attn_norm.weight 0x22dfc0800 0x5000
57 blk.14.ffn_down.weight 0x22dfc5800 0x14000000
58 blk.14.ffn_gate.weight 0x241fc5800 0x14000000
59 blk.14.ffn_up.weight 0x255fc5800 0x14000000
60 blk.14.ffn_norm.weight 0x269fc5800 0x5000
61 blk.14.attn_k.weight 0x269fca800 0xa00000
62 blk.14.attn_output.weight 0x26a9ca800 0x2800000
63 blk.14.attn_q.weight 0x26d1ca800 0x2800000
64 blk.14.attn_v.weight 0x26f9ca800 0xa00000
65 blk.15.attn_norm.weight 0x2703ca800 0x5000
66 blk.15.ffn_down.weight 0x2703cf800 0x14000000
67 blk.15.ffn_gate.weight 0x2843cf800 0x14000000
68 blk.15.ffn_up.weight 0x2983cf800 0x14000000
69 blk.15.ffn_norm.weight 0x2ac3cf800 0x5000
70 blk.15.attn_k.weight 0x2ac3d4800 0xa00000
71 blk.15.attn_output.weight 0x2acdd4800 0x2800000
72 blk.15.attn_q.weight 0x2af5d4800 0x2800000
73 blk.15.attn_v.weight 0x2b1dd4800 0xa00000
74 blk.16.attn_norm.weight 0x2b27d4800 0x5000
75 blk.16.ffn_down.weight 0x2b27d9800 0x14000000
76 blk.16.ffn_gate.weight 0x2c67d9800 0x14000000
77 blk.16.ffn_up.weight 0x2da7d9800 0x14000000
78 blk.16.ffn_norm.weight 0x2ee7d9800 0x5000
79 blk.16.attn_k.weight 0x2ee7de800 0xa00000
80 blk.16.attn_output.weight 0x2ef1de800 0x2800000
81 blk.16.attn_q.weight 0x2f19de800 0x2800000
82 blk.16.attn_v.weight 0x2f41de800 0xa00000
83 blk.17.attn_norm.weight 0x2f4bde800 0x5000
84 blk.17.ffn_down.weight 0x2f4be3800 0x14000000
85 blk.17.ffn_gate.weight 0x308be3800 0x14000000
86 blk.17.ffn_up.weight 0x31cbe3800 0x14000000
87 blk.17.ffn_norm.weight 0x330be3800 0x5000
88 blk.17.attn_k.weight 0x330be8800 0xa00000
89 blk.17.attn_output.weight 0x3315e8800 0x2800000
90 blk.17.attn_q.weight 0x333de8800 0x2800000
91 blk.17.attn_v.weight 0x3365e8800 0xa00000
92 blk.18.attn_norm.weight 0x336fe8800 0x5000
93 blk.18.ffn_down.weight 0x336fed800 0x14000000
94 blk.18.ffn_gate.weight 0x34afed800 0x14000000
95 blk.18.ffn_up.weight 0x35efed800 0x14000000
96 blk.18.ffn_norm.weight 0x372fed800 0x5000
97 blk.18.attn_k.weight 0x372ff2800 0xa00000
98 blk.18.attn_output.weight 0x3739f2800 0x2800000
99 blk.18.attn_q.weight 0x3761f2800 0x2800000
100 blk.18.attn_v.weight 0x3789f2800 0xa00000
101 blk.19.attn_norm.weight 0x3793f2800 0x5000
102 blk.19.ffn_down.weight 0x3793f7800 0x14000000
103 blk.19.ffn_gate.weight 0x38d3f7800 0x14000000
104 blk.19.ffn_up.weight 0x3a13f7800 0x14000000
105 blk.19.ffn_norm.weight 0x3b53f7800 0x5000
106 blk.19.attn_k.weight 0x3b53fc800 0xa00000
107 blk.19.attn_output.weight 0x3b5dfc800 0x2800000
108 blk.19.attn_q.weight 0x3b85fc800 0x2800000
109 blk.19.attn_v.weight 0x3badfc800 0xa00000
110 blk.2.attn_norm.weight 0x3bb7fc800 0x5000
111 blk.2.ffn_down.weight 0x3bb801800 0x14000000
112 blk.2.ffn_gate.weight 0x3cf801800 0x14000000
113 blk.2.ffn_up.weight 0x3e3801800 0x14000000
114 blk.2.ffn_norm.weight 0x3f7801800 0x5000
115 blk.2.attn_k.weight 0x3f7806800 0xa00000
116 blk.2.attn_output.weight 0x3f8206800 0x2800000
117 blk.2.attn_q.weight 0x3faa06800 0x2800000
118 blk.2.attn_v.weight 0x3fd206800 0xa00000
119 blk.20.attn_norm.weight 0x3fdc06800 0x5000
120 blk.20.ffn_down.weight 0x3fdc0b800 0x14000000
121 blk.20.ffn_gate.weight 0x411c0b800 0x14000000
122 blk.20.ffn_up.weight 0x425c0b800 0x14000000
123 blk.20.ffn_norm.weight 0x439c0b800 0x5000
124 blk.20.attn_k.weight 0x439c10800 0xa00000
125 blk.20.attn_output.weight 0x43a610800 0x2800000
126 blk.20.attn_q.weight 0x43ce10800 0x2800000
127 blk.20.attn_v.weight 0x43f610800 0xa00000
128 blk.21.attn_norm.weight 0x440010800 0x5000
129 blk.21.ffn_down.weight 0x440015800 0x14000000
130 blk.21.ffn_gate.weight 0x454015800 0x14000000
131 blk.21.ffn_up.weight 0x468015800 0x14000000
132 blk.21.ffn_norm.weight 0x47c015800 0x5000
133 blk.21.attn_k.weight 0x47c01a800 0xa00000
134 blk.21.attn_output.weight 0x47ca1a800 0x2800000
135 blk.21.attn_q.weight 0x47f21a800 0x2800000
136 blk.21.attn_v.weight 0x481a1a800 0xa00000
137 blk.22.attn_norm.weight 0x48241a800 0x5000
138 blk.22.ffn_down.weight 0x48241f800 0x14000000
139 blk.22.ffn_gate.weight 0x49641f800 0x14000000
140 blk.22.ffn_up.weight 0x4aa41f800 0x14000000
141 blk.22.ffn_norm.weight 0x4be41f800 0x5000
142 blk.22.attn_k.weight 0x4be424800 0xa00000
143 blk.22.attn_output.weight 0x4bee24800 0x2800000
144 blk.22.attn_q.weight 0x4c1624800 0x2800000
145 blk.22.attn_v.weight 0x4c3e24800 0xa00000
146 blk.23.attn_norm.weight 0x4c4824800 0x5000
147 blk.23.ffn_down.weight 0x4c4829800 0x14000000
148 blk.23.ffn_gate.weight 0x4d8829800 0x14000000
149 blk.23.ffn_up.weight 0x4ec829800 0x14000000
150 blk.23.ffn_norm.weight 0x500829800 0x5000
151 blk.23.attn_k.weight 0x50082e800 0xa00000
152 blk.23.attn_output.weight 0x50122e800 0x2800000
153 blk.23.attn_q.weight 0x503a2e800 0x2800000
154 blk.23.attn_v.weight 0x50622e800 0xa00000
155 blk.24.attn_norm.weight 0x506c2e800 0x5000
156 blk.24.ffn_down.weight 0x506c33800 0x14000000
157 blk.24.ffn_gate.weight 0x51ac33800 0x14000000
158 blk.24.ffn_up.weight 0x52ec33800 0x14000000
159 blk.24.ffn_norm.weight 0x542c33800 0x5000
160 blk.24.attn_k.weight 0x542c38800 0xa00000
161 blk.24.attn_output.weight 0x543638800 0x2800000
162 blk.24.attn_q.weight 0x545e38800 0x2800000
163 blk.24.attn_v.weight 0x548638800 0xa00000
164 blk.25.attn_norm.weight 0x549038800 0x5000
165 blk.25.ffn_down.weight 0x54903d800 0x14000000
166 blk.25.ffn_gate.weight 0x55d03d800 0x14000000
167 blk.25.ffn_up.weight 0x57103d800 0x14000000
168 blk.25.ffn_norm.weight 0x58503d800 0x5000
169 blk.25.attn_k.weight 0x585042800 0xa00000
170 blk.25.attn_output.weight 0x585a42800 0x2800000
171 blk.25.attn_q.weight 0x588242800 0x2800000
172 blk.25.attn_v.weight 0x58aa42800 0xa00000
173 blk.26.attn_norm.weight 0x58b442800 0x5000
174 blk.26.ffn_down.weight 0x58b447800 0x14000000
175 blk.26.ffn_gate.weight 0x59f447800 0x14000000
176 blk.26.ffn_up.weight 0x5b3447800 0x14000000
177 blk.26.ffn_norm.weight 0x5c7447800 0x5000
178 blk.26.attn_k.weight 0x5c744c800 0xa00000
179 blk.26.attn_output.weight 0x5c7e4c800 0x2800000
180 blk.26.attn_q.weight 0x5ca64c800 0x2800000
181 blk.26.attn_v.weight 0x5cce4c800 0xa00000
182 blk.27.attn_norm.weight 0x5cd84c800 0x5000
183 blk.27.ffn_down.weight 0x5cd851800 0x14000000
184 blk.27.ffn_gate.weight 0x5e1851800 0x14000000
185 blk.27.ffn_up.weight 0x5f5851800 0x14000000
186 blk.27.ffn_norm.weight 0x609851800 0x5000
187 blk.27.attn_k.weight 0x609856800 0xa00000
188 blk.27.attn_output.weight 0x60a256800 0x2800000
189 blk.27.attn_q.weight 0x60ca56800 0x2800000
190 blk.27.attn_v.weight 0x60f256800 0xa00000
191 blk.28.attn_norm.weight 0x60fc56800 0x5000
192 blk.28.ffn_down.weight 0x60fc5b800 0x14000000
193 blk.28.ffn_gate.weight 0x623c5b800 0x14000000
194 blk.28.ffn_up.weight 0x637c5b800 0x14000000
195 blk.28.ffn_norm.weight 0x64bc5b800 0x5000
196 blk.28.attn_k.weight 0x64bc60800 0xa00000
197 blk.28.attn_output.weight 0x64c660800 0x2800000
198 blk.28.attn_q.weight 0x64ee60800 0x2800000
199 blk.28.attn_v.weight 0x651660800 0xa00000
200 blk.29.attn_norm.weight 0x652060800 0x5000
201 blk.29.ffn_down.weight 0x652065800 0x14000000
202 blk.29.ffn_gate.weight 0x666065800 0x14000000
203 blk.29.ffn_up.weight 0x67a065800 0x14000000
204 blk.29.ffn_norm.weight 0x68e065800 0x5000
205 blk.29.attn_k.weight 0x68e06a800 0xa00000
206 blk.29.attn_output.weight 0x68ea6a800 0x2800000
207 blk.29.attn_q.weight 0x69126a800 0x2800000
208 blk.29.attn_v.weight 0x693a6a800 0xa00000
209 blk.3.attn_norm.weight 0x69446a800 0x5000
210 blk.3.ffn_down.weight 0x69446f800 0x14000000
211 blk.3.ffn_gate.weight 0x6a846f800 0x14000000
212 blk.3.ffn_up.weight 0x6bc46f800 0x14000000
213 blk.3.ffn_norm.weight 0x6d046f800 0x5000
214 blk.3.attn_k.weight 0x6d0474800 0xa00000
215 blk.3.attn_output.weight 0x6d0e74800 0x2800000
216 blk.3.attn_q.weight 0x6d3674800 0x2800000
217 blk.3.attn_v.weight 0x6d5e74800 0xa00000
218 blk.30.attn_norm.weight 0x6d6874800 0x5000
219 blk.30.ffn_down.weight 0x6d6879800 0x14000000
220 blk.30.ffn_gate.weight 0x6ea879800 0x14000000
221 blk.30.ffn_up.weight 0x6fe879800 0x14000000
222 blk.30.ffn_norm.weight 0x712879800 0x5000
223 blk.30.attn_k.weight 0x71287e800 0xa00000
224 blk.30.attn_output.weight 0x71327e800 0x2800000
225 blk.30.attn_q.weight 0x715a7e800 0x2800000
226 blk.30.attn_v.weight 0x71827e800 0xa00000
227 blk.31.attn_norm.weight 0x718c7e800 0x5000
228 blk.31.ffn_down.weight 0x718c83800 0x14000000
229 blk.31.ffn_gate.weight 0x72cc83800 0x14000000
230 blk.31.ffn_up.weight 0x740c83800 0x14000000
231 blk.31.ffn_norm.weight 0x754c83800 0x5000
232 blk.31.attn_k.weight 0x754c88800 0xa00000
233 blk.31.attn_output.weight 0x755688800 0x2800000
234 blk.31.attn_q.weight 0x757e88800 0x2800000
235 blk.31.attn_v.weight 0x75a688800 0xa00000
236 blk.32.attn_norm.weight 0x75b088800 0x5000
237 blk.32.ffn_down.weight 0x75b08d800 0x14000000
238 blk.32.ffn_gate.weight 0x76f08d800 0x14000000
239 blk.32.ffn_up.weight 0x78308d800 0x14000000
240 blk.32.ffn_norm.weight 0x79708d800 0x5000
241 blk.32.attn_k.weight 0x797092800 0xa00000
242 blk.32.attn_output.weight 0x797a92800 0x2800000
243 blk.32.attn_q.weight 0x79a292800 0x2800000
244 blk.32.attn_v.weight 0x79ca92800 0xa00000
245 blk.33.attn_norm.weight 0x79d492800 0x5000
246 blk.33.ffn_down.weight 0x79d497800 0x14000000
247 blk.33.ffn_gate.weight 0x7b1497800 0x14000000
248 blk.33.ffn_up.weight 0x7c5497800 0x14000000
249 blk.33.ffn_norm.weight 0x7d9497800 0x5000
250 blk.33.attn_k.weight 0x7d949c800 0xa00000
251 blk.33.attn_output.weight 0x7d9e9c800 0x2800000
252 blk.33.attn_q.weight 0x7dc69c800 0x2800000
253 blk.33.attn_v.weight 0x7dee9c800 0xa00000
254 blk.34.attn_norm.weight 0x7df89c800 0x5000
255 blk.34.ffn_down.weight 0x7df8a1800 0x14000000
256 blk.34.ffn_gate.weight 0x7f38a1800 0x14000000
257 blk.34.ffn_up.weight 0x8078a1800 0x14000000
258 blk.34.ffn_norm.weight 0x81b8a1800 0x5000
259 blk.34.attn_k.weight 0x81b8a6800 0xa00000
260 blk.34.attn_output.weight 0x81c2a6800 0x2800000
261 blk.34.attn_q.weight 0x81eaa6800 0x2800000
262 blk.34.attn_v.weight 0x8212a6800 0xa00000
263 blk.35.attn_norm.weight 0x821ca6800 0x5000
264 blk.35.ffn_down.weight 0x821cab800 0x14000000
265 blk.35.ffn_gate.weight 0x835cab800 0x14000000
266 blk.35.ffn_up.weight 0x849cab800 0x14000000
267 blk.35.ffn_norm.weight 0x85dcab800 0x5000
268 blk.35.attn_k.weight 0x85dcb0800 0xa00000
269 blk.35.attn_output.weight 0x85e6b0800 0x2800000
270 blk.35.attn_q.weight 0x860eb0800 0x2800000
271 blk.35.attn_v.weight 0x8636b0800 0xa00000
272 blk.36.attn_norm.weight 0x8640b0800 0x5000
273 blk.36.ffn_down.weight 0x8640b5800 0x14000000
274 blk.36.ffn_gate.weight 0x8780b5800 0x14000000
275 blk.36.ffn_up.weight 0x88c0b5800 0x14000000
276 blk.36.ffn_norm.weight 0x8a00b5800 0x5000
277 blk.36.attn_k.weight 0x8a00ba800 0xa00000
278 blk.36.attn_output.weight 0x8a0aba800 0x2800000
279 blk.36.attn_q.weight 0x8a32ba800 0x2800000
280 blk.36.attn_v.weight 0x8a5aba800 0xa00000
281 blk.37.attn_norm.weight 0x8a64ba800 0x5000
282 blk.37.ffn_down.weight 0x8a64bf800 0x14000000
283 blk.37.ffn_gate.weight 0x8ba4bf800 0x14000000
284 blk.37.ffn_up.weight 0x8ce4bf800 0x14000000
285 blk.37.ffn_norm.weight 0x8e24bf800 0x5000
286 blk.37.attn_k.weight 0x8e24c4800 0xa00000
287 blk.37.attn_output.weight 0x8e2ec4800 0x2800000
288 blk.37.attn_q.weight 0x8e56c4800 0x2800000
289 blk.37.attn_v.weight 0x8e7ec4800 0xa00000
290 blk.38.attn_norm.weight 0x8e88c4800 0x5000
291 blk.38.ffn_down.weight 0x8e88c9800 0x14000000
292 blk.38.ffn_gate.weight 0x8fc8c9800 0x14000000
293 blk.38.ffn_up.weight 0x9108c9800 0x14000000
294 blk.38.ffn_norm.weight 0x9248c9800 0x5000
295 blk.38.attn_k.weight 0x9248ce800 0xa00000
296 blk.38.attn_output.weight 0x9252ce800 0x2800000
297 blk.38.attn_q.weight 0x927ace800 0x2800000
298 blk.38.attn_v.weight 0x92a2ce800 0xa00000
299 blk.39.attn_norm.weight 0x92acce800 0x5000
300 blk.39.ffn_down.weight 0x92acd3800 0x14000000
301 blk.39.ffn_gate.weight 0x93ecd3800 0x14000000
302 blk.39.ffn_up.weight 0x952cd3800 0x14000000
303 blk.39.ffn_norm.weight 0x966cd3800 0x5000
304 blk.39.attn_k.weight 0x966cd8800 0xa00000
305 blk.39.attn_output.weight 0x9676d8800 0x2800000
306 blk.39.attn_q.weight 0x969ed8800 0x2800000
307 blk.39.attn_v.weight 0x96c6d8800 0xa00000
308 blk.4.attn_norm.weight 0x96d0d8800 0x5000
309 blk.4.ffn_down.weight 0x96d0dd800 0x14000000
310 blk.4.ffn_gate.weight 0x9810dd800 0x14000000
311 blk.4.ffn_up.weight 0x9950dd800 0x14000000
312 blk.4.ffn_norm.weight 0x9a90dd800 0x5000
313 blk.4.attn_k.weight 0x9a90e2800 0xa00000
314 blk.4.attn_output.weight 0x9a9ae2800 0x2800000
315 blk.4.attn_q.weight 0x9ac2e2800 0x2800000
316 blk.4.attn_v.weight 0x9aeae2800 0xa00000
317 blk.5.attn_norm.weight 0x9af4e2800 0x5000
318 blk.5.ffn_down.weight 0x9af4e7800 0x14000000
319 blk.5.ffn_gate.weight 0x9c34e7800 0x14000000
320 blk.5.ffn_up.weight 0x9d74e7800 0x14000000
321 blk.5.ffn_norm.weight 0x9eb4e7800 0x5000
322 blk.5.attn_k.weight 0x9eb4ec800 0xa00000
323 blk.5.attn_output.weight 0x9ebeec800 0x2800000
324 blk.5.attn_q.weight 0x9ee6ec800 0x2800000
325 blk.5.attn_v.weight 0x9f0eec800 0xa00000
326 blk.6.attn_norm.weight 0x9f18ec800 0x5000
327 blk.6.ffn_down.weight 0x9f18f1800 0x14000000
328 blk.6.ffn_gate.weight 0xa058f1800 0x14000000
329 blk.6.ffn_up.weight 0xa198f1800 0x14000000
330 blk.6.ffn_norm.weight 0xa2d8f1800 0x5000
331 blk.6.attn_k.weight 0xa2d8f6800 0xa00000
332 blk.6.attn_output.weight 0xa2e2f6800 0x2800000
333 blk.6.attn_q.weight 0xa30af6800 0x2800000
334 blk.6.attn_v.weight 0xa332f6800 0xa00000
335 blk.7.attn_norm.weight 0xa33cf6800 0x5000
336 blk.7.ffn_down.weight 0xa33cfb800 0x14000000
337 blk.7.ffn_gate.weight 0xa47cfb800 0x14000000
338 blk.7.ffn_up.weight 0xa5bcfb800 0x14000000
339 blk.7.ffn_norm.weight 0xa6fcfb800 0x5000
340 blk.7.attn_k.weight 0xa6fd00800 0xa00000
341 blk.7.attn_output.weight 0xa70700800 0x2800000
342 blk.7.attn_q.weight 0xa72f00800 0x2800000
343 blk.7.attn_v.weight 0xa75700800 0xa00000
344 blk.8.attn_norm.weight 0xa76100800 0x5000
345 blk.8.ffn_down.weight 0xa76105800 0x14000000
346 blk.8.ffn_gate.weight 0xa8a105800 0x14000000
347 blk.8.ffn_up.weight 0xa9e105800 0x14000000
348 blk.8.ffn_norm.weight 0xab2105800 0x5000
349 blk.8.attn_k.weight 0xab210a800 0xa00000
350 blk.8.attn_output.weight 0xab2b0a800 0x2800000
351 blk.8.attn_q.weight 0xab530a800 0x2800000
352 blk.8.attn_v.weight 0xab7b0a800 0xa00000
353 blk.9.attn_norm.weight 0xab850a800 0x5000
354 blk.9.ffn_down.weight 0xab850f800 0x14000000
355 blk.9.ffn_gate.weight 0xacc50f800 0x14000000
356 blk.9.ffn_up.weight 0xae050f800 0x14000000
357 blk.9.ffn_norm.weight 0xaf450f800 0x5000
358 blk.9.attn_k.weight 0xaf4514800 0xa00000
359 blk.9.attn_output.weight 0xaf4f14800 0x2800000
360 blk.9.attn_q.weight 0xaf7714800 0x2800000
361 blk.9.attn_v.weight 0xaf9f14800 0xa00000
362 output_norm.weight 0xafa914800 0x5000

Base Tensor Group : ~1B Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
0 output.weight Output (W) (~671M) 671088640 5120 x 131072 x 1 x 1 F16
1 token_embd.weight Token Embedding (W) (~671M) 671088640 5120 x 131072 x 1 x 1 F16
362 output_norm.weight Output Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
  • Total elements in base: ( ~1B) 1342182400
  • Percentage of total elements: 5.69%

Block 0 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
2 blk.0.attn_norm.weight Block 0 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
3 blk.0.ffn_down.weight Block 0 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
4 blk.0.ffn_gate.weight Block 0 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
5 blk.0.ffn_up.weight Block 0 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
6 blk.0.ffn_norm.weight Block 0 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
7 blk.0.attn_k.weight Block 0 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
8 blk.0.attn_output.weight Block 0 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
9 blk.0.attn_q.weight Block 0 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
10 blk.0.attn_v.weight Block 0 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.0: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 1 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
11 blk.1.attn_norm.weight Block 1 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
12 blk.1.ffn_down.weight Block 1 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
13 blk.1.ffn_gate.weight Block 1 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
14 blk.1.ffn_up.weight Block 1 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
15 blk.1.ffn_norm.weight Block 1 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
16 blk.1.attn_k.weight Block 1 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
17 blk.1.attn_output.weight Block 1 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
18 blk.1.attn_q.weight Block 1 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
19 blk.1.attn_v.weight Block 1 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.1: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 10 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
20 blk.10.attn_norm.weight Block 10 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
21 blk.10.ffn_down.weight Block 10 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
22 blk.10.ffn_gate.weight Block 10 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
23 blk.10.ffn_up.weight Block 10 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
24 blk.10.ffn_norm.weight Block 10 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
25 blk.10.attn_k.weight Block 10 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
26 blk.10.attn_output.weight Block 10 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
27 blk.10.attn_q.weight Block 10 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
28 blk.10.attn_v.weight Block 10 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.10: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 11 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
29 blk.11.attn_norm.weight Block 11 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
30 blk.11.ffn_down.weight Block 11 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
31 blk.11.ffn_gate.weight Block 11 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
32 blk.11.ffn_up.weight Block 11 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
33 blk.11.ffn_norm.weight Block 11 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
34 blk.11.attn_k.weight Block 11 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
35 blk.11.attn_output.weight Block 11 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
36 blk.11.attn_q.weight Block 11 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
37 blk.11.attn_v.weight Block 11 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.11: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 12 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
38 blk.12.attn_norm.weight Block 12 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
39 blk.12.ffn_down.weight Block 12 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
40 blk.12.ffn_gate.weight Block 12 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
41 blk.12.ffn_up.weight Block 12 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
42 blk.12.ffn_norm.weight Block 12 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
43 blk.12.attn_k.weight Block 12 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
44 blk.12.attn_output.weight Block 12 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
45 blk.12.attn_q.weight Block 12 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
46 blk.12.attn_v.weight Block 12 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.12: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 13 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
47 blk.13.attn_norm.weight Block 13 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
48 blk.13.ffn_down.weight Block 13 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
49 blk.13.ffn_gate.weight Block 13 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
50 blk.13.ffn_up.weight Block 13 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
51 blk.13.ffn_norm.weight Block 13 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
52 blk.13.attn_k.weight Block 13 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
53 blk.13.attn_output.weight Block 13 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
54 blk.13.attn_q.weight Block 13 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
55 blk.13.attn_v.weight Block 13 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.13: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 14 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
56 blk.14.attn_norm.weight Block 14 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
57 blk.14.ffn_down.weight Block 14 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
58 blk.14.ffn_gate.weight Block 14 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
59 blk.14.ffn_up.weight Block 14 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
60 blk.14.ffn_norm.weight Block 14 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
61 blk.14.attn_k.weight Block 14 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
62 blk.14.attn_output.weight Block 14 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
63 blk.14.attn_q.weight Block 14 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
64 blk.14.attn_v.weight Block 14 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.14: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 15 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
65 blk.15.attn_norm.weight Block 15 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
66 blk.15.ffn_down.weight Block 15 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
67 blk.15.ffn_gate.weight Block 15 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
68 blk.15.ffn_up.weight Block 15 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
69 blk.15.ffn_norm.weight Block 15 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
70 blk.15.attn_k.weight Block 15 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
71 blk.15.attn_output.weight Block 15 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
72 blk.15.attn_q.weight Block 15 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
73 blk.15.attn_v.weight Block 15 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.15: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 16 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
74 blk.16.attn_norm.weight Block 16 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
75 blk.16.ffn_down.weight Block 16 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
76 blk.16.ffn_gate.weight Block 16 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
77 blk.16.ffn_up.weight Block 16 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
78 blk.16.ffn_norm.weight Block 16 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
79 blk.16.attn_k.weight Block 16 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
80 blk.16.attn_output.weight Block 16 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
81 blk.16.attn_q.weight Block 16 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
82 blk.16.attn_v.weight Block 16 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.16: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 17 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
83 blk.17.attn_norm.weight Block 17 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
84 blk.17.ffn_down.weight Block 17 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
85 blk.17.ffn_gate.weight Block 17 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
86 blk.17.ffn_up.weight Block 17 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
87 blk.17.ffn_norm.weight Block 17 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
88 blk.17.attn_k.weight Block 17 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
89 blk.17.attn_output.weight Block 17 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
90 blk.17.attn_q.weight Block 17 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
91 blk.17.attn_v.weight Block 17 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.17: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 18 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
92 blk.18.attn_norm.weight Block 18 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
93 blk.18.ffn_down.weight Block 18 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
94 blk.18.ffn_gate.weight Block 18 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
95 blk.18.ffn_up.weight Block 18 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
96 blk.18.ffn_norm.weight Block 18 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
97 blk.18.attn_k.weight Block 18 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
98 blk.18.attn_output.weight Block 18 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
99 blk.18.attn_q.weight Block 18 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
100 blk.18.attn_v.weight Block 18 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.18: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 19 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
101 blk.19.attn_norm.weight Block 19 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
102 blk.19.ffn_down.weight Block 19 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
103 blk.19.ffn_gate.weight Block 19 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
104 blk.19.ffn_up.weight Block 19 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
105 blk.19.ffn_norm.weight Block 19 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
106 blk.19.attn_k.weight Block 19 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
107 blk.19.attn_output.weight Block 19 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
108 blk.19.attn_q.weight Block 19 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
109 blk.19.attn_v.weight Block 19 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.19: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 2 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
110 blk.2.attn_norm.weight Block 2 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
111 blk.2.ffn_down.weight Block 2 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
112 blk.2.ffn_gate.weight Block 2 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
113 blk.2.ffn_up.weight Block 2 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
114 blk.2.ffn_norm.weight Block 2 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
115 blk.2.attn_k.weight Block 2 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
116 blk.2.attn_output.weight Block 2 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
117 blk.2.attn_q.weight Block 2 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
118 blk.2.attn_v.weight Block 2 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.2: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 20 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
119 blk.20.attn_norm.weight Block 20 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
120 blk.20.ffn_down.weight Block 20 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
121 blk.20.ffn_gate.weight Block 20 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
122 blk.20.ffn_up.weight Block 20 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
123 blk.20.ffn_norm.weight Block 20 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
124 blk.20.attn_k.weight Block 20 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
125 blk.20.attn_output.weight Block 20 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
126 blk.20.attn_q.weight Block 20 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
127 blk.20.attn_v.weight Block 20 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.20: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 21 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
128 blk.21.attn_norm.weight Block 21 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
129 blk.21.ffn_down.weight Block 21 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
130 blk.21.ffn_gate.weight Block 21 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
131 blk.21.ffn_up.weight Block 21 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
132 blk.21.ffn_norm.weight Block 21 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
133 blk.21.attn_k.weight Block 21 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
134 blk.21.attn_output.weight Block 21 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
135 blk.21.attn_q.weight Block 21 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
136 blk.21.attn_v.weight Block 21 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.21: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 22 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
137 blk.22.attn_norm.weight Block 22 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
138 blk.22.ffn_down.weight Block 22 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
139 blk.22.ffn_gate.weight Block 22 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
140 blk.22.ffn_up.weight Block 22 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
141 blk.22.ffn_norm.weight Block 22 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
142 blk.22.attn_k.weight Block 22 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
143 blk.22.attn_output.weight Block 22 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
144 blk.22.attn_q.weight Block 22 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
145 blk.22.attn_v.weight Block 22 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.22: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 23 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
146 blk.23.attn_norm.weight Block 23 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
147 blk.23.ffn_down.weight Block 23 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
148 blk.23.ffn_gate.weight Block 23 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
149 blk.23.ffn_up.weight Block 23 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
150 blk.23.ffn_norm.weight Block 23 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
151 blk.23.attn_k.weight Block 23 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
152 blk.23.attn_output.weight Block 23 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
153 blk.23.attn_q.weight Block 23 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
154 blk.23.attn_v.weight Block 23 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.23: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 24 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
155 blk.24.attn_norm.weight Block 24 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
156 blk.24.ffn_down.weight Block 24 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
157 blk.24.ffn_gate.weight Block 24 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
158 blk.24.ffn_up.weight Block 24 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
159 blk.24.ffn_norm.weight Block 24 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
160 blk.24.attn_k.weight Block 24 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
161 blk.24.attn_output.weight Block 24 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
162 blk.24.attn_q.weight Block 24 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
163 blk.24.attn_v.weight Block 24 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.24: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 25 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
164 blk.25.attn_norm.weight Block 25 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
165 blk.25.ffn_down.weight Block 25 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
166 blk.25.ffn_gate.weight Block 25 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
167 blk.25.ffn_up.weight Block 25 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
168 blk.25.ffn_norm.weight Block 25 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
169 blk.25.attn_k.weight Block 25 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
170 blk.25.attn_output.weight Block 25 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
171 blk.25.attn_q.weight Block 25 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
172 blk.25.attn_v.weight Block 25 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.25: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 26 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
173 blk.26.attn_norm.weight Block 26 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
174 blk.26.ffn_down.weight Block 26 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
175 blk.26.ffn_gate.weight Block 26 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
176 blk.26.ffn_up.weight Block 26 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
177 blk.26.ffn_norm.weight Block 26 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
178 blk.26.attn_k.weight Block 26 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
179 blk.26.attn_output.weight Block 26 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
180 blk.26.attn_q.weight Block 26 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
181 blk.26.attn_v.weight Block 26 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.26: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 27 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
182 blk.27.attn_norm.weight Block 27 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
183 blk.27.ffn_down.weight Block 27 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
184 blk.27.ffn_gate.weight Block 27 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
185 blk.27.ffn_up.weight Block 27 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
186 blk.27.ffn_norm.weight Block 27 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
187 blk.27.attn_k.weight Block 27 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
188 blk.27.attn_output.weight Block 27 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
189 blk.27.attn_q.weight Block 27 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
190 blk.27.attn_v.weight Block 27 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.27: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 28 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
191 blk.28.attn_norm.weight Block 28 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
192 blk.28.ffn_down.weight Block 28 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
193 blk.28.ffn_gate.weight Block 28 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
194 blk.28.ffn_up.weight Block 28 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
195 blk.28.ffn_norm.weight Block 28 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
196 blk.28.attn_k.weight Block 28 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
197 blk.28.attn_output.weight Block 28 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
198 blk.28.attn_q.weight Block 28 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
199 blk.28.attn_v.weight Block 28 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.28: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 29 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
200 blk.29.attn_norm.weight Block 29 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
201 blk.29.ffn_down.weight Block 29 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
202 blk.29.ffn_gate.weight Block 29 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
203 blk.29.ffn_up.weight Block 29 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
204 blk.29.ffn_norm.weight Block 29 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
205 blk.29.attn_k.weight Block 29 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
206 blk.29.attn_output.weight Block 29 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
207 blk.29.attn_q.weight Block 29 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
208 blk.29.attn_v.weight Block 29 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.29: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 3 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
209 blk.3.attn_norm.weight Block 3 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
210 blk.3.ffn_down.weight Block 3 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
211 blk.3.ffn_gate.weight Block 3 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
212 blk.3.ffn_up.weight Block 3 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
213 blk.3.ffn_norm.weight Block 3 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
214 blk.3.attn_k.weight Block 3 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
215 blk.3.attn_output.weight Block 3 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
216 blk.3.attn_q.weight Block 3 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
217 blk.3.attn_v.weight Block 3 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.3: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 30 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
218 blk.30.attn_norm.weight Block 30 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
219 blk.30.ffn_down.weight Block 30 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
220 blk.30.ffn_gate.weight Block 30 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
221 blk.30.ffn_up.weight Block 30 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
222 blk.30.ffn_norm.weight Block 30 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
223 blk.30.attn_k.weight Block 30 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
224 blk.30.attn_output.weight Block 30 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
225 blk.30.attn_q.weight Block 30 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
226 blk.30.attn_v.weight Block 30 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.30: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 31 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
227 blk.31.attn_norm.weight Block 31 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
228 blk.31.ffn_down.weight Block 31 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
229 blk.31.ffn_gate.weight Block 31 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
230 blk.31.ffn_up.weight Block 31 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
231 blk.31.ffn_norm.weight Block 31 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
232 blk.31.attn_k.weight Block 31 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
233 blk.31.attn_output.weight Block 31 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
234 blk.31.attn_q.weight Block 31 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
235 blk.31.attn_v.weight Block 31 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.31: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 32 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
236 blk.32.attn_norm.weight Block 32 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
237 blk.32.ffn_down.weight Block 32 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
238 blk.32.ffn_gate.weight Block 32 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
239 blk.32.ffn_up.weight Block 32 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
240 blk.32.ffn_norm.weight Block 32 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
241 blk.32.attn_k.weight Block 32 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
242 blk.32.attn_output.weight Block 32 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
243 blk.32.attn_q.weight Block 32 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
244 blk.32.attn_v.weight Block 32 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.32: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 33 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
245 blk.33.attn_norm.weight Block 33 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
246 blk.33.ffn_down.weight Block 33 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
247 blk.33.ffn_gate.weight Block 33 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
248 blk.33.ffn_up.weight Block 33 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
249 blk.33.ffn_norm.weight Block 33 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
250 blk.33.attn_k.weight Block 33 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
251 blk.33.attn_output.weight Block 33 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
252 blk.33.attn_q.weight Block 33 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
253 blk.33.attn_v.weight Block 33 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.33: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 34 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
254 blk.34.attn_norm.weight Block 34 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
255 blk.34.ffn_down.weight Block 34 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
256 blk.34.ffn_gate.weight Block 34 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
257 blk.34.ffn_up.weight Block 34 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
258 blk.34.ffn_norm.weight Block 34 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
259 blk.34.attn_k.weight Block 34 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
260 blk.34.attn_output.weight Block 34 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
261 blk.34.attn_q.weight Block 34 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
262 blk.34.attn_v.weight Block 34 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.34: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 35 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
263 blk.35.attn_norm.weight Block 35 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
264 blk.35.ffn_down.weight Block 35 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
265 blk.35.ffn_gate.weight Block 35 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
266 blk.35.ffn_up.weight Block 35 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
267 blk.35.ffn_norm.weight Block 35 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
268 blk.35.attn_k.weight Block 35 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
269 blk.35.attn_output.weight Block 35 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
270 blk.35.attn_q.weight Block 35 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
271 blk.35.attn_v.weight Block 35 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.35: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 36 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
272 blk.36.attn_norm.weight Block 36 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
273 blk.36.ffn_down.weight Block 36 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
274 blk.36.ffn_gate.weight Block 36 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
275 blk.36.ffn_up.weight Block 36 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
276 blk.36.ffn_norm.weight Block 36 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
277 blk.36.attn_k.weight Block 36 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
278 blk.36.attn_output.weight Block 36 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
279 blk.36.attn_q.weight Block 36 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
280 blk.36.attn_v.weight Block 36 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.36: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 37 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
281 blk.37.attn_norm.weight Block 37 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
282 blk.37.ffn_down.weight Block 37 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
283 blk.37.ffn_gate.weight Block 37 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
284 blk.37.ffn_up.weight Block 37 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
285 blk.37.ffn_norm.weight Block 37 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
286 blk.37.attn_k.weight Block 37 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
287 blk.37.attn_output.weight Block 37 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
288 blk.37.attn_q.weight Block 37 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
289 blk.37.attn_v.weight Block 37 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.37: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 38 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
290 blk.38.attn_norm.weight Block 38 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
291 blk.38.ffn_down.weight Block 38 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
292 blk.38.ffn_gate.weight Block 38 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
293 blk.38.ffn_up.weight Block 38 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
294 blk.38.ffn_norm.weight Block 38 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
295 blk.38.attn_k.weight Block 38 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
296 blk.38.attn_output.weight Block 38 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
297 blk.38.attn_q.weight Block 38 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
298 blk.38.attn_v.weight Block 38 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.38: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 39 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
299 blk.39.attn_norm.weight Block 39 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
300 blk.39.ffn_down.weight Block 39 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
301 blk.39.ffn_gate.weight Block 39 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
302 blk.39.ffn_up.weight Block 39 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
303 blk.39.ffn_norm.weight Block 39 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
304 blk.39.attn_k.weight Block 39 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
305 blk.39.attn_output.weight Block 39 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
306 blk.39.attn_q.weight Block 39 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
307 blk.39.attn_v.weight Block 39 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.39: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 4 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
308 blk.4.attn_norm.weight Block 4 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
309 blk.4.ffn_down.weight Block 4 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
310 blk.4.ffn_gate.weight Block 4 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
311 blk.4.ffn_up.weight Block 4 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
312 blk.4.ffn_norm.weight Block 4 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
313 blk.4.attn_k.weight Block 4 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
314 blk.4.attn_output.weight Block 4 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
315 blk.4.attn_q.weight Block 4 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
316 blk.4.attn_v.weight Block 4 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.4: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 5 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
317 blk.5.attn_norm.weight Block 5 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
318 blk.5.ffn_down.weight Block 5 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
319 blk.5.ffn_gate.weight Block 5 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
320 blk.5.ffn_up.weight Block 5 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
321 blk.5.ffn_norm.weight Block 5 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
322 blk.5.attn_k.weight Block 5 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
323 blk.5.attn_output.weight Block 5 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
324 blk.5.attn_q.weight Block 5 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
325 blk.5.attn_v.weight Block 5 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.5: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 6 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
326 blk.6.attn_norm.weight Block 6 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
327 blk.6.ffn_down.weight Block 6 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
328 blk.6.ffn_gate.weight Block 6 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
329 blk.6.ffn_up.weight Block 6 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
330 blk.6.ffn_norm.weight Block 6 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
331 blk.6.attn_k.weight Block 6 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
332 blk.6.attn_output.weight Block 6 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
333 blk.6.attn_q.weight Block 6 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
334 blk.6.attn_v.weight Block 6 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.6: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 7 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
335 blk.7.attn_norm.weight Block 7 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
336 blk.7.ffn_down.weight Block 7 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
337 blk.7.ffn_gate.weight Block 7 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
338 blk.7.ffn_up.weight Block 7 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
339 blk.7.ffn_norm.weight Block 7 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
340 blk.7.attn_k.weight Block 7 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
341 blk.7.attn_output.weight Block 7 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
342 blk.7.attn_q.weight Block 7 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
343 blk.7.attn_v.weight Block 7 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.7: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 8 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
344 blk.8.attn_norm.weight Block 8 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
345 blk.8.ffn_down.weight Block 8 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
346 blk.8.ffn_gate.weight Block 8 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
347 blk.8.ffn_up.weight Block 8 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
348 blk.8.ffn_norm.weight Block 8 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
349 blk.8.attn_k.weight Block 8 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
350 blk.8.attn_output.weight Block 8 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
351 blk.8.attn_q.weight Block 8 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
352 blk.8.attn_v.weight Block 8 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.8: (~556M) 555755520
  • Percentage of total elements: 2.36%

Block 9 Tensor Group : ~556M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
353 blk.9.attn_norm.weight Block 9 Attention Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
354 blk.9.ffn_down.weight Block 9 Feed-Forward Network "Down" (W) (~168M) 167772160 32768 x 5120 x 1 x 1 F16
355 blk.9.ffn_gate.weight Block 9 Feed-Forward Network "Gate" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
356 blk.9.ffn_up.weight Block 9 Feed-Forward Network "Up" (W) (~168M) 167772160 5120 x 32768 x 1 x 1 F16
357 blk.9.ffn_norm.weight Block 9 Feed-Forward Network Normalization (W) ( ~5K) 5120 5120 x 1 x 1 x 1 F32
358 blk.9.attn_k.weight Block 9 Attention Key (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
359 blk.9.attn_output.weight Block 9 Attention Output (W) ( ~21M) 20971520 4096 x 5120 x 1 x 1 F16
360 blk.9.attn_q.weight Block 9 Attention Query (W) ( ~21M) 20971520 5120 x 4096 x 1 x 1 F16
361 blk.9.attn_v.weight Block 9 Attention Value (W) ( ~5M) 5242880 5120 x 1024 x 1 x 1 F16
  • Total elements in blk.9: (~556M) 555755520
  • Percentage of total elements: 2.36%