========================================================================================
  QUINTUS WEIGHT AUDIT
========================================================================================
  <REDACTED_ON_PURPOSE>  <REDACTED_ON_PURPOSE>
  base model      : Qwen/Qwen3-1.7B-Base
  distilled model : iamrahulreddy/Quintus
  alpha           : 0.3
  device          : cuda  |  dtype: torch.bfloat16
  python          : 3.12.13  |  torch: 2.11.0+cu128
========================================================================================

[01] Resolve checkpoints
  Downloading 'iamrahulreddy/Quintus' from HuggingFace Hub...
  Done in 0.2s
  base model commit  : <REDACTED_ON_PURPOSE>
  distilled commit   : <REDACTED_ON_PURPOSE>
  snapshot root      : <HF_CACHE_DIR>/snapshots

  Filename                                                     Size  Modified
  .gitattributes                                          1.533 KiB  2026-06-14 09:19
  README.md                                               5.388 KiB  2026-06-14 09:19
  added_tokens.json                                           707 B  2026-06-14 09:19
  chat_template.jinja                                     4.070 KiB  2026-06-14 09:19
  config.json                                             1.328 KiB  2026-06-14 09:19
  generation_config.json                                      143 B  2026-06-14 09:19
  model.safetensors                                       3.205 GiB  2026-06-14 09:21
  special_tokens_map.json                                     613 B  2026-06-14 09:19
  tokenizer.json                                         10.893 MiB  2026-06-14 09:19
  tokenizer_config.json                                   5.277 KiB  2026-06-14 09:19
  vocab.json                                              2.648 MiB  2026-06-14 09:19
  total                                                   3.218 GiB

[02] Architecture configuration
  Loading base config...
  Loading distilled config...

  -- Base
  label                    : base  (Qwen/Qwen3-1.7B-Base)
  model_type               : qwen3
  architecture             : Qwen3ForCausalLM

  Vocabulary
    vocab_size             : 151,936
    bos / eos / pad        : 151643 / 151643 / None

  Positional encoding
    max_position_embeddings: 32,768
    rope_theta             : N/A
    rope_scaling           : None

  Transformer dimensions
    hidden_size            : 2048
    num_hidden_layers      : 28
    intermediate_size      : 6144

  Attention
    num_attention_heads    : 16
    num_key_value_heads    : 8
    head_dim               : 128
    GQA ratio              : 2:1
    attention_bias         : False
    use_qk_norm            : True
    sliding_window         : None

  Feed-forward
    hidden_act             : silu
    mlp_bias               : False

  Misc
    rms_norm_eps           : 1e-06
    tie_word_embeddings    : True
    use_cache              : True
    torch_dtype            : float32
    initializer_range      : 0.02

  -- Distilled
  label                    : distilled  (iamrahulreddy/Quintus)
  model_type               : qwen3
  architecture             : Qwen3ForCausalLM

  Vocabulary
    vocab_size             : 151,936
    bos / eos / pad        : 151643 / 151643 / None

  Positional encoding
    max_position_embeddings: 32,768
    rope_theta             : N/A
    rope_scaling           : None

  Transformer dimensions
    hidden_size            : 2048
    num_hidden_layers      : 28
    intermediate_size      : 6144

  Attention
    num_attention_heads    : 16
    num_key_value_heads    : 8
    head_dim               : 128
    GQA ratio              : 2:1
    attention_bias         : False
    use_qk_norm            : True
    sliding_window         : None

  Feed-forward
    hidden_act             : silu
    mlp_bias               : False

  Misc
    rms_norm_eps           : 1e-06
    tie_word_embeddings    : True
    use_cache              : True
    torch_dtype            : float32
    initializer_range      : 0.02

  -- Config diff  (ignoring: _name_or_path, transformers_version)
  No differences — configs identical (expected for same-architecture KD).

[03] Parameter accounting

  -- Base
  base
    raw (all named)        : 2,031,739,904  (2.031740 B)
    embedding              : 311,164,928  (311.164928 M)
    lm_head                : 311,164,928  (311.164928 M)
    tied                   : True
    unique (deduped)       : 1,720,574,976  (1.720575 B)
    non-embedding          : 1,409,410,048  (1.409410 B)
    per layer (approx)     : 50,336,000

  -- Distilled
  distilled
    raw (all named)        : 2,031,739,904  (2.031740 B)
    embedding              : 311,164,928  (311.164928 M)
    lm_head                : 311,164,928  (311.164928 M)
    tied                   : True
    unique (deduped)       : 1,720,574,976  (1.720575 B)
    non-embedding          : 1,409,410,048  (1.409410 B)
    per layer (approx)     : 50,336,000

  -- Delta
  unique param delta     : +0  (+0.0000 %)
  non-embed param delta  : +0

[04] Load weights
  device: cuda  |  dtype: torch.bfloat16
  Loading base model   : Qwen/Qwen3-1.7B-Base
  Done in 15.4s
  Loading distilled    : iamrahulreddy/Quintus
  Done in 13.1s
  base tensors         : 311
  distilled tensors    : 311
  weight tying confirmed (embed == lm_head): True
  base weight memory   : 3.784 GiB
  distilled memory     : 3.784 GiB

[05] Per-tensor weight statistics  (distilled)
  Layer                                                                Shape                      Mean      Std      Min      Max  Sparse   KurtD   OutlR    RowL2  DeadR
  --------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  model.embed_tokens.weight                                            [151936, 2048]          -0.0001   0.0354  -0.3613   0.2988  0.0000   -0.00  0.0003   1.5789      0
  model.layers.0.self_attn.q_proj.weight                               [2048, 2048]            -0.0000   0.0356  -0.6641   0.6680  0.0000   -0.00  0.0017   1.5730      0
  model.layers.0.self_attn.k_proj.weight                               [1024, 2048]             0.0000   0.0362  -0.8320   0.6172  0.0000   -0.00  0.0019   1.5972      0
  model.layers.0.self_attn.v_proj.weight                               [1024, 2048]             0.0000   0.0340  -0.2275   0.2227  0.0000   -0.00  0.0005   1.5332      0
  model.layers.0.self_attn.o_proj.weight                               [2048, 2048]             0.0000   0.0328  -0.4512   0.5156  0.0000   -0.00  0.0031   1.4652      0
  model.layers.0.self_attn.q_norm.weight                               [128]                    1.6407   0.8456  -0.4355   6.9688  0.0000    0.00  0.0156      nan    N/A
  model.layers.0.self_attn.k_norm.weight                               [128]                    3.0374   6.5282   0.0679  68.0000  0.0000    0.00  0.0234      nan    N/A
  model.layers.0.mlp.gate_proj.weight                                  [6144, 2048]             0.0002   0.0390  -0.6484   0.6367  0.0000   -0.00  0.0012   1.7427      0
  model.layers.0.mlp.up_proj.weight                                    [6144, 2048]            -0.0000   0.0318  -0.4160   0.3945  0.0000   -0.00  0.0004   1.4374      0
  model.layers.0.mlp.down_proj.weight                                  [2048, 6144]             0.0000   0.0336  -0.6367   0.5156  0.0000   -0.00  0.0007   2.6247      0
  model.layers.0.input_layernorm.weight                                [2048]                   0.0895   0.0521   0.0576   1.0078  0.0000    0.00  0.0142      nan    N/A
  model.layers.0.post_attention_layernorm.weight                       [2048]                   0.1976   0.1459  -0.6406   0.8047  0.0000    0.00  0.0005      nan    N/A
  model.layers.1.self_attn.q_proj.weight                               [2048, 2048]            -0.0000   0.0349  -0.5039   0.6094  0.0000   -0.00  0.0027   1.5187      0
  model.layers.1.self_attn.k_proj.weight                               [1024, 2048]             0.0000   0.0341  -0.5078   0.4082  0.0000   -0.00  0.0022   1.4952      0
  model.layers.1.self_attn.v_proj.weight                               [1024, 2048]             0.0000   0.0350  -0.2305   0.2617  0.0000   -0.00  0.0005   1.5767      0
  model.layers.1.self_attn.o_proj.weight                               [2048, 2048]            -0.0000   0.0329  -0.3398   0.3711  0.0000   -0.00  0.0011   1.4763      0
  model.layers.1.self_attn.q_norm.weight                               [128]                    1.4082   0.7994  -2.1875   4.5938  0.0000    0.00  0.0078      nan    N/A
  model.layers.1.self_attn.k_norm.weight                               [128]                    2.1907   2.8109  -2.9688  30.6250  0.0000    0.00  0.0078      nan    N/A
  model.layers.1.mlp.gate_proj.weight                                  [6144, 2048]             0.0001   0.0453  -0.4043   0.5273  0.0000   -0.00  0.0007   1.9884      0
  model.layers.1.mlp.up_proj.weight                                    [6144, 2048]            -0.0000   0.0289  -0.4688   0.5234  0.0000   -0.00  0.0014   1.2540      0
  model.layers.1.mlp.down_proj.weight                                  [2048, 6144]            -0.0000   0.0294  -0.6289   0.5312  0.0000   -0.00  0.0014   2.2993      0
  model.layers.1.input_layernorm.weight                                [2048]                   0.0283   0.0940  -0.3125   0.5703  0.0000    0.00  0.0029      nan    N/A
  model.layers.1.post_attention_layernorm.weight                       [2048]                   0.5232   0.1638  -1.1953   5.2500  0.0000   -0.00  0.0068      nan    N/A
  model.layers.2.self_attn.q_proj.weight                               [2048, 2048]             0.0000   0.0347  -0.5156   0.6406  0.0000   -0.00  0.0029   1.5091      0
  model.layers.2.self_attn.k_proj.weight                               [1024, 2048]            -0.0000   0.0338  -0.2910   0.3613  0.0000   -0.00  0.0022   1.4704      0
  model.layers.2.self_attn.v_proj.weight                               [1024, 2048]             0.0000   0.0349  -0.2119   0.2422  0.0000   -0.00  0.0007   1.5588      0
  model.layers.2.self_attn.o_proj.weight                               [2048, 2048]            -0.0000   0.0319  -0.4121   0.4414  0.0000   -0.00  0.0018   1.4350      0
  model.layers.2.self_attn.q_norm.weight                               [128]                    1.4967   0.7808  -0.1245   4.9688  0.0000    0.00  0.0156      nan    N/A
  model.layers.2.self_attn.k_norm.weight                               [128]                    2.3197   3.2107  -0.0057  35.5000  0.0000    0.00  0.0078      nan    N/A
  model.layers.2.mlp.gate_proj.weight                                  [6144, 2048]             0.0000   0.0469  -0.4805   0.5000  0.0000   -0.00  0.0005   2.0895      0
  model.layers.2.mlp.up_proj.weight                                    [6144, 2048]             0.0000   0.0299  -0.5859   0.7031  0.0000   -0.00  0.0007   1.3430      0
  model.layers.2.mlp.down_proj.weight                                  [2048, 6144]            -0.0000   0.0309  -0.8203   0.6445  0.0000   -0.00  0.0007   2.4146      0
  model.layers.2.input_layernorm.weight                                [2048]                   0.1257   0.0469  -0.2061   0.6133  0.0000   -0.00  0.0210      nan    N/A
  model.layers.2.post_attention_layernorm.weight                       [2048]                   0.5019   0.1475  -0.5039   5.5000  0.0000    0.13  0.0029      nan    N/A
  model.layers.3.self_attn.q_proj.weight                               [2048, 2048]            -0.0000   0.0352  -0.6797   0.5078  0.0000   -0.00  0.0031   1.5280      0
  model.layers.3.self_attn.k_proj.weight                               [1024, 2048]            -0.0000   0.0345  -0.6484   0.7969  0.0000   -0.00  0.0025   1.4915      0
  model.layers.3.self_attn.v_proj.weight                               [1024, 2048]            -0.0000   0.0371  -0.3926   0.4844  0.0000   -0.00  0.0007   1.6652      0
  model.layers.3.self_attn.o_proj.weight                               [2048, 2048]            -0.0000   0.0338  -0.4004   0.4453  0.0000   -0.00  0.0012   1.5201      0
  model.layers.3.self_attn.q_norm.weight                               [128]                    1.8103   0.8711   0.0004   7.0000  0.0000    0.00  0.0156      nan    N/A
  model.layers.3.self_attn.k_norm.weight                               [128]                    1.7619   1.9032  -0.0018  20.2500  0.0000   -0.00  0.0078      nan    N/A
  model.layers.3.mlp.gate_proj.weight                                  [6144, 2048]            -0.0000   0.0463  -0.5352   0.4258  0.0000   -0.00  0.0005   2.0712      0
  model.layers.3.mlp.up_proj.weight                                    [6144, 2048]             0.0000   0.0306  -0.4414   0.4688  0.0000   -0.00  0.0005   1.3758      0
  model.layers.3.mlp.down_proj.weight                                  [2048, 6144]             0.0000   0.0308  -0.4160   0.6641  0.0000   -0.00  0.0005   2.4052      0
  model.layers.3.input_layernorm.weight                                [2048]                   0.3191   0.1033  -0.6133   1.5625  0.0000    0.00  0.0229      nan    N/A
  model.layers.3.post_attention_layernorm.weight                       [2048]                   0.5742   0.1119  -0.5938   2.0000  0.0000    0.00  0.0068      nan    N/A
  model.layers.4.self_attn.q_proj.weight                               [2048, 2048]             0.0000   0.0346  -0.4961   0.4805  0.0000   -0.00  0.0028   1.5056      0
  model.layers.4.self_attn.k_proj.weight                               [1024, 2048]             0.0000   0.0334  -0.5742   0.4141  0.0000   -0.00  0.0023   1.4488      0
  model.layers.4.self_attn.v_proj.weight                               [1024, 2048]            -0.0000   0.0364  -0.2402   0.2441  0.0000   -0.00  0.0006   1.6342      0
  model.layers.4.self_attn.o_proj.weight                               [2048, 2048]            -0.0000   0.0331  -0.6250   0.6641  0.0000   -0.00  0.0013   1.4884      0
  model.layers.4.self_attn.q_norm.weight                               [128]                    1.0624   0.5927   0.0052   3.7500  0.0000    0.00  0.0078      nan    N/A
  model.layers.4.self_attn.k_norm.weight                               [128]                    3.1512   3.6376  -0.0025  36.5000  0.0000    0.00  0.0156      nan    N/A
  model.layers.4.mlp.gate_proj.weight                                  [6144, 2048]            -0.0000   0.0449  -0.6172   0.6328  0.0000   -0.00  0.0005   2.0082      0
  model.layers.4.mlp.up_proj.weight                                    [6144, 2048]             0.0000   0.0309  -0.4668   0.4395  0.0000   -0.00  0.0005   1.3882      0
  model.layers.4.mlp.down_proj.weight                                  [2048, 6144]            -0.0000   0.0308  -0.5898   0.6328  0.0000   -0.00  0.0006   2.4082      0
  model.layers.4.input_layernorm.weight                                [2048]                   0.3279   0.0863  -0.5820   1.2969  0.0000    0.00  0.0186      nan    N/A
  model.layers.4.post_attention_layernorm.weight                       [2048]                   0.6251   0.1225  -0.8164   1.6094  0.0000   -0.00  0.0068      nan    N/A
  model.layers.5.self_attn.q_proj.weight                               [2048, 2048]             0.0000   0.0358  -0.4375   0.5195  0.0000   -0.00  0.0026   1.5625      0
  model.layers.5.self_attn.k_proj.weight                               [1024, 2048]            -0.0000   0.0339  -0.5469   0.5781  0.0000   -0.00  0.0016   1.4858      0
  model.layers.5.self_attn.v_proj.weight                               [1024, 2048]            -0.0000   0.0353  -0.2891   0.2656  0.0000   -0.00  0.0014   1.5872      0
  model.layers.5.self_attn.o_proj.weight                               [2048, 2048]             0.0000   0.0333  -0.4961   0.4395  0.0000   -0.00  0.0024   1.4987      0
  model.layers.5.self_attn.q_norm.weight                               [128]                    1.5458   1.0519  -0.0007   6.2812  0.0000   -0.00  0.0312      nan    N/A
  model.layers.5.self_attn.k_norm.weight                               [128]                    2.4593   3.5832   0.0013  31.1250  0.0000    0.00  0.0234      nan    N/A
  model.layers.5.mlp.gate_proj.weight                                  [6144, 2048]             0.0000   0.0400  -0.7227   0.7539  0.0000   -0.00  0.0009   1.7845      0
  model.layers.5.mlp.up_proj.weight                                    [6144, 2048]             0.0000   0.0331  -0.6602   0.5352  0.0000   -0.00  0.0004   1.4943      0
  model.layers.5.mlp.down_proj.weight                                  [2048, 6144]            -0.0000   0.0329  -0.6211   0.6758  0.0000   -0.00  0.0006   2.5677      0
  model.layers.5.input_layernorm.weight                                [2048]                   0.3916   0.0926  -0.9727   1.4219  0.0000    0.00  0.0210      nan    N/A
  model.layers.5.post_attention_layernorm.weight                       [2048]                   0.5236   0.0931  -0.3320   0.9688  0.0000   -0.00  0.0029      nan    N/A
  model.layers.6.self_attn.q_proj.weight                               [2048, 2048]             0.0000   0.0349  -0.4180   0.5859  0.0000   -0.00  0.0049   1.4738      0
  model.layers.6.self_attn.k_proj.weight                               [1024, 2048]             0.0001   0.0331  -0.5781   0.6094  0.0000   -0.00  0.0032   1.3997      0
  model.layers.6.self_attn.v_proj.weight                               [1024, 2048]             0.0000   0.0360  -0.2168   0.2207  0.0000   -0.00  0.0007   1.6123      0
  model.layers.6.self_attn.o_proj.weight                               [2048, 2048]             0.0000   0.0320  -0.5039   0.7539  0.0000   -0.00  0.0015   1.4365      0
  model.layers.6.self_attn.q_norm.weight                               [128]                    1.7113   1.0610  -0.0048   7.0312  0.0000   -0.00  0.0156      nan    N/A
  model.layers.6.self_attn.k_norm.weight                               [128]                    2.4032   2.1293  -0.0004  19.2500  0.0000    0.00  0.0156      nan    N/A
  model.layers.6.mlp.gate_proj.weight                                  [6144, 2048]            -0.0001   0.0394  -0.7031   0.6484  0.0000   -0.00  0.0010   1.7528      0
  model.layers.6.mlp.up_proj.weight                                    [6144, 2048]            -0.0000   0.0333  -0.4863   0.3652  0.0000   -0.00  0.0005   1.4986      0
  model.layers.6.mlp.down_proj.weight                                  [2048, 6144]            -0.0000   0.0324  -0.8672   0.8359  0.0000   -0.00  0.0008   2.5322      0
  model.layers.6.input_layernorm.weight                                [2048]                   0.3556   0.1117  -0.5938   1.3750  0.0000    0.00  0.0254      nan    N/A
  model.layers.6.post_attention_layernorm.weight                       [2048]                   0.5847   0.1243  -0.6328   1.2656  0.0000    0.00  0.0034      nan    N/A
  model.layers.7.self_attn.q_proj.weight                               [2048, 2048]             0.0000   0.0338  -0.4238   0.3887  0.0000   -0.00  0.0025   1.4765      0
  model.layers.7.self_attn.k_proj.weight                               [1024, 2048]             0.0000   0.0319  -0.3340   0.2617  0.0000   -0.00  0.0019   1.3969      0
  model.layers.7.self_attn.v_proj.weight                               [1024, 2048]             0.0000   0.0350  -0.2578   0.3203  0.0000   -0.00  0.0007   1.5745      0
  model.layers.7.self_attn.o_proj.weight                               [2048, 2048]            -0.0000   0.0322  -0.5742   0.5039  0.0000   -0.00  0.0012   1.4427      0
  model.layers.7.self_attn.q_norm.weight                               [128]                    1.5685   0.9296   0.0188   6.5312  0.0000    0.00  0.0156      nan    N/A
  model.layers.7.self_attn.k_norm.weight                               [128]                    1.8889   2.0411  -0.0031  19.2500  0.0000   -0.00  0.0078      nan    N/A
  model.layers.7.mlp.gate_proj.weight                                  [6144, 2048]            -0.0000   0.0406  -0.4727   0.5078  0.0000   -0.00  0.0008   1.7994      0
  model.layers.7.mlp.up_proj.weight                                    [6144, 2048]             0.0000   0.0328  -0.5430   0.4570  0.0000   -0.00  0.0006   1.4743      0
  model.layers.7.mlp.down_proj.weight                                  [2048, 6144]             0.0000   0.0319  -0.9805   0.6133  0.0000   -0.00  0.0007   2.4845      0
  model.layers.7.input_layernorm.weight                                [2048]                   0.5572   0.1534  -0.5820   3.5469  0.0000    0.00  0.0107      nan    N/A
  model.layers.7.post_attention_layernorm.weight                       [2048]                   0.6644   0.1445  -0.6914   1.5234  0.0000   -0.00  0.0029      nan    N/A
  model.layers.8.self_attn.q_proj.weight                               [2048, 2048]             0.0000   0.0347  -0.4434   0.4473  0.0000   -0.00  0.0042   1.4826      0
  model.layers.8.self_attn.k_proj.weight                               [1024, 2048]            -0.0000   0.0326  -0.8008   0.8750  0.0000   -0.00  0.0031   1.4049      0
  model.layers.8.self_attn.v_proj.weight                               [1024, 2048]            -0.0000   0.0354  -0.1914   0.2285  0.0000   -0.00  0.0004   1.5972      0
  model.layers.8.self_attn.o_proj.weight                               [2048, 2048]            -0.0000   0.0316  -0.5039   0.4746  0.0000   -0.00  0.0010   1.4136      0
  model.layers.8.self_attn.q_norm.weight                               [128]                    1.4452   0.5422  -0.0425   4.6562  0.0000   -0.00  0.0078      nan    N/A
  model.layers.8.self_attn.k_norm.weight                               [128]                    1.6409   0.8953  -0.0139   7.6875  0.0000    0.00  0.0078      nan    N/A
  model.layers.8.mlp.gate_proj.weight                                  [6144, 2048]            -0.0001   0.0388  -0.8125   0.8789  0.0000   -0.00  0.0010   1.7171      0
  model.layers.8.mlp.up_proj.weight                                    [6144, 2048]            -0.0000   0.0332  -0.5000   0.4355  0.0000   -0.00  0.0005   1.4933      0
  model.layers.8.mlp.down_proj.weight                                  [2048, 6144]             0.0000   0.0328  -0.6406   0.5391  0.0000   -0.00  0.0008   2.5532      0
  model.layers.8.input_layernorm.weight                                [2048]                   0.6002   0.1623   0.0991   1.9766  0.0000    0.00  0.0200      nan    N/A
  model.layers.8.post_attention_layernorm.weight                       [2048]                   0.6617   0.1351  -0.3613   1.3203  0.0000    0.00  0.0010      nan    N/A
  model.layers.9.self_attn.q_proj.weight                               [2048, 2048]             0.0000   0.0351  -0.4961   0.4453  0.0000   -0.00  0.0042   1.5042      0
  model.layers.9.self_attn.k_proj.weight                               [1024, 2048]            -0.0000   0.0324  -0.5508   0.5547  0.0000   -0.00  0.0024   1.4137      0
  model.layers.9.self_attn.v_proj.weight                               [1024, 2048]             0.0000   0.0347  -0.3086   0.3320  0.0000   -0.00  0.0012   1.5503      0
  model.layers.9.self_attn.o_proj.weight                               [2048, 2048]             0.0000   0.0327  -0.5547   0.5078  0.0000   -0.00  0.0025   1.4583      0
  model.layers.9.self_attn.q_norm.weight                               [128]                    1.6585   1.1838  -0.0251   6.7812  0.0000    0.00  0.0234      nan    N/A
  model.layers.9.self_attn.k_norm.weight                               [128]                    1.9812   1.8305  -0.0009  12.3750  0.0000   -0.00  0.0312      nan    N/A
  model.layers.9.mlp.gate_proj.weight                                  [6144, 2048]             0.0000   0.0386  -0.5586   0.5000  0.0000   -0.00  0.0011   1.7093      0
  model.layers.9.mlp.up_proj.weight                                    [6144, 2048]            -0.0000   0.0346  -0.4746   0.3633  0.0000   -0.00  0.0006   1.5537      0
  model.layers.9.mlp.down_proj.weight                                  [2048, 6144]            -0.0000   0.0330  -0.8047   0.5430  0.0000   -0.00  0.0009   2.5660      0
  model.layers.9.input_layernorm.weight                                [2048]                   0.7933   0.2096   0.0057   3.0312  0.0000    0.00  0.0098      nan    N/A
  model.layers.9.post_attention_layernorm.weight                       [2048]                   0.6711   0.1426  -0.0014   1.3750  0.0000   -0.00  0.0010      nan    N/A
  model.layers.10.self_attn.q_proj.weight                              [2048, 2048]             0.0000   0.0346  -0.4316   0.3926  0.0000   -0.00  0.0026   1.5106      0
  model.layers.10.self_attn.k_proj.weight                              [1024, 2048]             0.0000   0.0332  -0.2432   0.2676  0.0000   -0.00  0.0018   1.4593      0
  model.layers.10.self_attn.v_proj.weight                              [1024, 2048]             0.0000   0.0345  -0.2217   0.1992  0.0000   -0.00  0.0006   1.5555      0
  model.layers.10.self_attn.o_proj.weight                              [2048, 2048]            -0.0000   0.0319  -0.4688   0.5625  0.0000   -0.00  0.0016   1.4223      0
  model.layers.10.self_attn.q_norm.weight                              [128]                    1.7754   1.3316   0.0095   8.4375  0.0000    0.00  0.0312      nan    N/A
  model.layers.10.self_attn.k_norm.weight                              [128]                    1.8918   2.0176  -0.0007  20.7500  0.0000    0.00  0.0156      nan    N/A
  model.layers.10.mlp.gate_proj.weight                                 [6144, 2048]            -0.0000   0.0381  -0.7266   0.6289  0.0000   -0.00  0.0016   1.6789      0
  model.layers.10.mlp.up_proj.weight                                   [6144, 2048]            -0.0000   0.0347  -0.6133   0.6055  0.0000   -0.00  0.0008   1.5587      0
  model.layers.10.mlp.down_proj.weight                                 [2048, 6144]             0.0000   0.0328  -0.7148   0.6406  0.0000   -0.00  0.0012   2.5447      0
  model.layers.10.input_layernorm.weight                               [2048]                   0.7780   0.2307  -0.1816   3.1406  0.0000    0.00  0.0073      nan    N/A
  model.layers.10.post_attention_layernorm.weight                      [2048]                   0.6835   0.1354  -0.0013   1.2891  0.0000   -0.00  0.0015      nan    N/A
  model.layers.11.self_attn.q_proj.weight                              [2048, 2048]            -0.0000   0.0367  -0.5273   0.4961  0.0000   -0.00  0.0055   1.5396      0
  model.layers.11.self_attn.k_proj.weight                              [1024, 2048]             0.0000   0.0335  -0.4043   0.4062  0.0000   -0.00  0.0028   1.4380      0
  model.layers.11.self_attn.v_proj.weight                              [1024, 2048]             0.0000   0.0328  -0.2578   0.3027  0.0000   -0.00  0.0012   1.4683      0
  model.layers.11.self_attn.o_proj.weight                              [2048, 2048]            -0.0000   0.0303  -0.6797   0.6797  0.0000   -0.00  0.0028   1.3430      0
  model.layers.11.self_attn.q_norm.weight                              [128]                    1.7712   0.6162  -0.0052   3.1250  0.0000    0.00  0.0000      nan    N/A
  model.layers.11.self_attn.k_norm.weight                              [128]                    2.0542   1.0209  -0.0045   5.8125  0.0000   -0.00  0.0156      nan    N/A
  model.layers.11.mlp.gate_proj.weight                                 [6144, 2048]             0.0000   0.0362  -0.5703   0.5586  0.0000   -0.00  0.0013   1.6086      0
  model.layers.11.mlp.up_proj.weight                                   [6144, 2048]             0.0000   0.0355  -0.4160   0.4160  0.0000   -0.00  0.0007   1.5934      0
  model.layers.11.mlp.down_proj.weight                                 [2048, 6144]             0.0000   0.0339  -0.7695   0.8281  0.0000   -0.00  0.0010   2.6350      0
  model.layers.11.input_layernorm.weight                               [2048]                   1.0137   0.4236   0.0009   6.5000  0.0000    0.00  0.0117      nan    N/A
  model.layers.11.post_attention_layernorm.weight                      [2048]                   0.7500   0.2023  -0.6133   2.4375  0.0000    0.00  0.0015      nan    N/A
  model.layers.12.self_attn.q_proj.weight                              [2048, 2048]            -0.0000   0.0351  -0.4121   0.4121  0.0000   -0.00  0.0038   1.5119      0
  model.layers.12.self_attn.k_proj.weight                              [1024, 2048]            -0.0000   0.0326  -0.4414   0.4531  0.0000   -0.00  0.0027   1.4140      0
  model.layers.12.self_attn.v_proj.weight                              [1024, 2048]            -0.0000   0.0337  -0.2451   0.2676  0.0000   -0.00  0.0007   1.5215      0
  model.layers.12.self_attn.o_proj.weight                              [2048, 2048]             0.0000   0.0303  -0.3926   0.5508  0.0000   -0.00  0.0012   1.3480      0
  model.layers.12.self_attn.q_norm.weight                              [128]                    1.7989   0.7431  -0.0101   3.8594  0.0000    0.00  0.0000      nan    N/A
  model.layers.12.self_attn.k_norm.weight                              [128]                    1.7795   1.8132  -0.0039  20.1250  0.0000    0.00  0.0078      nan    N/A
  model.layers.12.mlp.gate_proj.weight                                 [6144, 2048]             0.0000   0.0353  -0.5117   0.4824  0.0000   -0.00  0.0013   1.5627      0
  model.layers.12.mlp.up_proj.weight                                   [6144, 2048]             0.0000   0.0354  -0.5273   0.4805  0.0000   -0.00  0.0008   1.5855      0
  model.layers.12.mlp.down_proj.weight                                 [2048, 6144]            -0.0000   0.0339  -0.8164   0.6250  0.0000   -0.00  0.0010   2.6305      0
  model.layers.12.input_layernorm.weight                               [2048]                   1.2048   0.5195  -0.7461   6.4062  0.0000   -0.00  0.0103      nan    N/A
  model.layers.12.post_attention_layernorm.weight                      [2048]                   0.7572   0.2071  -0.6055   1.6484  0.0000   -0.00  0.0020      nan    N/A
  model.layers.13.self_attn.q_proj.weight                              [2048, 2048]             0.0000   0.0355  -0.4121   0.4414  0.0000   -0.00  0.0057   1.4901      0
  model.layers.13.self_attn.k_proj.weight                              [1024, 2048]             0.0000   0.0311  -0.2969   0.3027  0.0000   -0.00  0.0024   1.3504      0
  model.layers.13.self_attn.v_proj.weight                              [1024, 2048]             0.0000   0.0336  -0.2197   0.2100  0.0000   -0.00  0.0005   1.5133      0
  model.layers.13.self_attn.o_proj.weight                              [2048, 2048]            -0.0000   0.0294  -0.4746   0.3711  0.0000   -0.00  0.0011   1.3055      0
  model.layers.13.self_attn.q_norm.weight                              [128]                    1.8504   0.9921  -0.1807   6.1875  0.0000    0.00  0.0234      nan    N/A
  model.layers.13.self_attn.k_norm.weight                              [128]                    1.7791   0.8772  -0.0124   4.9062  0.0000    0.00  0.0078      nan    N/A
  model.layers.13.mlp.gate_proj.weight                                 [6144, 2048]             0.0000   0.0341  -0.6523   0.6484  0.0000   -0.00  0.0017   1.5077      0
  model.layers.13.mlp.up_proj.weight                                   [6144, 2048]            -0.0000   0.0344  -0.4180   0.5859  0.0000   -0.00  0.0008   1.5437      0
  model.layers.13.mlp.down_proj.weight                                 [2048, 6144]             0.0000   0.0330  -0.9844   0.6992  0.0000   -0.00  0.0011   2.5615      0
  model.layers.13.input_layernorm.weight                               [2048]                   1.3235   0.6034  -0.8984   8.0625  0.0000    0.00  0.0112      nan    N/A
  model.layers.13.post_attention_layernorm.weight                      [2048]                   0.7902   0.2272  -0.0023   1.7266  0.0000    0.00  0.0049      nan    N/A
  model.layers.14.self_attn.q_proj.weight                              [2048, 2048]            -0.0000   0.0344  -0.4062   0.3945  0.0000   -0.00  0.0051   1.4512      0
  model.layers.14.self_attn.k_proj.weight                              [1024, 2048]             0.0000   0.0308  -0.4531   0.5156  0.0000   -0.00  0.0032   1.3290      0
  model.layers.14.self_attn.v_proj.weight                              [1024, 2048]             0.0000   0.0327  -0.1914   0.1934  0.0000   -0.00  0.0006   1.4757      0
  model.layers.14.self_attn.o_proj.weight                              [2048, 2048]             0.0000   0.0294  -0.5195   0.4395  0.0000   -0.00  0.0012   1.3063      0
  model.layers.14.self_attn.q_norm.weight                              [128]                    1.7915   1.0480  -0.0293   6.3750  0.0000    0.00  0.0312      nan    N/A
  model.layers.14.self_attn.k_norm.weight                              [128]                    1.8330   1.2059  -0.0398  12.1875  0.0000   -0.00  0.0078      nan    N/A
  model.layers.14.mlp.gate_proj.weight                                 [6144, 2048]             0.0000   0.0326  -0.7891   0.6875  0.0000   -0.00  0.0018   1.4436      0
  model.layers.14.mlp.up_proj.weight                                   [6144, 2048]             0.0000   0.0334  -0.5586   0.4570  0.0000   -0.00  0.0009   1.4960      0
  model.layers.14.mlp.down_proj.weight                                 [2048, 6144]            -0.0000   0.0328  -0.6719   0.6484  0.0000   -0.00  0.0011   2.5438      0
  model.layers.14.input_layernorm.weight                               [2048]                   1.6904   0.7990  -0.9961  10.3750  0.0000   -0.00  0.0107      nan    N/A
  model.layers.14.post_attention_layernorm.weight                      [2048]                   0.8251   0.2290  -0.0019   1.9688  0.0000   -0.00  0.0054      nan    N/A
  model.layers.15.self_attn.q_proj.weight                              [2048, 2048]            -0.0000   0.0352  -0.3848   0.4277  0.0000   -0.00  0.0054   1.4810      0
  model.layers.15.self_attn.k_proj.weight                              [1024, 2048]            -0.0000   0.0295  -0.2891   0.2969  0.0000   -0.00  0.0024   1.2812      0
  model.layers.15.self_attn.v_proj.weight                              [1024, 2048]             0.0001   0.0326  -0.4355   0.4141  0.0000   -0.00  0.0012   1.4678      0
  model.layers.15.self_attn.o_proj.weight                              [2048, 2048]             0.0000   0.0301  -0.4727   0.4473  0.0000   -0.00  0.0013   1.3374      0
  model.layers.15.self_attn.q_norm.weight                              [128]                    1.4976   0.6046  -0.0320   2.9062  0.0000   -0.00  0.0000      nan    N/A
  model.layers.15.self_attn.k_norm.weight                              [128]                    2.2923   2.5574  -0.0043  22.0000  0.0000   -0.00  0.0234      nan    N/A
  model.layers.15.mlp.gate_proj.weight                                 [6144, 2048]             0.0000   0.0328  -0.7383   0.6914  0.0000   -0.00  0.0019   1.4522      0
  model.layers.15.mlp.up_proj.weight                                   [6144, 2048]             0.0000   0.0341  -0.4570   0.5547  0.0000   -0.00  0.0010   1.5281      0
  model.layers.15.mlp.down_proj.weight                                 [2048, 6144]             0.0000   0.0329  -0.7461   0.7148  0.0000   -0.00  0.0011   2.5531      0
  model.layers.15.input_layernorm.weight                               [2048]                   2.3836   0.9875  -1.6250  14.4375  0.0000    0.00  0.0088      nan    N/A
  model.layers.15.post_attention_layernorm.weight                      [2048]                   0.8280   0.1941  -0.0019   1.9141  0.0000   -0.00  0.0078      nan    N/A
  model.layers.16.self_attn.q_proj.weight                              [2048, 2048]            -0.0000   0.0376  -0.4414   0.5039  0.0000   -0.00  0.0076   1.5383      0
  model.layers.16.self_attn.k_proj.weight                              [1024, 2048]            -0.0000   0.0306  -1.0312   0.8477  0.0000   -0.00  0.0041   1.2903      0
  model.layers.16.self_attn.v_proj.weight                              [1024, 2048]             0.0000   0.0328  -0.1973   0.2305  0.0000   -0.00  0.0008   1.4732      0
  model.layers.16.self_attn.o_proj.weight                              [2048, 2048]             0.0000   0.0301  -0.5000   0.4258  0.0000   -0.00  0.0016   1.3393      0
  model.layers.16.self_attn.q_norm.weight                              [128]                    1.6256   0.6673  -0.1147   3.5000  0.0000   -0.00  0.0000      nan    N/A
  model.layers.16.self_attn.k_norm.weight                              [128]                    2.0342   1.1452  -0.1045   7.8750  0.0000   -0.00  0.0078      nan    N/A
  model.layers.16.mlp.gate_proj.weight                                 [6144, 2048]             0.0000   0.0326  -0.4863   0.5273  0.0000   -0.00  0.0018   1.4504      0
  model.layers.16.mlp.up_proj.weight                                   [6144, 2048]            -0.0000   0.0350  -0.5742   0.5820  0.0000   -0.00  0.0011   1.5721      0
  model.layers.16.mlp.down_proj.weight                                 [2048, 6144]             0.0000   0.0332  -0.7578   0.6016  0.0000   -0.00  0.0014   2.5963      0
  model.layers.16.input_layernorm.weight                               [2048]                   2.2985   0.8804   0.0021  16.2500  0.0000    0.00  0.0103      nan    N/A
  model.layers.16.post_attention_layernorm.weight                      [2048]                   0.8429   0.1426  -0.0032   2.1719  0.0000    0.00  0.0122      nan    N/A
  model.layers.17.self_attn.q_proj.weight                              [2048, 2048]            -0.0000   0.0359  -0.3926   0.3906  0.0000   -0.00  0.0052   1.5199      0
  model.layers.17.self_attn.k_proj.weight                              [1024, 2048]            -0.0000   0.0305  -0.4199   0.4023  0.0000   -0.00  0.0020   1.3363      0
  model.layers.17.self_attn.v_proj.weight                              [1024, 2048]            -0.0000   0.0340  -0.2480   0.2539  0.0000   -0.00  0.0012   1.5262      0
  model.layers.17.self_attn.o_proj.weight                              [2048, 2048]             0.0000   0.0305  -0.4102   0.3633  0.0000   -0.00  0.0020   1.3648      0
  model.layers.17.self_attn.q_norm.weight                              [128]                    1.7546   1.1692  -0.0264   9.4375  0.0000    0.00  0.0156      nan    N/A
  model.layers.17.self_attn.k_norm.weight                              [128]                    1.9915   1.5669  -0.0845  16.0000  0.0000    0.00  0.0078      nan    N/A
  model.layers.17.mlp.gate_proj.weight                                 [6144, 2048]             0.0000   0.0325  -0.6172   0.6172  0.0000   -0.00  0.0017   1.4493      0
  model.layers.17.mlp.up_proj.weight                                   [6144, 2048]            -0.0000   0.0364  -0.3984   0.3887  0.0000   -0.00  0.0010   1.6309      0
  model.layers.17.mlp.down_proj.weight                                 [2048, 6144]            -0.0000   0.0343  -0.6797   0.5547  0.0000   -0.00  0.0009   2.6680      0
  model.layers.17.input_layernorm.weight                               [2048]                   4.0729   1.4421  -2.2188  33.7500  0.0000   -0.00  0.0098      nan    N/A
  model.layers.17.post_attention_layernorm.weight                      [2048]                   1.0375   0.2185  -0.0027   2.7656  0.0000   -0.00  0.0117      nan    N/A
  model.layers.18.self_attn.q_proj.weight                              [2048, 2048]            -0.0000   0.0370  -0.4414   0.4707  0.0000   -0.00  0.0061   1.5392      0
  model.layers.18.self_attn.k_proj.weight                              [1024, 2048]            -0.0000   0.0297  -0.4922   0.4395  0.0000   -0.00  0.0026   1.2818      0
  model.layers.18.self_attn.v_proj.weight                              [1024, 2048]             0.0000   0.0321  -0.2461   0.3145  0.0000   -0.00  0.0008   1.4507      0
  model.layers.18.self_attn.o_proj.weight                              [2048, 2048]            -0.0000   0.0287  -0.3672   0.4844  0.0000   -0.00  0.0013   1.2860      0
  model.layers.18.self_attn.q_norm.weight                              [128]                    1.5350   0.5170  -0.0645   2.6562  0.0000   -0.00  0.0000      nan    N/A
  model.layers.18.self_attn.k_norm.weight                              [128]                    1.8658   0.9406  -0.1133   6.3125  0.0000   -0.00  0.0078      nan    N/A
  model.layers.18.mlp.gate_proj.weight                                 [6144, 2048]             0.0001   0.0329  -0.5273   0.5625  0.0000   -0.00  0.0017   1.4636      0
  model.layers.18.mlp.up_proj.weight                                   [6144, 2048]             0.0000   0.0365  -0.3809   0.4062  0.0000   -0.00  0.0010   1.6349      0
  model.layers.18.mlp.down_proj.weight                                 [2048, 6144]            -0.0000   0.0352  -0.6484   0.7305  0.0000   -0.00  0.0009   2.7382      0
  model.layers.18.input_layernorm.weight                               [2048]                   4.1920   1.5139  -2.2500  27.1250  0.0000    0.00  0.0161      nan    N/A
  model.layers.18.post_attention_layernorm.weight                      [2048]                   1.1746   0.2241  -0.0031   3.2031  0.0000    0.00  0.0127      nan    N/A
  model.layers.19.self_attn.q_proj.weight                              [2048, 2048]            -0.0000   0.0382  -0.6016   0.4648  0.0000   -0.00  0.0079   1.5478      0
  model.layers.19.self_attn.k_proj.weight                              [1024, 2048]             0.0000   0.0299  -0.5156   0.4922  0.0000   -0.00  0.0025   1.2970      0
  model.layers.19.self_attn.v_proj.weight                              [1024, 2048]             0.0000   0.0320  -0.2812   0.3906  0.0000   -0.00  0.0021   1.4382      0
  model.layers.19.self_attn.o_proj.weight                              [2048, 2048]             0.0000   0.0298  -0.3457   0.4219  0.0000   -0.00  0.0014   1.3383      0
  model.layers.19.self_attn.q_norm.weight                              [128]                    1.6152   0.6862  -0.0435   3.8906  0.0000    0.00  0.0156      nan    N/A
  model.layers.19.self_attn.k_norm.weight                              [128]                    1.8694   1.0279  -0.0339   7.4062  0.0000   -0.00  0.0156      nan    N/A
  model.layers.19.mlp.gate_proj.weight                                 [6144, 2048]             0.0001   0.0334  -0.6719   0.6289  0.0000   -0.00  0.0020   1.4815      0
  model.layers.19.mlp.up_proj.weight                                   [6144, 2048]            -0.0000   0.0371  -0.9062   0.6562  0.0000   -0.00  0.0011   1.6638      0
  model.layers.19.mlp.down_proj.weight                                 [2048, 6144]             0.0000   0.0363  -0.7070   0.6719  0.0000   -0.00  0.0010   2.8286      0
  model.layers.19.input_layernorm.weight                               [2048]                   5.4223   2.2384  -0.0017  70.0000  0.0000   -0.00  0.0142      nan    N/A
  model.layers.19.post_attention_layernorm.weight                      [2048]                   1.2960   0.2268  -0.0038   3.7969  0.0000   -0.00  0.0142      nan    N/A
  model.layers.20.self_attn.q_proj.weight                              [2048, 2048]            -0.0000   0.0387  -0.4453   0.5703  0.0000   -0.00  0.0075   1.5911      0
  model.layers.20.self_attn.k_proj.weight                              [1024, 2048]             0.0000   0.0313  -0.3730   0.3574  0.0000   -0.00  0.0024   1.3673      0
  model.layers.20.self_attn.v_proj.weight                              [1024, 2048]            -0.0000   0.0333  -0.2812   0.3594  0.0000   -0.00  0.0022   1.4952      0
  model.layers.20.self_attn.o_proj.weight                              [2048, 2048]             0.0000   0.0311  -0.5586   0.6758  0.0000   -0.00  0.0016   1.3934      0
  model.layers.20.self_attn.q_norm.weight                              [128]                    1.6370   0.6676  -0.0091   4.4062  0.0000    0.00  0.0156      nan    N/A
  model.layers.20.self_attn.k_norm.weight                              [128]                    1.7924   1.0205  -0.0199   8.2500  0.0000   -0.00  0.0156      nan    N/A
  model.layers.20.mlp.gate_proj.weight                                 [6144, 2048]             0.0001   0.0347  -0.6328   0.4922  0.0000   -0.00  0.0017   1.5405      0
  model.layers.20.mlp.up_proj.weight                                   [6144, 2048]            -0.0000   0.0381  -0.5312   0.6797  0.0000   -0.00  0.0009   1.7075      0
  model.layers.20.mlp.down_proj.weight                                 [2048, 6144]             0.0000   0.0372  -0.6250   0.8047  0.0000   -0.00  0.0008   2.8967      0
  model.layers.20.input_layernorm.weight                               [2048]                   6.3076   2.1956   0.0019  29.2500  0.0000    0.00  0.0229      nan    N/A
  model.layers.20.post_attention_layernorm.weight                      [2048]                   1.4658   0.2796  -0.4805   4.4375  0.0000    0.00  0.0122      nan    N/A
  model.layers.21.self_attn.q_proj.weight                              [2048, 2048]            -0.0000   0.0388  -0.4688   0.4258  0.0000   -0.00  0.0064   1.6170      0
  model.layers.21.self_attn.k_proj.weight                              [1024, 2048]             0.0000   0.0320  -0.6484   0.8789  0.0000   -0.00  0.0027   1.3681      0
  model.layers.21.self_attn.v_proj.weight                              [1024, 2048]             0.0000   0.0360  -0.3535   0.3496  0.0000   -0.00  0.0037   1.5851      0
  model.layers.21.self_attn.o_proj.weight                              [2048, 2048]             0.0000   0.0327  -0.3242   0.3066  0.0000   -0.00  0.0021   1.4665      0
  model.layers.21.self_attn.q_norm.weight                              [128]                    1.4818   0.6233  -0.0381   2.7344  0.0000    0.00  0.0000      nan    N/A
  model.layers.21.self_attn.k_norm.weight                              [128]                    1.6387   1.1450  -0.0481   9.3750  0.0000    0.00  0.0078      nan    N/A
  model.layers.21.mlp.gate_proj.weight                                 [6144, 2048]             0.0000   0.0355  -0.6133   0.6250  0.0000   -0.00  0.0011   1.5840      0
  model.layers.21.mlp.up_proj.weight                                   [6144, 2048]            -0.0000   0.0387  -0.6992   0.8242  0.0000   -0.00  0.0007   1.7390      0
  model.layers.21.mlp.down_proj.weight                                 [2048, 6144]             0.0000   0.0379  -0.7812   0.8398  0.0000   -0.00  0.0006   2.9408      0
  model.layers.21.input_layernorm.weight                               [2048]                   7.8696   3.1144   0.0018  44.2500  0.0000    0.00  0.0229      nan    N/A
  model.layers.21.post_attention_layernorm.weight                      [2048]                   1.6247   0.2874  -0.0024   4.4375  0.0000   -0.00  0.0098      nan    N/A
  model.layers.22.self_attn.q_proj.weight                              [2048, 2048]            -0.0000   0.0378  -0.4062   0.4102  0.0000   -0.00  0.0056   1.5827      0
  model.layers.22.self_attn.k_proj.weight                              [1024, 2048]             0.0000   0.0317  -0.3711   0.4043  0.0000   -0.00  0.0027   1.3675      0
  model.layers.22.self_attn.v_proj.weight                              [1024, 2048]            -0.0000   0.0355  -0.3086   0.3047  0.0000   -0.00  0.0014   1.5926      0
  model.layers.22.self_attn.o_proj.weight                              [2048, 2048]             0.0000   0.0324  -0.3984   0.5234  0.0000   -0.00  0.0011   1.4517      0
  model.layers.22.self_attn.q_norm.weight                              [128]                    1.4677   0.6570  -0.0283   6.2812  0.0000    0.00  0.0078      nan    N/A
  model.layers.22.self_attn.k_norm.weight                              [128]                    1.5249   0.8786  -0.0417   5.0938  0.0000   -0.00  0.0234      nan    N/A
  model.layers.22.mlp.gate_proj.weight                                 [6144, 2048]             0.0001   0.0361  -0.7266   0.7188  0.0000   -0.00  0.0008   1.6176      0
  model.layers.22.mlp.up_proj.weight                                   [6144, 2048]             0.0000   0.0391  -0.5977   0.9453  0.0000   -0.00  0.0005   1.7588      0
  model.layers.22.mlp.down_proj.weight                                 [2048, 6144]            -0.0000   0.0386  -1.2031   1.1406  0.0000   -0.00  0.0005   2.9804      0
  model.layers.22.input_layernorm.weight                               [2048]                   9.1258   4.4278   0.8594  55.0000  0.0000    0.00  0.0259      nan    N/A
  model.layers.22.post_attention_layernorm.weight                      [2048]                   1.8063   0.2904  -0.0029   4.8438  0.0000    0.00  0.0103      nan    N/A
  model.layers.23.self_attn.q_proj.weight                              [2048, 2048]            -0.0000   0.0353  -0.4531   0.4492  0.0000   -0.00  0.0042   1.5161      0
  model.layers.23.self_attn.k_proj.weight                              [1024, 2048]            -0.0000   0.0313  -0.3457   0.3906  0.0000   -0.00  0.0022   1.3741      0
  model.layers.23.self_attn.v_proj.weight                              [1024, 2048]             0.0001   0.0348  -0.3438   0.2988  0.0000   -0.00  0.0015   1.5687      0
  model.layers.23.self_attn.o_proj.weight                              [2048, 2048]             0.0000   0.0326  -0.2969   0.4004  0.0000   -0.00  0.0009   1.4652      0
  model.layers.23.self_attn.q_norm.weight                              [128]                    1.4275   0.5005   0.0052   2.5625  0.0000    0.00  0.0000      nan    N/A
  model.layers.23.self_attn.k_norm.weight                              [128]                    2.0273   1.2693  -0.0050   9.0000  0.0000    0.00  0.0078      nan    N/A
  model.layers.23.mlp.gate_proj.weight                                 [6144, 2048]            -0.0000   0.0366  -0.7305   0.8555  0.0000   -0.00  0.0006   1.6415      0
  model.layers.23.mlp.up_proj.weight                                   [6144, 2048]            -0.0000   0.0392  -0.7852   0.6836  0.0000   -0.00  0.0004   1.7657      0
  model.layers.23.mlp.down_proj.weight                                 [2048, 6144]            -0.0000   0.0390  -0.9844   1.1797  0.0000   -0.00  0.0005   3.0050      0
  model.layers.23.input_layernorm.weight                               [2048]                  10.4963   5.4583   0.0018  74.5000  0.0000    0.00  0.0220      nan    N/A
  model.layers.23.post_attention_layernorm.weight                      [2048]                   2.0123   0.3104  -0.0019   5.1875  0.0000   -0.00  0.0112      nan    N/A
  model.layers.24.self_attn.q_proj.weight                              [2048, 2048]             0.0000   0.0369  -0.3828   0.4375  0.0000   -0.00  0.0057   1.5583      0
  model.layers.24.self_attn.k_proj.weight                              [1024, 2048]            -0.0000   0.0312  -0.3535   0.3047  0.0000   -0.00  0.0030   1.3521      0
  model.layers.24.self_attn.v_proj.weight                              [1024, 2048]             0.0000   0.0350  -0.2539   0.3438  0.0000   -0.00  0.0010   1.5803      0
  model.layers.24.self_attn.o_proj.weight                              [2048, 2048]             0.0000   0.0333  -0.3066   0.2812  0.0000   -0.00  0.0007   1.4859      0
  model.layers.24.self_attn.q_norm.weight                              [128]                    1.3816   0.5937  -0.0349   2.8125  0.0000    0.00  0.0000      nan    N/A
  model.layers.24.self_attn.k_norm.weight                              [128]                    2.0919   1.4716  -0.0908   8.0625  0.0000   -0.00  0.0156      nan    N/A
  model.layers.24.mlp.gate_proj.weight                                 [6144, 2048]             0.0000   0.0362  -0.8164   0.8711  0.0000   -0.00  0.0005   1.6259      0
  model.layers.24.mlp.up_proj.weight                                   [6144, 2048]             0.0000   0.0394  -0.5664   0.5742  0.0000   -0.00  0.0004   1.7753      0
  model.layers.24.mlp.down_proj.weight                                 [2048, 6144]             0.0000   0.0391  -0.8789   1.0859  0.0000   -0.00  0.0005   3.0185      0
  model.layers.24.input_layernorm.weight                               [2048]                  13.5919   9.3049  -4.0938  95.5000  0.0000    0.00  0.0220      nan    N/A
  model.layers.24.post_attention_layernorm.weight                      [2048]                   2.0885   0.3191  -0.0020   5.4375  0.0000    0.00  0.0137      nan    N/A
  model.layers.25.self_attn.q_proj.weight                              [2048, 2048]             0.0000   0.0359  -0.4219   0.5078  0.0000   -0.00  0.0050   1.5183      0
  model.layers.25.self_attn.k_proj.weight                              [1024, 2048]             0.0000   0.0302  -0.3594   0.3223  0.0000   -0.00  0.0026   1.3119      0
  model.layers.25.self_attn.v_proj.weight                              [1024, 2048]             0.0000   0.0388  -0.3223   0.2988  0.0000   -0.00  0.0010   1.7480      0
  model.layers.25.self_attn.o_proj.weight                              [2048, 2048]            -0.0000   0.0350  -0.2949   0.2988  0.0000   -0.00  0.0007   1.5653      0
  model.layers.25.self_attn.q_norm.weight                              [128]                    1.4685   0.6075   0.0018   3.1562  0.0000   -0.00  0.0000      nan    N/A
  model.layers.25.self_attn.k_norm.weight                              [128]                    1.8986   1.2900  -0.0150   5.7188  0.0000    0.00  0.0000      nan    N/A
  model.layers.25.mlp.gate_proj.weight                                 [6144, 2048]            -0.0000   0.0357  -0.7695   0.6953  0.0000   -0.00  0.0006   1.6021      0
  model.layers.25.mlp.up_proj.weight                                   [6144, 2048]             0.0000   0.0395  -0.7109   0.8867  0.0000   -0.00  0.0004   1.7789      0
  model.layers.25.mlp.down_proj.weight                                 [2048, 6144]            -0.0000   0.0388  -0.8516   0.9336  0.0000   -0.00  0.0004   3.0089      0
  model.layers.25.input_layernorm.weight                               [2048]                  15.9484  13.3387  -4.4688 121.5000  0.0000    0.00  0.0317      nan    N/A
  model.layers.25.post_attention_layernorm.weight                      [2048]                   2.2062   0.4000  -0.0029   9.3125  0.0000   -0.00  0.0151      nan    N/A
  model.layers.26.self_attn.q_proj.weight                              [2048, 2048]             0.0000   0.0389  -0.3672   0.4121  0.0000   -0.00  0.0054   1.6131      0
  model.layers.26.self_attn.k_proj.weight                              [1024, 2048]            -0.0000   0.0293  -0.2432   0.3145  0.0000   -0.00  0.0025   1.2736      0
  model.layers.26.self_attn.v_proj.weight                              [1024, 2048]             0.0000   0.0377  -0.3262   0.2432  0.0000   -0.00  0.0007   1.6897      0
  model.layers.26.self_attn.o_proj.weight                              [2048, 2048]            -0.0000   0.0348  -0.6719   0.6094  0.0000   -0.00  0.0020   1.5532      0
  model.layers.26.self_attn.q_norm.weight                              [128]                    1.2684   0.7311  -0.0376   3.5000  0.0000    0.00  0.0078      nan    N/A
  model.layers.26.self_attn.k_norm.weight                              [128]                    1.5801   1.0784  -0.0698   4.6250  0.0000    0.00  0.0000      nan    N/A
  model.layers.26.mlp.gate_proj.weight                                 [6144, 2048]             0.0000   0.0354  -0.5664   0.6797  0.0000   -0.00  0.0006   1.5891      0
  model.layers.26.mlp.up_proj.weight                                   [6144, 2048]             0.0000   0.0392  -0.5547   0.6094  0.0000   -0.00  0.0004   1.7659      0
  model.layers.26.mlp.down_proj.weight                                 [2048, 6144]            -0.0000   0.0381  -1.2109   0.8555  0.0000   -0.00  0.0006   2.9697      0
  model.layers.26.input_layernorm.weight                               [2048]                  17.9357  16.5038  -5.9688 136.0000  0.0000    0.00  0.0488      nan    N/A
  model.layers.26.post_attention_layernorm.weight                      [2048]                   2.4546   0.7710  -0.0022  25.5000  0.0000   -0.01  0.0117      nan    N/A
  model.layers.27.self_attn.q_proj.weight                              [2048, 2048]             0.0000   0.0355  -0.4297   0.5508  0.0000   -0.00  0.0024   1.5582      0
  model.layers.27.self_attn.k_proj.weight                              [1024, 2048]            -0.0000   0.0339  -1.6250   0.9023  0.0000   -0.00  0.0010   1.5101      0
  model.layers.27.self_attn.v_proj.weight                              [1024, 2048]            -0.0000   0.0376  -0.2373   0.2852  0.0000   -0.00  0.0004   1.6919      0
  model.layers.27.self_attn.o_proj.weight                              [2048, 2048]            -0.0000   0.0351  -0.5664   0.5273  0.0000   -0.00  0.0007   1.5805      0
  model.layers.27.self_attn.q_norm.weight                              [128]                    1.5298   0.6679  -0.0913   4.3125  0.0000    0.00  0.0234      nan    N/A
  model.layers.27.self_attn.k_norm.weight                              [128]                    1.7309   1.1485  -1.1172  10.1875  0.0000    0.00  0.0078      nan    N/A
  model.layers.27.mlp.gate_proj.weight                                 [6144, 2048]            -0.0001   0.0374  -0.7109   0.6719  0.0000   -0.00  0.0008   1.6732      0
  model.layers.27.mlp.up_proj.weight                                   [6144, 2048]            -0.0000   0.0387  -0.4609   1.0156  0.0000   -0.00  0.0006   1.7331      0
  model.layers.27.mlp.down_proj.weight                                 [2048, 6144]            -0.0000   0.0354  -1.0234   1.0234  0.0000   -0.00  0.0009   2.7653      0
  model.layers.27.input_layernorm.weight                               [2048]                  17.2849   4.6932   1.5312  77.5000  0.0000    0.00  0.0215      nan    N/A
  model.layers.27.post_attention_layernorm.weight                      [2048]                   3.0747   3.4869  -0.0024 153.0000  0.0000    0.00  0.0020      nan    N/A
  model.norm.weight                                                    [2048]                   2.2139   0.5169  -0.0461  12.0625  0.0000    0.00  0.0142      nan    N/A
  lm_head.weight                                                       [151936, 2048]          -0.0001   0.0354  -0.3613   0.2988  0.0000   -0.00  0.0003   1.5789      0

[06] Layer-type aggregated statistics  (distilled)
  Type               Count           Params   AvgMean    AvgStd  AvgSparse  AvgKurtD
  ----------------------------------------------------------------------------------
  attn_k                28       58,720,256  -0.00000   0.03208    0.00003    -0.000
  attn_knorm            28            3,584   2.01822   1.85875    0.00000     0.000
  attn_o                28      117,440,512  -0.00000   0.03193    0.00003    -0.000
  attn_q                28      117,440,512  -0.00000   0.03597    0.00003    -0.000
  attn_qnorm            28            3,584   1.57232   0.78864    0.00000     0.000
  attn_v                28       58,720,256   0.00001   0.03476    0.00003    -0.000
  embedding              1      311,164,928  -0.00011   0.03539    0.00001    -0.001
  final_norm             1            2,048   2.21386   0.51687    0.00000     0.000
  layernorm             56          114,688   2.81740   1.41555    0.00000     0.002
  lm_head                1      311,164,928  -0.00011   0.03539    0.00001    -0.001
  mlp_down              28      352,321,536   0.00000   0.03432    0.00002    -0.000
  mlp_gate              28      352,321,536   0.00002   0.03742    0.00002    -0.000
  mlp_up                28      352,321,536  -0.00000   0.03512    0.00002    -0.000

[07] Per-transformer-block breakdown  (distilled)
   Blk  Sublayer                            Shape                         L2     AbsMn       Std   Sparse     RowL2
  -------------------------------------------------------------------------------------------------------------------
     0  input_layernorm                     [2048]                     4.685   0.08947   0.05210  0.00000       nan
     0  self_attn.q_proj                    [2048, 2048]              72.901   0.02744   0.03559  0.00003   1.57298
     0  self_attn.k_proj                    [1024, 2048]              52.450   0.02796   0.03622  0.00003   1.59720
     0  self_attn.v_proj                    [1024, 2048]              49.252   0.02678   0.03401  0.00002   1.53317
     0  self_attn.o_proj                    [2048, 2048]              67.170   0.02499   0.03280  0.00003   1.46516
     0  self_attn.q_norm                    [128]                     20.866   1.64822   0.84563  0.00000       nan
     0  self_attn.k_norm                    [128]                     81.200   3.03745   6.52823  0.00000       nan
     0  post_attention_layernorm            [2048]                    11.116   0.24129   0.14591  0.00000       nan
     0  mlp.gate_proj                       [6144, 2048]             138.356   0.03043   0.03902  0.00002   1.74270
     0  mlp.up_proj                         [6144, 2048]             112.786   0.02509   0.03180  0.00002   1.43743
     0  mlp.down_proj                       [2048, 6144]             119.234   0.02647   0.03362  0.00002   2.62472

     1  input_layernorm                     [2048]                     4.442   0.09140   0.09403  0.00000       nan
     1  self_attn.q_proj                    [2048, 2048]              71.518   0.02654   0.03492  0.00003   1.51872
     1  self_attn.k_proj                    [1024, 2048]              49.452   0.02613   0.03415  0.00003   1.49522
     1  self_attn.v_proj                    [1024, 2048]              50.624   0.02751   0.03496  0.00003   1.57671
     1  self_attn.o_proj                    [2048, 2048]              67.294   0.02560   0.03286  0.00003   1.47633
     1  self_attn.q_norm                    [128]                     18.303   1.45519   0.79937  0.00000       nan
     1  self_attn.k_norm                    [128]                     40.221   2.25925   2.81085  0.00000       nan
     1  post_attention_layernorm            [2048]                    24.810   0.52890   0.16376  0.00000       nan
     1  mlp.gate_proj                       [6144, 2048]             160.751   0.03478   0.04534  0.00002   1.98843
     1  mlp.up_proj                         [6144, 2048]             102.435   0.02192   0.02889  0.00004   1.25396
     1  mlp.down_proj                       [2048, 6144]             104.364   0.02248   0.02943  0.00003   2.29928

     2  input_layernorm                     [2048]                     6.071   0.12772   0.04693  0.00000       nan
     2  self_attn.q_proj                    [2048, 2048]              71.066   0.02627   0.03470  0.00003   1.50912
     2  self_attn.k_proj                    [1024, 2048]              49.005   0.02568   0.03384  0.00004   1.47036
     2  self_attn.v_proj                    [1024, 2048]              50.551   0.02718   0.03491  0.00003   1.55884
     2  self_attn.o_proj                    [2048, 2048]              65.435   0.02473   0.03195  0.00003   1.43501
     2  self_attn.q_norm                    [128]                     19.083   1.49875   0.78076  0.00000       nan
     2  self_attn.k_norm                    [128]                     44.698   2.31981   3.21067  0.00000       nan
     2  post_attention_layernorm            [2048]                    23.675   0.50428   0.14753  0.00000       nan
     2  mlp.gate_proj                       [6144, 2048]             166.168   0.03672   0.04686  0.00002   2.08946
     2  mlp.up_proj                         [6144, 2048]             105.972   0.02350   0.02988  0.00003   1.34295
     2  mlp.down_proj                       [2048, 6144]             109.577   0.02421   0.03090  0.00003   2.41458

     3  input_layernorm                     [2048]                    15.178   0.32260   0.10331  0.00000       nan
     3  self_attn.q_proj                    [2048, 2048]              72.017   0.02663   0.03516  0.00003   1.52801
     3  self_attn.k_proj                    [1024, 2048]              50.006   0.02596   0.03453  0.00002   1.49155
     3  self_attn.v_proj                    [1024, 2048]              53.659   0.02896   0.03705  0.00003   1.66516
     3  self_attn.o_proj                    [2048, 2048]              69.315   0.02641   0.03384  0.00002   1.52010
     3  self_attn.q_norm                    [128]                     22.712   1.81032   0.87110  0.00000       nan
     3  self_attn.k_norm                    [128]                     29.281   1.76197   1.90323  0.00000       nan
     3  post_attention_layernorm            [2048]                    26.474   0.57677   0.11189  0.00000       nan
     3  mlp.gate_proj                       [6144, 2048]             164.227   0.03641   0.04631  0.00002   2.07124
     3  mlp.up_proj                         [6144, 2048]             108.473   0.02410   0.03059  0.00003   1.37582
     3  mlp.down_proj                       [2048, 6144]             109.132   0.02419   0.03077  0.00003   2.40517

     4  input_layernorm                     [2048]                    15.345   0.33078   0.08625  0.00000       nan
     4  self_attn.q_proj                    [2048, 2048]              70.937   0.02631   0.03464  0.00003   1.50560
     4  self_attn.k_proj                    [1024, 2048]              48.394   0.02525   0.03342  0.00003   1.44879
     4  self_attn.v_proj                    [1024, 2048]              52.777   0.02844   0.03644  0.00002   1.63425
     4  self_attn.o_proj                    [2048, 2048]              67.797   0.02570   0.03310  0.00003   1.48844
     4  self_attn.q_norm                    [128]                     13.751   1.06239   0.59266  0.00000       nan
     4  self_attn.k_norm                    [128]                     54.329   3.15128   3.63764  0.00000       nan
     4  post_attention_layernorm            [2048]                    28.826   0.62692   0.12249  0.00000       nan
     4  mlp.gate_proj                       [6144, 2048]             159.325   0.03529   0.04493  0.00002   2.00817
     4  mlp.up_proj                         [6144, 2048]             109.481   0.02430   0.03087  0.00003   1.38822
     4  mlp.down_proj                       [2048, 6144]             109.280   0.02420   0.03082  0.00003   2.40825

     5  input_layernorm                     [2048]                    18.209   0.39405   0.09262  0.00000       nan
     5  self_attn.q_proj                    [2048, 2048]              73.221   0.02723   0.03575  0.00003   1.56245
     5  self_attn.k_proj                    [1024, 2048]              49.084   0.02599   0.03389  0.00002   1.48577
     5  self_attn.v_proj                    [1024, 2048]              51.053   0.02729   0.03525  0.00003   1.58715
     5  self_attn.o_proj                    [2048, 2048]              68.198   0.02546   0.03330  0.00003   1.49868
     5  self_attn.q_norm                    [128]                     21.127   1.54580   1.05185  0.00000       nan
     5  self_attn.k_norm                    [128]                     49.039   2.45930   3.58324  0.00000       nan
     5  post_attention_layernorm            [2048]                    24.066   0.52422   0.09310  0.00000       nan
     5  mlp.gate_proj                       [6144, 2048]             141.749   0.03130   0.03997  0.00002   1.78453
     5  mlp.up_proj                         [6144, 2048]             117.417   0.02614   0.03311  0.00002   1.49425
     5  mlp.down_proj                       [2048, 6144]             116.503   0.02585   0.03285  0.00003   2.56775

     6  input_layernorm                     [2048]                    16.867   0.35778   0.11165  0.00000       nan
     6  self_attn.q_proj                    [2048, 2048]              71.503   0.02570   0.03491  0.00003   1.47382
     6  self_attn.k_proj                    [1024, 2048]              47.997   0.02440   0.03314  0.00004   1.39967
     6  self_attn.v_proj                    [1024, 2048]              52.187   0.02799   0.03604  0.00002   1.61226
     6  self_attn.o_proj                    [2048, 2048]              65.590   0.02470   0.03202  0.00003   1.43655
     6  self_attn.q_norm                    [128]                     22.756   1.71137   1.06104  0.00000       nan
     6  self_attn.k_norm                    [128]                     36.264   2.40319   2.12933  0.00000       nan
     6  post_attention_layernorm            [2048]                    27.050   0.58757   0.12428  0.00000       nan
     6  mlp.gate_proj                       [6144, 2048]             139.719   0.03072   0.03940  0.00002   1.75281
     6  mlp.up_proj                         [6144, 2048]             117.968   0.02621   0.03326  0.00002   1.49864
     6  mlp.down_proj                       [2048, 6144]             115.044   0.02540   0.03244  0.00003   2.53218

     7  input_layernorm                     [2048]                    26.151   0.55813   0.15338  0.00000       nan
     7  self_attn.q_proj                    [2048, 2048]              69.230   0.02582   0.03380  0.00003   1.47648
     7  self_attn.k_proj                    [1024, 2048]              46.204   0.02446   0.03190  0.00003   1.39686
     7  self_attn.v_proj                    [1024, 2048]              50.629   0.02729   0.03496  0.00002   1.57454
     7  self_attn.o_proj                    [2048, 2048]              65.907   0.02504   0.03218  0.00003   1.44267
     7  self_attn.q_norm                    [128]                     20.607   1.56852   0.92958  0.00000       nan
     7  self_attn.k_norm                    [128]                     31.397   1.88898   2.04107  0.00000       nan
     7  post_attention_layernorm            [2048]                    30.772   0.66610   0.14451  0.00000       nan
     7  mlp.gate_proj                       [6144, 2048]             144.020   0.03162   0.04061  0.00002   1.79936
     7  mlp.up_proj                         [6144, 2048]             116.378   0.02579   0.03282  0.00003   1.47432
     7  mlp.down_proj                       [2048, 6144]             113.238   0.02495   0.03193  0.00003   2.48453

     8  input_layernorm                     [2048]                    28.138   0.60021   0.16233  0.00000       nan
     8  self_attn.q_proj                    [2048, 2048]              70.971   0.02579   0.03465  0.00003   1.48264
     8  self_attn.k_proj                    [1024, 2048]              47.226   0.02438   0.03261  0.00003   1.40493
     8  self_attn.v_proj                    [1024, 2048]              51.273   0.02780   0.03540  0.00002   1.59720
     8  self_attn.o_proj                    [2048, 2048]              64.765   0.02465   0.03162  0.00003   1.41357
     8  self_attn.q_norm                    [128]                     17.455   1.44596   0.54216  0.00000       nan
     8  self_attn.k_norm                    [128]                     21.129   1.64117   0.89530  0.00000       nan
     8  post_attention_layernorm            [2048]                    30.563   0.66207   0.13507  0.00000       nan
     8  mlp.gate_proj                       [6144, 2048]             137.524   0.03013   0.03878  0.00002   1.71706
     8  mlp.up_proj                         [6144, 2048]             117.760   0.02611   0.03321  0.00002   1.49330
     8  mlp.down_proj                       [2048, 6144]             116.430   0.02562   0.03283  0.00002   2.55323

     9  input_layernorm                     [2048]                    37.133   0.79334   0.20959  0.00000       nan
     9  self_attn.q_proj                    [2048, 2048]              71.985   0.02616   0.03515  0.00003   1.50425
     9  self_attn.k_proj                    [1024, 2048]              46.960   0.02450   0.03243  0.00003   1.41366
     9  self_attn.v_proj                    [1024, 2048]              50.199   0.02671   0.03466  0.00003   1.55031
     9  self_attn.o_proj                    [2048, 2048]              66.940   0.02478   0.03268  0.00003   1.45834
     9  self_attn.q_norm                    [128]                     23.023   1.65887   1.18381  0.00000       nan
     9  self_attn.k_norm                    [128]                     30.462   1.98125   1.83048  0.00000       nan
     9  post_attention_layernorm            [2048]                    31.048   0.67109   0.14262  0.00000       nan
     9  mlp.gate_proj                       [6144, 2048]             136.793   0.02997   0.03858  0.00002   1.70927
     9  mlp.up_proj                         [6144, 2048]             122.606   0.02717   0.03457  0.00002   1.55372
     9  mlp.down_proj                       [2048, 6144]             117.158   0.02569   0.03304  0.00003   2.56601

    10  input_layernorm                     [2048]                    36.724   0.77820   0.23073  0.00000       nan
    10  self_attn.q_proj                    [2048, 2048]              70.798   0.02641   0.03457  0.00002   1.51064
    10  self_attn.k_proj                    [1024, 2048]              48.128   0.02556   0.03323  0.00002   1.45925
    10  self_attn.v_proj                    [1024, 2048]              49.962   0.02700   0.03450  0.00003   1.55550
    10  self_attn.o_proj                    [2048, 2048]              65.289   0.02457   0.03188  0.00003   1.42228
    10  self_attn.q_norm                    [128]                     25.073   1.77536   1.33162  0.00000       nan
    10  self_attn.k_norm                    [128]                     31.226   1.89182   2.01759  0.00000       nan
    10  post_attention_layernorm            [2048]                    31.534   0.68355   0.13537  0.00000       nan
    10  mlp.gate_proj                       [6144, 2048]             135.160   0.02929   0.03812  0.00002   1.67895
    10  mlp.up_proj                         [6144, 2048]             123.157   0.02714   0.03473  0.00002   1.55874
    10  mlp.down_proj                       [2048, 6144]             116.162   0.02527   0.03276  0.00003   2.54469

    11  input_layernorm                     [2048]                    49.718   1.01369   0.42364  0.00000       nan
    11  self_attn.q_proj                    [2048, 2048]              75.117   0.02664   0.03668  0.00002   1.53962
    11  self_attn.k_proj                    [1024, 2048]              48.444   0.02482   0.03345  0.00003   1.43803
    11  self_attn.v_proj                    [1024, 2048]              47.471   0.02513   0.03278  0.00003   1.46825
    11  self_attn.o_proj                    [2048, 2048]              61.987   0.02241   0.03027  0.00003   1.34302
    11  self_attn.q_norm                    [128]                     21.207   1.77125   0.61618  0.00000       nan
    11  self_attn.k_norm                    [128]                     25.933   2.05436   1.02094  0.00000       nan
    11  post_attention_layernorm            [2048]                    35.154   0.75061   0.20229  0.00000       nan
    11  mlp.gate_proj                       [6144, 2048]             128.540   0.02813   0.03624  0.00002   1.60858
    11  mlp.up_proj                         [6144, 2048]             125.940   0.02782   0.03551  0.00003   1.59340
    11  mlp.down_proj                       [2048, 6144]             120.142   0.02632   0.03388  0.00002   2.63503

    12  input_layernorm                     [2048]                    59.373   1.20554   0.51949  0.00000       nan
    12  self_attn.q_proj                    [2048, 2048]              71.912   0.02639   0.03511  0.00003   1.51193
    12  self_attn.k_proj                    [1024, 2048]              47.209   0.02461   0.03260  0.00004   1.41397
    12  self_attn.v_proj                    [1024, 2048]              48.870   0.02639   0.03374  0.00003   1.52150
    12  self_attn.o_proj                    [2048, 2048]              62.022   0.02345   0.03028  0.00003   1.34798
    12  self_attn.q_norm                    [128]                     22.008   1.79909   0.74309  0.00000       nan
    12  self_attn.k_norm                    [128]                     28.686   1.77965   1.81323  0.00000       nan
    12  post_attention_layernorm            [2048]                    35.525   0.75779   0.20713  0.00000       nan
    12  mlp.gate_proj                       [6144, 2048]             125.090   0.02734   0.03527  0.00002   1.56271
    12  mlp.up_proj                         [6144, 2048]             125.435   0.02766   0.03537  0.00002   1.58552
    12  mlp.down_proj                       [2048, 6144]             120.075   0.02627   0.03386  0.00003   2.63047

    13  input_layernorm                     [2048]                    65.825   1.32442   0.60343  0.00000       nan
    13  self_attn.q_proj                    [2048, 2048]              72.686   0.02604   0.03549  0.00002   1.49013
    13  self_attn.k_proj                    [1024, 2048]              44.996   0.02354   0.03107  0.00003   1.35037
    13  self_attn.v_proj                    [1024, 2048]              48.618   0.02629   0.03357  0.00003   1.51328
    13  self_attn.o_proj                    [2048, 2048]              60.200   0.02273   0.02939  0.00003   1.30548
    13  self_attn.q_norm                    [128]                     23.733   1.85438   0.99210  0.00000       nan
    13  self_attn.k_norm                    [128]                     22.425   1.77937   0.87723  0.00000       nan
    13  post_attention_layernorm            [2048]                    37.208   0.79018   0.22723  0.00000       nan
    13  mlp.gate_proj                       [6144, 2048]             120.812   0.02631   0.03407  0.00003   1.50769
    13  mlp.up_proj                         [6144, 2048]             122.147   0.02692   0.03444  0.00003   1.54372
    13  mlp.down_proj                       [2048, 6144]             117.098   0.02556   0.03302  0.00003   2.56153

    14  input_layernorm                     [2048]                    84.611   1.69142   0.79895  0.00000       nan
    14  self_attn.q_proj                    [2048, 2048]              70.351   0.02527   0.03435  0.00003   1.45125
    14  self_attn.k_proj                    [1024, 2048]              44.671   0.02308   0.03085  0.00003   1.32900
    14  self_attn.v_proj                    [1024, 2048]              47.417   0.02557   0.03274  0.00003   1.47567
    14  self_attn.o_proj                    [2048, 2048]              60.245   0.02268   0.02941  0.00003   1.30631
    14  self_attn.q_norm                    [128]                     23.459   1.79195   1.04803  0.00000       nan
    14  self_attn.k_norm                    [128]                     24.794   1.83374   1.20587  0.00000       nan
    14  post_attention_layernorm            [2048]                    38.752   0.82513   0.22901  0.00000       nan
    14  mlp.gate_proj                       [6144, 2048]             115.589   0.02511   0.03260  0.00003   1.44356
    14  mlp.up_proj                         [6144, 2048]             118.312   0.02602   0.03336  0.00002   1.49604
    14  mlp.down_proj                       [2048, 6144]             116.246   0.02533   0.03278  0.00003   2.54382

    15  input_layernorm                     [2048]                   116.758   2.38523   0.98750  0.00000       nan
    15  self_attn.q_proj                    [2048, 2048]              72.109   0.02581   0.03521  0.00003   1.48097
    15  self_attn.k_proj                    [1024, 2048]              42.654   0.02237   0.02945  0.00003   1.28119
    15  self_attn.v_proj                    [1024, 2048]              47.263   0.02518   0.03264  0.00003   1.46780
    15  self_attn.o_proj                    [2048, 2048]              61.616   0.02314   0.03008  0.00003   1.33738
    15  self_attn.q_norm                    [128]                     18.262   1.49814   0.60459  0.00000       nan
    15  self_attn.k_norm                    [128]                     38.772   2.29244   2.55740  0.00000       nan
    15  post_attention_layernorm            [2048]                    38.488   0.82803   0.19409  0.00000       nan
    15  mlp.gate_proj                       [6144, 2048]             116.356   0.02523   0.03281  0.00003   1.45222
    15  mlp.up_proj                         [6144, 2048]             120.975   0.02656   0.03411  0.00003   1.52807
    15  mlp.down_proj                       [2048, 6144]             116.761   0.02546   0.03293  0.00003   2.55310

    16  input_layernorm                     [2048]                   111.385   2.29854   0.88037  0.00000       nan
    16  self_attn.q_proj                    [2048, 2048]              77.028   0.02664   0.03761  0.00003   1.53832
    16  self_attn.k_proj                    [1024, 2048]              44.274   0.02238   0.03057  0.00003   1.29034
    16  self_attn.v_proj                    [1024, 2048]              47.469   0.02541   0.03278  0.00002   1.47323
    16  self_attn.o_proj                    [2048, 2048]              61.723   0.02308   0.03014  0.00003   1.33927
    16  self_attn.q_norm                    [128]                     19.869   1.62765   0.66726  0.00000       nan
    16  self_attn.k_norm                    [128]                     26.386   2.03656   1.14519  0.00000       nan
    16  post_attention_layernorm            [2048]                    38.689   0.84295   0.14264  0.00000       nan
    16  mlp.gate_proj                       [6144, 2048]             115.659   0.02505   0.03262  0.00003   1.45039
    16  mlp.up_proj                         [6144, 2048]             124.277   0.02710   0.03504  0.00002   1.57209
    16  mlp.down_proj                       [2048, 6144]             117.831   0.02567   0.03323  0.00003   2.59632

    17  input_layernorm                     [2048]                   195.528   4.07512   1.44214  0.00000       nan
    17  self_attn.q_proj                    [2048, 2048]              73.600   0.02634   0.03594  0.00002   1.51986
    17  self_attn.k_proj                    [1024, 2048]              44.185   0.02334   0.03051  0.00002   1.33630
    17  self_attn.v_proj                    [1024, 2048]              49.308   0.02630   0.03405  0.00004   1.52620
    17  self_attn.o_proj                    [2048, 2048]              62.506   0.02339   0.03052  0.00003   1.36478
    17  self_attn.q_norm                    [128]                     23.826   1.75517   1.16920  0.00000       nan
    17  self_attn.k_norm                    [128]                     28.626   1.99286   1.56686  0.00000       nan
    17  post_attention_layernorm            [2048]                    47.981   1.03749   0.21853  0.00000       nan
    17  mlp.gate_proj                       [6144, 2048]             115.404   0.02511   0.03254  0.00003   1.44933
    17  mlp.up_proj                         [6144, 2048]             129.050   0.02835   0.03639  0.00002   1.63090
    17  mlp.down_proj                       [2048, 6144]             121.534   0.02669   0.03427  0.00003   2.66801

    18  input_layernorm                     [2048]                   201.693   4.19416   1.51388  0.00000       nan
    18  self_attn.q_proj                    [2048, 2048]              75.780   0.02675   0.03700  0.00003   1.53918
    18  self_attn.k_proj                    [1024, 2048]              43.076   0.02232   0.02974  0.00003   1.28183
    18  self_attn.v_proj                    [1024, 2048]              46.493   0.02501   0.03210  0.00003   1.45073
    18  self_attn.o_proj                    [2048, 2048]              58.817   0.02227   0.02872  0.00003   1.28603
    18  self_attn.q_norm                    [128]                     18.318   1.53600   0.51702  0.00000       nan
    18  self_attn.k_norm                    [128]                     23.621   1.86847   0.94057  0.00000       nan
    18  post_attention_layernorm            [2048]                    54.117   1.17464   0.22414  0.00000       nan
    18  mlp.gate_proj                       [6144, 2048]             116.670   0.02534   0.03290  0.00003   1.46360
    18  mlp.up_proj                         [6144, 2048]             129.370   0.02840   0.03648  0.00003   1.63489
    18  mlp.down_proj                       [2048, 6144]             124.809   0.02738   0.03519  0.00002   2.73824

    19  input_layernorm                     [2048]                   265.464   5.42234   2.23838  0.00000       nan
    19  self_attn.q_proj                    [2048, 2048]              78.144   0.02687   0.03816  0.00003   1.54781
    19  self_attn.k_proj                    [1024, 2048]              43.280   0.02255   0.02989  0.00003   1.29700
    19  self_attn.v_proj                    [1024, 2048]              46.370   0.02445   0.03202  0.00002   1.43815
    19  self_attn.o_proj                    [2048, 2048]              61.106   0.02311   0.02984  0.00003   1.33827
    19  self_attn.q_norm                    [128]                     19.842   1.61686   0.68621  0.00000       nan
    19  self_attn.k_norm                    [128]                     24.114   1.87106   1.02789  0.00000       nan
    19  post_attention_layernorm            [2048]                    59.540   1.29598   0.22682  0.00000       nan
    19  mlp.gate_proj                       [6144, 2048]             118.280   0.02563   0.03335  0.00003   1.48151
    19  mlp.up_proj                         [6144, 2048]             131.654   0.02883   0.03712  0.00002   1.66379
    19  mlp.down_proj                       [2048, 6144]             128.726   0.02814   0.03630  0.00002   2.82859

    20  input_layernorm                     [2048]                   302.241   6.30764   2.19555  0.00000       nan
    20  self_attn.q_proj                    [2048, 2048]              79.257   0.02737   0.03870  0.00003   1.59106
    20  self_attn.k_proj                    [1024, 2048]              45.357   0.02364   0.03132  0.00003   1.36726
    20  self_attn.v_proj                    [1024, 2048]              48.235   0.02536   0.03331  0.00002   1.49516
    20  self_attn.o_proj                    [2048, 2048]              63.751   0.02390   0.03113  0.00002   1.39336
    20  self_attn.q_norm                    [128]                     19.990   1.63712   0.66759  0.00000       nan
    20  self_attn.k_norm                    [128]                     23.313   1.79316   1.02053  0.00000       nan
    20  post_attention_layernorm            [2048]                    67.529   1.46625   0.27958  0.00000       nan
    20  mlp.gate_proj                       [6144, 2048]             122.886   0.02678   0.03465  0.00003   1.54046
    20  mlp.up_proj                         [6144, 2048]             134.977   0.02972   0.03806  0.00002   1.70752
    20  mlp.down_proj                       [2048, 6144]             132.033   0.02897   0.03723  0.00002   2.89669

    21  input_layernorm                     [2048]                   382.999   7.86958   3.11439  0.00000       nan
    21  self_attn.q_proj                    [2048, 2048]              79.468   0.02798   0.03880  0.00003   1.61697
    21  self_attn.k_proj                    [1024, 2048]              46.374   0.02374   0.03202  0.00003   1.36808
    21  self_attn.v_proj                    [1024, 2048]              52.175   0.02694   0.03603  0.00003   1.58513
    21  self_attn.o_proj                    [2048, 2048]              67.043   0.02514   0.03273  0.00003   1.46654
    21  self_attn.q_norm                    [128]                     18.176   1.48291   0.62333  0.00000       nan
    21  self_attn.k_norm                    [128]                     22.588   1.64089   1.14500  0.00000       nan
    21  post_attention_layernorm            [2048]                    74.667   1.62471   0.28739  0.00000       nan
    21  mlp.gate_proj                       [6144, 2048]             125.949   0.02768   0.03551  0.00002   1.58401
    21  mlp.up_proj                         [6144, 2048]             137.228   0.03036   0.03870  0.00002   1.73898
    21  mlp.down_proj                       [2048, 6144]             134.445   0.02958   0.03791  0.00002   2.94081

    22  input_layernorm                     [2048]                   459.010   9.12582   4.42780  0.00000       nan
    22  self_attn.q_proj                    [2048, 2048]              77.429   0.02735   0.03781  0.00003   1.58268
    22  self_attn.k_proj                    [1024, 2048]              45.910   0.02373   0.03170  0.00004   1.36746
    22  self_attn.v_proj                    [1024, 2048]              51.373   0.02718   0.03547  0.00002   1.59256
    22  self_attn.o_proj                    [2048, 2048]              66.308   0.02509   0.03238  0.00002   1.45173
    22  self_attn.q_norm                    [128]                     18.181   1.46818   0.65705  0.00000       nan
    22  self_attn.k_norm                    [128]                     19.892   1.52586   0.87863  0.00000       nan
    22  post_attention_layernorm            [2048]                    82.794   1.80634   0.29039  0.00000       nan
    22  mlp.gate_proj                       [6144, 2048]             128.172   0.02834   0.03614  0.00002   1.61757
    22  mlp.up_proj                         [6144, 2048]             138.566   0.03079   0.03907  0.00002   1.75877
    22  mlp.down_proj                       [2048, 6144]             136.816   0.03010   0.03858  0.00002   2.98038

    23  input_layernorm                     [2048]                   535.369  10.49632   5.45829  0.00000       nan
    23  self_attn.q_proj                    [2048, 2048]              72.309   0.02620   0.03531  0.00003   1.51609
    23  self_attn.k_proj                    [1024, 2048]              45.286   0.02374   0.03127  0.00004   1.37414
    23  self_attn.v_proj                    [1024, 2048]              50.443   0.02670   0.03483  0.00003   1.56868
    23  self_attn.o_proj                    [2048, 2048]              66.829   0.02537   0.03263  0.00002   1.46522
    23  self_attn.q_norm                    [128]                     17.107   1.42749   0.50050  0.00000       nan
    23  self_attn.k_norm                    [128]                     27.031   2.02744   1.26929  0.00000       nan
    23  post_attention_layernorm            [2048]                    92.141   2.01226   0.31043  0.00000       nan
    23  mlp.gate_proj                       [6144, 2048]             129.759   0.02883   0.03659  0.00002   1.64149
    23  mlp.up_proj                         [6144, 2048]             139.033   0.03096   0.03921  0.00002   1.76572
    23  mlp.down_proj                       [2048, 6144]             138.182   0.03038   0.03897  0.00002   3.00505

    24  input_layernorm                     [2048]                   745.373  13.59591   9.30494  0.00000       nan
    24  self_attn.q_proj                    [2048, 2048]              75.594   0.02681   0.03691  0.00003   1.55826
    24  self_attn.k_proj                    [1024, 2048]              45.236   0.02332   0.03124  0.00003   1.35214
    24  self_attn.v_proj                    [1024, 2048]              50.747   0.02695   0.03504  0.00003   1.58034
    24  self_attn.o_proj                    [2048, 2048]              68.236   0.02596   0.03332  0.00002   1.48595
    24  self_attn.q_norm                    [128]                     17.003   1.38267   0.59374  0.00000       nan
    24  self_attn.k_norm                    [128]                     28.900   2.09434   1.47164  0.00000       nan
    24  post_attention_layernorm            [2048]                    95.613   2.08855   0.31907  0.00000       nan
    24  mlp.gate_proj                       [6144, 2048]             128.384   0.02856   0.03620  0.00002   1.62588
    24  mlp.up_proj                         [6144, 2048]             139.729   0.03114   0.03940  0.00002   1.77526
    24  mlp.down_proj                       [2048, 6144]             138.658   0.03053   0.03910  0.00002   3.01847

    25  input_layernorm                     [2048]                   940.804  15.95273  13.33869  0.00000       nan
    25  self_attn.q_proj                    [2048, 2048]              73.422   0.02611   0.03585  0.00003   1.51835
    25  self_attn.k_proj                    [1024, 2048]              43.686   0.02266   0.03017  0.00003   1.31192
    25  self_attn.v_proj                    [1024, 2048]              56.192   0.02986   0.03880  0.00002   1.74803
    25  self_attn.o_proj                    [2048, 2048]              71.640   0.02723   0.03498  0.00002   1.56534
    25  self_attn.q_norm                    [128]                     17.970   1.46855   0.60751  0.00000       nan
    25  self_attn.k_norm                    [128]                     25.937   1.89888   1.29004  0.00000       nan
    25  post_attention_layernorm            [2048]                   101.466   2.20616   0.40003  0.00000       nan
    25  mlp.gate_proj                       [6144, 2048]             126.516   0.02814   0.03567  0.00002   1.60211
    25  mlp.up_proj                         [6144, 2048]             140.019   0.03120   0.03948  0.00002   1.77892
    25  mlp.down_proj                       [2048, 6144]             137.531   0.03044   0.03878  0.00002   3.00895

    26  input_layernorm                     [2048]                  1102.892  17.94151  16.50383  0.00000       nan
    26  self_attn.q_proj                    [2048, 2048]              79.566   0.02773   0.03885  0.00003   1.61306
    26  self_attn.k_proj                    [1024, 2048]              42.406   0.02208   0.02928  0.00004   1.27361
    26  self_attn.v_proj                    [1024, 2048]              54.631   0.02919   0.03772  0.00002   1.68969
    26  self_attn.o_proj                    [2048, 2048]              71.338   0.02655   0.03483  0.00002   1.55324
    26  self_attn.q_norm                    [128]                     16.547   1.26933   0.73112  0.00000       nan
    26  self_attn.k_norm                    [128]                     21.617   1.58323   1.07843  0.00000       nan
    26  post_attention_layernorm            [2048]                   116.430   2.45460   0.77095  0.00000       nan
    26  mlp.gate_proj                       [6144, 2048]             125.452   0.02789   0.03537  0.00002   1.58906
    26  mlp.up_proj                         [6144, 2048]             139.144   0.03096   0.03924  0.00002   1.76593
    26  mlp.down_proj                       [2048, 6144]             135.120   0.02999   0.03810  0.00002   2.96974

    27  input_layernorm                     [2048]                   810.531  17.28486   4.69322  0.00000       nan
    27  self_attn.q_proj                    [2048, 2048]              72.699   0.02733   0.03550  0.00002   1.55820
    27  self_attn.k_proj                    [1024, 2048]              49.078   0.02647   0.03389  0.00002   1.51007
    27  self_attn.v_proj                    [1024, 2048]              54.406   0.02942   0.03757  0.00003   1.69188
    27  self_attn.o_proj                    [2048, 2048]              71.960   0.02762   0.03514  0.00002   1.58045
    27  self_attn.q_norm                    [128]                     18.873   1.53224   0.66789  0.00000       nan
    27  self_attn.k_norm                    [128]                     23.474   1.75737   1.14853  0.00000       nan
    27  post_attention_layernorm            [2048]                   210.356   3.07466   3.48695  0.00000       nan
    27  mlp.gate_proj                       [6144, 2048]             132.785   0.02938   0.03744  0.00002   1.67322
    27  mlp.up_proj                         [6144, 2048]             137.233   0.03040   0.03870  0.00002   1.73311
    27  mlp.down_proj                       [2048, 6144]             125.503   0.02759   0.03539  0.00002   2.76534


[08] Isotropy analysis  (distilled, 2D tensors only)
  Sampling up to 2048 rows per layer.
  Score near 0 = isotropic (healthy).  Score near 1 = representation collapse.

  Layer                                                                Shape                     Score
  ------------------------------------------------------------------------------------------------------
  model.embed_tokens.weight                                            [151936, 2048]         0.053880
  model.layers.0.self_attn.q_proj.weight                               [2048, 2048]          -0.000024
  model.layers.0.self_attn.k_proj.weight                               [1024, 2048]          -0.000105
  model.layers.0.self_attn.v_proj.weight                               [1024, 2048]           0.000003
  model.layers.0.self_attn.o_proj.weight                               [2048, 2048]           0.000003
  model.layers.0.mlp.gate_proj.weight                                  [6144, 2048]           0.004988
  model.layers.0.mlp.up_proj.weight                                    [6144, 2048]          -0.000027
  model.layers.0.mlp.down_proj.weight                                  [2048, 6144]          -0.000010
  model.layers.1.self_attn.q_proj.weight                               [2048, 2048]           0.000003
  model.layers.1.self_attn.k_proj.weight                               [1024, 2048]           0.000002
  model.layers.1.self_attn.v_proj.weight                               [1024, 2048]          -0.000032
  model.layers.1.self_attn.o_proj.weight                               [2048, 2048]          -0.000009
  model.layers.1.mlp.gate_proj.weight                                  [6144, 2048]           0.022451
  model.layers.1.mlp.up_proj.weight                                    [6144, 2048]           0.000007
  model.layers.1.mlp.down_proj.weight                                  [2048, 6144]          -0.000011
  model.layers.2.self_attn.q_proj.weight                               [2048, 2048]           0.000066
  model.layers.2.self_attn.k_proj.weight                               [1024, 2048]          -0.000001
  model.layers.2.self_attn.v_proj.weight                               [1024, 2048]           0.000026
  model.layers.2.self_attn.o_proj.weight                               [2048, 2048]          -0.000055
  model.layers.2.mlp.gate_proj.weight                                  [6144, 2048]           0.006968
  model.layers.2.mlp.up_proj.weight                                    [6144, 2048]           0.000021
  model.layers.2.mlp.down_proj.weight                                  [2048, 6144]           0.000004
  model.layers.3.self_attn.q_proj.weight                               [2048, 2048]           0.000014
  model.layers.3.self_attn.k_proj.weight                               [1024, 2048]          -0.000067
  model.layers.3.self_attn.v_proj.weight                               [1024, 2048]          -0.000037
  model.layers.3.self_attn.o_proj.weight                               [2048, 2048]           0.000007
  model.layers.3.mlp.gate_proj.weight                                  [6144, 2048]           0.007741
  model.layers.3.mlp.up_proj.weight                                    [6144, 2048]           0.000003
  model.layers.3.mlp.down_proj.weight                                  [2048, 6144]           0.000014
  model.layers.4.self_attn.q_proj.weight                               [2048, 2048]           0.000045
  model.layers.4.self_attn.k_proj.weight                               [1024, 2048]          -0.000034
  model.layers.4.self_attn.v_proj.weight                               [1024, 2048]           0.000003
  model.layers.4.self_attn.o_proj.weight                               [2048, 2048]          -0.000006
  model.layers.4.mlp.gate_proj.weight                                  [6144, 2048]           0.011344
  model.layers.4.mlp.up_proj.weight                                    [6144, 2048]           0.000008
  model.layers.4.mlp.down_proj.weight                                  [2048, 6144]           0.000012
  model.layers.5.self_attn.q_proj.weight                               [2048, 2048]           0.000046
  model.layers.5.self_attn.k_proj.weight                               [1024, 2048]          -0.000004
  model.layers.5.self_attn.v_proj.weight                               [1024, 2048]           0.000016
  model.layers.5.self_attn.o_proj.weight                               [2048, 2048]           0.000011
  model.layers.5.mlp.gate_proj.weight                                  [6144, 2048]           0.006708
  model.layers.5.mlp.up_proj.weight                                    [6144, 2048]          -0.000022
  model.layers.5.mlp.down_proj.weight                                  [2048, 6144]           0.000015
  model.layers.6.self_attn.q_proj.weight                               [2048, 2048]           0.000093
  model.layers.6.self_attn.k_proj.weight                               [1024, 2048]          -0.000075
  model.layers.6.self_attn.v_proj.weight                               [1024, 2048]          -0.000043
  model.layers.6.self_attn.o_proj.weight                               [2048, 2048]           0.000007
  model.layers.6.mlp.gate_proj.weight                                  [6144, 2048]           0.007193
  model.layers.6.mlp.up_proj.weight                                    [6144, 2048]          -0.000010
  model.layers.6.mlp.down_proj.weight                                  [2048, 6144]           0.000017
  model.layers.7.self_attn.q_proj.weight                               [2048, 2048]           0.000059
  model.layers.7.self_attn.k_proj.weight                               [1024, 2048]          -0.000034
  model.layers.7.self_attn.v_proj.weight                               [1024, 2048]          -0.000037
  model.layers.7.self_attn.o_proj.weight                               [2048, 2048]           0.000010
  model.layers.7.mlp.gate_proj.weight                                  [6144, 2048]           0.009189
  model.layers.7.mlp.up_proj.weight                                    [6144, 2048]           0.000011
  model.layers.7.mlp.down_proj.weight                                  [2048, 6144]          -0.000001
  model.layers.8.self_attn.q_proj.weight                               [2048, 2048]           0.000037
  model.layers.8.self_attn.k_proj.weight                               [1024, 2048]          -0.000017
  model.layers.8.self_attn.v_proj.weight                               [1024, 2048]          -0.000040
  model.layers.8.self_attn.o_proj.weight                               [2048, 2048]           0.000012
  model.layers.8.mlp.gate_proj.weight                                  [6144, 2048]           0.006754
  model.layers.8.mlp.up_proj.weight                                    [6144, 2048]           0.000022
  model.layers.8.mlp.down_proj.weight                                  [2048, 6144]           0.000009
  model.layers.9.self_attn.q_proj.weight                               [2048, 2048]           0.000025
  model.layers.9.self_attn.k_proj.weight                               [1024, 2048]           0.000040
  model.layers.9.self_attn.v_proj.weight                               [1024, 2048]           0.000018
  model.layers.9.self_attn.o_proj.weight                               [2048, 2048]           0.000007
  model.layers.9.mlp.gate_proj.weight                                  [6144, 2048]           0.005668
  model.layers.9.mlp.up_proj.weight                                    [6144, 2048]           0.000008
  model.layers.9.mlp.down_proj.weight                                  [2048, 6144]           0.000004
  model.layers.10.self_attn.q_proj.weight                              [2048, 2048]           0.000032
  model.layers.10.self_attn.k_proj.weight                              [1024, 2048]          -0.000022
  model.layers.10.self_attn.v_proj.weight                              [1024, 2048]           0.000004
  model.layers.10.self_attn.o_proj.weight                              [2048, 2048]           0.000008
  model.layers.10.mlp.gate_proj.weight                                 [6144, 2048]           0.004282
  model.layers.10.mlp.up_proj.weight                                   [6144, 2048]          -0.000008
  model.layers.10.mlp.down_proj.weight                                 [2048, 6144]           0.000002
  model.layers.11.self_attn.q_proj.weight                              [2048, 2048]           0.000071
  model.layers.11.self_attn.k_proj.weight                              [1024, 2048]          -0.000073
  model.layers.11.self_attn.v_proj.weight                              [1024, 2048]           0.000048
  model.layers.11.self_attn.o_proj.weight                              [2048, 2048]           0.000011
  model.layers.11.mlp.gate_proj.weight                                 [6144, 2048]           0.003255
  model.layers.11.mlp.up_proj.weight                                   [6144, 2048]          -0.000026
  model.layers.11.mlp.down_proj.weight                                 [2048, 6144]          -0.000023
  model.layers.12.self_attn.q_proj.weight                              [2048, 2048]           0.000031
  model.layers.12.self_attn.k_proj.weight                              [1024, 2048]          -0.000078
  model.layers.12.self_attn.v_proj.weight                              [1024, 2048]           0.000001
  model.layers.12.self_attn.o_proj.weight                              [2048, 2048]          -0.000010
  model.layers.12.mlp.gate_proj.weight                                 [6144, 2048]           0.002464
  model.layers.12.mlp.up_proj.weight                                   [6144, 2048]           0.000018
  model.layers.12.mlp.down_proj.weight                                 [2048, 6144]          -0.000003
  model.layers.13.self_attn.q_proj.weight                              [2048, 2048]           0.000075
  model.layers.13.self_attn.k_proj.weight                              [1024, 2048]          -0.000093
  model.layers.13.self_attn.v_proj.weight                              [1024, 2048]           0.000004
  model.layers.13.self_attn.o_proj.weight                              [2048, 2048]           0.000006
  model.layers.13.mlp.gate_proj.weight                                 [6144, 2048]           0.002376
  model.layers.13.mlp.up_proj.weight                                   [6144, 2048]           0.000014
  model.layers.13.mlp.down_proj.weight                                 [2048, 6144]           0.000006
  model.layers.14.self_attn.q_proj.weight                              [2048, 2048]           0.000035
  model.layers.14.self_attn.k_proj.weight                              [1024, 2048]          -0.000052
  model.layers.14.self_attn.v_proj.weight                              [1024, 2048]           0.000018
  model.layers.14.self_attn.o_proj.weight                              [2048, 2048]           0.000013
  model.layers.14.mlp.gate_proj.weight                                 [6144, 2048]           0.002415
  model.layers.14.mlp.up_proj.weight                                   [6144, 2048]           0.000037
  model.layers.14.mlp.down_proj.weight                                 [2048, 6144]           0.000017
  model.layers.15.self_attn.q_proj.weight                              [2048, 2048]           0.000125
  model.layers.15.self_attn.k_proj.weight                              [1024, 2048]           0.000015
  model.layers.15.self_attn.v_proj.weight                              [1024, 2048]          -0.000009
  model.layers.15.self_attn.o_proj.weight                              [2048, 2048]           0.000011
  model.layers.15.mlp.gate_proj.weight                                 [6144, 2048]           0.002226
  model.layers.15.mlp.up_proj.weight                                   [6144, 2048]          -0.000013
  model.layers.15.mlp.down_proj.weight                                 [2048, 6144]          -0.000011
  model.layers.16.self_attn.q_proj.weight                              [2048, 2048]           0.000119
  model.layers.16.self_attn.k_proj.weight                              [1024, 2048]           0.000000
  model.layers.16.self_attn.v_proj.weight                              [1024, 2048]          -0.000033
  model.layers.16.self_attn.o_proj.weight                              [2048, 2048]           0.000011
  model.layers.16.mlp.gate_proj.weight                                 [6144, 2048]           0.002481
  model.layers.16.mlp.up_proj.weight                                   [6144, 2048]          -0.000019
  model.layers.16.mlp.down_proj.weight                                 [2048, 6144]          -0.000020
  model.layers.17.self_attn.q_proj.weight                              [2048, 2048]           0.000154
  model.layers.17.self_attn.k_proj.weight                              [1024, 2048]           0.000096
  model.layers.17.self_attn.v_proj.weight                              [1024, 2048]          -0.000025
  model.layers.17.self_attn.o_proj.weight                              [2048, 2048]           0.000005
  model.layers.17.mlp.gate_proj.weight                                 [6144, 2048]           0.003676
  model.layers.17.mlp.up_proj.weight                                   [6144, 2048]           0.000009
  model.layers.17.mlp.down_proj.weight                                 [2048, 6144]           0.000008
  model.layers.18.self_attn.q_proj.weight                              [2048, 2048]           0.000152
  model.layers.18.self_attn.k_proj.weight                              [1024, 2048]          -0.000039
  model.layers.18.self_attn.v_proj.weight                              [1024, 2048]          -0.000016
  model.layers.18.self_attn.o_proj.weight                              [2048, 2048]          -0.000004
  model.layers.18.mlp.gate_proj.weight                                 [6144, 2048]           0.004275
  model.layers.18.mlp.up_proj.weight                                   [6144, 2048]          -0.000034
  model.layers.18.mlp.down_proj.weight                                 [2048, 6144]           0.000024
  model.layers.19.self_attn.q_proj.weight                              [2048, 2048]           0.000093
  model.layers.19.self_attn.k_proj.weight                              [1024, 2048]          -0.000042
  model.layers.19.self_attn.v_proj.weight                              [1024, 2048]          -0.000006
  model.layers.19.self_attn.o_proj.weight                              [2048, 2048]          -0.000030
  model.layers.19.mlp.gate_proj.weight                                 [6144, 2048]           0.005028
  model.layers.19.mlp.up_proj.weight                                   [6144, 2048]           0.000002
  model.layers.19.mlp.down_proj.weight                                 [2048, 6144]          -0.000000
  model.layers.20.self_attn.q_proj.weight                              [2048, 2048]           0.000107
  model.layers.20.self_attn.k_proj.weight                              [1024, 2048]           0.000033
  model.layers.20.self_attn.v_proj.weight                              [1024, 2048]           0.000032
  model.layers.20.self_attn.o_proj.weight                              [2048, 2048]           0.000024
  model.layers.20.mlp.gate_proj.weight                                 [6144, 2048]           0.006397
  model.layers.20.mlp.up_proj.weight                                   [6144, 2048]           0.000010
  model.layers.20.mlp.down_proj.weight                                 [2048, 6144]           0.000013
  model.layers.21.self_attn.q_proj.weight                              [2048, 2048]           0.000059
  model.layers.21.self_attn.k_proj.weight                              [1024, 2048]          -0.000016
  model.layers.21.self_attn.v_proj.weight                              [1024, 2048]           0.000006
  model.layers.21.self_attn.o_proj.weight                              [2048, 2048]           0.000017
  model.layers.21.mlp.gate_proj.weight                                 [6144, 2048]           0.007248
  model.layers.21.mlp.up_proj.weight                                   [6144, 2048]          -0.000014
  model.layers.21.mlp.down_proj.weight                                 [2048, 6144]          -0.000005
  model.layers.22.self_attn.q_proj.weight                              [2048, 2048]           0.000028
  model.layers.22.self_attn.k_proj.weight                              [1024, 2048]          -0.000029
  model.layers.22.self_attn.v_proj.weight                              [1024, 2048]          -0.000045
  model.layers.22.self_attn.o_proj.weight                              [2048, 2048]           0.000050
  model.layers.22.mlp.gate_proj.weight                                 [6144, 2048]           0.008393
  model.layers.22.mlp.up_proj.weight                                   [6144, 2048]           0.000002
  model.layers.22.mlp.down_proj.weight                                 [2048, 6144]           0.000002
  model.layers.23.self_attn.q_proj.weight                              [2048, 2048]           0.000058
  model.layers.23.self_attn.k_proj.weight                              [1024, 2048]          -0.000055
  model.layers.23.self_attn.v_proj.weight                              [1024, 2048]           0.000044
  model.layers.23.self_attn.o_proj.weight                              [2048, 2048]          -0.000004
  model.layers.23.mlp.gate_proj.weight                                 [6144, 2048]           0.010053
  model.layers.23.mlp.up_proj.weight                                   [6144, 2048]          -0.000030
  model.layers.23.mlp.down_proj.weight                                 [2048, 6144]           0.000003
  model.layers.24.self_attn.q_proj.weight                              [2048, 2048]           0.000135
  model.layers.24.self_attn.k_proj.weight                              [1024, 2048]          -0.000025
  model.layers.24.self_attn.v_proj.weight                              [1024, 2048]           0.000060
  model.layers.24.self_attn.o_proj.weight                              [2048, 2048]           0.000004
  model.layers.24.mlp.gate_proj.weight                                 [6144, 2048]           0.008813
  model.layers.24.mlp.up_proj.weight                                   [6144, 2048]           0.000034
  model.layers.24.mlp.down_proj.weight                                 [2048, 6144]          -0.000001
  model.layers.25.self_attn.q_proj.weight                              [2048, 2048]           0.000192
  model.layers.25.self_attn.k_proj.weight                              [1024, 2048]           0.000218
  model.layers.25.self_attn.v_proj.weight                              [1024, 2048]          -0.000024
  model.layers.25.self_attn.o_proj.weight                              [2048, 2048]          -0.000006
  model.layers.25.mlp.gate_proj.weight                                 [6144, 2048]           0.006651
  model.layers.25.mlp.up_proj.weight                                   [6144, 2048]          -0.000004
  model.layers.25.mlp.down_proj.weight                                 [2048, 6144]           0.000001
  model.layers.26.self_attn.q_proj.weight                              [2048, 2048]           0.000148
  model.layers.26.self_attn.k_proj.weight                              [1024, 2048]          -0.000014
  model.layers.26.self_attn.v_proj.weight                              [1024, 2048]           0.000003
  model.layers.26.self_attn.o_proj.weight                              [2048, 2048]           0.000006
  model.layers.26.mlp.gate_proj.weight                                 [6144, 2048]           0.006363
  model.layers.26.mlp.up_proj.weight                                   [6144, 2048]           0.000014
  model.layers.26.mlp.down_proj.weight                                 [2048, 6144]           0.000014
  model.layers.27.self_attn.q_proj.weight                              [2048, 2048]           0.000033
  model.layers.27.self_attn.k_proj.weight                              [1024, 2048]          -0.000059
  model.layers.27.self_attn.v_proj.weight                              [1024, 2048]          -0.000051
  model.layers.27.self_attn.o_proj.weight                              [2048, 2048]           0.000006
  model.layers.27.mlp.gate_proj.weight                                 [6144, 2048]           0.009203
  model.layers.27.mlp.up_proj.weight                                   [6144, 2048]          -0.000008
  model.layers.27.mlp.down_proj.weight                                 [2048, 6144]           0.000004
  lm_head.weight                                                       [151936, 2048]         0.053880

  Global (across 198 2D layers)
    mean : 0.001484
    min  : -0.000105
    max  : 0.053880

[09] Base vs distilled divergence  (all shared layers)
  Shared tensors: 311

  Layer                                                                 MaxDelta  MeanDelta   L2Delta   CosSim   RelErr  SNR_dB  Chg
  ---------------------------------------------------------------------------------------------------------------------------------------
  lm_head.weight                                                         0.00835   0.000098    8.2285  0.99991  0.00353   37.60    Y
  model.embed_tokens.weight                                              0.00835   0.000098    8.2285  0.99991  0.00353   37.60    Y
  model.layers.0.input_layernorm.weight                                  0.00000   0.000000    0.0000  1.00000  0.00000  253.41    N
  model.layers.0.mlp.down_proj.weight                                    0.00291   0.000030    0.4447  0.99999  0.00112   48.57    Y
  model.layers.0.mlp.gate_proj.weight                                    0.00278   0.000026    0.4056  1.00000  0.00084   50.66    Y
  model.layers.0.mlp.up_proj.weight                                      0.00301   0.000031    0.4568  0.99999  0.00125   47.85    Y
  model.layers.0.post_attention_layernorm.weight                         0.00154   0.000006    0.0033  1.00000  0.00003   70.61    Y
  model.layers.0.self_attn.k_norm.weight                                 0.00000   0.000000    0.0000  1.00000  0.00000  278.19    N
  model.layers.0.self_attn.k_proj.weight                                 0.00240   0.000026    0.1657  1.00000  0.00094   50.01    Y
  model.layers.0.self_attn.o_proj.weight                                 0.00251   0.000031    0.2599  0.99999  0.00125   48.25    Y
  model.layers.0.self_attn.q_norm.weight                                 0.00018   0.000001    0.0002  1.00000  0.00000  101.13    Y
  model.layers.0.self_attn.q_proj.weight                                 0.00270   0.000027    0.2340  0.99999  0.00099   49.87    Y
  model.layers.0.self_attn.v_proj.weight                                 0.00266   0.000029    0.1828  0.99999  0.00110   48.61    Y
  model.layers.1.input_layernorm.weight                                  0.00000   0.000000    0.0000  1.00000  0.00000  252.95    N
  model.layers.1.mlp.down_proj.weight                                    0.00307   0.000039    0.5086  0.99999  0.00175   46.25    Y
  model.layers.1.mlp.gate_proj.weight                                    0.00272   0.000026    0.4024  1.00000  0.00076   52.03    Y
  model.layers.1.mlp.up_proj.weight                                      0.00310   0.000043    0.5255  0.99999  0.00198   45.80    Y
  model.layers.1.post_attention_layernorm.weight                         0.00060   0.000001    0.0008  1.00000  0.00000   89.51    Y
  model.layers.1.self_attn.k_norm.weight                                 0.00000   0.000000    0.0000  1.00000  0.00000  272.09    N
  model.layers.1.self_attn.k_proj.weight                                 0.00246   0.000030    0.1827  0.99999  0.00116   48.65    Y
  model.layers.1.self_attn.o_proj.weight                                 0.00259   0.000031    0.2623  0.99999  0.00121   48.18    Y
  model.layers.1.self_attn.q_norm.weight                                 0.00000   0.000000    0.0000  1.00000  0.00000  265.25    N
  model.layers.1.self_attn.q_proj.weight                                 0.00296   0.000029    0.2489  0.99999  0.00111   49.17    Y
  model.layers.1.self_attn.v_proj.weight                                 0.00258   0.000029    0.1793  0.99999  0.00104   49.01    Y
  model.layers.10.input_layernorm.weight                                 0.00000   0.000000    0.0000  1.00000  0.00000  271.30    N
  model.layers.10.mlp.down_proj.weight                                   0.00322   0.000044    0.5900  0.99999  0.00173   45.89    Y
  model.layers.10.mlp.gate_proj.weight                                   0.00347   0.000036    0.5295  0.99999  0.00124   48.14    Y
  model.layers.10.mlp.up_proj.weight                                     0.00333   0.000039    0.5501  0.99999  0.00143   47.00    Y
  model.layers.10.post_attention_layernorm.weight                        0.00214   0.000002    0.0028  1.00000  0.00000   80.97    Y
  model.layers.10.self_attn.k_norm.weight                                0.00071   0.000013    0.0009  1.00000  0.00001   90.93    Y
  model.layers.10.self_attn.k_proj.weight                                0.00366   0.000039    0.2194  0.99999  0.00151   46.82    Y
  model.layers.10.self_attn.o_proj.weight                                0.00312   0.000044    0.3382  0.99999  0.00179   45.71    Y
  model.layers.10.self_attn.q_norm.weight                                0.00000   0.000000    0.0000  1.00000  0.00000  267.98    N
  model.layers.10.self_attn.q_proj.weight                                0.00278   0.000038    0.3137  0.99999  0.00146   47.07    Y
  model.layers.10.self_attn.v_proj.weight                                0.00298   0.000039    0.2219  0.99999  0.00144   47.05    Y
  model.layers.11.input_layernorm.weight                                 0.00099   0.000000    0.0010  1.00000  0.00000   93.98    Y
  model.layers.11.mlp.down_proj.weight                                   0.00330   0.000042    0.5863  0.99999  0.00160   46.23    Y
  model.layers.11.mlp.gate_proj.weight                                   0.00365   0.000038    0.5469  0.99999  0.00136   47.43    Y
  model.layers.11.mlp.up_proj.weight                                     0.00340   0.000039    0.5542  0.99999  0.00139   47.13    Y
  model.layers.11.post_attention_layernorm.weight                        0.00206   0.000002    0.0028  1.00000  0.00000   82.01    Y
  model.layers.11.self_attn.k_norm.weight                                0.00061   0.000006    0.0006  1.00000  0.00000   92.39    Y
  model.layers.11.self_attn.k_proj.weight                                0.00314   0.000044    0.2314  0.99999  0.00175   46.42    Y
  model.layers.11.self_attn.o_proj.weight                                0.00332   0.000050    0.3651  0.99998  0.00224   44.60    Y
  model.layers.11.self_attn.q_norm.weight                                0.00000   0.000000    0.0000  1.00000  0.00000  266.53    N
  model.layers.11.self_attn.q_proj.weight                                0.00308   0.000042    0.3231  0.99999  0.00157   47.33    Y
  model.layers.11.self_attn.v_proj.weight                                0.00307   0.000047    0.2479  0.99999  0.00187   45.64    Y
  model.layers.12.input_layernorm.weight                                 0.00122   0.000001    0.0012  1.00000  0.00000   93.71    Y
  model.layers.12.mlp.down_proj.weight                                   0.00352   0.000044    0.6074  0.99999  0.00168   45.92    Y
  model.layers.12.mlp.gate_proj.weight                                   0.00385   0.000041    0.5746  0.99999  0.00149   46.76    Y
  model.layers.12.mlp.up_proj.weight                                     0.00341   0.000040    0.5733  0.99999  0.00145   46.80    Y
  model.layers.12.post_attention_layernorm.weight                        0.00207   0.000003    0.0035  1.00000  0.00000   80.16    Y
  model.layers.12.self_attn.k_norm.weight                                0.00067   0.000020    0.0012  1.00000  0.00001   87.90    Y
  model.layers.12.self_attn.k_proj.weight                                0.00305   0.000043    0.2318  0.99999  0.00173   46.18    Y
  model.layers.12.self_attn.o_proj.weight                                0.00328   0.000049    0.3622  0.99998  0.00207   44.67    Y
  model.layers.12.self_attn.q_norm.weight                                0.00000   0.000000    0.0000  1.00000  0.00000  266.85    N
  model.layers.12.self_attn.q_proj.weight                                0.00286   0.000040    0.3211  0.99999  0.00153   47.00    Y
  model.layers.12.self_attn.v_proj.weight                                0.00271   0.000042    0.2334  0.99999  0.00158   46.42    Y
  model.layers.13.input_layernorm.weight                                 0.00095   0.000000    0.0009  1.00000  0.00000   96.85    Y
  model.layers.13.mlp.down_proj.weight                                   0.00358   0.000047    0.6369  0.99999  0.00184   45.29    Y
  model.layers.13.mlp.gate_proj.weight                                   0.00328   0.000043    0.5999  0.99999  0.00165   46.08    Y
  model.layers.13.mlp.up_proj.weight                                     0.00400   0.000042    0.5943  0.99999  0.00157   46.26    Y
  model.layers.13.post_attention_layernorm.weight                        0.00226   0.000002    0.0030  1.00000  0.00000   81.90    Y
  model.layers.13.self_attn.k_norm.weight                                0.00053   0.000014    0.0008  1.00000  0.00001   89.00    Y
  model.layers.13.self_attn.k_proj.weight                                0.00278   0.000044    0.2374  0.99999  0.00188   45.55    Y
  model.layers.13.self_attn.o_proj.weight                                0.00347   0.000051    0.3762  0.99998  0.00225   44.08    Y
  model.layers.13.self_attn.q_norm.weight                                0.00000   0.000000    0.0000  1.00000  0.00000  267.51    N
  model.layers.13.self_attn.q_proj.weight                                0.00315   0.000042    0.3262  0.99999  0.00160   46.96    Y
  model.layers.13.self_attn.v_proj.weight                                0.00307   0.000042    0.2380  0.99999  0.00161   46.20    Y
  model.layers.14.input_layernorm.weight                                 0.00164   0.000001    0.0016  1.00000  0.00000   94.24    Y
  model.layers.14.mlp.down_proj.weight                                   0.00346   0.000049    0.6642  0.99998  0.00195   44.86    Y
  model.layers.14.mlp.gate_proj.weight                                   0.00356   0.000047    0.6357  0.99998  0.00189   45.20    Y
  model.layers.14.mlp.up_proj.weight                                     0.00351   0.000045    0.6247  0.99999  0.00175   45.55    Y
  model.layers.14.post_attention_layernorm.weight                        0.00279   0.000003    0.0041  1.00000  0.00000   79.52    Y
  model.layers.14.self_attn.k_norm.weight                                0.00095   0.000017    0.0012  1.00000  0.00001   86.20    Y
  model.layers.14.self_attn.k_proj.weight                                0.00294   0.000047    0.2449  0.99998  0.00202   45.22    Y
  model.layers.14.self_attn.o_proj.weight                                0.00346   0.000053    0.3908  0.99998  0.00234   43.76    Y
  model.layers.14.self_attn.q_norm.weight                                0.00000   0.000000    0.0000  1.00000  0.00000  267.41    N
  model.layers.14.self_attn.q_proj.weight                                0.00294   0.000044    0.3352  0.99999  0.00173   46.44    Y
  model.layers.14.self_attn.v_proj.weight                                0.00388   0.000045    0.2499  0.99999  0.00177   45.56    Y
  model.layers.15.input_layernorm.weight                                 0.00200   0.000001    0.0020  1.00000  0.00000   95.34    Y
  model.layers.15.mlp.down_proj.weight                                   0.00398   0.000051    0.6876  0.99998  0.00201   44.60    Y
  model.layers.15.mlp.gate_proj.weight                                   0.00379   0.000049    0.6628  0.99998  0.00195   44.89    Y
  model.layers.15.mlp.up_proj.weight                                     0.00357   0.000047    0.6463  0.99999  0.00176   45.45    Y
  model.layers.15.post_attention_layernorm.weight                        0.00395   0.000004    0.0054  1.00000  0.00001   77.13    Y
  model.layers.15.self_attn.k_norm.weight                                0.00094   0.000012    0.0011  1.00000  0.00001   90.67    Y
  model.layers.15.self_attn.k_proj.weight                                0.00307   0.000052    0.2690  0.99998  0.00233   44.00    Y
  model.layers.15.self_attn.o_proj.weight                                0.00349   0.000056    0.4069  0.99998  0.00240   43.60    Y
  model.layers.15.self_attn.q_norm.weight                                0.00089   0.000009    0.0009  1.00000  0.00001   86.00    Y
  model.layers.15.self_attn.q_proj.weight                                0.00331   0.000050    0.3798  0.99999  0.00193   45.57    Y
  model.layers.15.self_attn.v_proj.weight                                0.00330   0.000051    0.2726  0.99998  0.00202   44.78    Y
  model.layers.16.input_layernorm.weight                                 0.00205   0.000001    0.0020  1.00000  0.00000   94.71    Y
  model.layers.16.mlp.down_proj.weight                                   0.00372   0.000051    0.6956  0.99998  0.00198   44.58    Y
  model.layers.16.mlp.gate_proj.weight                                   0.00354   0.000052    0.6944  0.99998  0.00209   44.43    Y
  model.layers.16.mlp.up_proj.weight                                     0.00415   0.000049    0.6728  0.99999  0.00180   45.33    Y
  model.layers.16.post_attention_layernorm.weight                        0.00370   0.000004    0.0052  1.00000  0.00001   77.43    Y
  model.layers.16.self_attn.k_norm.weight                                0.00192   0.000021    0.0020  1.00000  0.00001   82.55    Y
  model.layers.16.self_attn.k_proj.weight                                0.00351   0.000053    0.2705  0.99998  0.00236   44.28    Y
  model.layers.16.self_attn.o_proj.weight                                0.00360   0.000056    0.4125  0.99998  0.00244   43.50    Y
  model.layers.16.self_attn.q_norm.weight                                0.00018   0.000001    0.0002  1.00000  0.00000  100.71    Y
  model.layers.16.self_attn.q_proj.weight                                0.00307   0.000048    0.3639  0.99999  0.00178   46.51    Y
  model.layers.16.self_attn.v_proj.weight                                0.00346   0.000049    0.2702  0.99998  0.00194   44.89    Y
  model.layers.17.input_layernorm.weight                                 0.00184   0.000001    0.0018  1.00000  0.00000  100.52    Y
  model.layers.17.mlp.down_proj.weight                                   0.00383   0.000050    0.6924  0.99998  0.00186   44.89    Y
  model.layers.17.mlp.gate_proj.weight                                   0.00432   0.000052    0.6950  0.99998  0.00207   44.41    Y
  model.layers.17.mlp.up_proj.weight                                     0.00419   0.000046    0.6594  0.99999  0.00163   45.83    Y
  model.layers.17.post_attention_layernorm.weight                        0.00394   0.000007    0.0073  1.00000  0.00001   76.41    Y
  model.layers.17.self_attn.k_norm.weight                                0.00046   0.000007    0.0006  1.00000  0.00000   93.71    Y
  model.layers.17.self_attn.k_proj.weight                                0.00357   0.000053    0.2760  0.99998  0.00227   44.09    Y
  model.layers.17.self_attn.o_proj.weight                                0.00359   0.000059    0.4249  0.99998  0.00250   43.35    Y
  model.layers.17.self_attn.q_norm.weight                                0.00000   0.000000    0.0000  1.00000  0.00000  267.54    N
  model.layers.17.self_attn.q_proj.weight                                0.00327   0.000051    0.3870  0.99999  0.00195   45.58    Y
  model.layers.17.self_attn.v_proj.weight                                0.00369   0.000054    0.2911  0.99998  0.00205   44.58    Y
  model.layers.18.input_layernorm.weight                                 0.00153   0.000001    0.0015  1.00000  0.00000  102.37    Y
  model.layers.18.mlp.down_proj.weight                                   0.00391   0.000049    0.6975  0.99998  0.00180   45.06    Y
  model.layers.18.mlp.gate_proj.weight                                   0.00538   0.000052    0.7063  0.99998  0.00207   44.36    Y
  model.layers.18.mlp.up_proj.weight                                     0.00451   0.000047    0.6702  0.99999  0.00165   45.71    Y
  model.layers.18.post_attention_layernorm.weight                        0.00314   0.000004    0.0046  1.00000  0.00000   81.37    Y
  model.layers.18.self_attn.k_norm.weight                                0.00085   0.000010    0.0010  1.00000  0.00001   87.85    Y
  model.layers.18.self_attn.k_proj.weight                                0.00306   0.000053    0.2766  0.99998  0.00240   43.85    Y
  model.layers.18.self_attn.o_proj.weight                                0.00364   0.000059    0.4308  0.99997  0.00263   42.70    Y
  model.layers.18.self_attn.q_norm.weight                                0.00012   0.000001    0.0001  1.00000  0.00000  103.53    Y
  model.layers.18.self_attn.q_proj.weight                                0.00320   0.000048    0.3734  0.99999  0.00179   46.15    Y
  model.layers.18.self_attn.v_proj.weight                                0.00336   0.000053    0.2861  0.99998  0.00211   44.22    Y
  model.layers.19.input_layernorm.weight                                 0.00160   0.000001    0.0016  1.00000  0.00000  104.39    Y
  model.layers.19.mlp.down_proj.weight                                   0.00453   0.000050    0.7104  0.99998  0.00177   45.17    Y
  model.layers.19.mlp.gate_proj.weight                                   0.00410   0.000053    0.7214  0.99998  0.00208   44.30    Y
  model.layers.19.mlp.up_proj.weight                                     0.00522   0.000048    0.6880  0.99999  0.00165   45.64    Y
  model.layers.19.post_attention_layernorm.weight                        0.00383   0.000004    0.0054  1.00000  0.00000   80.83    Y
  model.layers.19.self_attn.k_norm.weight                                0.00066   0.000010    0.0008  1.00000  0.00001   89.56    Y
  model.layers.19.self_attn.k_proj.weight                                0.00372   0.000056    0.2909  0.99998  0.00248   43.45    Y
  model.layers.19.self_attn.o_proj.weight                                0.00391   0.000060    0.4523  0.99997  0.00262   42.61    Y
  model.layers.19.self_attn.q_norm.weight                                0.00000   0.000000    0.0000  1.00000  0.00000  265.95    N
  model.layers.19.self_attn.q_proj.weight                                0.00365   0.000054    0.4036  0.99999  0.00199   45.74    Y
  model.layers.19.self_attn.v_proj.weight                                0.00397   0.000061    0.3253  0.99998  0.00251   43.08    Y
  model.layers.2.input_layernorm.weight                                  0.00003   0.000000    0.0000  1.00000  0.00000  105.97    Y
  model.layers.2.mlp.down_proj.weight                                    0.00309   0.000034    0.4665  0.99999  0.00140   47.42    Y
  model.layers.2.mlp.gate_proj.weight                                    0.00284   0.000022    0.3744  1.00000  0.00061   52.95    Y
  model.layers.2.mlp.up_proj.weight                                      0.00284   0.000035    0.4758  0.99999  0.00148   46.96    Y
  model.layers.2.post_attention_layernorm.weight                         0.00164   0.000002    0.0024  1.00000  0.00000   79.71    Y
  model.layers.2.self_attn.k_norm.weight                                 0.00000   0.000000    0.0000  1.00000  0.00000  273.01    N
  model.layers.2.self_attn.k_proj.weight                                 0.00269   0.000032    0.1865  0.99999  0.00123   48.39    Y
  model.layers.2.self_attn.o_proj.weight                                 0.00289   0.000032    0.2681  0.99999  0.00131   47.75    Y
  model.layers.2.self_attn.q_norm.weight                                 0.00000   0.000000    0.0000  1.00000  0.00000  265.61    N
  model.layers.2.self_attn.q_proj.weight                                 0.00246   0.000030    0.2516  0.99999  0.00114   49.02    Y
  model.layers.2.self_attn.v_proj.weight                                 0.00323   0.000029    0.1817  0.99999  0.00108   48.89    Y
  model.layers.20.input_layernorm.weight                                 0.00168   0.000001    0.0017  1.00000  0.00000  105.09    Y
  model.layers.20.mlp.down_proj.weight                                   0.00505   0.000047    0.6766  0.99999  0.00161   45.81    Y
  model.layers.20.mlp.gate_proj.weight                                   0.00436   0.000050    0.6897  0.99998  0.00185   45.02    Y
  model.layers.20.mlp.up_proj.weight                                     0.00443   0.000045    0.6601  0.99999  0.00151   46.22    Y
  model.layers.20.post_attention_layernorm.weight                        0.00268   0.000002    0.0033  1.00000  0.00000   86.18    Y
  model.layers.20.self_attn.k_norm.weight                                0.00015   0.000002    0.0002  1.00000  0.00000  103.31    Y
  model.layers.20.self_attn.k_proj.weight                                0.00351   0.000054    0.2880  0.99998  0.00231   43.95    Y
  model.layers.20.self_attn.o_proj.weight                                0.00394   0.000060    0.4510  0.99997  0.00250   43.01    Y
  model.layers.20.self_attn.q_norm.weight                                0.00000   0.000000    0.0000  1.00000  0.00000  266.02    N
  model.layers.20.self_attn.q_proj.weight                                0.00348   0.000052    0.3948  0.99999  0.00189   46.05    Y
  model.layers.20.self_attn.v_proj.weight                                0.00371   0.000060    0.3226  0.99998  0.00235   43.49    Y
  model.layers.21.input_layernorm.weight                                 0.00166   0.000001    0.0017  1.00000  0.00000  107.25    Y
  model.layers.21.mlp.down_proj.weight                                   0.00422   0.000045    0.6490  0.99999  0.00151   46.33    Y
  model.layers.21.mlp.gate_proj.weight                                   0.00478   0.000045    0.6403  0.99999  0.00163   45.88    Y
  model.layers.21.mlp.up_proj.weight                                     0.00461   0.000041    0.6157  0.99999  0.00136   46.96    Y
  model.layers.21.post_attention_layernorm.weight                        0.00370   0.000004    0.0047  1.00000  0.00000   84.07    Y
  model.layers.21.self_attn.k_norm.weight                                0.00037   0.000012    0.0006  1.00000  0.00001   90.93    Y
  model.layers.21.self_attn.k_proj.weight                                0.00367   0.000055    0.2872  0.99998  0.00234   44.16    Y
  model.layers.21.self_attn.o_proj.weight                                0.00447   0.000059    0.4621  0.99998  0.00236   43.23    Y
  model.layers.21.self_attn.q_norm.weight                                0.00053   0.000011    0.0008  1.00000  0.00001   87.66    Y
  model.layers.21.self_attn.q_proj.weight                                0.00397   0.000048    0.3821  0.99999  0.00172   46.36    Y
  model.layers.21.self_attn.v_proj.weight                                0.00406   0.000061    0.3313  0.99998  0.00226   43.94    Y
  model.layers.22.input_layernorm.weight                                 0.00000   0.000000    0.0000  1.00000  0.00000  293.24    N
  model.layers.22.mlp.down_proj.weight                                   0.00450   0.000042    0.6191  0.99999  0.00141   46.89    Y
  model.layers.22.mlp.gate_proj.weight                                   0.00479   0.000042    0.5993  0.99999  0.00148   46.60    Y
  model.layers.22.mlp.up_proj.weight                                     0.00500   0.000039    0.5853  0.99999  0.00127   47.49    Y
  model.layers.22.post_attention_layernorm.weight                        0.00310   0.000004    0.0049  1.00000  0.00000   84.63    Y
  model.layers.22.self_attn.k_norm.weight                                0.00101   0.000026    0.0015  1.00000  0.00002   82.39    Y
  model.layers.22.self_attn.k_proj.weight                                0.00382   0.000058    0.3024  0.99998  0.00243   43.63    Y
  model.layers.22.self_attn.o_proj.weight                                0.00444   0.000066    0.4993  0.99997  0.00263   42.46    Y
  model.layers.22.self_attn.q_norm.weight                                0.00073   0.000012    0.0010  1.00000  0.00001   85.04    Y
  model.layers.22.self_attn.q_proj.weight                                0.00376   0.000055    0.4232  0.99999  0.00202   45.25    Y
  model.layers.22.self_attn.v_proj.weight                                0.00386   0.000064    0.3426  0.99998  0.00237   43.52    Y
  model.layers.23.input_layernorm.weight                                 0.00183   0.000001    0.0018  1.00000  0.00000  109.30    Y
  model.layers.23.mlp.down_proj.weight                                   0.00555   0.000042    0.6098  0.99999  0.00138   47.11    Y
  model.layers.23.mlp.gate_proj.weight                                   0.00572   0.000040    0.5813  0.99999  0.00140   46.98    Y
  model.layers.23.mlp.up_proj.weight                                     0.00586   0.000038    0.5674  0.99999  0.00123   47.79    Y
  model.layers.23.post_attention_layernorm.weight                        0.00265   0.000003    0.0038  1.00000  0.00000   87.79    Y
  model.layers.23.self_attn.k_norm.weight                                0.00066   0.000015    0.0009  1.00000  0.00001   89.57    Y
  model.layers.23.self_attn.k_proj.weight                                0.00394   0.000060    0.3171  0.99998  0.00254   43.10    Y
  model.layers.23.self_attn.o_proj.weight                                0.00522   0.000063    0.4985  0.99997  0.00247   42.55    Y
  model.layers.23.self_attn.q_norm.weight                                0.00000   0.000000    0.0000  1.00000  0.00000  264.66    N
  model.layers.23.self_attn.q_proj.weight                                0.00385   0.000056    0.4306  0.99998  0.00214   44.50    Y
  model.layers.23.self_attn.v_proj.weight                                0.00435   0.000066    0.3622  0.99997  0.00248   42.88    Y
  model.layers.24.input_layernorm.weight                                 0.00000   0.000000    0.0000  1.00000  0.00000  297.45    N
  model.layers.24.mlp.down_proj.weight                                   0.00490   0.000043    0.6213  0.99999  0.00139   46.98    Y
  model.layers.24.mlp.gate_proj.weight                                   0.00485   0.000041    0.5920  0.99999  0.00144   46.73    Y
  model.layers.24.mlp.up_proj.weight                                     0.00537   0.000039    0.5803  0.99999  0.00125   47.64    Y
  model.layers.24.post_attention_layernorm.weight                        0.00253   0.000003    0.0038  1.00000  0.00000   87.93    Y
  model.layers.24.self_attn.k_norm.weight                                0.00043   0.000005    0.0004  1.00000  0.00000   96.20    Y
  model.layers.24.self_attn.k_proj.weight                                0.00393   0.000061    0.3163  0.99998  0.00260   43.11    Y
  model.layers.24.self_attn.o_proj.weight                                0.00444   0.000064    0.5073  0.99997  0.00248   42.58    Y
  model.layers.24.self_attn.q_norm.weight                                0.00000   0.000000    0.0000  1.00000  0.00000  264.61    N
  model.layers.24.self_attn.q_proj.weight                                0.00375   0.000054    0.4165  0.99998  0.00200   45.18    Y
  model.layers.24.self_attn.v_proj.weight                                0.00462   0.000071    0.3789  0.99997  0.00262   42.54    Y
  model.layers.25.input_layernorm.weight                                 0.00000   0.000000    0.0000  1.00000  0.00000  299.47    N
  model.layers.25.mlp.down_proj.weight                                   0.00560   0.000043    0.6433  0.99999  0.00143   46.60    Y
  model.layers.25.mlp.gate_proj.weight                                   0.00534   0.000044    0.6358  0.99999  0.00157   45.98    Y
  model.layers.25.mlp.up_proj.weight                                     0.00577   0.000040    0.6062  0.99999  0.00129   47.27    Y
  model.layers.25.post_attention_layernorm.weight                        0.00295   0.000004    0.0044  1.00000  0.00000   87.35    Y
  model.layers.25.self_attn.k_norm.weight                                0.00090   0.000018    0.0013  1.00000  0.00001   86.33    Y
  model.layers.25.self_attn.k_proj.weight                                0.00451   0.000069    0.3509  0.99997  0.00306   41.90    Y
  model.layers.25.self_attn.o_proj.weight                                0.00551   0.000070    0.5722  0.99997  0.00257   41.95    Y
  model.layers.25.self_attn.q_norm.weight                                0.00082   0.000007    0.0008  1.00000  0.00000   86.70    Y
  model.layers.25.self_attn.q_proj.weight                                0.00470   0.000063    0.4760  0.99998  0.00241   43.76    Y
  model.layers.25.self_attn.v_proj.weight                                0.00507   0.000072    0.4023  0.99997  0.00242   42.90    Y
  model.layers.26.input_layernorm.weight                                 0.00000   0.000000    0.0000  1.00000  0.00000  300.85    N
  model.layers.26.mlp.down_proj.weight                                   0.00528   0.000045    0.6640  0.99999  0.00150   46.17    Y
  model.layers.26.mlp.gate_proj.weight                                   0.00610   0.000047    0.6667  0.99999  0.00168   45.49    Y
  model.layers.26.mlp.up_proj.weight                                     0.00630   0.000042    0.6272  0.99999  0.00136   46.92    Y
  model.layers.26.post_attention_layernorm.weight                        0.00246   0.000003    0.0040  1.00000  0.00000   89.19    Y
  model.layers.26.self_attn.k_norm.weight                                0.00059   0.000009    0.0008  1.00000  0.00001   88.71    Y
  model.layers.26.self_attn.k_proj.weight                                0.00468   0.000078    0.3770  0.99996  0.00355   41.02    Y
  model.layers.26.self_attn.o_proj.weight                                0.00565   0.000078    0.5811  0.99997  0.00294   41.78    Y
  model.layers.26.self_attn.q_norm.weight                                0.00061   0.000012    0.0008  1.00000  0.00001   85.86    Y
  model.layers.26.self_attn.q_proj.weight                                0.00470   0.000069    0.4942  0.99998  0.00247   44.14    Y
  model.layers.26.self_attn.v_proj.weight                                0.00506   0.000076    0.4092  0.99997  0.00259   42.51    Y
  model.layers.27.input_layernorm.weight                                 0.00000   0.000000    0.0000  1.00000  0.00000  298.18    N
  model.layers.27.mlp.down_proj.weight                                   0.00674   0.000048    0.6742  0.99999  0.00173   45.40    Y
  model.layers.27.mlp.gate_proj.weight                                   0.00656   0.000044    0.6610  0.99999  0.00150   46.06    Y
  model.layers.27.mlp.up_proj.weight                                     0.00679   0.000041    0.6234  0.99999  0.00137   46.86    Y
  model.layers.27.post_attention_layernorm.weight                        0.00257   0.000002    0.0035  1.00000  0.00000   95.48    Y
  model.layers.27.self_attn.k_norm.weight                                0.00079   0.000015    0.0010  1.00000  0.00001   87.28    Y
  model.layers.27.self_attn.k_proj.weight                                0.00404   0.000056    0.3102  0.99998  0.00212   43.99    Y
  model.layers.27.self_attn.o_proj.weight                                0.00525   0.000063    0.4910  0.99998  0.00229   43.32    Y
  model.layers.27.self_attn.q_norm.weight                                0.00000   0.000000    0.0000  1.00000  0.00000  265.52    N
  model.layers.27.self_attn.q_proj.weight                                0.00464   0.000056    0.4464  0.99998  0.00207   44.24    Y
  model.layers.27.self_attn.v_proj.weight                                0.00400   0.000062    0.3477  0.99998  0.00210   43.89    Y
  model.layers.3.input_layernorm.weight                                  0.00000   0.000000    0.0000  1.00000  0.00000  263.62    N
  model.layers.3.mlp.down_proj.weight                                    0.00308   0.000035    0.4759  0.99999  0.00144   47.21    Y
  model.layers.3.mlp.gate_proj.weight                                    0.00297   0.000023    0.3753  1.00000  0.00062   52.82    Y
  model.layers.3.mlp.up_proj.weight                                      0.00294   0.000034    0.4731  0.99999  0.00143   47.21    Y
  model.layers.3.post_attention_layernorm.weight                         0.00071   0.000001    0.0010  1.00000  0.00000   88.33    Y
  model.layers.3.self_attn.k_norm.weight                                 0.00082   0.000031    0.0016  1.00000  0.00002   85.17    Y
  model.layers.3.self_attn.k_proj.weight                                 0.00264   0.000031    0.1822  0.99999  0.00120   48.77    Y
  model.layers.3.self_attn.o_proj.weight                                 0.00283   0.000030    0.2614  0.99999  0.00114   48.47    Y
  model.layers.3.self_attn.q_norm.weight                                 0.00144   0.000011    0.0014  1.00000  0.00001   83.93    Y
  model.layers.3.self_attn.q_proj.weight                                 0.00288   0.000030    0.2514  0.99999  0.00111   49.14    Y
  model.layers.3.self_attn.v_proj.weight                                 0.00283   0.000028    0.1784  0.99999  0.00096   49.56    Y
  model.layers.4.input_layernorm.weight                                  0.00000   0.000000    0.0000  1.00000  0.00000  263.72    N
  model.layers.4.mlp.down_proj.weight                                    0.00302   0.000036    0.4926  0.99999  0.00148   46.92    Y
  model.layers.4.mlp.gate_proj.weight                                    0.00314   0.000024    0.3943  1.00000  0.00068   52.13    Y
  model.layers.4.mlp.up_proj.weight                                      0.00315   0.000035    0.4877  0.99999  0.00145   47.03    Y
  model.layers.4.post_attention_layernorm.weight                         0.00110   0.000001    0.0013  1.00000  0.00000   86.80    Y
  model.layers.4.self_attn.k_norm.weight                                 0.00041   0.000004    0.0004  1.00000  0.00000  102.18    Y
  model.layers.4.self_attn.k_proj.weight                                 0.00231   0.000034    0.1930  0.99999  0.00135   47.99    Y
  model.layers.4.self_attn.o_proj.weight                                 0.00297   0.000033    0.2751  0.99999  0.00128   47.83    Y
  model.layers.4.self_attn.q_norm.weight                                 0.00000   0.000000    0.0000  1.00000  0.00000  262.77    N
  model.layers.4.self_attn.q_proj.weight                                 0.00261   0.000032    0.2630  0.99999  0.00121   48.62    Y
  model.layers.4.self_attn.v_proj.weight                                 0.00288   0.000030    0.1861  0.99999  0.00106   49.05    Y
  model.layers.5.input_layernorm.weight                                  0.00000   0.000000    0.0000  1.00000  0.00000  265.21    N
  model.layers.5.mlp.down_proj.weight                                    0.00332   0.000034    0.4893  0.99999  0.00131   47.54    Y
  model.layers.5.mlp.gate_proj.weight                                    0.00289   0.000028    0.4340  1.00000  0.00089   50.28    Y
  model.layers.5.mlp.up_proj.weight                                      0.00301   0.000033    0.4778  0.99999  0.00126   47.81    Y
  model.layers.5.post_attention_layernorm.weight                         0.00134   0.000001    0.0015  1.00000  0.00000   83.84    Y
  model.layers.5.self_attn.k_norm.weight                                 0.00109   0.000009    0.0011  1.00000  0.00000   93.01    Y
  model.layers.5.self_attn.k_proj.weight                                 0.00241   0.000033    0.1896  0.99999  0.00126   48.26    Y
  model.layers.5.self_attn.o_proj.weight                                 0.00307   0.000035    0.2836  0.99999  0.00136   47.62    Y
  model.layers.5.self_attn.q_norm.weight                                 0.00098   0.000008    0.0010  1.00000  0.00000   86.69    Y
  model.layers.5.self_attn.q_proj.weight                                 0.00267   0.000032    0.2687  0.99999  0.00117   48.71    Y
  model.layers.5.self_attn.v_proj.weight                                 0.00282   0.000033    0.1941  0.99999  0.00120   48.40    Y
  model.layers.6.input_layernorm.weight                                  0.00000   0.000000    0.0000  1.00000  0.00000  264.54    N
  model.layers.6.mlp.down_proj.weight                                    0.00294   0.000036    0.5101  0.99999  0.00142   47.07    Y
  model.layers.6.mlp.gate_proj.weight                                    0.00287   0.000029    0.4471  0.99999  0.00094   49.90    Y
  model.layers.6.mlp.up_proj.weight                                      0.00273   0.000034    0.4868  0.99999  0.00129   47.69    Y
  model.layers.6.post_attention_layernorm.weight                         0.00194   0.000002    0.0025  1.00000  0.00000   80.77    Y
  model.layers.6.self_attn.k_norm.weight                                 0.00061   0.000012    0.0008  1.00000  0.00000   93.05    Y
  model.layers.6.self_attn.k_proj.weight                                 0.00246   0.000039    0.2090  0.99999  0.00160   47.22    Y
  model.layers.6.self_attn.o_proj.weight                                 0.00284   0.000037    0.2948  0.99999  0.00149   46.95    Y
  model.layers.6.self_attn.q_norm.weight                                 0.00023   0.000002    0.0002  1.00000  0.00000   99.95    Y
  model.layers.6.self_attn.q_proj.weight                                 0.00280   0.000035    0.2785  0.99999  0.00135   48.19    Y
  model.layers.6.self_attn.v_proj.weight                                 0.00285   0.000032    0.1911  0.99999  0.00114   48.73    Y
  model.layers.7.input_layernorm.weight                                  0.00000   0.000000    0.0000  1.00000  0.00000  268.35    N
  model.layers.7.mlp.down_proj.weight                                    0.00320   0.000038    0.5276  0.99999  0.00153   46.64    Y
  model.layers.7.mlp.gate_proj.weight                                    0.00307   0.000029    0.4485  1.00000  0.00091   50.14    Y
  model.layers.7.mlp.up_proj.weight                                      0.00301   0.000035    0.5010  0.99999  0.00136   47.32    Y
  model.layers.7.post_attention_layernorm.weight                         0.00055   0.000000    0.0006  1.00000  0.00000   94.82    Y
  model.layers.7.self_attn.k_norm.weight                                 0.00106   0.000019    0.0012  1.00000  0.00001   88.06    Y
  model.layers.7.self_attn.k_proj.weight                                 0.00264   0.000036    0.2023  0.99999  0.00148   47.17    Y
  model.layers.7.self_attn.o_proj.weight                                 0.00292   0.000037    0.2979  0.99999  0.00146   46.90    Y
  model.layers.7.self_attn.q_norm.weight                                 0.00000   0.000000    0.0000  1.00000  0.00000  266.28    N
  model.layers.7.self_attn.q_proj.weight                                 0.00261   0.000035    0.2821  0.99999  0.00134   47.80    Y
  model.layers.7.self_attn.v_proj.weight                                 0.00237   0.000033    0.1980  0.99999  0.00123   48.16    Y
  model.layers.8.input_layernorm.weight                                  0.00000   0.000000    0.0000  1.00000  0.00000  268.99    N
  model.layers.8.mlp.down_proj.weight                                    0.00306   0.000039    0.5414  0.99999  0.00153   46.65    Y
  model.layers.8.mlp.gate_proj.weight                                    0.00285   0.000032    0.4766  0.99999  0.00105   49.21    Y
  model.layers.8.mlp.up_proj.weight                                      0.00293   0.000037    0.5160  0.99999  0.00140   47.17    Y
  model.layers.8.post_attention_layernorm.weight                         0.00121   0.000002    0.0019  1.00000  0.00000   84.34    Y
  model.layers.8.self_attn.k_norm.weight                                 0.00036   0.000013    0.0006  1.00000  0.00001   90.26    Y
  model.layers.8.self_attn.k_proj.weight                                 0.00277   0.000036    0.2029  0.99999  0.00148   47.34    Y
  model.layers.8.self_attn.o_proj.weight                                 0.00294   0.000038    0.3006  0.99999  0.00153   46.67    Y
  model.layers.8.self_attn.q_norm.weight                                 0.00035   0.000003    0.0004  1.00000  0.00000   93.93    Y
  model.layers.8.self_attn.q_proj.weight                                 0.00281   0.000034    0.2798  0.99999  0.00133   48.09    Y
  model.layers.8.self_attn.v_proj.weight                                 0.00269   0.000033    0.1968  0.99999  0.00117   48.32    Y
  model.layers.9.input_layernorm.weight                                  0.00000   0.000000    0.0000  1.00000  0.00000  271.40    N
  model.layers.9.mlp.down_proj.weight                                    0.00362   0.000041    0.5626  0.99999  0.00159   46.37    Y
  model.layers.9.mlp.gate_proj.weight                                    0.00291   0.000033    0.4943  0.99999  0.00110   48.84    Y
  model.layers.9.mlp.up_proj.weight                                      0.00317   0.000037    0.5275  0.99999  0.00135   47.33    Y
  model.layers.9.post_attention_layernorm.weight                         0.00178   0.000002    0.0025  1.00000  0.00000   82.01    Y
  model.layers.9.self_attn.k_norm.weight                                 0.00100   0.000015    0.0011  1.00000  0.00001   88.79    Y
  model.layers.9.self_attn.k_proj.weight                                 0.00293   0.000038    0.2147  0.99999  0.00157   46.80    Y
  model.layers.9.self_attn.o_proj.weight                                 0.00284   0.000042    0.3233  0.99999  0.00169   46.32    Y
  model.layers.9.self_attn.q_norm.weight                                 0.00060   0.000005    0.0006  1.00000  0.00000   91.63    Y
  model.layers.9.self_attn.q_proj.weight                                 0.00291   0.000037    0.2997  0.99999  0.00142   47.61    Y
  model.layers.9.self_attn.v_proj.weight                                 0.00263   0.000038    0.2159  0.99999  0.00144   47.33    Y
  model.norm.weight                                                      0.00000   0.000000    0.0000  1.00000  0.00000  280.25    N

  Changed  : 277 / 311
  Unchanged: 34 / 311
  Unchanged (first 10): ['model.layers.0.input_layernorm.weight', 'model.layers.0.self_attn.k_norm.weight', 'model.layers.1.input_layernorm.weight', 'model.layers.1.self_attn.k_norm.weight', 'model.layers.1.self_attn.q_norm.weight', 'model.layers.10.input_layernorm.weight', 'model.layers.10.self_attn.q_norm.weight', 'model.layers.11.self_attn.q_norm.weight', 'model.layers.12.self_attn.q_norm.weight', 'model.layers.13.self_attn.q_norm.weight']

  Note: Unchanged tensors are primarily normalization layers (input_layernorm, q_norm, k_norm, model.norm).
        This demonstrates that the SFT/KD process modified the primary semantic projection weights
        (attention and MLP projections) while preserving basic layer scaling characteristics.

[10] Cosine similarity distribution histogram
  Range                   Count  Histogram
  [    -inf,   0.90000)       0
  [ 0.90000,   0.99000)       0
  [ 0.99000,   0.99900)       0
  [ 0.99900,   0.99990)       0
  [ 0.99990,   0.99999)     127  ############################
  [ 0.99999,   1.00001)     184  ########################################
  [ 1.00001,   1.00100)       0
  [ 1.00100,       inf)       0

[11] Attention geometry per transformer block
  Query heads: 16  |  KV heads: 8  |  head_dim: 128  |  GQA: 2:1

   Blk  Q shape              K shape              V shape              O shape                  Q L2     K L2     V L2     O L2
  ----------------------------------------------------------------------------------------------------------------------------------
     0  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           72.901   52.450   49.252   67.170
     1  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           71.518   49.452   50.624   67.294
     2  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           71.066   49.005   50.551   65.435
     3  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           72.017   50.006   53.659   69.315
     4  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           70.937   48.394   52.777   67.797
     5  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           73.221   49.084   51.053   68.198
     6  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           71.503   47.997   52.187   65.590
     7  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           69.230   46.204   50.629   65.907
     8  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           70.971   47.226   51.273   64.765
     9  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           71.985   46.960   50.199   66.940
    10  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           70.798   48.128   49.962   65.289
    11  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           75.117   48.444   47.471   61.987
    12  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           71.912   47.209   48.870   62.022
    13  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           72.686   44.996   48.618   60.200
    14  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           70.351   44.671   47.417   60.245
    15  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           72.109   42.654   47.263   61.616
    16  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           77.028   44.274   47.469   61.723
    17  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           73.600   44.185   49.308   62.506
    18  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           75.780   43.076   46.493   58.817
    19  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           78.144   43.280   46.370   61.106
    20  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           79.257   45.357   48.235   63.751
    21  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           79.468   46.374   52.175   67.043
    22  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           77.429   45.910   51.373   66.308
    23  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           72.309   45.286   50.443   66.829
    24  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           75.594   45.236   50.747   68.236
    25  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           73.422   43.686   56.192   71.640
    26  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           79.566   42.406   54.631   71.338
    27  [2048, 2048]         [1024, 2048]         [1024, 2048]         [2048, 2048]           72.699   49.078   54.406   71.960

[12] MLP feed-forward geometry per transformer block
  intermediate_size: 6144  |  activation: silu

   Blk  Gate shape             Up shape               Down shape              Gate L2    Up L2   Down L2   GateSp     UpSp     DnSp
  ---------------------------------------------------------------------------------------------------------------------------------------
     0  [6144, 2048]           [6144, 2048]           [2048, 6144]            138.356  112.786   119.234  0.00002  0.00002  0.00002
     1  [6144, 2048]           [6144, 2048]           [2048, 6144]            160.751  102.435   104.364  0.00002  0.00004  0.00003
     2  [6144, 2048]           [6144, 2048]           [2048, 6144]            166.168  105.972   109.577  0.00002  0.00003  0.00003
     3  [6144, 2048]           [6144, 2048]           [2048, 6144]            164.227  108.473   109.132  0.00002  0.00003  0.00003
     4  [6144, 2048]           [6144, 2048]           [2048, 6144]            159.325  109.481   109.280  0.00002  0.00003  0.00003
     5  [6144, 2048]           [6144, 2048]           [2048, 6144]            141.749  117.417   116.503  0.00002  0.00002  0.00003
     6  [6144, 2048]           [6144, 2048]           [2048, 6144]            139.719  117.968   115.044  0.00002  0.00002  0.00003
     7  [6144, 2048]           [6144, 2048]           [2048, 6144]            144.020  116.378   113.238  0.00002  0.00003  0.00003
     8  [6144, 2048]           [6144, 2048]           [2048, 6144]            137.524  117.760   116.430  0.00002  0.00002  0.00002
     9  [6144, 2048]           [6144, 2048]           [2048, 6144]            136.793  122.606   117.158  0.00002  0.00002  0.00003
    10  [6144, 2048]           [6144, 2048]           [2048, 6144]            135.160  123.157   116.162  0.00002  0.00002  0.00003
    11  [6144, 2048]           [6144, 2048]           [2048, 6144]            128.540  125.940   120.142  0.00002  0.00003  0.00002
    12  [6144, 2048]           [6144, 2048]           [2048, 6144]            125.090  125.435   120.075  0.00002  0.00002  0.00003
    13  [6144, 2048]           [6144, 2048]           [2048, 6144]            120.812  122.147   117.098  0.00003  0.00003  0.00003
    14  [6144, 2048]           [6144, 2048]           [2048, 6144]            115.589  118.312   116.246  0.00003  0.00002  0.00003
    15  [6144, 2048]           [6144, 2048]           [2048, 6144]            116.356  120.975   116.761  0.00003  0.00003  0.00003
    16  [6144, 2048]           [6144, 2048]           [2048, 6144]            115.659  124.277   117.831  0.00003  0.00002  0.00003
    17  [6144, 2048]           [6144, 2048]           [2048, 6144]            115.404  129.050   121.534  0.00003  0.00002  0.00003
    18  [6144, 2048]           [6144, 2048]           [2048, 6144]            116.670  129.370   124.809  0.00003  0.00003  0.00002
    19  [6144, 2048]           [6144, 2048]           [2048, 6144]            118.280  131.654   128.726  0.00003  0.00002  0.00002
    20  [6144, 2048]           [6144, 2048]           [2048, 6144]            122.886  134.977   132.033  0.00003  0.00002  0.00002
    21  [6144, 2048]           [6144, 2048]           [2048, 6144]            125.949  137.228   134.445  0.00002  0.00002  0.00002
    22  [6144, 2048]           [6144, 2048]           [2048, 6144]            128.172  138.566   136.816  0.00002  0.00002  0.00002
    23  [6144, 2048]           [6144, 2048]           [2048, 6144]            129.759  139.033   138.182  0.00002  0.00002  0.00002
    24  [6144, 2048]           [6144, 2048]           [2048, 6144]            128.384  139.729   138.658  0.00002  0.00002  0.00002
    25  [6144, 2048]           [6144, 2048]           [2048, 6144]            126.516  140.019   137.531  0.00002  0.00002  0.00002
    26  [6144, 2048]           [6144, 2048]           [2048, 6144]            125.452  139.144   135.120  0.00002  0.00002  0.00002
    27  [6144, 2048]           [6144, 2048]           [2048, 6144]            132.785  137.233   125.503  0.00002  0.00002  0.00002

[13] Weight health diagnostics

  Sparsity > 10%
    none

  |Kurtosis Delta| > 5.0
    none

  Outlier ratio > 1%
    model.layers.0.self_attn.q_norm.weight                                  outlier_ratio=0.01562
    model.layers.0.self_attn.k_norm.weight                                  outlier_ratio=0.02344
    model.layers.0.input_layernorm.weight                                   outlier_ratio=0.01416
    model.layers.2.self_attn.q_norm.weight                                  outlier_ratio=0.01562
    model.layers.2.input_layernorm.weight                                   outlier_ratio=0.02100
    model.layers.3.self_attn.q_norm.weight                                  outlier_ratio=0.01562
    model.layers.3.input_layernorm.weight                                   outlier_ratio=0.02295
    model.layers.4.self_attn.k_norm.weight                                  outlier_ratio=0.01562
    model.layers.4.input_layernorm.weight                                   outlier_ratio=0.01855
    model.layers.5.self_attn.q_norm.weight                                  outlier_ratio=0.03125
    model.layers.5.self_attn.k_norm.weight                                  outlier_ratio=0.02344
    model.layers.5.input_layernorm.weight                                   outlier_ratio=0.02100
    model.layers.6.self_attn.q_norm.weight                                  outlier_ratio=0.01562
    model.layers.6.self_attn.k_norm.weight                                  outlier_ratio=0.01562
    model.layers.6.input_layernorm.weight                                   outlier_ratio=0.02539
    model.layers.7.self_attn.q_norm.weight                                  outlier_ratio=0.01562
    model.layers.7.input_layernorm.weight                                   outlier_ratio=0.01074
    model.layers.8.input_layernorm.weight                                   outlier_ratio=0.02002
    model.layers.9.self_attn.q_norm.weight                                  outlier_ratio=0.02344
    model.layers.9.self_attn.k_norm.weight                                  outlier_ratio=0.03125
    model.layers.10.self_attn.q_norm.weight                                 outlier_ratio=0.03125
    model.layers.10.self_attn.k_norm.weight                                 outlier_ratio=0.01562
    model.layers.11.self_attn.k_norm.weight                                 outlier_ratio=0.01562
    model.layers.11.input_layernorm.weight                                  outlier_ratio=0.01172
    model.layers.12.input_layernorm.weight                                  outlier_ratio=0.01025
    model.layers.13.self_attn.q_norm.weight                                 outlier_ratio=0.02344
    model.layers.13.input_layernorm.weight                                  outlier_ratio=0.01123
    model.layers.14.self_attn.q_norm.weight                                 outlier_ratio=0.03125
    model.layers.14.input_layernorm.weight                                  outlier_ratio=0.01074
    model.layers.15.self_attn.k_norm.weight                                 outlier_ratio=0.02344
    model.layers.16.input_layernorm.weight                                  outlier_ratio=0.01025
    model.layers.16.post_attention_layernorm.weight                         outlier_ratio=0.01221
    model.layers.17.self_attn.q_norm.weight                                 outlier_ratio=0.01562
    model.layers.17.post_attention_layernorm.weight                         outlier_ratio=0.01172
    model.layers.18.input_layernorm.weight                                  outlier_ratio=0.01611
    model.layers.18.post_attention_layernorm.weight                         outlier_ratio=0.01270
    model.layers.19.self_attn.q_norm.weight                                 outlier_ratio=0.01562
    model.layers.19.self_attn.k_norm.weight                                 outlier_ratio=0.01562
    model.layers.19.input_layernorm.weight                                  outlier_ratio=0.01416
    model.layers.19.post_attention_layernorm.weight                         outlier_ratio=0.01416
    model.layers.20.self_attn.q_norm.weight                                 outlier_ratio=0.01562
    model.layers.20.self_attn.k_norm.weight                                 outlier_ratio=0.01562
    model.layers.20.input_layernorm.weight                                  outlier_ratio=0.02295
    model.layers.20.post_attention_layernorm.weight                         outlier_ratio=0.01221
    model.layers.21.input_layernorm.weight                                  outlier_ratio=0.02295
    model.layers.22.self_attn.k_norm.weight                                 outlier_ratio=0.02344
    model.layers.22.input_layernorm.weight                                  outlier_ratio=0.02588
    model.layers.22.post_attention_layernorm.weight                         outlier_ratio=0.01025
    model.layers.23.input_layernorm.weight                                  outlier_ratio=0.02197
    model.layers.23.post_attention_layernorm.weight                         outlier_ratio=0.01123
    model.layers.24.self_attn.k_norm.weight                                 outlier_ratio=0.01562
    model.layers.24.input_layernorm.weight                                  outlier_ratio=0.02197
    model.layers.24.post_attention_layernorm.weight                         outlier_ratio=0.01367
    model.layers.25.input_layernorm.weight                                  outlier_ratio=0.03174
    model.layers.25.post_attention_layernorm.weight                         outlier_ratio=0.01514
    model.layers.26.input_layernorm.weight                                  outlier_ratio=0.04883
    model.layers.26.post_attention_layernorm.weight                         outlier_ratio=0.01172
    model.layers.27.self_attn.q_norm.weight                                 outlier_ratio=0.02344
    model.layers.27.input_layernorm.weight                                  outlier_ratio=0.02148
    model.norm.weight                                                       outlier_ratio=0.01416

  Dead rows (L2 < 1e-6)
    none

  Low cosine sim vs base (<0.95)
    none

  Low SNR vs base (< 20 dB)
    none

  Note on kurtosis delta: Kurtosis values are reported as the difference (delta) compared to the base model.
  A high kurtosis delta on tiny vectors (like norm/q-k-norm vectors of size 128) is statistically expected
  due to small sample sizes and does not indicate a model health or representation collapse issue.

[14] Executive summary
  shared tensors                   : 311
  tensors changed vs base          : 277 / 311
  cosine similarity                : mean = 0.999991 | median = 0.999992 | p10 = 0.999980 | p90 = 1.000000
  relative error                   : mean = 0.001093 | median = 0.001293 | p10 = 0.000000 | p90 = 0.002338
  SNR dB                           : mean = 81.86 | median = 47.79 | p10 = 43.89 | p90 = 262.77
  high-sparsity layers (>10%)      : 0
  heavy-tail layers (|kurt_d|>5.0) : 0
  dead-row layers                  : 0
  low-cos layers (<0.95)           : 0
  low-SNR layers (<20 dB)          : 0
  distillation alpha               : 0.3

  checkpoint size on disk          : 3.218 GiB
  base weights in memory           : 3.784 GiB
  distilled weights in memory      : 3.784 GiB

========================================================================================
  END OF REPORT
========================================================================================
