Model: ModelCloud/Qwen1.5-1.8B-Chat-GPTQ-4bits-dynamic-cfg-with-lm_head-symTrue Source: Original Platform
630 B
630 B
| 1 | layer | module | loss | damp | time |
|---|---|---|---|---|---|
| 2 | 0 | self_attn.k_proj | 0.38340 | 0.01000 | 0.985 |
| 3 | 0 | self_attn.v_proj | 0.00899 | 0.01000 | 0.332 |
| 4 | 0 | self_attn.q_proj | 0.52614 | 0.01000 | 0.327 |
| 5 | 0 | self_attn.o_proj | 0.00033 | 0.01000 | 0.326 |
| 6 | 0 | mlp.up_proj | 1.90171 | 0.01000 | 0.348 |
| 7 | 0 | mlp.gate_proj | 1.81718 | 0.01000 | 0.366 |
| 8 | 0 | mlp.down_proj | 0.04200 | 0.01000 | 0.950 |
| 9 | 1 | self_attn.k_proj | 0.00821 | 0.01000 | 0.338 |
| 10 | 1 | self_attn.v_proj | 0.00108 | 0.01000 | 0.329 |
| 11 | 1 | self_attn.q_proj | 0.00811 | 0.01000 | 0.329 |
| 12 | 1 | self_attn.o_proj | 0.00003 | 0.01000 | 0.347 |
| 13 | 1 | mlp.up_proj | 0.00614 | 0.01000 | 0.370 |
| 14 | 1 | mlp.gate_proj | 0.00698 | 0.01000 | 0.383 |
| 15 | 1 | mlp.down_proj | 0.00040 | 0.01000 | 0.944 |
| 16 | 24 | lm_head | 23.30706 | 0.01000 | 4.334 |