frankenstallm/source/eval/outputs/ppl_3b_val.txt

/usr/local/lib/python3.12/dist-packages/torch/library.py:356: UserWarning: Warning only once for all operators,  other operators may also be overridden.
  Overriding a previously registered kernel for the same operator and the same dispatch key
  operator: flash_attn::_flash_attn_backward(Tensor dout, Tensor q, Tensor k, Tensor v, Tensor out, Tensor softmax_lse, Tensor(a6!)? dq, Tensor(a7!)? dk, Tensor(a8!)? dv, float dropout_p, float softmax_scale, bool causal, SymInt window_size_left, SymInt window_size_right, float softcap, Tensor? alibi_slopes, bool deterministic, Tensor? rng_state=None) -> Tensor
    registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922
  dispatch key: ADInplaceOrView
  previous kernel: no debug info
       new kernel: registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922 (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/core/dispatch/OperatorEntry.cpp:208.)
  self.m.impl(
Loading model from: checkpoints/korean_3b_fp8_run1/checkpoint-0057000
Model parameters: 3015.4M

Perplexity config: seq_len=2048, stride=512, batch_size=32
Loaded 75,681,623 tokens from data/3b_val.bin

Evaluating perplexity:   0%|          | 0/4620 [00:00<?, ?batch/s]
Evaluating perplexity:   0%|          | 1/4620 [00:02<3:00:51,  2.35s/batch]
Evaluating perplexity:   0%|          | 2/4620 [00:04<2:50:04,  2.21s/batch]
Evaluating perplexity:   0%|          | 3/4620 [00:06<2:45:28,  2.15s/batch]
Evaluating perplexity:   0%|          | 4/4620 [00:08<2:43:16,  2.12s/batch]
Evaluating perplexity:   0%|          | 5/4620 [00:10<2:42:40,  2.12s/batch]
Evaluating perplexity:   0%|          | 6/4620 [00:12<2:42:48,  2.12s/batch]
Evaluating perplexity:   0%|          | 7/4620 [00:14<2:43:15,  2.12s/batch]
Evaluating perplexity:   0%|          | 8/4620 [00:17<2:43:41,  2.13s/batch]
Evaluating perplexity:   0%|          | 9/4620 [00:19<2:44:03,  2.13s/batch]
Evaluating perplexity:   0%|          | 10/4620 [00:21<2:44:39,  2.14s/batch]
Evaluating perplexity:   0%|          | 11/4620 [00:23<2:44:45,  2.14s/batch]
Evaluating perplexity:   0%|          | 12/4620 [00:25<2:44:44,  2.15s/batch]
Evaluating perplexity:   0%|          | 13/4620 [00:27<2:44:12,  2.14s/batch]
Evaluating perplexity:   0%|          | 14/4620 [00:30<2:44:50,  2.15s/batch]
Evaluating perplexity:   0%|          | 15/4620 [00:32<2:45:25,  2.16s/batch]
Evaluating perplexity:   0%|          | 16/4620 [00:34<2:45:19,  2.15s/batch]
Evaluating perplexity:   0%|          | 17/4620 [00:36<2:45:29,  2.16s/batch]
Evaluating perplexity:   0%|          | 18/4620 [00:39<3:05:57,  2.42s/batch]
Evaluating perplexity:   0%|          | 19/4620 [00:41<2:58:42,  2.33s/batch]
Evaluating perplexity:   0%|          | 20/4620 [00:43<2:54:44,  2.28s/batch]
Evaluating perplexity:   0%|          | 21/4620 [00:45<2:51:13,  2.23s/batch]
Evaluating perplexity:   0%|          | 22/4620 [00:48<2:48:49,  2.20s/batch]
Evaluating perplexity:   0%|          | 23/4620 [00:50<2:47:13,  2.18s/batch]
Evaluating perplexity:   1%|          | 24/4620 [00:52<2:45:42,  2.16s/batch]
Evaluating perplexity:   1%|          | 25/4620 [00:54<2:44:41,  2.15s/batch]
Evaluating perplexity:   1%|          | 26/4620 [00:56<2:44:52,  2.15s/batch]
Evaluating perplexity:   1%|          | 27/4620 [00:58<2:44:54,  2.15s/batch]
Evaluating perplexity:   1%|          | 28/4620 [01:00<2:44:47,  2.15s/batch]
Evaluating perplexity:   1%|          | 29/4620 [01:03<2:44:45,  2.15s/batch]
Evaluating perplexity:   1%|          | 30/4620 [01:05<2:43:59,  2.14s/batch]
Evaluating perplexity:   1%|          | 31/4620 [01:07<2:44:17,  2.15s/batch]
Evaluating perplexity:   1%|          | 32/4620 [01:09<2:44:21,  2.15s/batch]
Evaluating perplexity:   1%|          | 33/4620 [01:11<2:44:31,  2.15s/batch]
Evaluating perplexity:   1%|          | 34/4620 [01:13<2:45:05,  2.16s/batch]
Evaluating perplexity:   1%|          | 35/4620 [01:15<2:44:20,  2.15s/batch]
Evaluating perplexity:   1%|          | 36/4620 [01:18<2:44:27,  2.15s/batch]
Evaluating perplexity:   1%|          | 37/4620 [01:20<2:44:08,  2.15s/batch]
Evaluating perplexity:   1%|          | 38/4620 [01:22<2:44:06,  2.15s/batch]
Evaluating perplexity:   1%|          | 39/4620 [01:24<2:44:14,  2.15s/batch]
Evaluating perplexity:   1%|          | 40/4620 [01:26<2:44:11,  2.15s/batch]
Evaluating perplexity:   1%|          | 41/4620 [01:28<2:43:26,  2.14s/batch]
Evaluating perplexity:   1%|          | 42/4620 [01:30<2:43:19,  2.14s/batch]
Evaluating perplexity:   1%|          | 43/4620 [01:33<2:43:15,  2.14s/batch]
Evaluating perplexity:   1%|          | 44/4620 [01:35<2:42:47,  2.13s/batch]
Evaluating perplexity:   1%|          | 45/4620 [01:37<2:42:45,  2.13s/batch]
Evaluating perplexity:   1%|          | 46/4620 [01:39<2:42:32,  2.13s/batch]
Evaluating perplexity:   1%|          | 47/4620 [01:41<2:43:10,  2.14s/batch]
Evaluating perplexity:   1%|          | 48/4620 [01:43<2:42:50,  2.14s/batch]
Evaluating perplexity:   1%|          | 49/4620 [01:45<2:42:40,  2.14s/batch]
Evaluating perplexity:   1%|          | 50/4620 [01:48<2:42:54,  2.14s/batch]
Evaluating perplexity:   1%|          | 51/4620 [01:50<2:42:33,  2.13s/batch]
Evaluating perplexity:   1%|          | 52/4620 [01:52<2:4