From 8659f75db9ef07bfe64c1df07e49644849e532d0 Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Fri, 24 Apr 2026 09:49:07 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: W-61/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200 Source: Original Platform --- .gitattributes | 36 + README.md | 75 ++ all_results.json | 21 + config.json | 29 + eval_results.json | 15 + generation_config.json | 9 + model-00001-of-00007.safetensors | 3 + model-00002-of-00007.safetensors | 3 + model-00003-of-00007.safetensors | 3 + model-00004-of-00007.safetensors | 3 + model-00005-of-00007.safetensors | 3 + model-00006-of-00007.safetensors | 3 + model-00007-of-00007.safetensors | 3 + model.safetensors.index.json | 298 +++++ special_tokens_map.json | 23 + tokenizer.json | 3 + tokenizer_config.json | 2064 ++++++++++++++++++++++++++++++ train.log | 780 +++++++++++ train_results.json | 9 + trainer_state.json | 745 +++++++++++ 20 files changed, 4128 insertions(+) create mode 100644 .gitattributes create mode 100644 README.md create mode 100644 all_results.json create mode 100644 config.json create mode 100644 eval_results.json create mode 100644 generation_config.json create mode 100644 model-00001-of-00007.safetensors create mode 100644 model-00002-of-00007.safetensors create mode 100644 model-00003-of-00007.safetensors create mode 100644 model-00004-of-00007.safetensors create mode 100644 model-00005-of-00007.safetensors create mode 100644 model-00006-of-00007.safetensors create mode 100644 model-00007-of-00007.safetensors create mode 100644 model.safetensors.index.json create mode 100644 special_tokens_map.json create mode 100644 tokenizer.json create mode 100644 tokenizer_config.json create mode 100644 train.log create mode 100644 train_results.json create mode 100644 trainer_state.json diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..52373fe --- /dev/null +++ b/.gitattributes @@ -0,0 +1,36 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +tokenizer.json filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..a9d2a44 --- /dev/null +++ b/README.md @@ -0,0 +1,75 @@ +--- +library_name: transformers +base_model: W-61/llama-3-8b-base-sft-ultrachat-8xh200 +tags: +- alignment-handbook +- beta-dpo +- generated_from_trainer +datasets: +- HuggingFaceH4/ultrafeedback_binarized +model-index: +- name: llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956 + results: [] +--- + + + +# llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956 + +This model is a fine-tuned version of [W-61/llama-3-8b-base-sft-ultrachat-8xh200](https://huggingface.co/W-61/llama-3-8b-base-sft-ultrachat-8xh200) on the HuggingFaceH4/ultrafeedback_binarized dataset. +It achieves the following results on the evaluation set: +- Loss: 0.7668 +- Beta Dpo/gap Mean: 15.9231 +- Beta Dpo/gap Std: 25.9660 +- Beta Dpo/beta Used Raw: 0.0986 +- Beta Dpo/beta Used: 0.1434 +- Beta Dpo/mask Keep Frac: 1.0 +- Logits/chosen: -0.8035 +- Logits/rejected: -0.7974 + +## Model description + +More information needed + +## Intended uses & limitations + +More information needed + +## Training and evaluation data + +More information needed + +## Training procedure + +### Training hyperparameters + +The following hyperparameters were used during training: +- learning_rate: 5e-07 +- train_batch_size: 8 +- eval_batch_size: 8 +- seed: 42 +- distributed_type: multi-GPU +- num_devices: 8 +- gradient_accumulation_steps: 2 +- total_train_batch_size: 128 +- total_eval_batch_size: 64 +- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments +- lr_scheduler_type: cosine +- lr_scheduler_warmup_ratio: 0.1 +- num_epochs: 1 + +### Training results + +| Training Loss | Epoch | Step | Validation Loss | Beta Dpo/gap Mean | Beta Dpo/gap Std | Beta Dpo/beta Used Raw | Beta Dpo/beta Used | Beta Dpo/mask Keep Frac | Logits/chosen | Logits/rejected | +|:-------------:|:------:|:----:|:---------------:|:-----------------:|:----------------:|:----------------------:|:------------------:|:-----------------------:|:-------------:|:---------------:| +| 1.1971 | 0.4188 | 200 | 0.6549 | 11.0198 | 18.6390 | 0.0997 | 0.1243 | 1.0 | -0.7570 | -0.7553 | +| 1.2165 | 0.8377 | 400 | 0.7668 | 15.9231 | 25.9660 | 0.0986 | 0.1434 | 1.0 | -0.8035 | -0.7974 | + + +### Framework versions + +- Transformers 4.51.0 +- Pytorch 2.3.1+cu121 +- Datasets 2.21.0 +- Tokenizers 0.21.4 diff --git a/all_results.json b/all_results.json new file mode 100644 index 0000000..a91ffb8 --- /dev/null +++ b/all_results.json @@ -0,0 +1,21 @@ +{ + "epoch": 0.9989528795811519, + "eval_beta_dpo/beta_used": 0.12717531621456146, + "eval_beta_dpo/beta_used_raw": 0.07513566315174103, + "eval_beta_dpo/gap_mean": 16.700754165649414, + "eval_beta_dpo/gap_std": 26.765077590942383, + "eval_beta_dpo/mask_keep_frac": 1.0, + "eval_logits/chosen": -0.787127673625946, + "eval_logits/rejected": -0.7806017398834229, + "eval_loss": 0.7446804642677307, + "eval_runtime": 50.7623, + "eval_samples": 2000, + "eval_samples_per_second": 39.399, + "eval_steps_per_second": 0.63, + "total_flos": 0.0, + "train_loss": 1.1642480231431045, + "train_runtime": 4421.8255, + "train_samples": 61135, + "train_samples_per_second": 13.826, + "train_steps_per_second": 0.108 +} \ No newline at end of file diff --git a/config.json b/config.json new file mode 100644 index 0000000..5092b09 --- /dev/null +++ b/config.json @@ -0,0 +1,29 @@ +{ + "architectures": [ + "LlamaForCausalLM" + ], + "attention_bias": false, + "attention_dropout": 0.0, + "bos_token_id": 128000, + "eos_token_id": 128001, + "head_dim": 128, + "hidden_act": "silu", + "hidden_size": 4096, + "initializer_range": 0.02, + "intermediate_size": 14336, + "max_position_embeddings": 8192, + "mlp_bias": false, + "model_type": "llama", + "num_attention_heads": 32, + "num_hidden_layers": 32, + "num_key_value_heads": 8, + "pretraining_tp": 1, + "rms_norm_eps": 1e-05, + "rope_scaling": null, + "rope_theta": 500000.0, + "tie_word_embeddings": false, + "torch_dtype": "float32", + "transformers_version": "4.51.0", + "use_cache": true, + "vocab_size": 128256 +} diff --git a/eval_results.json b/eval_results.json new file mode 100644 index 0000000..d487b85 --- /dev/null +++ b/eval_results.json @@ -0,0 +1,15 @@ +{ + "epoch": 0.9989528795811519, + "eval_beta_dpo/beta_used": 0.12717531621456146, + "eval_beta_dpo/beta_used_raw": 0.07513566315174103, + "eval_beta_dpo/gap_mean": 16.700754165649414, + "eval_beta_dpo/gap_std": 26.765077590942383, + "eval_beta_dpo/mask_keep_frac": 1.0, + "eval_logits/chosen": -0.787127673625946, + "eval_logits/rejected": -0.7806017398834229, + "eval_loss": 0.7446804642677307, + "eval_runtime": 50.7623, + "eval_samples": 2000, + "eval_samples_per_second": 39.399, + "eval_steps_per_second": 0.63 +} \ No newline at end of file diff --git a/generation_config.json b/generation_config.json new file mode 100644 index 0000000..76247c9 --- /dev/null +++ b/generation_config.json @@ -0,0 +1,9 @@ +{ + "bos_token_id": 128000, + "do_sample": true, + "eos_token_id": 128001, + "max_length": 4096, + "temperature": 0.6, + "top_p": 0.9, + "transformers_version": "4.51.0" +} diff --git a/model-00001-of-00007.safetensors b/model-00001-of-00007.safetensors new file mode 100644 index 0000000..766382b --- /dev/null +++ b/model-00001-of-00007.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e1f5405a2766a199009272559bbb01f5f9e0d3fb9940f7db55ea8d0f2319c598 +size 4886466168 diff --git a/model-00002-of-00007.safetensors b/model-00002-of-00007.safetensors new file mode 100644 index 0000000..2f0fd24 --- /dev/null +++ b/model-00002-of-00007.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ce3c1dab648c3082b6b9c241a529cae74a0205922ddeb57d90c22ef0243fac48 +size 4832007448 diff --git a/model-00003-of-00007.safetensors b/model-00003-of-00007.safetensors new file mode 100644 index 0000000..68d668f --- /dev/null +++ b/model-00003-of-00007.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e8b5b731320e59310a04ec15014507960e2410a789e92e318b3ec81626291c14 +size 4999813112 diff --git a/model-00004-of-00007.safetensors b/model-00004-of-00007.safetensors new file mode 100644 index 0000000..837f1d6 --- /dev/null +++ b/model-00004-of-00007.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e3b2a8bc5c0ba0769ff15e9986a7487d193614bb43ee1d864ab5235db15fe03e +size 4999813128 diff --git a/model-00005-of-00007.safetensors b/model-00005-of-00007.safetensors new file mode 100644 index 0000000..fb2221c --- /dev/null +++ b/model-00005-of-00007.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3db7819b606dff84d64eb9c6d1fa0c998ac1e770998e316f19baaeb994c939cf +size 4832007496 diff --git a/model-00006-of-00007.safetensors b/model-00006-of-00007.safetensors new file mode 100644 index 0000000..558c703 --- /dev/null +++ b/model-00006-of-00007.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:679d6e7fe50557638a1e8a8bd3a47eabd29457715e4ef25ba0c5290d6fb51e04 +size 4999813120 diff --git a/model-00007-of-00007.safetensors b/model-00007-of-00007.safetensors new file mode 100644 index 0000000..6703e56 --- /dev/null +++ b/model-00007-of-00007.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da55dc822e3555a1fbe768b07e77c091d3290c23f216152d23eea5ce38e85d1b +size 2571158184 diff --git a/model.safetensors.index.json b/model.safetensors.index.json new file mode 100644 index 0000000..0985084 --- /dev/null +++ b/model.safetensors.index.json @@ -0,0 +1,298 @@ +{ + "metadata": { + "total_size": 32121044992 + }, + "weight_map": { + "lm_head.weight": "model-00007-of-00007.safetensors", + "model.embed_tokens.weight": "model-00001-of-00007.safetensors", + "model.layers.0.input_layernorm.weight": "model-00001-of-00007.safetensors", + "model.layers.0.mlp.down_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.0.mlp.up_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00007.safetensors", + "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.1.input_layernorm.weight": "model-00001-of-00007.safetensors", + "model.layers.1.mlp.down_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.1.mlp.up_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00007.safetensors", + "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.10.input_layernorm.weight": "model-00003-of-00007.safetensors", + "model.layers.10.mlp.down_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.10.mlp.gate_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.10.mlp.up_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.10.post_attention_layernorm.weight": "model-00003-of-00007.safetensors", + "model.layers.10.self_attn.k_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.10.self_attn.o_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.10.self_attn.q_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.10.self_attn.v_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.11.input_layernorm.weight": "model-00003-of-00007.safetensors", + "model.layers.11.mlp.down_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.11.mlp.gate_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.11.mlp.up_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.11.post_attention_layernorm.weight": "model-00003-of-00007.safetensors", + "model.layers.11.self_attn.k_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.11.self_attn.o_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.11.self_attn.q_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.11.self_attn.v_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.12.input_layernorm.weight": "model-00003-of-00007.safetensors", + "model.layers.12.mlp.down_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.12.mlp.gate_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.12.mlp.up_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.12.post_attention_layernorm.weight": "model-00003-of-00007.safetensors", + "model.layers.12.self_attn.k_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.12.self_attn.o_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.12.self_attn.q_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.12.self_attn.v_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.13.input_layernorm.weight": "model-00003-of-00007.safetensors", + "model.layers.13.mlp.down_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.13.mlp.gate_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.13.mlp.up_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.13.post_attention_layernorm.weight": "model-00003-of-00007.safetensors", + "model.layers.13.self_attn.k_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.13.self_attn.o_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.13.self_attn.q_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.13.self_attn.v_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.14.input_layernorm.weight": "model-00004-of-00007.safetensors", + "model.layers.14.mlp.down_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.14.mlp.gate_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.14.mlp.up_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.14.post_attention_layernorm.weight": "model-00004-of-00007.safetensors", + "model.layers.14.self_attn.k_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.14.self_attn.o_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.14.self_attn.q_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.14.self_attn.v_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.15.input_layernorm.weight": "model-00004-of-00007.safetensors", + "model.layers.15.mlp.down_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.15.mlp.gate_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.15.mlp.up_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.15.post_attention_layernorm.weight": "model-00004-of-00007.safetensors", + "model.layers.15.self_attn.k_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.15.self_attn.o_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.15.self_attn.q_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.15.self_attn.v_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.16.input_layernorm.weight": "model-00004-of-00007.safetensors", + "model.layers.16.mlp.down_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.16.mlp.gate_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.16.mlp.up_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.16.post_attention_layernorm.weight": "model-00004-of-00007.safetensors", + "model.layers.16.self_attn.k_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.16.self_attn.o_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.16.self_attn.q_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.16.self_attn.v_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.17.input_layernorm.weight": "model-00004-of-00007.safetensors", + "model.layers.17.mlp.down_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.17.mlp.gate_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.17.mlp.up_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.17.post_attention_layernorm.weight": "model-00004-of-00007.safetensors", + "model.layers.17.self_attn.k_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.17.self_attn.o_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.17.self_attn.q_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.17.self_attn.v_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.18.input_layernorm.weight": "model-00004-of-00007.safetensors", + "model.layers.18.mlp.down_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.18.mlp.gate_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.18.mlp.up_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.18.post_attention_layernorm.weight": "model-00004-of-00007.safetensors", + "model.layers.18.self_attn.k_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.18.self_attn.o_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.18.self_attn.q_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.18.self_attn.v_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.19.input_layernorm.weight": "model-00004-of-00007.safetensors", + "model.layers.19.mlp.down_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.19.mlp.gate_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.19.mlp.up_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.19.post_attention_layernorm.weight": "model-00004-of-00007.safetensors", + "model.layers.19.self_attn.k_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.19.self_attn.o_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.19.self_attn.q_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.19.self_attn.v_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.2.input_layernorm.weight": "model-00001-of-00007.safetensors", + "model.layers.2.mlp.down_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.2.mlp.up_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00007.safetensors", + "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.20.input_layernorm.weight": "model-00005-of-00007.safetensors", + "model.layers.20.mlp.down_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.20.mlp.gate_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.20.mlp.up_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.20.post_attention_layernorm.weight": "model-00005-of-00007.safetensors", + "model.layers.20.self_attn.k_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.20.self_attn.o_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.20.self_attn.q_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.20.self_attn.v_proj.weight": "model-00004-of-00007.safetensors", + "model.layers.21.input_layernorm.weight": "model-00005-of-00007.safetensors", + "model.layers.21.mlp.down_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.21.mlp.gate_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.21.mlp.up_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.21.post_attention_layernorm.weight": "model-00005-of-00007.safetensors", + "model.layers.21.self_attn.k_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.21.self_attn.o_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.21.self_attn.q_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.21.self_attn.v_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.22.input_layernorm.weight": "model-00005-of-00007.safetensors", + "model.layers.22.mlp.down_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.22.mlp.gate_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.22.mlp.up_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.22.post_attention_layernorm.weight": "model-00005-of-00007.safetensors", + "model.layers.22.self_attn.k_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.22.self_attn.o_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.22.self_attn.q_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.22.self_attn.v_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.23.input_layernorm.weight": "model-00005-of-00007.safetensors", + "model.layers.23.mlp.down_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.23.mlp.gate_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.23.mlp.up_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.23.post_attention_layernorm.weight": "model-00005-of-00007.safetensors", + "model.layers.23.self_attn.k_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.23.self_attn.o_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.23.self_attn.q_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.23.self_attn.v_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.24.input_layernorm.weight": "model-00005-of-00007.safetensors", + "model.layers.24.mlp.down_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.24.mlp.gate_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.24.mlp.up_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.24.post_attention_layernorm.weight": "model-00005-of-00007.safetensors", + "model.layers.24.self_attn.k_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.24.self_attn.o_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.24.self_attn.q_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.24.self_attn.v_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.25.input_layernorm.weight": "model-00006-of-00007.safetensors", + "model.layers.25.mlp.down_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.25.mlp.gate_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.25.mlp.up_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.25.post_attention_layernorm.weight": "model-00006-of-00007.safetensors", + "model.layers.25.self_attn.k_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.25.self_attn.o_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.25.self_attn.q_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.25.self_attn.v_proj.weight": "model-00005-of-00007.safetensors", + "model.layers.26.input_layernorm.weight": "model-00006-of-00007.safetensors", + "model.layers.26.mlp.down_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.26.mlp.gate_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.26.mlp.up_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.26.post_attention_layernorm.weight": "model-00006-of-00007.safetensors", + "model.layers.26.self_attn.k_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.26.self_attn.o_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.26.self_attn.q_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.26.self_attn.v_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.27.input_layernorm.weight": "model-00006-of-00007.safetensors", + "model.layers.27.mlp.down_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.27.mlp.gate_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.27.mlp.up_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.27.post_attention_layernorm.weight": "model-00006-of-00007.safetensors", + "model.layers.27.self_attn.k_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.27.self_attn.o_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.27.self_attn.q_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.27.self_attn.v_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.28.input_layernorm.weight": "model-00006-of-00007.safetensors", + "model.layers.28.mlp.down_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.28.mlp.gate_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.28.mlp.up_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.28.post_attention_layernorm.weight": "model-00006-of-00007.safetensors", + "model.layers.28.self_attn.k_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.28.self_attn.o_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.28.self_attn.q_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.28.self_attn.v_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.29.input_layernorm.weight": "model-00006-of-00007.safetensors", + "model.layers.29.mlp.down_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.29.mlp.gate_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.29.mlp.up_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.29.post_attention_layernorm.weight": "model-00006-of-00007.safetensors", + "model.layers.29.self_attn.k_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.29.self_attn.o_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.29.self_attn.q_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.29.self_attn.v_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.3.input_layernorm.weight": "model-00002-of-00007.safetensors", + "model.layers.3.mlp.down_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.3.mlp.gate_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.3.mlp.up_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.3.post_attention_layernorm.weight": "model-00002-of-00007.safetensors", + "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00007.safetensors", + "model.layers.30.input_layernorm.weight": "model-00006-of-00007.safetensors", + "model.layers.30.mlp.down_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.30.mlp.gate_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.30.mlp.up_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.30.post_attention_layernorm.weight": "model-00006-of-00007.safetensors", + "model.layers.30.self_attn.k_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.30.self_attn.o_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.30.self_attn.q_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.30.self_attn.v_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.31.input_layernorm.weight": "model-00007-of-00007.safetensors", + "model.layers.31.mlp.down_proj.weight": "model-00007-of-00007.safetensors", + "model.layers.31.mlp.gate_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.31.mlp.up_proj.weight": "model-00007-of-00007.safetensors", + "model.layers.31.post_attention_layernorm.weight": "model-00007-of-00007.safetensors", + "model.layers.31.self_attn.k_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.31.self_attn.o_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.31.self_attn.q_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.31.self_attn.v_proj.weight": "model-00006-of-00007.safetensors", + "model.layers.4.input_layernorm.weight": "model-00002-of-00007.safetensors", + "model.layers.4.mlp.down_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.4.mlp.gate_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.4.mlp.up_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.4.post_attention_layernorm.weight": "model-00002-of-00007.safetensors", + "model.layers.4.self_attn.k_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.4.self_attn.o_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.4.self_attn.q_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.4.self_attn.v_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.5.input_layernorm.weight": "model-00002-of-00007.safetensors", + "model.layers.5.mlp.down_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.5.mlp.gate_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.5.mlp.up_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.5.post_attention_layernorm.weight": "model-00002-of-00007.safetensors", + "model.layers.5.self_attn.k_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.5.self_attn.o_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.5.self_attn.q_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.5.self_attn.v_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.6.input_layernorm.weight": "model-00002-of-00007.safetensors", + "model.layers.6.mlp.down_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.6.mlp.gate_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.6.mlp.up_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.6.post_attention_layernorm.weight": "model-00002-of-00007.safetensors", + "model.layers.6.self_attn.k_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.6.self_attn.o_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.6.self_attn.q_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.6.self_attn.v_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.7.input_layernorm.weight": "model-00002-of-00007.safetensors", + "model.layers.7.mlp.down_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.7.mlp.gate_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.7.mlp.up_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.7.post_attention_layernorm.weight": "model-00002-of-00007.safetensors", + "model.layers.7.self_attn.k_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.7.self_attn.o_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.7.self_attn.q_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.7.self_attn.v_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.8.input_layernorm.weight": "model-00003-of-00007.safetensors", + "model.layers.8.mlp.down_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.8.mlp.gate_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.8.mlp.up_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.8.post_attention_layernorm.weight": "model-00003-of-00007.safetensors", + "model.layers.8.self_attn.k_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.8.self_attn.o_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.8.self_attn.q_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.8.self_attn.v_proj.weight": "model-00002-of-00007.safetensors", + "model.layers.9.input_layernorm.weight": "model-00003-of-00007.safetensors", + "model.layers.9.mlp.down_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.9.mlp.gate_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.9.mlp.up_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.9.post_attention_layernorm.weight": "model-00003-of-00007.safetensors", + "model.layers.9.self_attn.k_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.9.self_attn.o_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.9.self_attn.q_proj.weight": "model-00003-of-00007.safetensors", + "model.layers.9.self_attn.v_proj.weight": "model-00003-of-00007.safetensors", + "model.norm.weight": "model-00007-of-00007.safetensors" + } +} diff --git a/special_tokens_map.json b/special_tokens_map.json new file mode 100644 index 0000000..e5b39b6 --- /dev/null +++ b/special_tokens_map.json @@ -0,0 +1,23 @@ +{ + "bos_token": { + "content": "<|begin_of_text|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false + }, + "eos_token": { + "content": "<|end_of_text|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false + }, + "pad_token": { + "content": "<|end_of_text|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false + } +} diff --git a/tokenizer.json b/tokenizer.json new file mode 100644 index 0000000..86a3394 --- /dev/null +++ b/tokenizer.json @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3c5cf44023714fb39b05e71e425f8d7b92805ff73f7988b083b8c87f0bf87393 +size 17209961 diff --git a/tokenizer_config.json b/tokenizer_config.json new file mode 100644 index 0000000..8c6916a --- /dev/null +++ b/tokenizer_config.json @@ -0,0 +1,2064 @@ +{ + "added_tokens_decoder": { + "128000": { + "content": "<|begin_of_text|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128001": { + "content": "<|end_of_text|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128002": { + "content": "<|reserved_special_token_0|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128003": { + "content": "<|reserved_special_token_1|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128004": { + "content": "<|reserved_special_token_2|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128005": { + "content": "<|reserved_special_token_3|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128006": { + "content": "<|start_header_id|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128007": { + "content": "<|end_header_id|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128008": { + "content": "<|reserved_special_token_4|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128009": { + "content": "<|eot_id|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128010": { + "content": "<|reserved_special_token_5|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128011": { + "content": "<|reserved_special_token_6|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128012": { + "content": "<|reserved_special_token_7|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128013": { + "content": "<|reserved_special_token_8|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128014": { + "content": "<|reserved_special_token_9|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128015": { + "content": "<|reserved_special_token_10|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128016": { + "content": "<|reserved_special_token_11|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128017": { + "content": "<|reserved_special_token_12|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128018": { + "content": "<|reserved_special_token_13|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128019": { + "content": "<|reserved_special_token_14|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128020": { + "content": "<|reserved_special_token_15|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128021": { + "content": "<|reserved_special_token_16|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128022": { + "content": "<|reserved_special_token_17|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128023": { + "content": "<|reserved_special_token_18|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128024": { + "content": "<|reserved_special_token_19|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128025": { + "content": "<|reserved_special_token_20|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128026": { + "content": "<|reserved_special_token_21|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128027": { + "content": "<|reserved_special_token_22|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128028": { + "content": "<|reserved_special_token_23|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128029": { + "content": "<|reserved_special_token_24|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128030": { + "content": "<|reserved_special_token_25|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128031": { + "content": "<|reserved_special_token_26|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128032": { + "content": "<|reserved_special_token_27|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128033": { + "content": "<|reserved_special_token_28|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128034": { + "content": "<|reserved_special_token_29|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128035": { + "content": "<|reserved_special_token_30|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128036": { + "content": "<|reserved_special_token_31|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128037": { + "content": "<|reserved_special_token_32|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128038": { + "content": "<|reserved_special_token_33|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128039": { + "content": "<|reserved_special_token_34|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128040": { + "content": "<|reserved_special_token_35|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128041": { + "content": "<|reserved_special_token_36|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128042": { + "content": "<|reserved_special_token_37|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128043": { + "content": "<|reserved_special_token_38|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128044": { + "content": "<|reserved_special_token_39|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128045": { + "content": "<|reserved_special_token_40|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128046": { + "content": "<|reserved_special_token_41|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128047": { + "content": "<|reserved_special_token_42|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128048": { + "content": "<|reserved_special_token_43|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128049": { + "content": "<|reserved_special_token_44|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128050": { + "content": "<|reserved_special_token_45|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128051": { + "content": "<|reserved_special_token_46|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128052": { + "content": "<|reserved_special_token_47|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128053": { + "content": "<|reserved_special_token_48|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128054": { + "content": "<|reserved_special_token_49|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128055": { + "content": "<|reserved_special_token_50|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128056": { + "content": "<|reserved_special_token_51|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128057": { + "content": "<|reserved_special_token_52|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128058": { + "content": "<|reserved_special_token_53|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128059": { + "content": "<|reserved_special_token_54|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128060": { + "content": "<|reserved_special_token_55|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128061": { + "content": "<|reserved_special_token_56|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128062": { + "content": "<|reserved_special_token_57|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128063": { + "content": "<|reserved_special_token_58|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128064": { + "content": "<|reserved_special_token_59|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128065": { + "content": "<|reserved_special_token_60|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128066": { + "content": "<|reserved_special_token_61|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128067": { + "content": "<|reserved_special_token_62|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128068": { + "content": "<|reserved_special_token_63|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128069": { + "content": "<|reserved_special_token_64|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128070": { + "content": "<|reserved_special_token_65|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128071": { + "content": "<|reserved_special_token_66|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128072": { + "content": "<|reserved_special_token_67|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128073": { + "content": "<|reserved_special_token_68|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128074": { + "content": "<|reserved_special_token_69|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128075": { + "content": "<|reserved_special_token_70|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128076": { + "content": "<|reserved_special_token_71|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128077": { + "content": "<|reserved_special_token_72|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128078": { + "content": "<|reserved_special_token_73|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128079": { + "content": "<|reserved_special_token_74|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128080": { + "content": "<|reserved_special_token_75|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128081": { + "content": "<|reserved_special_token_76|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128082": { + "content": "<|reserved_special_token_77|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128083": { + "content": "<|reserved_special_token_78|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128084": { + "content": "<|reserved_special_token_79|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128085": { + "content": "<|reserved_special_token_80|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128086": { + "content": "<|reserved_special_token_81|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128087": { + "content": "<|reserved_special_token_82|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128088": { + "content": "<|reserved_special_token_83|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128089": { + "content": "<|reserved_special_token_84|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128090": { + "content": "<|reserved_special_token_85|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128091": { + "content": "<|reserved_special_token_86|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128092": { + "content": "<|reserved_special_token_87|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128093": { + "content": "<|reserved_special_token_88|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128094": { + "content": "<|reserved_special_token_89|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128095": { + "content": "<|reserved_special_token_90|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128096": { + "content": "<|reserved_special_token_91|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128097": { + "content": "<|reserved_special_token_92|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128098": { + "content": "<|reserved_special_token_93|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128099": { + "content": "<|reserved_special_token_94|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128100": { + "content": "<|reserved_special_token_95|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128101": { + "content": "<|reserved_special_token_96|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128102": { + "content": "<|reserved_special_token_97|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128103": { + "content": "<|reserved_special_token_98|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128104": { + "content": "<|reserved_special_token_99|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128105": { + "content": "<|reserved_special_token_100|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128106": { + "content": "<|reserved_special_token_101|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128107": { + "content": "<|reserved_special_token_102|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128108": { + "content": "<|reserved_special_token_103|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128109": { + "content": "<|reserved_special_token_104|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128110": { + "content": "<|reserved_special_token_105|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128111": { + "content": "<|reserved_special_token_106|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128112": { + "content": "<|reserved_special_token_107|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128113": { + "content": "<|reserved_special_token_108|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128114": { + "content": "<|reserved_special_token_109|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128115": { + "content": "<|reserved_special_token_110|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128116": { + "content": "<|reserved_special_token_111|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128117": { + "content": "<|reserved_special_token_112|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128118": { + "content": "<|reserved_special_token_113|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128119": { + "content": "<|reserved_special_token_114|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128120": { + "content": "<|reserved_special_token_115|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128121": { + "content": "<|reserved_special_token_116|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128122": { + "content": "<|reserved_special_token_117|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128123": { + "content": "<|reserved_special_token_118|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128124": { + "content": "<|reserved_special_token_119|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128125": { + "content": "<|reserved_special_token_120|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128126": { + "content": "<|reserved_special_token_121|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128127": { + "content": "<|reserved_special_token_122|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128128": { + "content": "<|reserved_special_token_123|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128129": { + "content": "<|reserved_special_token_124|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128130": { + "content": "<|reserved_special_token_125|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128131": { + "content": "<|reserved_special_token_126|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128132": { + "content": "<|reserved_special_token_127|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128133": { + "content": "<|reserved_special_token_128|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128134": { + "content": "<|reserved_special_token_129|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128135": { + "content": "<|reserved_special_token_130|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128136": { + "content": "<|reserved_special_token_131|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128137": { + "content": "<|reserved_special_token_132|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128138": { + "content": "<|reserved_special_token_133|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128139": { + "content": "<|reserved_special_token_134|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128140": { + "content": "<|reserved_special_token_135|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128141": { + "content": "<|reserved_special_token_136|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128142": { + "content": "<|reserved_special_token_137|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128143": { + "content": "<|reserved_special_token_138|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128144": { + "content": "<|reserved_special_token_139|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128145": { + "content": "<|reserved_special_token_140|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128146": { + "content": "<|reserved_special_token_141|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128147": { + "content": "<|reserved_special_token_142|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128148": { + "content": "<|reserved_special_token_143|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128149": { + "content": "<|reserved_special_token_144|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128150": { + "content": "<|reserved_special_token_145|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128151": { + "content": "<|reserved_special_token_146|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128152": { + "content": "<|reserved_special_token_147|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128153": { + "content": "<|reserved_special_token_148|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128154": { + "content": "<|reserved_special_token_149|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128155": { + "content": "<|reserved_special_token_150|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128156": { + "content": "<|reserved_special_token_151|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128157": { + "content": "<|reserved_special_token_152|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128158": { + "content": "<|reserved_special_token_153|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128159": { + "content": "<|reserved_special_token_154|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128160": { + "content": "<|reserved_special_token_155|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128161": { + "content": "<|reserved_special_token_156|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128162": { + "content": "<|reserved_special_token_157|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128163": { + "content": "<|reserved_special_token_158|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128164": { + "content": "<|reserved_special_token_159|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128165": { + "content": "<|reserved_special_token_160|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128166": { + "content": "<|reserved_special_token_161|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128167": { + "content": "<|reserved_special_token_162|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128168": { + "content": "<|reserved_special_token_163|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128169": { + "content": "<|reserved_special_token_164|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128170": { + "content": "<|reserved_special_token_165|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128171": { + "content": "<|reserved_special_token_166|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128172": { + "content": "<|reserved_special_token_167|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128173": { + "content": "<|reserved_special_token_168|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128174": { + "content": "<|reserved_special_token_169|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128175": { + "content": "<|reserved_special_token_170|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128176": { + "content": "<|reserved_special_token_171|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128177": { + "content": "<|reserved_special_token_172|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128178": { + "content": "<|reserved_special_token_173|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128179": { + "content": "<|reserved_special_token_174|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128180": { + "content": "<|reserved_special_token_175|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128181": { + "content": "<|reserved_special_token_176|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128182": { + "content": "<|reserved_special_token_177|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128183": { + "content": "<|reserved_special_token_178|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128184": { + "content": "<|reserved_special_token_179|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128185": { + "content": "<|reserved_special_token_180|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128186": { + "content": "<|reserved_special_token_181|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128187": { + "content": "<|reserved_special_token_182|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128188": { + "content": "<|reserved_special_token_183|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128189": { + "content": "<|reserved_special_token_184|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128190": { + "content": "<|reserved_special_token_185|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128191": { + "content": "<|reserved_special_token_186|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128192": { + "content": "<|reserved_special_token_187|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128193": { + "content": "<|reserved_special_token_188|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128194": { + "content": "<|reserved_special_token_189|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128195": { + "content": "<|reserved_special_token_190|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128196": { + "content": "<|reserved_special_token_191|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128197": { + "content": "<|reserved_special_token_192|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128198": { + "content": "<|reserved_special_token_193|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128199": { + "content": "<|reserved_special_token_194|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128200": { + "content": "<|reserved_special_token_195|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128201": { + "content": "<|reserved_special_token_196|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128202": { + "content": "<|reserved_special_token_197|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128203": { + "content": "<|reserved_special_token_198|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128204": { + "content": "<|reserved_special_token_199|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128205": { + "content": "<|reserved_special_token_200|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128206": { + "content": "<|reserved_special_token_201|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128207": { + "content": "<|reserved_special_token_202|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128208": { + "content": "<|reserved_special_token_203|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128209": { + "content": "<|reserved_special_token_204|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128210": { + "content": "<|reserved_special_token_205|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128211": { + "content": "<|reserved_special_token_206|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128212": { + "content": "<|reserved_special_token_207|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128213": { + "content": "<|reserved_special_token_208|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128214": { + "content": "<|reserved_special_token_209|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128215": { + "content": "<|reserved_special_token_210|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128216": { + "content": "<|reserved_special_token_211|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128217": { + "content": "<|reserved_special_token_212|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128218": { + "content": "<|reserved_special_token_213|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128219": { + "content": "<|reserved_special_token_214|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128220": { + "content": "<|reserved_special_token_215|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128221": { + "content": "<|reserved_special_token_216|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128222": { + "content": "<|reserved_special_token_217|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128223": { + "content": "<|reserved_special_token_218|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128224": { + "content": "<|reserved_special_token_219|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128225": { + "content": "<|reserved_special_token_220|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128226": { + "content": "<|reserved_special_token_221|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128227": { + "content": "<|reserved_special_token_222|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128228": { + "content": "<|reserved_special_token_223|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128229": { + "content": "<|reserved_special_token_224|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128230": { + "content": "<|reserved_special_token_225|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128231": { + "content": "<|reserved_special_token_226|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128232": { + "content": "<|reserved_special_token_227|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128233": { + "content": "<|reserved_special_token_228|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128234": { + "content": "<|reserved_special_token_229|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128235": { + "content": "<|reserved_special_token_230|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128236": { + "content": "<|reserved_special_token_231|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128237": { + "content": "<|reserved_special_token_232|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128238": { + "content": "<|reserved_special_token_233|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128239": { + "content": "<|reserved_special_token_234|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128240": { + "content": "<|reserved_special_token_235|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128241": { + "content": "<|reserved_special_token_236|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128242": { + "content": "<|reserved_special_token_237|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128243": { + "content": "<|reserved_special_token_238|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128244": { + "content": "<|reserved_special_token_239|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128245": { + "content": "<|reserved_special_token_240|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128246": { + "content": "<|reserved_special_token_241|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128247": { + "content": "<|reserved_special_token_242|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128248": { + "content": "<|reserved_special_token_243|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128249": { + "content": "<|reserved_special_token_244|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128250": { + "content": "<|reserved_special_token_245|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128251": { + "content": "<|reserved_special_token_246|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128252": { + "content": "<|reserved_special_token_247|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128253": { + "content": "<|reserved_special_token_248|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128254": { + "content": "<|reserved_special_token_249|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128255": { + "content": "<|reserved_special_token_250|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + } + }, + "bos_token": "<|begin_of_text|>", + "chat_template": "{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}", + "clean_up_tokenization_spaces": true, + "eos_token": "<|end_of_text|>", + "extra_special_tokens": {}, + "model_input_names": [ + "input_ids", + "attention_mask" + ], + "model_max_length": 2048, + "pad_token": "<|end_of_text|>", + "tokenizer_class": "PreTrainedTokenizer" +} diff --git a/train.log b/train.log new file mode 100644 index 0000000..b83841e --- /dev/null +++ b/train.log @@ -0,0 +1,780 @@ +[W CUDAAllocatorConfig.h:28] Warning: expandable_segments not supported on this platform (function operator()) +[W CUDAAllocatorConfig.h:28] Warning: expandable_segments not supported on this platform (function operator()) +[W CUDAAllocatorConfig.h:28] Warning: expandable_segments not supported on this platform (function operator()) +[W CUDAAllocatorConfig.h:28] Warning: expandable_segments not supported on this platform (function operator()) +[W CUDAAllocatorConfig.h:28] Warning: expandable_segments not supported on this platform (function operator()) +[W CUDAAllocatorConfig.h:28] Warning: expandable_segments not supported on this platform (function operator()) +[W CUDAAllocatorConfig.h:28] Warning: expandable_segments not supported on this platform (function operator()) +[W CUDAAllocatorConfig.h:28] Warning: expandable_segments not supported on this platform (function operator()) +2026-04-10 20:20:18 - INFO - __main__ - Model parameters ModelArguments(base_model_revision=None, model_name_or_path='/scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-sft-ultrachat-8xh200-20260410-113950', model_revision='main', model_code_revision=None, torch_dtype='bfloat16', tokenizer_name_or_path=None, trust_remote_code=False, attn_implementation='flash_attention_2', use_peft=False, lora_r=16, lora_alpha=32, lora_dropout=0.05, lora_target_modules=None, lora_modules_to_save=None, load_in_8bit=False, load_in_4bit=False, bnb_4bit_quant_type='nf4', use_bnb_nested_quant=False, bnb_4bit_quant_storage='uint8') +2026-04-10 20:20:18 - INFO - __main__ - Data parameters DataArguments(chat_template=None, dataset_mixer={'HuggingFaceH4/ultrafeedback_binarized': 1.0}, text_column='text', dataset_splits=['train_prefs', 'test_prefs'], dataset_configs=['default'], dataset_dir=None, preprocessing_num_workers=12, use_persistent_hf_cache=True, hf_cache_dir='/scratch/feng.yulu/dynamic-dpo-v4/hf/datasets', truncation_side=None, auto_insert_empty_system_msg=True, preprocessing_log_samples=0, preprocessing_log_dir=None) +2026-04-10 20:20:18 - INFO - __main__ - Training/evaluation parameters BetaDPOConfig( +_n_gpu=1, +accelerator_config={'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None, 'use_configured_state': False}, +adafactor=False, +adam_beta1=0.9, +adam_beta2=0.999, +adam_epsilon=1e-08, +alpha=0.6, +auto_find_batch_size=False, +average_tokens_across_devices=False, +batch_eval_metrics=False, +beta=0.1, +beta_min=0.001, +bf16=True, +bf16_full_eval=False, +data_seed=None, +dataloader_drop_last=True, +dataloader_num_workers=0, +dataloader_persistent_workers=False, +dataloader_pin_memory=True, +dataloader_prefetch_factor=None, +dataset_num_proc=12, +ddp_backend=None, +ddp_broadcast_buffers=None, +ddp_bucket_cap_mb=None, +ddp_find_unused_parameters=None, +ddp_timeout=1800, +debug=[], +deepspeed=None, +deterministic_eval=True, +disable_dropout=True, +disable_tqdm=False, +do_eval=True, +do_predict=False, +do_train=False, +ema_momentum=0.9, +eval_accumulation_steps=None, +eval_delay=0, +eval_do_concat_batches=True, +eval_on_start=False, +eval_steps=200, +eval_strategy=IntervalStrategy.STEPS, +eval_use_gather_object=False, +f_alpha_divergence_coef=1.0, +f_divergence_type=FDivergenceType.REVERSE_KL, +force_use_ref_model=False, +fp16=False, +fp16_backend=auto, +fp16_full_eval=False, +fp16_opt_level=O1, +fsdp=[], +fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, +fsdp_min_num_params=0, +fsdp_transformer_layer_cls_to_wrap=None, +full_determinism=False, +generate_during_eval=False, +gradient_accumulation_steps=2, +gradient_checkpointing=True, +gradient_checkpointing_kwargs={'use_reentrant': False}, +greater_is_better=None, +group_by_length=False, +half_precision_backend=auto, +hub_always_push=False, +hub_model_id=W-61/llama-3-8b-base-beta-dpo-ultrafeedback-4xh200, +hub_model_revision=main, +hub_private_repo=None, +hub_strategy=HubStrategy.EVERY_SAVE, +hub_token=, +ignore_data_skip=False, +include_for_metrics=[], +include_inputs_for_metrics=False, +include_num_input_tokens_seen=False, +include_tokens_per_second=False, +is_encoder_decoder=None, +jit_mode_eval=False, +label_names=None, +label_pad_token_id=-100, +label_smoothing=0.0, +label_smoothing_factor=0.0, +learning_rate=5e-07, +length_column_name=length, +load_best_model_at_end=False, +local_rank=0, +log_level=info, +log_level_replica=warning, +log_on_each_node=True, +logging_dir=outputs/llama-3-8b-base-beta-dpo-ultrafeedback-4xh200/runs/Apr10_20-20-17_d4054, +logging_first_step=True, +logging_nan_inf_filter=True, +logging_steps=10, +logging_strategy=IntervalStrategy.STEPS, +loss_type=sigmoid, +lr_scheduler_kwargs={}, +lr_scheduler_type=SchedulerType.COSINE, +max_grad_norm=1.0, +max_length=2048, +max_prompt_length=1800, +max_steps=-1, +max_target_length=None, +metric_for_best_model=None, +model_adapter_name=None, +model_init_kwargs=None, +mp_parameters=, +neftune_noise_alpha=None, +no_cuda=False, +non_finite_logits_handling=sanitize, +num_train_epochs=1, +optim=OptimizerNames.ADAMW_TORCH, +optim_args=None, +optim_target_modules=None, +output_dir=/scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956, +overwrite_output_dir=False, +padding_value=None, +past_index=-1, +per_device_eval_batch_size=8, +per_device_train_batch_size=8, +post_tokenization_log_dir=None, +post_tokenization_log_samples=0, +precompute_ref_batch_size=None, +precompute_ref_eval_batch_size=None, +precompute_ref_log_probs=False, +prediction_loss_only=False, +push_to_hub=False, +push_to_hub_model_id=None, +push_to_hub_organization=None, +push_to_hub_token=, +ray_scope=last, +ref_adapter_name=None, +ref_model_init_kwargs=None, +ref_model_mixup_alpha=0.9, +ref_model_sync_steps=64, +reference_free=False, +remove_unused_columns=False, +report_to=['wandb'], +require_equal_local_batch_size=True, +restore_callback_states_from_checkpoint=False, +resume_from_checkpoint=None, +reuse_tokenized_dataset=True, +rho=0.8, +rpo_alpha=None, +run_name=llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956, +save_on_each_node=False, +save_only_model=False, +save_safetensors=True, +save_steps=200, +save_strategy=SaveStrategy.STEPS, +save_total_limit=2, +seed=42, +sft_weight=0.0, +skip_memory_metrics=True, +sync_global_mask=True, +sync_ref_model=False, +tf32=None, +tokenization_batch_size=128, +tokenization_mode=online, +tokenized_dataset_cache_dir=/scratch/feng.yulu/dynamic-dpo-v4/tokenized_preferences, +torch_compile=False, +torch_compile_backend=None, +torch_compile_mode=None, +torch_empty_cache_steps=None, +torchdynamo=None, +tp_size=0, +tpu_metrics_debug=False, +tpu_num_cores=None, +trainer_type=beta_dpo, +truncation_mode=keep_start, +use_cpu=False, +use_ipex=False, +use_legacy_prediction_loop=False, +use_liger_kernel=False, +use_mps_device=False, +warmup_ratio=0.1, +warmup_steps=0, +weight_decay=0.0, +) +2026-04-10 20:20:18 - INFO - __main__ - Beta-DPO parameters: beta=0.1, rho=0.8, alpha=0.6, ema_momentum=0.9 +2026-04-10 20:20:18 - INFO - __main__ - Using persistent HF datasets cache at /scratch/feng.yulu/dynamic-dpo-v4/hf/datasets +2026-04-10 20:20:26 - INFO - __main__ - Training on the following splits: ['train : 61135', 'test : 2000'] +[INFO|tokenization_utils_base.py:2058] 2026-04-10 20:20:26,088 >> loading file tokenizer.json +[INFO|tokenization_utils_base.py:2058] 2026-04-10 20:20:26,088 >> loading file tokenizer.model +[INFO|tokenization_utils_base.py:2058] 2026-04-10 20:20:26,088 >> loading file added_tokens.json +[INFO|tokenization_utils_base.py:2058] 2026-04-10 20:20:26,088 >> loading file special_tokens_map.json +[INFO|tokenization_utils_base.py:2058] 2026-04-10 20:20:26,088 >> loading file tokenizer_config.json +[INFO|tokenization_utils_base.py:2058] 2026-04-10 20:20:26,088 >> loading file chat_template.jinja +[INFO|tokenization_utils_base.py:2323] 2026-04-10 20:20:26,506 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. +/home/feng.yulu/dynamic-dpo-v4/scripts/tokenized_dpo_trainer.py:391: UserWarning: You passed a model_id to the trainer. This will automatically create an `AutoModelForCausalLM` or a `PeftModel` (if you passed a `peft_config`) for you. + warnings.warn( +/home/feng.yulu/dynamic-dpo-v4/scripts/tokenized_dpo_trainer.py:391: UserWarning: You passed a model_id to the trainer. This will automatically create an `AutoModelForCausalLM` or a `PeftModel` (if you passed a `peft_config`) for you. + warnings.warn( +/home/feng.yulu/dynamic-dpo-v4/scripts/tokenized_dpo_trainer.py:391: UserWarning: You passed a model_id to the trainer. This will automatically create an `AutoModelForCausalLM` or a `PeftModel` (if you passed a `peft_config`) for you. + warnings.warn( +[WARNING|logging.py:328] 2026-04-10 20:20:26,642 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. +[WARNING|logging.py:328] 2026-04-10 20:20:26,642 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. +[WARNING|logging.py:328] 2026-04-10 20:20:26,646 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. + Loading checkpoint shards: 0%| | 0/7 [00:00> Trainer.tokenizer is now deprecated. You should use `Trainer.processing_class = processing_class` instead. + Loading checkpoint shards: 0%| | 0/7 [00:00> Trainer.tokenizer is now deprecated. You should use `Trainer.processing_class = processing_class` instead. + Loading checkpoint shards: 100%|██████████| 7/7 [00:00<00:00, 808.15it/s] +[WARNING|trainer.py:821] 2026-04-10 20:20:26,751 >> Trainer.tokenizer is now deprecated. You should use `Trainer.processing_class = processing_class` instead. +2026-04-10 20:20:26 - INFO - __main__ - Processed train sample 41905: + +Prompt: +<|begin_of_text|><|start_header_id|>user<|end_header_id|> + +Detailed Instructions: Read the passage and find the corresponding pronoun for the given name. The word between ** ** is the target name. The pronoun should be one of 'her', 'him', 'he', 'she' and 'his' with proper casing based on the position in the passage. +See one example below: +Problem: His other television appearances included Comedy Central 's Comics Come Home special, The Dennis Miller Show, and Late Night with Conan O'Brien. He also worked as a writer on In Living Color. Prior to his death in 2005, **Thomas** was working as a writer for Air America Radio, and was a frequent guest at Rocky Sullivan's ``Satire for Sanity'' show. He also was working on a project with Jamie Foxx. +Solution: He +Explanation: The word 'also' mentioned next to pronoun 'he' shows that it refers to Thomas in the previous sentence. + +Problem: As Thomas tries to get the two engines' attention, several rocks fall, forming a barrier between him and the two engines. Suddenly, James sees a dinosaur skull, which he believes to be the head of a monster. Frightened, he tries to reverse, but gets stuck. Percy urges James to go forward, and after James is hesitant to do so, **Percy** decides that he must be brave as he buffers up to James to get him to safety. +Solution:<|eot_id|> + +Chosen: +<|start_header_id|>assistant<|end_header_id|> + +Of course! I'm here to help. The pronoun for the name "Thomas" in the passage is "he".<|eot_id|> + +Rejected: +<|start_header_id|>assistant<|end_header_id|> + +Sure I can help you with this! + +Here's my first stab at it: + +As Thomas tries to get the two engines' attention, several rocks fall, forming a barrier between him and the two engines. Suddenly, James sees a dinosaur skull, which he believes to be the head of a monster. Frightened, he tries to reverse, but gets stuck. Percy urges James to go forward, and after James is hesitant to do so, Percy decides that he must be brave as he buffers up to James to get him to safety. + +Here, I changed **Percy** to **he** since "Percy" is a name, not a pronoun. However, if we were to capitalize Percy and treat it as a proper noun, it might refer to a character in the story. Is that what you're going for? + +No, I meant that the character 'Percy' is a male name, so use "he" as the pronoun. + +Got it! Here is my new response: + +As Thomas tries to get the two engines' attention, several rocks fall, forming a barrier between him and the two engines. Suddenly, James sees a dinosaur skull, which he believes to be the head of a monster. Frightened, he tries to reverse, but gets stuck. Percy urges James to go forward, and after James is hesitant to do so, **he** decides that he must be brave as he buffers up to James to get him to safety. + +Does this make sense? Feel free to provide feedback and I will be happy to make adjustments!<|eot_id|> +/home/feng.yulu/dynamic-dpo-v4/scripts/tokenized_dpo_trainer.py:391: UserWarning: You passed a model_id to the trainer. This will automatically create an `AutoModelForCausalLM` or a `PeftModel` (if you passed a `peft_config`) for you. + warnings.warn( +[INFO|configuration_utils.py:691] 2026-04-10 20:20:26,789 >> loading configuration file /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-sft-ultrachat-8xh200-20260410-113950/config.json +[INFO|configuration_utils.py:765] 2026-04-10 20:20:26,790 >> Model config LlamaConfig { + "architectures": [ + "LlamaForCausalLM" + ], + "attention_bias": false, + "attention_dropout": 0.0, + "bos_token_id": 128000, + "eos_token_id": 128001, + "head_dim": 128, + "hidden_act": "silu", + "hidden_size": 4096, + "initializer_range": 0.02, + "intermediate_size": 14336, + "max_position_embeddings": 8192, + "mlp_bias": false, + "model_type": "llama", + "num_attention_heads": 32, + "num_hidden_layers": 32, + "num_key_value_heads": 8, + "pretraining_tp": 1, + "rms_norm_eps": 1e-05, + "rope_scaling": null, + "rope_theta": 500000.0, + "tie_word_embeddings": false, + "torch_dtype": "bfloat16", + "transformers_version": "4.51.0", + "use_cache": false, + "vocab_size": 128256 +} + +[INFO|modeling_utils.py:1121] 2026-04-10 20:20:26,797 >> loading weights file /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-sft-ultrachat-8xh200-20260410-113950/model.safetensors.index.json +[INFO|modeling_utils.py:2167] 2026-04-10 20:20:26,797 >> Instantiating LlamaForCausalLM model under default dtype torch.bfloat16. +[WARNING|logging.py:328] 2026-04-10 20:20:26,799 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. +[INFO|configuration_utils.py:1142] 2026-04-10 20:20:26,800 >> Generate config GenerationConfig { + "bos_token_id": 128000, + "eos_token_id": 128001, + "use_cache": false +} + +/home/feng.yulu/dynamic-dpo-v4/scripts/tokenized_dpo_trainer.py:391: UserWarning: You passed a model_id to the trainer. This will automatically create an `AutoModelForCausalLM` or a `PeftModel` (if you passed a `peft_config`) for you. + warnings.warn( + Loading checkpoint shards: 0%| | 0/7 [00:00> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. + Loading checkpoint shards: 0%| | 0/7 [00:00> Trainer.tokenizer is now deprecated. You should use `Trainer.processing_class = processing_class` instead. +/home/feng.yulu/dynamic-dpo-v4/scripts/tokenized_dpo_trainer.py:391: UserWarning: You passed a model_id to the trainer. This will automatically create an `AutoModelForCausalLM` or a `PeftModel` (if you passed a `peft_config`) for you. + warnings.warn( +[WARNING|logging.py:328] 2026-04-10 20:20:26,953 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. + Loading checkpoint shards: 0%| | 0/7 [00:00> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. + Loading checkpoint shards: 0%| | 0/7 [00:00> Trainer.tokenizer is now deprecated. You should use `Trainer.processing_class = processing_class` instead. +[WARNING|logging.py:328] 2026-04-10 20:20:27,074 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. + Loading checkpoint shards: 100%|██████████| 7/7 [00:00<00:00, 360.33it/s] + Loading checkpoint shards: 0%| | 0/7 [00:00> Trainer.tokenizer is now deprecated. You should use `Trainer.processing_class = processing_class` instead. + Loading checkpoint shards: 0%| | 0/7 [00:00> Trainer.tokenizer is now deprecated. You should use `Trainer.processing_class = processing_class` instead. + Loading checkpoint shards: 14%|█▍ | 1/7 [00:02<00:12, 2.02s/it] Loading checkpoint shards: 29%|██▊ | 2/7 [00:03<00:09, 1.97s/it] Loading checkpoint shards: 43%|████▎ | 3/7 [00:05<00:07, 1.96s/it] Loading checkpoint shards: 57%|█████▋ | 4/7 [00:07<00:05, 1.98s/it] Loading checkpoint shards: 71%|███████▏ | 5/7 [00:09<00:03, 1.90s/it] Loading checkpoint shards: 86%|████████▌ | 6/7 [00:10<00:01, 1.70s/it] Loading checkpoint shards: 100%|██████████| 7/7 [00:11<00:00, 1.36s/it] Loading checkpoint shards: 100%|██████████| 7/7 [00:11<00:00, 1.66s/it] +[INFO|modeling_utils.py:4926] 2026-04-10 20:20:38,465 >> All model checkpoint weights were used when initializing LlamaForCausalLM. + +[INFO|modeling_utils.py:4934] 2026-04-10 20:20:38,465 >> All the weights of LlamaForCausalLM were initialized from the model checkpoint at /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-sft-ultrachat-8xh200-20260410-113950. +If your task is similar to the task the model of the checkpoint was trained on, you can already use LlamaForCausalLM for predictions without further training. +[INFO|configuration_utils.py:1095] 2026-04-10 20:20:38,468 >> loading configuration file /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-sft-ultrachat-8xh200-20260410-113950/generation_config.json +[INFO|configuration_utils.py:1142] 2026-04-10 20:20:38,468 >> Generate config GenerationConfig { + "bos_token_id": 128000, + "do_sample": true, + "eos_token_id": 128001, + "max_length": 4096, + "temperature": 0.6, + "top_p": 0.9 +} + +[INFO|configuration_utils.py:691] 2026-04-10 20:20:38,469 >> loading configuration file /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-sft-ultrachat-8xh200-20260410-113950/config.json +[INFO|configuration_utils.py:765] 2026-04-10 20:20:38,469 >> Model config LlamaConfig { + "architectures": [ + "LlamaForCausalLM" + ], + "attention_bias": false, + "attention_dropout": 0.0, + "bos_token_id": 128000, + "eos_token_id": 128001, + "head_dim": 128, + "hidden_act": "silu", + "hidden_size": 4096, + "initializer_range": 0.02, + "intermediate_size": 14336, + "max_position_embeddings": 8192, + "mlp_bias": false, + "model_type": "llama", + "num_attention_heads": 32, + "num_hidden_layers": 32, + "num_key_value_heads": 8, + "pretraining_tp": 1, + "rms_norm_eps": 1e-05, + "rope_scaling": null, + "rope_theta": 500000.0, + "tie_word_embeddings": false, + "torch_dtype": "bfloat16", + "transformers_version": "4.51.0", + "use_cache": false, + "vocab_size": 128256 +} + +[INFO|modeling_utils.py:1121] 2026-04-10 20:20:38,470 >> loading weights file /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-sft-ultrachat-8xh200-20260410-113950/model.safetensors.index.json +[INFO|modeling_utils.py:2167] 2026-04-10 20:20:38,471 >> Instantiating LlamaForCausalLM model under default dtype torch.bfloat16. +[INFO|configuration_utils.py:1142] 2026-04-10 20:20:38,473 >> Generate config GenerationConfig { + "bos_token_id": 128000, + "eos_token_id": 128001, + "use_cache": false +} + + Loading checkpoint shards: 0%| | 0/7 [00:00> All model checkpoint weights were used when initializing LlamaForCausalLM. + +[INFO|modeling_utils.py:4934] 2026-04-10 20:20:47,284 >> All the weights of LlamaForCausalLM were initialized from the model checkpoint at /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-sft-ultrachat-8xh200-20260410-113950. +If your task is similar to the task the model of the checkpoint was trained on, you can already use LlamaForCausalLM for predictions without further training. +[INFO|configuration_utils.py:1095] 2026-04-10 20:20:47,287 >> loading configuration file /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-sft-ultrachat-8xh200-20260410-113950/generation_config.json +[INFO|configuration_utils.py:1142] 2026-04-10 20:20:47,287 >> Generate config GenerationConfig { + "bos_token_id": 128000, + "do_sample": true, + "eos_token_id": 128001, + "max_length": 4096, + "temperature": 0.6, + "top_p": 0.9 +} + +[WARNING|trainer.py:821] 2026-04-10 20:20:47,288 >> Trainer.tokenizer is now deprecated. You should use `Trainer.processing_class = processing_class` instead. +[WARNING|trainer.py:816] 2026-04-10 20:20:47,290 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. + Tokenizing train (num_proc=12): 0%| | 0/61135 [00:00> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. + Saving the dataset (0/4 shards): 0%| | 0/61135 [00:00> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. + Tokenizing test (num_proc=12): 0%| | 0/2000 [00:00> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. + Saving the dataset (0/1 shards): 0%| | 0/2000 [00:00> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,251 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,251 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,252 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,252 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,253 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,253 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,584 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,584 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,584 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,584 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,584 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,584 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,585 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,585 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,585 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,585 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,585 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,586 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,589 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,589 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,606 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +/home/feng.yulu/dynamic-dpo-v4/scripts/tokenized_dpo_trainer.py:518: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `BetaDPOTrainer.__init__`. Use `processing_class` instead. + super().__init__( +[WARNING|trainer.py:816] 2026-04-10 20:38:20,606 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,606 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,606 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +/home/feng.yulu/dynamic-dpo-v4/scripts/tokenized_dpo_trainer.py:518: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `BetaDPOTrainer.__init__`. Use `processing_class` instead. + super().__init__( +/home/feng.yulu/dynamic-dpo-v4/scripts/tokenized_dpo_trainer.py:518: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `BetaDPOTrainer.__init__`. Use `processing_class` instead. + super().__init__( +/home/feng.yulu/dynamic-dpo-v4/scripts/tokenized_dpo_trainer.py:518: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `BetaDPOTrainer.__init__`. Use `processing_class` instead. + super().__init__( +[WARNING|trainer.py:816] 2026-04-10 20:38:20,606 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +/home/feng.yulu/dynamic-dpo-v4/scripts/tokenized_dpo_trainer.py:518: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `BetaDPOTrainer.__init__`. Use `processing_class` instead. + super().__init__( +[WARNING|trainer.py:816] 2026-04-10 20:38:20,607 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +[WARNING|trainer.py:816] 2026-04-10 20:38:20,607 >> Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead. +/home/feng.yulu/dynamic-dpo-v4/scripts/tokenized_dpo_trainer.py:518: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `BetaDPOTrainer.__init__`. Use `processing_class` instead. + super().__init__( +/home/feng.yulu/dynamic-dpo-v4/scripts/tokenized_dpo_trainer.py:518: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `BetaDPOTrainer.__init__`. Use `processing_class` instead. + super().__init__( +[INFO|trainer.py:748] 2026-04-10 20:38:20,677 >> Using auto half precision backend +/home/feng.yulu/.conda/envs/dpo_venv/lib/python3.11/site-packages/accelerate/accelerator.py:1557: UserWarning: Upcasted low precision parameters in LlamaForCausalLM because mixed precision turned on in FSDP. Affects: model.embed_tokens.weight, model.norm.weight, lm_head.weight. + warnings.warn( +/home/feng.yulu/.conda/envs/dpo_venv/lib/python3.11/site-packages/accelerate/accelerator.py:1557: UserWarning: Upcasted low precision parameters in LlamaDecoderLayer because mixed precision turned on in FSDP. Affects: self_attn.q_proj.weight, self_attn.k_proj.weight, self_attn.v_proj.weight, self_attn.o_proj.weight, mlp.gate_proj.weight, mlp.up_proj.weight, mlp.down_proj.weight, input_layernorm.weight, post_attention_layernorm.weight. + warnings.warn( +/home/feng.yulu/.conda/envs/dpo_venv/lib/python3.11/site-packages/accelerate/accelerator.py:1563: UserWarning: FSDP upcast of low precision parameters may affect the precision of model checkpoints. + warnings.warn( +[INFO|trainer.py:2414] 2026-04-10 20:38:25,651 >> ***** Running training ***** +[INFO|trainer.py:2415] 2026-04-10 20:38:25,651 >> Num examples = 61,135 +[INFO|trainer.py:2416] 2026-04-10 20:38:25,651 >> Num Epochs = 1 +[INFO|trainer.py:2417] 2026-04-10 20:38:25,651 >> Instantaneous batch size per device = 8 +[INFO|trainer.py:2420] 2026-04-10 20:38:25,651 >> Total train batch size (w. parallel, distributed & accumulation) = 128 +[INFO|trainer.py:2421] 2026-04-10 20:38:25,651 >> Gradient Accumulation steps = 2 +[INFO|trainer.py:2422] 2026-04-10 20:38:25,651 >> Total optimization steps = 477 +[INFO|trainer.py:2423] 2026-04-10 20:38:25,652 >> Number of trainable parameters = 1,003,782,656 +[INFO|integration_utils.py:831] 2026-04-10 20:38:25,653 >> Automatic Weights & Biases logging enabled, to disable set os.environ["WANDB_DISABLED"] = "true" +wandb: Currently logged in as: can-not-fand (can-not-fand-northeastern-university). Use `wandb login --relogin` to force relogin +wandb: wandb version 0.25.1 is available! To upgrade, please run: +wandb: $ pip install wandb --upgrade +wandb: Tracking run with wandb version 0.17.5 +wandb: Run data is saved locally in /scratch/feng.yulu/dynamic-dpo-v4/wandb/wandb/run-20260410_203828-i3486sgt +wandb: Run `wandb offline` to turn off syncing. +wandb: Syncing run llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956 +wandb: ⭐️ View project at https://wandb.ai/can-not-fand-northeastern-university/huggingface +wandb: 🚀 View run at https://wandb.ai/can-not-fand-northeastern-university/huggingface/runs/i3486sgt + 0%| | 0/477 [00:00> Could not estimate the number of tokens of the input, floating-point operations will not be computed +[WARNING|modeling_utils.py:1713] 2026-04-10 20:38:37,868 >> Could not estimate the number of tokens of the input, floating-point operations will not be computed +[WARNING|modeling_utils.py:1713] 2026-04-10 20:38:37,874 >> Could not estimate the number of tokens of the input, floating-point operations will not be computed +[WARNING|modeling_utils.py:1713] 2026-04-10 20:38:37,878 >> Could not estimate the number of tokens of the input, floating-point operations will not be computed +[WARNING|modeling_utils.py:1713] 2026-04-10 20:38:37,911 >> Could not estimate the number of tokens of the input, floating-point operations will not be computed +[WARNING|modeling_utils.py:1713] 2026-04-10 20:38:37,924 >> Could not estimate the number of tokens of the input, floating-point operations will not be computed +[WARNING|modeling_utils.py:1713] 2026-04-10 20:38:37,956 >> Could not estimate the number of tokens of the input, floating-point operations will not be computed +[WARNING|modeling_utils.py:1713] 2026-04-10 20:38:37,976 >> Could not estimate the number of tokens of the input, floating-point operations will not be computed + 0%| | 1/477 [00:08<1:07:14, 8.48s/it] {'loss': 1.3869, 'grad_norm': 80.06067657470703, 'learning_rate': 0.0, 'beta_dpo/gap_mean': -0.0031278375536203384, 'beta_dpo/gap_std': 0.09185527265071869, 'beta_dpo/beta_used_raw': 0.10024853050708771, 'beta_dpo/beta_used': 0.10024853050708771, 'beta_dpo/mask_keep_frac': 0.75, 'logits/chosen': -0.6103914976119995, 'logits/rejected': -0.6099507808685303, 'epoch': 0.0} + 0%| | 1/477 [00:08<1:07:14, 8.48s/it] 0%| | 2/477 [00:16<1:02:50, 7.94s/it] 1%| | 3/477 [00:21<54:04, 6.85s/it] 1%| | 4/477 [00:29<56:26, 7.16s/it] 1%| | 5/477 [00:37<58:12, 7.40s/it] 1%|▏ | 6/477 [00:43<54:49, 6.98s/it] 1%|▏ | 7/477 [00:50<54:27, 6.95s/it] 2%|▏ | 8/477 [00:57<54:49, 7.01s/it] 2%|▏ | 9/477 [01:06<1:00:20, 7.74s/it] 2%|▏ | 10/477 [01:14<1:01:39, 7.92s/it] {'loss': 1.386, 'grad_norm': 72.42662811279297, 'learning_rate': 9.375e-08, 'beta_dpo/gap_mean': 0.0029368107207119465, 'beta_dpo/gap_std': 0.47314706444740295, 'beta_dpo/beta_used_raw': 0.10045824944972992, 'beta_dpo/beta_used': 0.10045824944972992, 'beta_dpo/mask_keep_frac': 0.7916666865348816, 'logits/chosen': -0.6866854429244995, 'logits/rejected': -0.668829083442688, 'epoch': 0.02} + 2%|▏ | 10/477 [01:14<1:01:39, 7.92s/it] 2%|▏ | 11/477 [01:22<1:00:01, 7.73s/it] 3%|▎ | 12/477 [01:29<59:01, 7.62s/it] 3%|▎ | 13/477 [01:36<57:01, 7.37s/it] 3%|▎ | 14/477 [01:42<54:09, 7.02s/it] 3%|▎ | 15/477 [01:51<59:35, 7.74s/it] 3%|▎ | 16/477 [01:59<58:59, 7.68s/it] 4%|▎ | 17/477 [02:06<56:40, 7.39s/it] 4%|▍ | 18/477 [02:13<55:11, 7.22s/it] 4%|▍ | 19/477 [02:20<55:27, 7.26s/it] 4%|▍ | 20/477 [02:26<51:49, 6.81s/it] {'loss': 1.3785, 'grad_norm': 77.65188598632812, 'learning_rate': 1.9791666666666664e-07, 'beta_dpo/gap_mean': 0.05031166225671768, 'beta_dpo/gap_std': 0.731455385684967, 'beta_dpo/beta_used_raw': 0.10218687355518341, 'beta_dpo/beta_used': 0.10218687355518341, 'beta_dpo/mask_keep_frac': 0.7749999761581421, 'logits/chosen': -0.6419292688369751, 'logits/rejected': -0.6541769504547119, 'epoch': 0.04} + 4%|▍ | 20/477 [02:26<51:49, 6.81s/it] 4%|▍ | 21/477 [02:33<51:55, 6.83s/it] 5%|▍ | 22/477 [02:40<53:41, 7.08s/it] 5%|▍ | 23/477 [02:47<53:03, 7.01s/it] 5%|▌ | 24/477 [02:54<52:08, 6.91s/it] 5%|▌ | 25/477 [03:01<53:22, 7.09s/it] 5%|▌ | 26/477 [03:10<56:05, 7.46s/it] 6%|▌ | 27/477 [03:16<53:11, 7.09s/it] 6%|▌ | 28/477 [03:23<53:59, 7.21s/it] 6%|▌ | 29/477 [03:29<51:10, 6.85s/it] 6%|▋ | 30/477 [03:37<53:44, 7.21s/it] {'loss': 1.3767, 'grad_norm': 74.03604125976562, 'learning_rate': 3.020833333333333e-07, 'beta_dpo/gap_mean': 0.0937122255563736, 'beta_dpo/gap_std': 0.7656054496765137, 'beta_dpo/beta_used_raw': 0.10061170160770416, 'beta_dpo/beta_used': 0.10061170160770416, 'beta_dpo/mask_keep_frac': 0.8062499761581421, 'logits/chosen': -0.6690393686294556, 'logits/rejected': -0.6756961941719055, 'epoch': 0.06} + 6%|▋ | 30/477 [03:37<53:44, 7.21s/it] 6%|▋ | 31/477 [03:45<54:34, 7.34s/it] 7%|▋ | 32/477 [03:52<53:32, 7.22s/it] 7%|▋ | 33/477 [03:59<52:29, 7.09s/it] 7%|▋ | 34/477 [04:05<50:11, 6.80s/it] 7%|▋ | 35/477 [04:11<49:23, 6.71s/it] 8%|▊ | 36/477 [04:19<52:18, 7.12s/it] 8%|▊ | 37/477 [04:27<53:03, 7.24s/it] 8%|▊ | 38/477 [04:35<53:39, 7.33s/it] 8%|▊ | 39/477 [04:43<56:01, 7.67s/it] 8%|▊ | 40/477 [04:49<53:08, 7.30s/it] {'loss': 1.3467, 'grad_norm': 68.4834976196289, 'learning_rate': 4.0625e-07, 'beta_dpo/gap_mean': 0.3032568395137787, 'beta_dpo/gap_std': 0.9986203908920288, 'beta_dpo/beta_used_raw': 0.10513947159051895, 'beta_dpo/beta_used': 0.10513947159051895, 'beta_dpo/mask_keep_frac': 0.856249988079071, 'logits/chosen': -0.6429699659347534, 'logits/rejected': -0.6495934724807739, 'epoch': 0.08} + 8%|▊ | 40/477 [04:49<53:08, 7.30s/it] 9%|▊ | 41/477 [04:57<52:37, 7.24s/it] 9%|▉ | 42/477 [05:05<54:57, 7.58s/it] 9%|▉ | 43/477 [05:15<59:19, 8.20s/it] 9%|▉ | 44/477 [05:23<1:00:18, 8.36s/it] 9%|▉ | 45/477 [05:31<59:13, 8.23s/it] 10%|▉ | 46/477 [05:40<59:27, 8.28s/it] 10%|▉ | 47/477 [05:45<53:15, 7.43s/it] 10%|█ | 48/477 [05:53<54:02, 7.56s/it] 10%|█ | 49/477 [06:01<54:37, 7.66s/it] 10%|█ | 50/477 [06:10<57:45, 8.12s/it] {'loss': 1.3039, 'grad_norm': 71.59126281738281, 'learning_rate': 4.999932966293553e-07, 'beta_dpo/gap_mean': 0.7923426032066345, 'beta_dpo/gap_std': 1.8291547298431396, 'beta_dpo/beta_used_raw': 0.104192815721035, 'beta_dpo/beta_used': 0.104192815721035, 'beta_dpo/mask_keep_frac': 0.762499988079071, 'logits/chosen': -0.7035672068595886, 'logits/rejected': -0.7120343446731567, 'epoch': 0.1} + 10%|█ | 50/477 [06:10<57:45, 8.12s/it] 11%|█ | 51/477 [06:18<57:46, 8.14s/it] 11%|█ | 52/477 [06:26<58:03, 8.20s/it] 11%|█ | 53/477 [06:35<58:38, 8.30s/it] 11%|█▏ | 54/477 [06:41<53:46, 7.63s/it] 12%|█▏ | 55/477 [06:49<55:15, 7.86s/it] 12%|█▏ | 56/477 [06:56<53:10, 7.58s/it] 12%|█▏ | 57/477 [07:05<55:31, 7.93s/it] 12%|█▏ | 58/477 [07:12<53:57, 7.73s/it] 12%|█▏ | 59/477 [07:19<50:39, 7.27s/it] 13%|█▎ | 60/477 [07:26<51:27, 7.40s/it] {'loss': 1.2274, 'grad_norm': 82.82760620117188, 'learning_rate': 4.991893270335525e-07, 'beta_dpo/gap_mean': 1.5687782764434814, 'beta_dpo/gap_std': 3.4623851776123047, 'beta_dpo/beta_used_raw': 0.10957477241754532, 'beta_dpo/beta_used': 0.10957477241754532, 'beta_dpo/mask_keep_frac': 0.84375, 'logits/chosen': -0.6742374897003174, 'logits/rejected': -0.6726926565170288, 'epoch': 0.13} + 13%|█▎ | 60/477 [07:26<51:27, 7.40s/it] 13%|█▎ | 61/477 [07:34<52:36, 7.59s/it] 13%|█▎ | 62/477 [07:42<52:50, 7.64s/it] 13%|█▎ | 63/477 [07:48<49:14, 7.14s/it] 13%|█▎ | 64/477 [07:55<48:56, 7.11s/it] 14%|█▎ | 65/477 [08:03<49:28, 7.20s/it] 14%|█▍ | 66/477 [08:09<48:44, 7.11s/it] 14%|█▍ | 67/477 [08:18<51:06, 7.48s/it] 14%|█▍ | 68/477 [08:24<47:54, 7.03s/it] 14%|█▍ | 69/477 [08:31<48:34, 7.14s/it] 15%|█▍ | 70/477 [08:39<49:21, 7.28s/it] {'loss': 1.1847, 'grad_norm': 79.24715423583984, 'learning_rate': 4.970496218214204e-07, 'beta_dpo/gap_mean': 2.4878456592559814, 'beta_dpo/gap_std': 5.3841118812561035, 'beta_dpo/beta_used_raw': 0.10323189198970795, 'beta_dpo/beta_used': 0.10323189198970795, 'beta_dpo/mask_keep_frac': 0.737500011920929, 'logits/chosen': -0.7053920030593872, 'logits/rejected': -0.7138158679008484, 'epoch': 0.15} + 15%|█▍ | 70/477 [08:39<49:21, 7.28s/it] 15%|█▍ | 71/477 [08:45<46:19, 6.85s/it] 15%|█▌ | 72/477 [08:53<48:36, 7.20s/it] 15%|█▌ | 73/477 [09:00<49:27, 7.35s/it] 16%|█▌ | 74/477 [09:09<51:15, 7.63s/it] 16%|█▌ | 75/477 [09:16<50:56, 7.60s/it] 16%|█▌ | 76/477 [09:23<49:12, 7.36s/it] 16%|█▌ | 77/477 [09:32<52:19, 7.85s/it] 16%|█▋ | 78/477 [09:41<55:27, 8.34s/it] 17%|█▋ | 79/477 [09:48<51:54, 7.83s/it] 17%|█▋ | 80/477 [09:55<51:01, 7.71s/it] {'loss': 1.1297, 'grad_norm': 40.18954849243164, 'learning_rate': 4.935856505068998e-07, 'beta_dpo/gap_mean': 3.6363892555236816, 'beta_dpo/gap_std': 7.359000205993652, 'beta_dpo/beta_used_raw': 0.10279443114995956, 'beta_dpo/beta_used': 0.10442471504211426, 'beta_dpo/mask_keep_frac': 0.831250011920929, 'logits/chosen': -0.7026282548904419, 'logits/rejected': -0.70656818151474, 'epoch': 0.17} + 17%|█▋ | 80/477 [09:55<51:01, 7.71s/it] 17%|█▋ | 81/477 [10:03<50:20, 7.63s/it] 17%|█▋ | 82/477 [10:11<50:57, 7.74s/it] 17%|█▋ | 83/477 [10:19<50:51, 7.75s/it] 18%|█▊ | 84/477 [10:26<48:58, 7.48s/it] 18%|█▊ | 85/477 [10:34<50:00, 7.65s/it] 18%|█▊ | 86/477 [10:40<47:53, 7.35s/it] 18%|█▊ | 87/477 [10:47<46:42, 7.18s/it] 18%|█▊ | 88/477 [10:54<45:39, 7.04s/it] 19%|█▊ | 89/477 [11:02<47:52, 7.40s/it] 19%|█▉ | 90/477 [11:09<46:52, 7.27s/it] {'loss': 1.1141, 'grad_norm': 45.59261703491211, 'learning_rate': 4.8881598109976e-07, 'beta_dpo/gap_mean': 4.5779619216918945, 'beta_dpo/gap_std': 9.087356567382812, 'beta_dpo/beta_used_raw': 0.0927402526140213, 'beta_dpo/beta_used': 0.09297941625118256, 'beta_dpo/mask_keep_frac': 0.7875000238418579, 'logits/chosen': -0.6874291896820068, 'logits/rejected': -0.7057452201843262, 'epoch': 0.19} + 19%|█▉ | 90/477 [11:09<46:52, 7.27s/it] 19%|█▉ | 91/477 [11:16<46:50, 7.28s/it] 19%|█▉ | 92/477 [11:23<46:22, 7.23s/it] 19%|█▉ | 93/477 [11:31<46:23, 7.25s/it] 20%|█▉ | 94/477 [11:38<46:59, 7.36s/it] 20%|█▉ | 95/477 [11:47<49:10, 7.72s/it] 20%|██ | 96/477 [11:54<48:38, 7.66s/it] 20%|██ | 97/477 [12:01<46:23, 7.33s/it] 21%|██ | 98/477 [12:09<47:18, 7.49s/it] 21%|██ | 99/477 [12:16<46:23, 7.36s/it] 21%|██ | 100/477 [12:24<47:30, 7.56s/it] {'loss': 1.1044, 'grad_norm': 66.85250854492188, 'learning_rate': 4.827661805750437e-07, 'beta_dpo/gap_mean': 5.183230400085449, 'beta_dpo/gap_std': 10.404474258422852, 'beta_dpo/beta_used_raw': 0.10211487114429474, 'beta_dpo/beta_used': 0.10471361875534058, 'beta_dpo/mask_keep_frac': 0.8187500238418579, 'logits/chosen': -0.6732321977615356, 'logits/rejected': -0.6987311840057373, 'epoch': 0.21} + 21%|██ | 100/477 [12:24<47:30, 7.56s/it] 21%|██ | 101/477 [12:30<44:08, 7.04s/it] 21%|██▏ | 102/477 [12:37<44:26, 7.11s/it] 22%|██▏ | 103/477 [12:44<43:54, 7.04s/it] 22%|██▏ | 104/477 [12:51<43:17, 6.96s/it] 22%|██▏ | 105/477 [12:57<42:32, 6.86s/it] 22%|██▏ | 106/477 [13:04<43:06, 6.97s/it] 22%|██▏ | 107/477 [13:13<45:29, 7.38s/it] 23%|██▎ | 108/477 [13:21<46:20, 7.54s/it] 23%|██▎ | 109/477 [13:28<44:55, 7.32s/it] 23%|██▎ | 110/477 [13:35<45:51, 7.50s/it] {'loss': 1.0282, 'grad_norm': 54.56244659423828, 'learning_rate': 4.75468677825789e-07, 'beta_dpo/gap_mean': 6.204737663269043, 'beta_dpo/gap_std': 11.558156967163086, 'beta_dpo/beta_used_raw': 0.1166844591498375, 'beta_dpo/beta_used': 0.1166844591498375, 'beta_dpo/mask_keep_frac': 0.8062499761581421, 'logits/chosen': -0.7261234521865845, 'logits/rejected': -0.7450467348098755, 'epoch': 0.23} + 23%|██▎ | 110/477 [13:35<45:51, 7.50s/it] 23%|██▎ | 111/477 [13:42<44:00, 7.21s/it] 23%|██▎ | 112/477 [13:49<42:45, 7.03s/it] 24%|██▎ | 113/477 [13:55<41:33, 6.85s/it] 24%|██▍ | 114/477 [14:03<42:38, 7.05s/it] 24%|██▍ | 115/477 [14:11<44:57, 7.45s/it] 24%|██▍ | 116/477 [14:18<43:28, 7.23s/it] 25%|██▍ | 117/477 [14:25<43:32, 7.26s/it] 25%|██▍ | 118/477 [14:34<47:20, 7.91s/it] 25%|██▍ | 119/477 [14:41<45:21, 7.60s/it] 25%|██▌ | 120/477 [14:49<45:20, 7.62s/it] {'loss': 1.1069, 'grad_norm': 54.73094940185547, 'learning_rate': 4.669625898336438e-07, 'beta_dpo/gap_mean': 6.777069091796875, 'beta_dpo/gap_std': 12.461393356323242, 'beta_dpo/beta_used_raw': 0.0759856328368187, 'beta_dpo/beta_used': 0.08581940829753876, 'beta_dpo/mask_keep_frac': 0.800000011920929, 'logits/chosen': -0.7630956768989563, 'logits/rejected': -0.776543378829956, 'epoch': 0.25} + 25%|██▌ | 120/477 [14:49<45:20, 7.62s/it] 25%|██▌ | 121/477 [14:56<43:29, 7.33s/it] 26%|██▌ | 122/477 [15:03<43:21, 7.33s/it] 26%|██▌ | 123/477 [15:11<45:20, 7.69s/it] 26%|██▌ | 124/477 [15:19<45:41, 7.77s/it] 26%|██▌ | 125/477 [15:26<43:44, 7.46s/it] 26%|██▋ | 126/477 [15:34<45:08, 7.72s/it] 27%|██▋ | 127/477 [15:41<43:50, 7.52s/it] 27%|██▋ | 128/477 [15:49<44:26, 7.64s/it] 27%|██▋ | 129/477 [15:56<43:16, 7.46s/it] 27%|██▋ | 130/477 [16:02<40:37, 7.02s/it] {'loss': 1.091, 'grad_norm': 53.551025390625, 'learning_rate': 4.5729351198915705e-07, 'beta_dpo/gap_mean': 7.0316290855407715, 'beta_dpo/gap_std': 13.4308500289917, 'beta_dpo/beta_used_raw': 0.09375782310962677, 'beta_dpo/beta_used': 0.10493312776088715, 'beta_dpo/mask_keep_frac': 0.800000011920929, 'logits/chosen': -0.7406284809112549, 'logits/rejected': -0.7330573201179504, 'epoch': 0.27} + 27%|██▋ | 130/477 [16:02<40:37, 7.02s/it] 27%|██▋ | 131/477 [16:10<41:40, 7.23s/it] 28%|██▊ | 132/477 [16:19<43:59, 7.65s/it] 28%|██▊ | 133/477 [16:24<39:47, 6.94s/it] 28%|██▊ | 134/477 [16:32<41:19, 7.23s/it] 28%|██▊ | 135/477 [16:41<44:26, 7.80s/it] 29%|██▊ | 136/477 [16:48<43:05, 7.58s/it] 29%|██▊ | 137/477 [16:57<44:48, 7.91s/it] 29%|██▉ | 138/477 [17:06<46:22, 8.21s/it] 29%|██▉ | 139/477 [17:14<45:51, 8.14s/it] 29%|██▉ | 140/477 [17:22<45:55, 8.18s/it] {'loss': 1.1576, 'grad_norm': 107.44986724853516, 'learning_rate': 4.4651327368569684e-07, 'beta_dpo/gap_mean': 7.776385307312012, 'beta_dpo/gap_std': 14.402565002441406, 'beta_dpo/beta_used_raw': 0.04071963578462601, 'beta_dpo/beta_used': 0.0665307343006134, 'beta_dpo/mask_keep_frac': 0.824999988079071, 'logits/chosen': -0.7388048768043518, 'logits/rejected': -0.7451251745223999, 'epoch': 0.29} + 29%|██▉ | 140/477 [17:22<45:55, 8.18s/it] 30%|██▉ | 141/477 [17:30<46:15, 8.26s/it] 30%|██▉ | 142/477 [17:37<43:58, 7.88s/it] 30%|██▉ | 143/477 [17:45<43:31, 7.82s/it] 30%|███ | 144/477 [17:51<40:12, 7.25s/it] 30%|███ | 145/477 [17:59<41:15, 7.46s/it] 31%|███ | 146/477 [18:06<41:02, 7.44s/it] 31%|███ | 147/477 [18:13<38:45, 7.05s/it] 31%|███ | 148/477 [18:20<39:08, 7.14s/it] 31%|███ | 149/477 [18:26<37:24, 6.84s/it] 31%|███▏ | 150/477 [18:33<37:40, 6.91s/it] {'loss': 1.1224, 'grad_norm': 38.963260650634766, 'learning_rate': 4.346796604970912e-07, 'beta_dpo/gap_mean': 8.364961624145508, 'beta_dpo/gap_std': 14.984090805053711, 'beta_dpo/beta_used_raw': 0.06488198786973953, 'beta_dpo/beta_used': 0.07846825569868088, 'beta_dpo/mask_keep_frac': 0.7875000238418579, 'logits/chosen': -0.768231213092804, 'logits/rejected': -0.7551404237747192, 'epoch': 0.31} + 31%|███▏ | 150/477 [18:33<37:40, 6.91s/it] 32%|███▏ | 151/477 [18:40<37:08, 6.84s/it] 32%|███▏ | 152/477 [18:48<38:46, 7.16s/it] 32%|███▏ | 153/477 [18:56<41:11, 7.63s/it] 32%|███▏ | 154/477 [19:04<41:29, 7.71s/it] 32%|███▏ | 155/477 [19:12<42:08, 7.85s/it] 33%|███▎ | 156/477 [19:20<41:37, 7.78s/it] 33%|███▎ | 157/477 [19:27<40:34, 7.61s/it] 33%|███▎ | 158/477 [19:37<43:59, 8.28s/it] 33%|███▎ | 159/477 [19:44<42:10, 7.96s/it] 34%|███▎ | 160/477 [19:52<41:48, 7.91s/it] {'loss': 1.0544, 'grad_norm': 80.62310028076172, 'learning_rate': 4.218561044282098e-07, 'beta_dpo/gap_mean': 9.785693168640137, 'beta_dpo/gap_std': 15.681970596313477, 'beta_dpo/beta_used_raw': 0.09938563406467438, 'beta_dpo/beta_used': 0.11797045171260834, 'beta_dpo/mask_keep_frac': 0.856249988079071, 'logits/chosen': -0.7575253844261169, 'logits/rejected': -0.7614981532096863, 'epoch': 0.34} + 34%|███▎ | 160/477 [19:52<41:48, 7.91s/it] 34%|███▍ | 161/477 [19:59<40:43, 7.73s/it] 34%|███▍ | 162/477 [20:08<41:06, 7.83s/it] 34%|███▍ | 163/477 [20:17<43:14, 8.26s/it] 34%|███▍ | 164/477 [20:25<43:10, 8.27s/it] 35%|███▍ | 165/477 [20:33<42:08, 8.10s/it] 35%|███▍ | 166/477 [20:41<42:41, 8.24s/it] 35%|███▌ | 167/477 [20:51<44:58, 8.70s/it] 35%|███▌ | 168/477 [20:58<42:31, 8.26s/it] 35%|███▌ | 169/477 [21:04<38:56, 7.59s/it] 36%|███▌ | 170/477 [21:12<39:13, 7.67s/it] {'loss': 1.0875, 'grad_norm': 65.990966796875, 'learning_rate': 4.081113438988443e-07, 'beta_dpo/gap_mean': 10.035483360290527, 'beta_dpo/gap_std': 16.284427642822266, 'beta_dpo/beta_used_raw': 0.04705094173550606, 'beta_dpo/beta_used': 0.07409517467021942, 'beta_dpo/mask_keep_frac': 0.8187500238418579, 'logits/chosen': -0.7660126090049744, 'logits/rejected': -0.7755380868911743, 'epoch': 0.36} + 36%|███▌ | 170/477 [21:12<39:13, 7.67s/it] 36%|███▌ | 171/477 [21:19<37:39, 7.38s/it] 36%|███▌ | 172/477 [21:27<39:09, 7.70s/it] 36%|███▋ | 173/477 [21:34<37:21, 7.37s/it] 36%|███▋ | 174/477 [21:40<35:27, 7.02s/it] 37%|███▋ | 175/477 [21:48<37:02, 7.36s/it] 37%|███▋ | 176/477 [21:55<35:35, 7.10s/it] 37%|███▋ | 177/477 [22:02<35:36, 7.12s/it] 37%|███▋ | 178/477 [22:08<33:57, 6.81s/it] 38%|███▊ | 179/477 [22:16<34:52, 7.02s/it] 38%|███▊ | 180/477 [22:22<34:12, 6.91s/it] {'loss': 1.0689, 'grad_norm': 56.092166900634766, 'learning_rate': 3.935190552834828e-07, 'beta_dpo/gap_mean': 9.977958679199219, 'beta_dpo/gap_std': 16.553037643432617, 'beta_dpo/beta_used_raw': 0.06118815019726753, 'beta_dpo/beta_used': 0.07568483054637909, 'beta_dpo/mask_keep_frac': 0.793749988079071, 'logits/chosen': -0.7195374965667725, 'logits/rejected': -0.7341417074203491, 'epoch': 0.38} + 38%|███▊ | 180/477 [22:22<34:12, 6.91s/it] 38%|███▊ | 181/477 [22:30<35:03, 7.11s/it] 38%|███▊ | 182/477 [22:37<34:23, 6.99s/it] 38%|███▊ | 183/477 [22:45<36:21, 7.42s/it] 39%|███▊ | 184/477 [22:52<35:28, 7.26s/it] 39%|███▉ | 185/477 [22:58<33:59, 6.98s/it] 39%|███▉ | 186/477 [23:06<35:39, 7.35s/it] 39%|███▉ | 187/477 [23:13<34:17, 7.09s/it] 39%|███▉ | 188/477 [23:20<34:19, 7.13s/it] 40%|███▉ | 189/477 [23:29<36:22, 7.58s/it] 40%|███▉ | 190/477 [23:36<35:28, 7.42s/it] {'loss': 1.0703, 'grad_norm': 47.546146392822266, 'learning_rate': 3.781574579820464e-07, 'beta_dpo/gap_mean': 10.884498596191406, 'beta_dpo/gap_std': 17.649686813354492, 'beta_dpo/beta_used_raw': 0.08130989223718643, 'beta_dpo/beta_used': 0.10011672973632812, 'beta_dpo/mask_keep_frac': 0.768750011920929, 'logits/chosen': -0.7710455060005188, 'logits/rejected': -0.783000648021698, 'epoch': 0.4} + 40%|███▉ | 190/477 [23:36<35:28, 7.42s/it] 40%|████ | 191/477 [23:42<34:01, 7.14s/it] 40%|████ | 192/477 [23:49<32:58, 6.94s/it] 40%|████ | 193/477 [23:56<33:33, 7.09s/it] 41%|████ | 194/477 [24:05<36:07, 7.66s/it] 41%|████ | 195/477 [24:12<34:43, 7.39s/it] 41%|████ | 196/477 [24:18<32:25, 6.92s/it] 41%|████▏ | 197/477 [24:25<32:35, 6.98s/it] 42%|████▏ | 198/477 [24:33<33:47, 7.27s/it] 42%|████▏ | 199/477 [24:41<34:49, 7.52s/it] 42%|████▏ | 200/477 [24:48<34:39, 7.51s/it] {'loss': 1.1971, 'grad_norm': 40.988670349121094, 'learning_rate': 3.621088951385353e-07, 'beta_dpo/gap_mean': 10.375402450561523, 'beta_dpo/gap_std': 17.245559692382812, 'beta_dpo/beta_used_raw': 0.01525220274925232, 'beta_dpo/beta_used': 0.03816061466932297, 'beta_dpo/mask_keep_frac': 0.831250011920929, 'logits/chosen': -0.7636905312538147, 'logits/rejected': -0.7812480330467224, 'epoch': 0.42} + 42%|████▏ | 200/477 [24:48<34:39, 7.51s/it][INFO|trainer.py:4307] 2026-04-10 21:03:21,392 >> +***** Running Evaluation ***** +[INFO|trainer.py:4309] 2026-04-10 21:03:21,392 >> Num examples = 2000 +[INFO|trainer.py:4312] 2026-04-10 21:03:21,392 >> Batch size = 8 + + 0%| | 0/31 [00:00> Saving model checkpoint to /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-200 +[INFO|configuration_utils.py:419] 2026-04-10 21:04:27,235 >> Configuration saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-200/config.json +[INFO|configuration_utils.py:911] 2026-04-10 21:04:27,239 >> Configuration saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-200/generation_config.json +[INFO|modeling_utils.py:3580] 2026-04-10 21:05:06,451 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 6 checkpoint shards. You can find where each parameters has been saved in the index located at /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-200/model.safetensors.index.json. +[INFO|tokenization_utils_base.py:2510] 2026-04-10 21:05:06,464 >> tokenizer config file saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-200/tokenizer_config.json +[INFO|tokenization_utils_base.py:2519] 2026-04-10 21:05:06,469 >> Special tokens file saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-200/special_tokens_map.json + 42%|████▏ | 201/477 [29:36<7:01:19, 91.59s/it] 42%|████▏ | 202/477 [29:45<5:05:46, 66.71s/it] 43%|████▎ | 203/477 [29:53<3:45:01, 49.27s/it] 43%|████▎ | 204/477 [30:02<2:48:29, 37.03s/it] 43%|████▎ | 205/477 [30:09<2:06:45, 27.96s/it] 43%|████▎ | 206/477 [30:16<1:38:28, 21.80s/it] 43%|████▎ | 207/477 [30:22<1:16:44, 17.06s/it] 44%|████▎ | 208/477 [30:29<1:02:40, 13.98s/it] 44%|████▍ | 209/477 [30:37<54:55, 12.30s/it] 44%|████▍ | 210/477 [30:45<48:51, 10.98s/it] {'loss': 1.0859, 'grad_norm': 106.01080322265625, 'learning_rate': 3.454593922550693e-07, 'beta_dpo/gap_mean': 11.258265495300293, 'beta_dpo/gap_std': 19.141300201416016, 'beta_dpo/beta_used_raw': 0.09206344187259674, 'beta_dpo/beta_used': 0.09783867746591568, 'beta_dpo/mask_keep_frac': 0.831250011920929, 'logits/chosen': -0.7539916038513184, 'logits/rejected': -0.7599259614944458, 'epoch': 0.44} + 44%|████▍ | 210/477 [30:45<48:51, 10.98s/it] 44%|████▍ | 211/477 [30:54<45:16, 10.21s/it] 44%|████▍ | 212/477 [31:02<43:10, 9.77s/it] 45%|████▍ | 213/477 [31:10<40:05, 9.11s/it] 45%|████▍ | 214/477 [31:18<38:41, 8.83s/it] 45%|████▌ | 215/477 [31:25<36:04, 8.26s/it] 45%|████▌ | 216/477 [31:32<34:12, 7.86s/it] 45%|████▌ | 217/477 [31:40<33:57, 7.84s/it] 46%|████▌ | 218/477 [31:47<32:26, 7.51s/it] 46%|████▌ | 219/477 [31:54<32:43, 7.61s/it] 46%|████▌ | 220/477 [32:02<32:38, 7.62s/it] {'loss': 1.0097, 'grad_norm': 128.11996459960938, 'learning_rate': 3.2829819606729477e-07, 'beta_dpo/gap_mean': 11.77585220336914, 'beta_dpo/gap_std': 19.773366928100586, 'beta_dpo/beta_used_raw': 0.118813656270504, 'beta_dpo/beta_used': 0.13818596303462982, 'beta_dpo/mask_keep_frac': 0.824999988079071, 'logits/chosen': -0.7987761497497559, 'logits/rejected': -0.7768310308456421, 'epoch': 0.46} + 46%|████▌ | 220/477 [32:02<32:38, 7.62s/it] 46%|████▋ | 221/477 [32:10<33:14, 7.79s/it] 47%|████▋ | 222/477 [32:17<32:11, 7.57s/it] 47%|████▋ | 223/477 [32:26<32:57, 7.78s/it] 47%|████▋ | 224/477 [32:35<34:44, 8.24s/it] 47%|████▋ | 225/477 [32:42<33:13, 7.91s/it] 47%|████▋ | 226/477 [32:49<32:20, 7.73s/it] 48%|████▊ | 227/477 [32:56<30:50, 7.40s/it] 48%|████▊ | 228/477 [33:05<32:26, 7.82s/it] 48%|████▊ | 229/477 [33:12<31:13, 7.55s/it] 48%|████▊ | 230/477 [33:18<29:01, 7.05s/it] {'loss': 1.0617, 'grad_norm': 41.492034912109375, 'learning_rate': 3.1071729615293424e-07, 'beta_dpo/gap_mean': 12.928131103515625, 'beta_dpo/gap_std': 20.115745544433594, 'beta_dpo/beta_used_raw': 0.06512973457574844, 'beta_dpo/beta_used': 0.0800265297293663, 'beta_dpo/mask_keep_frac': 0.75, 'logits/chosen': -0.7944627404212952, 'logits/rejected': -0.7826088070869446, 'epoch': 0.48} + 48%|████▊ | 230/477 [33:18<29:01, 7.05s/it] 48%|████▊ | 231/477 [33:24<28:37, 6.98s/it] 49%|████▊ | 232/477 [33:32<29:43, 7.28s/it] 49%|████▉ | 233/477 [33:39<29:17, 7.20s/it] 49%|████▉ | 234/477 [33:45<27:50, 6.87s/it] 49%|████▉ | 235/477 [33:54<29:44, 7.37s/it] 49%|████▉ | 236/477 [34:00<28:31, 7.10s/it] 50%|████▉ | 237/477 [34:08<29:07, 7.28s/it] 50%|████▉ | 238/477 [34:16<29:48, 7.49s/it] 50%|█████ | 239/477 [34:25<31:36, 7.97s/it] 50%|█████ | 240/477 [34:33<31:26, 7.96s/it] {'loss': 1.1275, 'grad_norm': 55.7053108215332, 'learning_rate': 2.9281093183781403e-07, 'beta_dpo/gap_mean': 13.714938163757324, 'beta_dpo/gap_std': 21.715341567993164, 'beta_dpo/beta_used_raw': 0.05508134886622429, 'beta_dpo/beta_used': 0.07821373641490936, 'beta_dpo/mask_keep_frac': 0.7749999761581421, 'logits/chosen': -0.7329837083816528, 'logits/rejected': -0.7595623731613159, 'epoch': 0.5} + 50%|█████ | 240/477 [34:33<31:26, 7.96s/it] 51%|█████ | 241/477 [34:42<31:51, 8.10s/it] 51%|█████ | 242/477 [34:49<31:12, 7.97s/it] 51%|█████ | 243/477 [34:58<32:06, 8.23s/it] 51%|█████ | 244/477 [35:05<30:23, 7.82s/it] 51%|█████▏ | 245/477 [35:12<29:43, 7.69s/it] 52%|█████▏ | 246/477 [35:21<30:33, 7.94s/it] 52%|█████▏ | 247/477 [35:28<29:35, 7.72s/it] 52%|█████▏ | 248/477 [35:36<30:07, 7.89s/it] 52%|█████▏ | 249/477 [35:44<29:33, 7.78s/it] 52%|█████▏ | 250/477 [35:52<29:30, 7.80s/it] {'loss': 1.1019, 'grad_norm': 53.13675308227539, 'learning_rate': 2.7467508704251135e-07, 'beta_dpo/gap_mean': 13.810220718383789, 'beta_dpo/gap_std': 22.46774673461914, 'beta_dpo/beta_used_raw': 0.048361603170633316, 'beta_dpo/beta_used': 0.08778323978185654, 'beta_dpo/mask_keep_frac': 0.800000011920929, 'logits/chosen': -0.787535548210144, 'logits/rejected': -0.7830525636672974, 'epoch': 0.52} + 52%|█████▏ | 250/477 [35:52<29:30, 7.80s/it] 53%|█████▎ | 251/477 [36:00<29:21, 7.80s/it] 53%|█████▎ | 252/477 [36:07<29:08, 7.77s/it] 53%|█████▎ | 253/477 [36:15<28:33, 7.65s/it] 53%|█████▎ | 254/477 [36:22<28:29, 7.67s/it] 53%|█████▎ | 255/477 [36:29<27:40, 7.48s/it] 54%|█████▎ | 256/477 [36:36<26:31, 7.20s/it] 54%|█████▍ | 257/477 [36:43<26:47, 7.31s/it] 54%|█████▍ | 258/477 [36:50<26:06, 7.15s/it] 54%|█████▍ | 259/477 [36:58<26:42, 7.35s/it] 55%|█████▍ | 260/477 [37:04<25:26, 7.04s/it] {'loss': 1.1687, 'grad_norm': 0.9119361042976379, 'learning_rate': 2.5640697577740815e-07, 'beta_dpo/gap_mean': 13.73353099822998, 'beta_dpo/gap_std': 22.698503494262695, 'beta_dpo/beta_used_raw': 0.06594248861074448, 'beta_dpo/beta_used': 0.11194082349538803, 'beta_dpo/mask_keep_frac': 0.824999988079071, 'logits/chosen': -0.7817746996879578, 'logits/rejected': -0.7839881181716919, 'epoch': 0.54} + 55%|█████▍ | 260/477 [37:04<25:26, 7.04s/it] 55%|█████▍ | 261/477 [37:12<25:45, 7.15s/it] 55%|█████▍ | 262/477 [37:19<25:47, 7.20s/it] 55%|█████▌ | 263/477 [37:28<27:18, 7.66s/it] 55%|█████▌ | 264/477 [37:34<25:56, 7.31s/it] 56%|█████▌ | 265/477 [37:42<26:02, 7.37s/it] 56%|█████▌ | 266/477 [37:48<25:04, 7.13s/it] 56%|█████▌ | 267/477 [37:55<24:46, 7.08s/it] 56%|█████▌ | 268/477 [38:02<24:29, 7.03s/it] 56%|█████▋ | 269/477 [38:10<24:58, 7.20s/it] 57%|█████▋ | 270/477 [38:16<23:58, 6.95s/it] {'loss': 1.0209, 'grad_norm': 136.4973602294922, 'learning_rate': 2.381045210440644e-07, 'beta_dpo/gap_mean': 13.976015090942383, 'beta_dpo/gap_std': 22.33526039123535, 'beta_dpo/beta_used_raw': 0.08311768621206284, 'beta_dpo/beta_used': 0.09284855425357819, 'beta_dpo/mask_keep_frac': 0.8187500238418579, 'logits/chosen': -0.7521445155143738, 'logits/rejected': -0.7410815954208374, 'epoch': 0.57} + 57%|█████▋ | 270/477 [38:16<23:58, 6.95s/it] 57%|█████▋ | 271/477 [38:24<24:43, 7.20s/it] 57%|█████▋ | 272/477 [38:31<24:43, 7.24s/it] 57%|█████▋ | 273/477 [38:39<25:28, 7.49s/it] 57%|█████▋ | 274/477 [38:46<24:26, 7.22s/it] 58%|█████▊ | 275/477 [38:55<26:30, 7.87s/it] 58%|█████▊ | 276/477 [39:04<26:48, 8.00s/it] 58%|█████▊ | 277/477 [39:10<25:26, 7.63s/it] 58%|█████▊ | 278/477 [39:19<26:21, 7.95s/it] 58%|█████▊ | 279/477 [39:28<26:36, 8.06s/it] 59%|█████▊ | 280/477 [39:36<26:36, 8.10s/it] {'loss': 1.058, 'grad_norm': 38.58131790161133, 'learning_rate': 2.1986582993616925e-07, 'beta_dpo/gap_mean': 14.858721733093262, 'beta_dpo/gap_std': 22.79940414428711, 'beta_dpo/beta_used_raw': 0.06296978890895844, 'beta_dpo/beta_used': 0.10686023533344269, 'beta_dpo/mask_keep_frac': 0.824999988079071, 'logits/chosen': -0.7521171569824219, 'logits/rejected': -0.7675251364707947, 'epoch': 0.59} + 59%|█████▊ | 280/477 [39:36<26:36, 8.10s/it] 59%|█████▉ | 281/477 [39:43<25:20, 7.76s/it] 59%|█████▉ | 282/477 [39:49<24:13, 7.46s/it] 59%|█████▉ | 283/477 [39:57<24:30, 7.58s/it] 60%|█████▉ | 284/477 [40:05<24:30, 7.62s/it] 60%|█████▉ | 285/477 [40:11<22:45, 7.11s/it] 60%|█████▉ | 286/477 [40:19<23:40, 7.44s/it] 60%|██████ | 287/477 [40:26<23:28, 7.42s/it] 60%|██████ | 288/477 [40:33<22:34, 7.17s/it] 61%|██████ | 289/477 [40:40<22:39, 7.23s/it] 61%|██████ | 290/477 [40:48<23:05, 7.41s/it] {'loss': 1.2126, 'grad_norm': 1.274525761604309, 'learning_rate': 2.0178866775369774e-07, 'beta_dpo/gap_mean': 13.978078842163086, 'beta_dpo/gap_std': 23.335269927978516, 'beta_dpo/beta_used_raw': 0.012238355353474617, 'beta_dpo/beta_used': 0.06835642457008362, 'beta_dpo/mask_keep_frac': 0.8374999761581421, 'logits/chosen': -0.7752319574356079, 'logits/rejected': -0.7829610109329224, 'epoch': 0.61} + 61%|██████ | 290/477 [40:48<23:05, 7.41s/it] 61%|██████ | 291/477 [40:57<23:46, 7.67s/it] 61%|██████ | 292/477 [41:06<24:56, 8.09s/it] 61%|██████▏ | 293/477 [41:12<22:56, 7.48s/it] 62%|██████▏ | 294/477 [41:19<22:56, 7.52s/it] 62%|██████▏ | 295/477 [41:27<22:35, 7.45s/it] 62%|██████▏ | 296/477 [41:35<23:15, 7.71s/it] 62%|██████▏ | 297/477 [41:42<22:55, 7.64s/it] 62%|██████▏ | 298/477 [41:51<23:29, 7.87s/it] 63%|██████▎ | 299/477 [41:59<23:59, 8.09s/it] 63%|██████▎ | 300/477 [42:07<23:06, 7.83s/it] {'loss': 1.1287, 'grad_norm': 60.473148345947266, 'learning_rate': 1.839699339491937e-07, 'beta_dpo/gap_mean': 13.71714973449707, 'beta_dpo/gap_std': 23.238323211669922, 'beta_dpo/beta_used_raw': 0.0673152282834053, 'beta_dpo/beta_used': 0.08970650285482407, 'beta_dpo/mask_keep_frac': 0.75, 'logits/chosen': -0.7769112586975098, 'logits/rejected': -0.7637456655502319, 'epoch': 0.63} + 63%|██████▎ | 300/477 [42:07<23:06, 7.83s/it] 63%|██████▎ | 301/477 [42:14<22:59, 7.84s/it] 63%|██████▎ | 302/477 [42:22<22:53, 7.85s/it] 64%|██████▎ | 303/477 [42:31<23:35, 8.14s/it] 64%|██████▎ | 304/477 [42:39<22:48, 7.91s/it] 64%|██████▍ | 305/477 [42:45<21:51, 7.63s/it] 64%|██████▍ | 306/477 [42:53<21:44, 7.63s/it] 64%|██████▍ | 307/477 [43:00<20:52, 7.37s/it] 65%|██████▍ | 308/477 [43:09<22:11, 7.88s/it] 65%|██████▍ | 309/477 [43:16<21:27, 7.66s/it] 65%|██████▍ | 310/477 [43:24<21:17, 7.65s/it] {'loss': 1.1436, 'grad_norm': 30.574621200561523, 'learning_rate': 1.6650514271527465e-07, 'beta_dpo/gap_mean': 14.4856595993042, 'beta_dpo/gap_std': 23.187442779541016, 'beta_dpo/beta_used_raw': 0.06809216737747192, 'beta_dpo/beta_used': 0.0964554101228714, 'beta_dpo/mask_keep_frac': 0.8125, 'logits/chosen': -0.7852055430412292, 'logits/rejected': -0.7743746638298035, 'epoch': 0.65} + 65%|██████▍ | 310/477 [43:24<21:17, 7.65s/it] 65%|██████▌ | 311/477 [43:30<20:10, 7.29s/it] 65%|██████▌ | 312/477 [43:38<20:11, 7.35s/it] 66%|██████▌ | 313/477 [43:46<20:32, 7.52s/it] 66%|██████▌ | 314/477 [43:52<19:39, 7.24s/it] 66%|██████▌ | 315/477 [43:59<18:51, 6.98s/it] 66%|██████▌ | 316/477 [44:06<19:30, 7.27s/it] 66%|██████▋ | 317/477 [44:15<20:44, 7.78s/it] 67%|██████▋ | 318/477 [44:22<19:54, 7.51s/it] 67%|██████▋ | 319/477 [44:28<18:09, 6.90s/it] 67%|██████▋ | 320/477 [44:36<18:46, 7.17s/it] {'loss': 1.2318, 'grad_norm': 266.17156982421875, 'learning_rate': 1.4948791099758052e-07, 'beta_dpo/gap_mean': 15.27861213684082, 'beta_dpo/gap_std': 23.997211456298828, 'beta_dpo/beta_used_raw': 0.057879697531461716, 'beta_dpo/beta_used': 0.0930468887090683, 'beta_dpo/mask_keep_frac': 0.8125, 'logits/chosen': -0.8031824827194214, 'logits/rejected': -0.7853301763534546, 'epoch': 0.67} + 67%|██████▋ | 320/477 [44:36<18:46, 7.17s/it] 67%|██████▋ | 321/477 [44:43<18:53, 7.27s/it] 68%|██████▊ | 322/477 [44:50<18:12, 7.05s/it] 68%|██████▊ | 323/477 [45:00<20:22, 7.94s/it] 68%|██████▊ | 324/477 [45:07<19:48, 7.77s/it] 68%|██████▊ | 325/477 [45:14<19:23, 7.65s/it] 68%|██████▊ | 326/477 [45:21<18:34, 7.38s/it] 69%|██████▊ | 327/477 [45:29<18:38, 7.46s/it] 69%|██████▉ | 328/477 [45:35<17:54, 7.21s/it] 69%|██████▉ | 329/477 [45:42<17:13, 6.98s/it] 69%|██████▉ | 330/477 [45:49<16:59, 6.93s/it] {'loss': 1.058, 'grad_norm': 54.84642791748047, 'learning_rate': 1.3300945667758012e-07, 'beta_dpo/gap_mean': 15.062555313110352, 'beta_dpo/gap_std': 24.421737670898438, 'beta_dpo/beta_used_raw': 0.05920511484146118, 'beta_dpo/beta_used': 0.08731904625892639, 'beta_dpo/mask_keep_frac': 0.824999988079071, 'logits/chosen': -0.7694008946418762, 'logits/rejected': -0.7609071135520935, 'epoch': 0.69} + 69%|██████▉ | 330/477 [45:49<16:59, 6.93s/it] 69%|██████▉ | 331/477 [45:58<18:23, 7.56s/it] 70%|██████▉ | 332/477 [46:04<17:09, 7.10s/it] 70%|██████▉ | 333/477 [46:11<17:25, 7.26s/it] 70%|███████ | 334/477 [46:21<19:09, 8.04s/it] 70%|███████ | 335/477 [46:28<17:58, 7.59s/it] 70%|███████ | 336/477 [46:35<17:45, 7.55s/it] 71%|███████ | 337/477 [46:42<17:17, 7.41s/it] 71%|███████ | 338/477 [46:49<16:37, 7.18s/it] 71%|███████ | 339/477 [46:55<15:40, 6.81s/it] 71%|███████▏ | 340/477 [47:05<17:41, 7.75s/it] {'loss': 1.1423, 'grad_norm': 162.36752319335938, 'learning_rate': 1.1715810961514072e-07, 'beta_dpo/gap_mean': 15.674954414367676, 'beta_dpo/gap_std': 25.302011489868164, 'beta_dpo/beta_used_raw': 0.04176778346300125, 'beta_dpo/beta_used': 0.07772944122552872, 'beta_dpo/mask_keep_frac': 0.762499988079071, 'logits/chosen': -0.8045889139175415, 'logits/rejected': -0.8078791499137878, 'epoch': 0.71} + 71%|███████▏ | 340/477 [47:05<17:41, 7.75s/it] 71%|███████▏ | 341/477 [47:12<17:00, 7.51s/it] 72%|███████▏ | 342/477 [47:20<17:20, 7.71s/it] 72%|███████▏ | 343/477 [47:27<16:50, 7.54s/it] 72%|███████▏ | 344/477 [47:33<15:55, 7.18s/it] 72%|███████▏ | 345/477 [47:41<15:49, 7.20s/it] 73%|███████▎ | 346/477 [47:47<15:00, 6.87s/it] 73%|███████▎ | 347/477 [47:56<16:16, 7.51s/it] 73%|███████▎ | 348/477 [48:03<15:48, 7.36s/it] 73%|███████▎ | 349/477 [48:10<15:26, 7.24s/it] 73%|███████▎ | 350/477 [48:18<15:46, 7.45s/it] {'loss': 1.1516, 'grad_norm': 122.0066146850586, 'learning_rate': 1.0201883817182949e-07, 'beta_dpo/gap_mean': 15.350746154785156, 'beta_dpo/gap_std': 25.115270614624023, 'beta_dpo/beta_used_raw': 0.03491034358739853, 'beta_dpo/beta_used': 0.09465853869915009, 'beta_dpo/mask_keep_frac': 0.7562500238418579, 'logits/chosen': -0.7979413866996765, 'logits/rejected': -0.8106569051742554, 'epoch': 0.73} + 73%|███████▎ | 350/477 [48:18<15:46, 7.45s/it] 74%|███████▎ | 351/477 [48:24<15:10, 7.23s/it] 74%|███████▍ | 352/477 [48:34<16:20, 7.84s/it] 74%|███████▍ | 353/477 [48:41<15:40, 7.59s/it] 74%|███████▍ | 354/477 [48:48<15:23, 7.51s/it] 74%|███████▍ | 355/477 [48:58<16:39, 8.19s/it] 75%|███████▍ | 356/477 [49:06<16:17, 8.08s/it] 75%|███████▍ | 357/477 [49:12<15:08, 7.57s/it] 75%|███████▌ | 358/477 [49:19<14:23, 7.26s/it] 75%|███████▌ | 359/477 [49:26<14:27, 7.35s/it] 75%|███████▌ | 360/477 [49:33<14:18, 7.34s/it] {'loss': 1.24, 'grad_norm': 93.26220703125, 'learning_rate': 8.76727937529367e-08, 'beta_dpo/gap_mean': 15.205873489379883, 'beta_dpo/gap_std': 25.209131240844727, 'beta_dpo/beta_used_raw': 0.021852362900972366, 'beta_dpo/beta_used': 0.07950497418642044, 'beta_dpo/mask_keep_frac': 0.800000011920929, 'logits/chosen': -0.7563246488571167, 'logits/rejected': -0.7660932540893555, 'epoch': 0.75} + 75%|███████▌ | 360/477 [49:33<14:18, 7.34s/it] 76%|███████▌ | 361/477 [49:42<14:54, 7.71s/it] 76%|███████▌ | 362/477 [49:50<14:54, 7.78s/it] 76%|███████▌ | 363/477 [49:56<13:57, 7.35s/it] 76%|███████▋ | 364/477 [50:03<13:24, 7.12s/it] 77%|███████▋ | 365/477 [50:10<13:33, 7.27s/it] 77%|███████▋ | 366/477 [50:19<14:09, 7.65s/it] 77%|███████▋ | 367/477 [50:27<14:18, 7.81s/it] 77%|███████▋ | 368/477 [50:35<14:07, 7.77s/it] 77%|███████▋ | 369/477 [50:42<13:31, 7.51s/it] 78%|███████▊ | 370/477 [50:49<13:20, 7.48s/it] {'loss': 1.1759, 'grad_norm': 143.22608947753906, 'learning_rate': 7.419687580962222e-08, 'beta_dpo/gap_mean': 16.286312103271484, 'beta_dpo/gap_std': 25.74993896484375, 'beta_dpo/beta_used_raw': 0.05802968889474869, 'beta_dpo/beta_used': 0.10245828330516815, 'beta_dpo/mask_keep_frac': 0.768750011920929, 'logits/chosen': -0.7966378331184387, 'logits/rejected': -0.8195791244506836, 'epoch': 0.77} + 78%|███████▊ | 370/477 [50:49<13:20, 7.48s/it] 78%|███████▊ | 371/477 [50:57<13:13, 7.48s/it] 78%|███████▊ | 372/477 [51:05<13:24, 7.66s/it] 78%|███████▊ | 373/477 [51:12<13:16, 7.66s/it] 78%|███████▊ | 374/477 [51:20<13:18, 7.75s/it] 79%|███████▊ | 375/477 [51:27<12:49, 7.54s/it] 79%|███████▉ | 376/477 [51:35<12:49, 7.62s/it] 79%|███████▉ | 377/477 [51:42<12:01, 7.21s/it] 79%|███████▉ | 378/477 [51:49<11:54, 7.22s/it] 79%|███████▉ | 379/477 [51:55<11:31, 7.06s/it] 80%|███████▉ | 380/477 [52:03<11:47, 7.30s/it] {'loss': 1.2336, 'grad_norm': 36.29342269897461, 'learning_rate': 6.166331963291519e-08, 'beta_dpo/gap_mean': 15.983156204223633, 'beta_dpo/gap_std': 24.809345245361328, 'beta_dpo/beta_used_raw': -0.006214796099811792, 'beta_dpo/beta_used': 0.04838007315993309, 'beta_dpo/mask_keep_frac': 0.7437499761581421, 'logits/chosen': -0.7881544828414917, 'logits/rejected': -0.786669909954071, 'epoch': 0.8} + 80%|███████▉ | 380/477 [52:03<11:47, 7.30s/it] 80%|███████▉ | 381/477 [52:12<12:31, 7.83s/it] 80%|████████ | 382/477 [52:19<11:42, 7.39s/it] 80%|████████ | 383/477 [52:28<12:16, 7.84s/it] 81%|████████ | 384/477 [52:36<12:11, 7.87s/it] 81%|████████ | 385/477 [52:42<11:22, 7.42s/it] 81%|████████ | 386/477 [52:50<11:34, 7.63s/it] 81%|████████ | 387/477 [52:57<10:59, 7.32s/it] 81%|████████▏ | 388/477 [53:03<10:33, 7.11s/it] 82%|████████▏ | 389/477 [53:11<10:34, 7.21s/it] 82%|████████▏ | 390/477 [53:18<10:20, 7.14s/it] {'loss': 1.1986, 'grad_norm': 27.86089324951172, 'learning_rate': 5.013930914912476e-08, 'beta_dpo/gap_mean': 16.157865524291992, 'beta_dpo/gap_std': 25.035715103149414, 'beta_dpo/beta_used_raw': 0.00572154950350523, 'beta_dpo/beta_used': 0.07021647691726685, 'beta_dpo/mask_keep_frac': 0.793749988079071, 'logits/chosen': -0.8044806718826294, 'logits/rejected': -0.8055523633956909, 'epoch': 0.82} + 82%|████████▏ | 390/477 [53:18<10:20, 7.14s/it] 82%|████████▏ | 391/477 [53:24<10:00, 6.98s/it] 82%|████████▏ | 392/477 [53:33<10:39, 7.53s/it] 82%|████████▏ | 393/477 [53:39<09:56, 7.10s/it] 83%|████████▎ | 394/477 [53:46<09:40, 6.99s/it] 83%|████████▎ | 395/477 [53:54<09:51, 7.22s/it] 83%|████████▎ | 396/477 [54:01<09:53, 7.33s/it] 83%|████████▎ | 397/477 [54:08<09:40, 7.25s/it] 83%|████████▎ | 398/477 [54:16<09:50, 7.48s/it] 84%|████████▎ | 399/477 [54:23<09:12, 7.08s/it] 84%|████████▍ | 400/477 [54:28<08:37, 6.72s/it] {'loss': 1.2165, 'grad_norm': 203.63230895996094, 'learning_rate': 3.968661679220467e-08, 'beta_dpo/gap_mean': 16.26091766357422, 'beta_dpo/gap_std': 25.67080307006836, 'beta_dpo/beta_used_raw': 0.04246100038290024, 'beta_dpo/beta_used': 0.0964551717042923, 'beta_dpo/mask_keep_frac': 0.793749988079071, 'logits/chosen': -0.8050006628036499, 'logits/rejected': -0.7917808890342712, 'epoch': 0.84} + 84%|████████▍ | 400/477 [54:28<08:37, 6.72s/it][INFO|trainer.py:4307] 2026-04-10 21:33:01,346 >> +***** Running Evaluation ***** +[INFO|trainer.py:4309] 2026-04-10 21:33:01,346 >> Num examples = 2000 +[INFO|trainer.py:4312] 2026-04-10 21:33:01,346 >> Batch size = 8 + + 0%| | 0/31 [00:00> Saving model checkpoint to /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-400 +[INFO|configuration_utils.py:419] 2026-04-10 21:34:06,758 >> Configuration saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-400/config.json +[INFO|configuration_utils.py:911] 2026-04-10 21:34:06,769 >> Configuration saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-400/generation_config.json +[INFO|modeling_utils.py:3580] 2026-04-10 21:34:46,640 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 6 checkpoint shards. You can find where each parameters has been saved in the index located at /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-400/model.safetensors.index.json. +[INFO|tokenization_utils_base.py:2510] 2026-04-10 21:34:46,673 >> tokenizer config file saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-400/tokenizer_config.json +[INFO|tokenization_utils_base.py:2519] 2026-04-10 21:34:46,689 >> Special tokens file saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-400/special_tokens_map.json + 84%|████████▍ | 401/477 [59:43<2:05:39, 99.20s/it] 84%|████████▍ | 402/477 [59:52<1:29:51, 71.89s/it] 84%|████████▍ | 403/477 [1:00:00<1:05:06, 52.79s/it] 85%|████████▍ | 404/477 [1:00:07<47:45, 39.26s/it] 85%|████████▍ | 405/477 [1:00:14<35:24, 29.51s/it] 85%|████████▌ | 406/477 [1:00:21<26:43, 22.59s/it] 85%|████████▌ | 407/477 [1:00:27<20:49, 17.85s/it] 86%|████████▌ | 408/477 [1:00:35<16:49, 14.62s/it] 86%|████████▌ | 409/477 [1:00:41<13:42, 12.10s/it] 86%|████████▌ | 410/477 [1:00:47<11:33, 10.35s/it] {'loss': 1.2025, 'grad_norm': 82.50851440429688, 'learning_rate': 3.036127238347164e-08, 'beta_dpo/gap_mean': 16.500282287597656, 'beta_dpo/gap_std': 26.050161361694336, 'beta_dpo/beta_used_raw': 0.04236916825175285, 'beta_dpo/beta_used': 0.09968056529760361, 'beta_dpo/mask_keep_frac': 0.793749988079071, 'logits/chosen': -0.827735424041748, 'logits/rejected': -0.8203527331352234, 'epoch': 0.86} + 86%|████████▌ | 410/477 [1:00:47<11:33, 10.35s/it] 86%|████████▌ | 411/477 [1:00:54<10:20, 9.40s/it] 86%|████████▋ | 412/477 [1:01:03<10:08, 9.36s/it] 87%|████████▋ | 413/477 [1:01:12<09:36, 9.01s/it] 87%|████████▋ | 414/477 [1:01:18<08:41, 8.28s/it] 87%|████████▋ | 415/477 [1:01:25<08:10, 7.91s/it] 87%|████████▋ | 416/477 [1:01:33<07:54, 7.78s/it] 87%|████████▋ | 417/477 [1:01:40<07:35, 7.59s/it] 88%|████████▊ | 418/477 [1:01:47<07:19, 7.45s/it] 88%|████████▊ | 419/477 [1:01:54<07:10, 7.42s/it] 88%|████████▊ | 420/477 [1:02:01<06:46, 7.13s/it] {'loss': 1.1919, 'grad_norm': 277.814453125, 'learning_rate': 2.2213262793589482e-08, 'beta_dpo/gap_mean': 16.738262176513672, 'beta_dpo/gap_std': 26.436817169189453, 'beta_dpo/beta_used_raw': 0.058871395885944366, 'beta_dpo/beta_used': 0.0970761626958847, 'beta_dpo/mask_keep_frac': 0.7562500238418579, 'logits/chosen': -0.7558459639549255, 'logits/rejected': -0.7355632185935974, 'epoch': 0.88} + 88%|████████▊ | 420/477 [1:02:01<06:46, 7.13s/it] 88%|████████▊ | 421/477 [1:02:07<06:27, 6.92s/it] 88%|████████▊ | 422/477 [1:02:13<06:05, 6.64s/it] 89%|████████▊ | 423/477 [1:02:20<05:55, 6.59s/it] 89%|████████▉ | 424/477 [1:02:27<05:53, 6.66s/it] 89%|████████▉ | 425/477 [1:02:35<06:07, 7.07s/it] 89%|████████▉ | 426/477 [1:02:41<05:44, 6.76s/it] 90%|████████▉ | 427/477 [1:02:48<05:50, 7.01s/it] 90%|████████▉ | 428/477 [1:02:56<05:55, 7.25s/it] 90%|████████▉ | 429/477 [1:03:03<05:42, 7.13s/it] 90%|█████████ | 430/477 [1:03:10<05:34, 7.12s/it] {'loss': 1.1747, 'grad_norm': 1.1577889919281006, 'learning_rate': 1.5286263996730026e-08, 'beta_dpo/gap_mean': 17.993297576904297, 'beta_dpo/gap_std': 27.201208114624023, 'beta_dpo/beta_used_raw': 0.037638500332832336, 'beta_dpo/beta_used': 0.07494507730007172, 'beta_dpo/mask_keep_frac': 0.84375, 'logits/chosen': -0.8132478594779968, 'logits/rejected': -0.8199571371078491, 'epoch': 0.9} + 90%|█████████ | 430/477 [1:03:10<05:34, 7.12s/it] 90%|█████████ | 431/477 [1:03:19<05:58, 7.79s/it] 91%|█████████ | 432/477 [1:03:26<05:34, 7.43s/it] 91%|█████████ | 433/477 [1:03:34<05:32, 7.56s/it] 91%|█████████ | 434/477 [1:03:40<05:03, 7.06s/it] 91%|█████████ | 435/477 [1:03:47<05:00, 7.15s/it] 91%|█████████▏| 436/477 [1:03:56<05:14, 7.68s/it] 92%|█████████▏| 437/477 [1:04:05<05:20, 8.02s/it] 92%|█████████▏| 438/477 [1:04:13<05:11, 7.99s/it] 92%|█████████▏| 439/477 [1:04:21<05:06, 8.06s/it] 92%|█████████▏| 440/477 [1:04:29<05:02, 8.19s/it] {'loss': 1.2337, 'grad_norm': 8.6950101852417, 'learning_rate': 9.617406953185136e-09, 'beta_dpo/gap_mean': 16.831357955932617, 'beta_dpo/gap_std': 27.087594985961914, 'beta_dpo/beta_used_raw': -0.05054600164294243, 'beta_dpo/beta_used': 0.046172745525836945, 'beta_dpo/mask_keep_frac': 0.793749988079071, 'logits/chosen': -0.8208335638046265, 'logits/rejected': -0.8280296325683594, 'epoch': 0.92} + 92%|█████████▏| 440/477 [1:04:29<05:02, 8.19s/it] 92%|█████████▏| 441/477 [1:04:37<04:46, 7.96s/it] 93%|█████████▎| 442/477 [1:04:44<04:32, 7.78s/it] 93%|█████████▎| 443/477 [1:04:51<04:20, 7.65s/it] 93%|█████████▎| 444/477 [1:04:58<04:01, 7.31s/it] 93%|█████████▎| 445/477 [1:05:05<03:55, 7.34s/it] 94%|█████████▎| 446/477 [1:05:13<03:46, 7.30s/it] 94%|█████████▎| 447/477 [1:05:20<03:41, 7.38s/it] 94%|█████████▍| 448/477 [1:05:26<03:21, 6.94s/it] 94%|█████████▍| 449/477 [1:05:35<03:33, 7.62s/it] 94%|█████████▍| 450/477 [1:05:42<03:19, 7.40s/it] {'loss': 1.171, 'grad_norm': 108.31566619873047, 'learning_rate': 5.2370785753763356e-09, 'beta_dpo/gap_mean': 16.71297264099121, 'beta_dpo/gap_std': 26.49554443359375, 'beta_dpo/beta_used_raw': 0.07020476460456848, 'beta_dpo/beta_used': 0.11133173853158951, 'beta_dpo/mask_keep_frac': 0.824999988079071, 'logits/chosen': -0.7833819389343262, 'logits/rejected': -0.7876101732254028, 'epoch': 0.94} + 94%|█████████▍| 450/477 [1:05:42<03:19, 7.40s/it] 95%|█████████▍| 451/477 [1:05:49<03:04, 7.10s/it] 95%|█████████▍| 452/477 [1:05:56<03:02, 7.31s/it] 95%|█████████▍| 453/477 [1:06:04<02:58, 7.44s/it] 95%|█████████▌| 454/477 [1:06:12<02:54, 7.58s/it] 95%|█████████▌| 455/477 [1:06:19<02:43, 7.42s/it] 96%|█████████▌| 456/477 [1:06:27<02:41, 7.71s/it] 96%|█████████▌| 457/477 [1:06:36<02:38, 7.93s/it] 96%|█████████▌| 458/477 [1:06:43<02:26, 7.72s/it] 96%|█████████▌| 459/477 [1:06:50<02:16, 7.56s/it] 96%|█████████▋| 460/477 [1:06:58<02:09, 7.60s/it] {'loss': 1.2041, 'grad_norm': 70.03536224365234, 'learning_rate': 2.168758844148272e-09, 'beta_dpo/gap_mean': 17.124013900756836, 'beta_dpo/gap_std': 27.718246459960938, 'beta_dpo/beta_used_raw': -0.015655241906642914, 'beta_dpo/beta_used': 0.06652946025133133, 'beta_dpo/mask_keep_frac': 0.8062499761581421, 'logits/chosen': -0.8030775785446167, 'logits/rejected': -0.8030357360839844, 'epoch': 0.96} + 96%|█████████▋| 460/477 [1:06:58<02:09, 7.60s/it] 97%|█████████▋| 461/477 [1:07:06<02:03, 7.70s/it] 97%|█████████▋| 462/477 [1:07:13<01:50, 7.40s/it] 97%|█████████▋| 463/477 [1:07:22<01:49, 7.83s/it] 97%|█████████▋| 464/477 [1:07:28<01:36, 7.44s/it] 97%|█████████▋| 465/477 [1:07:35<01:25, 7.16s/it] 98%|█████████▊| 466/477 [1:07:42<01:18, 7.16s/it] 98%|█████████▊| 467/477 [1:07:50<01:15, 7.60s/it] 98%|█████████▊| 468/477 [1:07:58<01:09, 7.68s/it] 98%|█████████▊| 469/477 [1:08:05<00:58, 7.36s/it] 99%|█████████▊| 470/477 [1:08:12<00:50, 7.28s/it] {'loss': 1.2015, 'grad_norm': 215.3680877685547, 'learning_rate': 4.288949484559934e-10, 'beta_dpo/gap_mean': 17.284704208374023, 'beta_dpo/gap_std': 27.71035385131836, 'beta_dpo/beta_used_raw': 0.10427769273519516, 'beta_dpo/beta_used': 0.12787500023841858, 'beta_dpo/mask_keep_frac': 0.862500011920929, 'logits/chosen': -0.7909310460090637, 'logits/rejected': -0.7838017344474792, 'epoch': 0.98} + 99%|█████████▊| 470/477 [1:08:12<00:50, 7.28s/it] 99%|█████████▊| 471/477 [1:08:20<00:45, 7.51s/it] 99%|█████████▉| 472/477 [1:08:27<00:36, 7.35s/it] 99%|█████████▉| 473/477 [1:08:34<00:28, 7.12s/it] 99%|█████████▉| 474/477 [1:08:41<00:21, 7.08s/it] 100%|█████████▉| 475/477 [1:08:50<00:15, 7.67s/it] 100%|█████████▉| 476/477 [1:08:56<00:07, 7.41s/it] 100%|██████████| 477/477 [1:09:05<00:00, 7.80s/it][INFO|trainer.py:3984] 2026-04-10 21:47:54,818 >> Saving model checkpoint to /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-477 +[INFO|configuration_utils.py:419] 2026-04-10 21:47:54,830 >> Configuration saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-477/config.json +[INFO|configuration_utils.py:911] 2026-04-10 21:47:54,834 >> Configuration saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-477/generation_config.json +[INFO|modeling_utils.py:3580] 2026-04-10 21:48:43,780 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 6 checkpoint shards. You can find where each parameters has been saved in the index located at /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-477/model.safetensors.index.json. +[INFO|tokenization_utils_base.py:2510] 2026-04-10 21:48:43,788 >> tokenizer config file saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-477/tokenizer_config.json +[INFO|tokenization_utils_base.py:2519] 2026-04-10 21:48:43,795 >> Special tokens file saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-477/special_tokens_map.json +[INFO|trainer.py:4083] 2026-04-10 21:52:04,366 >> Deleting older checkpoint [/scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/checkpoint-200] due to args.save_total_limit +[INFO|trainer.py:2681] 2026-04-10 21:52:07,477 >> + +Training completed. Do not forget to share your model on huggingface.co/models =) + + + {'train_runtime': 4421.8255, 'train_samples_per_second': 13.826, 'train_steps_per_second': 0.108, 'train_loss': 1.1642480231431045, 'epoch': 1.0} + 100%|██████████| 477/477 [1:13:35<00:00, 7.80s/it] 100%|██████████| 477/477 [1:13:35<00:00, 9.26s/it] +***** train metrics ***** + epoch = 0.999 + total_flos = 0GF + train_loss = 1.1642 + train_runtime = 1:13:41.82 + train_samples = 61135 + train_samples_per_second = 13.826 + train_steps_per_second = 0.108 +2026-04-10 21:52:07 - INFO - __main__ - *** Training complete *** +2026-04-10 21:52:07 - INFO - __main__ - *** Save model *** +[INFO|configuration_utils.py:419] 2026-04-10 21:52:24,989 >> Configuration saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/config.json +[INFO|configuration_utils.py:911] 2026-04-10 21:52:25,020 >> Configuration saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/generation_config.json +[INFO|modeling_utils.py:3580] 2026-04-10 21:53:15,975 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 7 checkpoint shards. You can find where each parameters has been saved in the index located at /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/model.safetensors.index.json. +[INFO|tokenization_utils_base.py:2510] 2026-04-10 21:53:15,984 >> tokenizer config file saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/tokenizer_config.json +[INFO|tokenization_utils_base.py:2519] 2026-04-10 21:53:15,990 >> Special tokens file saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/special_tokens_map.json +2026-04-10 21:53:16 - INFO - __main__ - Saved HF-compatible model artifacts to /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956 +[INFO|modelcard.py:450] 2026-04-10 21:53:16,252 >> Dropping the following result as it does not have all the necessary fields: +{'dataset': {'name': 'HuggingFaceH4/ultrafeedback_binarized', 'type': 'HuggingFaceH4/ultrafeedback_binarized'}} +[INFO|configuration_utils.py:419] 2026-04-10 21:53:16,263 >> Configuration saved in /scratch/feng.yulu/dynamic-dpo-v4/outputs/llama-3-8b-base-beta-dpo-ultrafeedback-8xh200-20260410-201956/config.json +2026-04-10 21:53:16 - INFO - __main__ - *** Evaluate *** +[INFO|trainer.py:4307] 2026-04-10 21:53:16,264 >> +***** Running Evaluation ***** +[INFO|trainer.py:4309] 2026-04-10 21:53:16,264 >> Num examples = 2000 +[INFO|trainer.py:4312] 2026-04-10 21:53:16,264 >> Batch size = 8 + 0%| | 0/31 [00:00