From d167527f563b4aa86375eb2d9fcd6c5dc64b4edf Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Mon, 20 Jul 2026 23:46:10 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: taozi555/MN-12B-Mag-Mell-R1-KTO Source: Original Platform --- .gitattributes | 36 + .ipynb_checkpoints/README-checkpoint.md | 74 + README.md | 74 + all_results.json | 20 + chat_template.jinja | 7 + config.json | 26 + eval_results.json | 15 + generation_config.json | 7 + model-00001-of-00005.safetensors | 3 + model-00002-of-00005.safetensors | 3 + model-00003-of-00005.safetensors | 3 + model-00004-of-00005.safetensors | 3 + model-00005-of-00005.safetensors | 3 + model.safetensors.index.json | 371 + special_tokens_map.json | 33 + tokenizer.json | 3 + tokenizer_config.json | 8020 ++++++++++++++++++++ train_results.json | 8 + trainer_log.jsonl | 596 ++ trainer_state.json | 8953 +++++++++++++++++++++++ training_args.bin | 3 + training_eval_loss.png | Bin 0 -> 24009 bytes training_loss.png | Bin 0 -> 61008 bytes training_rewards_chosen.png | Bin 0 -> 53405 bytes 24 files changed, 18261 insertions(+) create mode 100644 .gitattributes create mode 100644 .ipynb_checkpoints/README-checkpoint.md create mode 100644 README.md create mode 100644 all_results.json create mode 100644 chat_template.jinja create mode 100644 config.json create mode 100644 eval_results.json create mode 100644 generation_config.json create mode 100644 model-00001-of-00005.safetensors create mode 100644 model-00002-of-00005.safetensors create mode 100644 model-00003-of-00005.safetensors create mode 100644 model-00004-of-00005.safetensors create mode 100644 model-00005-of-00005.safetensors create mode 100644 model.safetensors.index.json create mode 100644 special_tokens_map.json create mode 100644 tokenizer.json create mode 100644 tokenizer_config.json create mode 100644 train_results.json create mode 100644 trainer_log.jsonl create mode 100644 trainer_state.json create mode 100644 training_args.bin create mode 100644 training_eval_loss.png create mode 100644 training_loss.png create mode 100644 training_rewards_chosen.png diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..52373fe --- /dev/null +++ b/.gitattributes @@ -0,0 +1,36 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +tokenizer.json filter=lfs diff=lfs merge=lfs -text diff --git a/.ipynb_checkpoints/README-checkpoint.md b/.ipynb_checkpoints/README-checkpoint.md new file mode 100644 index 0000000..faa5260 --- /dev/null +++ b/.ipynb_checkpoints/README-checkpoint.md @@ -0,0 +1,74 @@ +--- +library_name: transformers +license: other +base_model: inflatebot/MN-12B-Mag-Mell-R1 +tags: +- llama-factory +- full +- generated_from_trainer +model-index: +- name: sft + results: [] +--- + + + +# sft + +This model is a fine-tuned version of [inflatebot/MN-12B-Mag-Mell-R1](https://huggingface.co/inflatebot/MN-12B-Mag-Mell-R1) on the kto_rp dataset. +It achieves the following results on the evaluation set: +- Loss: 0.3763 +- Rewards/chosen: 0.6433 +- Logps/chosen: -219.3819 +- Logits/chosen: -923703389.9450 +- Rewards/rejected: -1.3013 +- Logps/rejected: -231.1543 +- Logits/rejected: -831308124.5957 +- Rewards/margins: 1.9446 +- Kl: 0.0 + +## Model description + +More information needed + +## Intended uses & limitations + +More information needed + +## Training and evaluation data + +More information needed + +## Training procedure + +### Training hyperparameters + +The following hyperparameters were used during training: +- learning_rate: 5e-07 +- train_batch_size: 1 +- eval_batch_size: 1 +- seed: 42 +- distributed_type: multi-GPU +- num_devices: 4 +- gradient_accumulation_steps: 8 +- total_train_batch_size: 32 +- total_eval_batch_size: 4 +- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments +- lr_scheduler_type: cosine +- lr_scheduler_warmup_ratio: 0.1 +- num_epochs: 1.0 + +### Training results + +| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Logps/chosen | Logits/chosen | Rewards/rejected | Logps/rejected | Logits/rejected | Rewards/margins | | +|:-------------:|:------:|:----:|:---------------:|:--------------:|:------------:|:---------------:|:----------------:|:--------------:|:---------------:|:---------------:|:---:| +| 0.3079 | 0.8421 | 500 | 0.3781 | 0.8047 | -217.7675 | -922359676.4771 | -1.0247 | -228.3881 | -826945158.3546 | 1.8294 | 0.0 | + + +### Framework versions + +- Transformers 4.55.0 +- Pytorch 2.5.1+cu124 +- Datasets 3.6.0 +- Tokenizers 0.21.1 diff --git a/README.md b/README.md new file mode 100644 index 0000000..faa5260 --- /dev/null +++ b/README.md @@ -0,0 +1,74 @@ +--- +library_name: transformers +license: other +base_model: inflatebot/MN-12B-Mag-Mell-R1 +tags: +- llama-factory +- full +- generated_from_trainer +model-index: +- name: sft + results: [] +--- + + + +# sft + +This model is a fine-tuned version of [inflatebot/MN-12B-Mag-Mell-R1](https://huggingface.co/inflatebot/MN-12B-Mag-Mell-R1) on the kto_rp dataset. +It achieves the following results on the evaluation set: +- Loss: 0.3763 +- Rewards/chosen: 0.6433 +- Logps/chosen: -219.3819 +- Logits/chosen: -923703389.9450 +- Rewards/rejected: -1.3013 +- Logps/rejected: -231.1543 +- Logits/rejected: -831308124.5957 +- Rewards/margins: 1.9446 +- Kl: 0.0 + +## Model description + +More information needed + +## Intended uses & limitations + +More information needed + +## Training and evaluation data + +More information needed + +## Training procedure + +### Training hyperparameters + +The following hyperparameters were used during training: +- learning_rate: 5e-07 +- train_batch_size: 1 +- eval_batch_size: 1 +- seed: 42 +- distributed_type: multi-GPU +- num_devices: 4 +- gradient_accumulation_steps: 8 +- total_train_batch_size: 32 +- total_eval_batch_size: 4 +- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments +- lr_scheduler_type: cosine +- lr_scheduler_warmup_ratio: 0.1 +- num_epochs: 1.0 + +### Training results + +| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Logps/chosen | Logits/chosen | Rewards/rejected | Logps/rejected | Logits/rejected | Rewards/margins | | +|:-------------:|:------:|:----:|:---------------:|:--------------:|:------------:|:---------------:|:----------------:|:--------------:|:---------------:|:---------------:|:---:| +| 0.3079 | 0.8421 | 500 | 0.3781 | 0.8047 | -217.7675 | -922359676.4771 | -1.0247 | -228.3881 | -826945158.3546 | 1.8294 | 0.0 | + + +### Framework versions + +- Transformers 4.55.0 +- Pytorch 2.5.1+cu124 +- Datasets 3.6.0 +- Tokenizers 0.21.1 diff --git a/all_results.json b/all_results.json new file mode 100644 index 0000000..a9156f0 --- /dev/null +++ b/all_results.json @@ -0,0 +1,20 @@ +{ + "epoch": 1.0, + "eval_logits/chosen": -923703389.9449542, + "eval_logits/rejected": -831308124.5957447, + "eval_logps/chosen": -219.38191657110093, + "eval_logps/rejected": -231.15425531914894, + "eval_loss": 0.37631505727767944, + "eval_rewards/chosen": 0.6432788393913059, + "eval_rewards/margins": 1.9445905758025428, + "eval_rewards/rejected": -1.3013117364112368, + "eval_runtime": 1556.3946, + "eval_samples_per_second": 0.643, + "eval_steps_per_second": 0.161, + "kl": 0.0, + "total_flos": 2.3597531637009613e+17, + "train_loss": 0.4100885201333348, + "train_runtime": 53730.3673, + "train_samples_per_second": 0.354, + "train_steps_per_second": 0.011 +} \ No newline at end of file diff --git a/chat_template.jinja b/chat_template.jinja new file mode 100644 index 0000000..85e8f5d --- /dev/null +++ b/chat_template.jinja @@ -0,0 +1,7 @@ +{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% endif %}{% if system_message is defined %}{{ '<|im_start|>system +' + system_message + '<|im_end|> +' }}{% endif %}{% for message in loop_messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ '<|im_start|>user +' + content + '<|im_end|> +<|im_start|>assistant +' }}{% elif message['role'] == 'assistant' %}{{ content + '<|im_end|> +' }}{% endif %}{% endfor %} \ No newline at end of file diff --git a/config.json b/config.json new file mode 100644 index 0000000..1df92db --- /dev/null +++ b/config.json @@ -0,0 +1,26 @@ +{ + "architectures": [ + "MistralForCausalLM" + ], + "attention_dropout": 0.0, + "bos_token_id": 1, + "eos_token_id": 15, + "head_dim": 128, + "hidden_act": "silu", + "hidden_size": 5120, + "initializer_range": 0.02, + "intermediate_size": 14336, + "max_position_embeddings": 1024000, + "model_type": "mistral", + "num_attention_heads": 32, + "num_hidden_layers": 40, + "num_key_value_heads": 8, + "rms_norm_eps": 1e-05, + "rope_theta": 1000000.0, + "sliding_window": null, + "tie_word_embeddings": false, + "torch_dtype": "bfloat16", + "transformers_version": "4.55.0", + "use_cache": false, + "vocab_size": 131072 +} diff --git a/eval_results.json b/eval_results.json new file mode 100644 index 0000000..daa8fbf --- /dev/null +++ b/eval_results.json @@ -0,0 +1,15 @@ +{ + "epoch": 1.0, + "eval_logits/chosen": -923703389.9449542, + "eval_logits/rejected": -831308124.5957447, + "eval_logps/chosen": -219.38191657110093, + "eval_logps/rejected": -231.15425531914894, + "eval_loss": 0.37631505727767944, + "eval_rewards/chosen": 0.6432788393913059, + "eval_rewards/margins": 1.9445905758025428, + "eval_rewards/rejected": -1.3013117364112368, + "eval_runtime": 1556.3946, + "eval_samples_per_second": 0.643, + "eval_steps_per_second": 0.161, + "kl": 0.0 +} \ No newline at end of file diff --git a/generation_config.json b/generation_config.json new file mode 100644 index 0000000..fa5ee72 --- /dev/null +++ b/generation_config.json @@ -0,0 +1,7 @@ +{ + "_from_model_config": true, + "bos_token_id": 1, + "eos_token_id": 15, + "transformers_version": "4.55.0", + "use_cache": false +} diff --git a/model-00001-of-00005.safetensors b/model-00001-of-00005.safetensors new file mode 100644 index 0000000..f73d862 --- /dev/null +++ b/model-00001-of-00005.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e8c7e75827daee4dce4b4cf9a6f465e9c60f253bb6bf3b74edc0c85726437078 +size 4865522496 diff --git a/model-00002-of-00005.safetensors b/model-00002-of-00005.safetensors new file mode 100644 index 0000000..b92048f --- /dev/null +++ b/model-00002-of-00005.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e74f252c782700a79b4c140d6a93cee415e4dd141017da9522a4d2036670f4bf +size 4907529424 diff --git a/model-00003-of-00005.safetensors b/model-00003-of-00005.safetensors new file mode 100644 index 0000000..0cb344b --- /dev/null +++ b/model-00003-of-00005.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bef3cf9139d3cc7595764651bdfb6ca9823dabd84c15e4216c5dd21d40ce9244 +size 4907529456 diff --git a/model-00004-of-00005.safetensors b/model-00004-of-00005.safetensors new file mode 100644 index 0000000..f46682f --- /dev/null +++ b/model-00004-of-00005.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d86fc077d411eb68f317ed59b13cac57f7b1d4e44e3974abe37775e9a3f20f1b +size 4907529456 diff --git a/model-00005-of-00005.safetensors b/model-00005-of-00005.safetensors new file mode 100644 index 0000000..cccf92e --- /dev/null +++ b/model-00005-of-00005.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f85aec4e0acc6860bc2ca9c2cbd7f80f7318b6dd55d3fcb4ddf897695d1cb672 +size 4907496272 diff --git a/model.safetensors.index.json b/model.safetensors.index.json new file mode 100644 index 0000000..ecc4d30 --- /dev/null +++ b/model.safetensors.index.json @@ -0,0 +1,371 @@ +{ + "metadata": { + "total_parameters": 414720, + "total_size": 24495564800 + }, + "weight_map": { + "lm_head.weight": "model-00005-of-00005.safetensors", + "model.embed_tokens.weight": "model-00001-of-00005.safetensors", + "model.layers.0.input_layernorm.weight": "model-00001-of-00005.safetensors", + "model.layers.0.mlp.down_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.0.mlp.up_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00005.safetensors", + "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.1.input_layernorm.weight": "model-00001-of-00005.safetensors", + "model.layers.1.mlp.down_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.1.mlp.up_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00005.safetensors", + "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.10.input_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.10.mlp.down_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.10.mlp.gate_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.10.mlp.up_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.10.self_attn.o_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.10.self_attn.v_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.11.input_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.11.mlp.down_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.11.mlp.up_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.12.input_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.12.mlp.down_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.12.mlp.gate_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.12.mlp.up_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.12.self_attn.o_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.13.input_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.13.mlp.down_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.13.mlp.gate_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.13.mlp.up_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.13.self_attn.k_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.13.self_attn.q_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.14.input_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.14.mlp.down_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.14.mlp.gate_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.14.mlp.up_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.14.self_attn.k_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.14.self_attn.q_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.15.input_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.15.mlp.down_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.15.mlp.gate_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.15.mlp.up_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.15.post_attention_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.16.input_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.16.mlp.down_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.16.mlp.gate_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.16.mlp.up_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.16.post_attention_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.16.self_attn.k_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.16.self_attn.o_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.16.self_attn.q_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.16.self_attn.v_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.17.input_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.17.mlp.down_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.17.mlp.gate_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.17.mlp.up_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.17.post_attention_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.17.self_attn.k_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.17.self_attn.o_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.17.self_attn.q_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.17.self_attn.v_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.18.input_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.18.mlp.down_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.18.mlp.gate_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.18.mlp.up_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.18.post_attention_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.18.self_attn.k_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.18.self_attn.o_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.18.self_attn.q_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.18.self_attn.v_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.19.input_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.19.mlp.down_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.19.mlp.gate_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.19.mlp.up_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.19.post_attention_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.19.self_attn.k_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.19.self_attn.o_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.19.self_attn.q_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.19.self_attn.v_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.2.input_layernorm.weight": "model-00001-of-00005.safetensors", + "model.layers.2.mlp.down_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.2.mlp.up_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00005.safetensors", + "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.20.input_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.20.mlp.down_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.20.mlp.gate_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.20.mlp.up_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.20.post_attention_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.20.self_attn.k_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.20.self_attn.o_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.20.self_attn.q_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.20.self_attn.v_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.21.input_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.21.mlp.down_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.21.mlp.gate_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.21.mlp.up_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.21.post_attention_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.21.self_attn.k_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.21.self_attn.o_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.21.self_attn.q_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.21.self_attn.v_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.22.input_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.22.mlp.down_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.22.mlp.gate_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.22.mlp.up_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.22.post_attention_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.22.self_attn.k_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.22.self_attn.o_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.22.self_attn.q_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.22.self_attn.v_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.23.input_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.23.mlp.down_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.23.mlp.gate_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.23.mlp.up_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00005.safetensors", + "model.layers.23.self_attn.k_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.23.self_attn.o_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.23.self_attn.q_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.23.self_attn.v_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.24.input_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.24.mlp.down_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.24.mlp.gate_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.24.mlp.up_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.24.post_attention_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.24.self_attn.k_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.24.self_attn.o_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.24.self_attn.q_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.24.self_attn.v_proj.weight": "model-00003-of-00005.safetensors", + "model.layers.25.input_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.25.mlp.down_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.25.mlp.gate_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.25.mlp.up_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.25.post_attention_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.25.self_attn.k_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.25.self_attn.o_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.25.self_attn.q_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.25.self_attn.v_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.26.input_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.26.mlp.down_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.26.mlp.gate_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.26.mlp.up_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.26.post_attention_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.26.self_attn.k_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.26.self_attn.o_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.26.self_attn.q_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.26.self_attn.v_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.27.input_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.27.mlp.down_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.27.mlp.gate_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.27.mlp.up_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.27.post_attention_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.27.self_attn.k_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.27.self_attn.o_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.27.self_attn.q_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.27.self_attn.v_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.28.input_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.28.mlp.down_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.28.mlp.gate_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.28.mlp.up_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.28.post_attention_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.28.self_attn.k_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.28.self_attn.o_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.28.self_attn.q_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.28.self_attn.v_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.29.input_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.29.mlp.down_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.29.mlp.gate_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.29.mlp.up_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.29.post_attention_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.29.self_attn.k_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.29.self_attn.o_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.29.self_attn.q_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.29.self_attn.v_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.3.input_layernorm.weight": "model-00001-of-00005.safetensors", + "model.layers.3.mlp.down_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.3.mlp.up_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00005.safetensors", + "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.30.input_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.30.mlp.down_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.30.mlp.gate_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.30.mlp.up_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.30.post_attention_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.30.self_attn.k_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.30.self_attn.o_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.30.self_attn.q_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.30.self_attn.v_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.31.input_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.31.mlp.down_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.31.mlp.gate_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.31.mlp.up_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.31.post_attention_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.31.self_attn.k_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.31.self_attn.o_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.31.self_attn.q_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.31.self_attn.v_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.32.input_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.32.mlp.down_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.32.mlp.gate_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.32.mlp.up_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.32.post_attention_layernorm.weight": "model-00004-of-00005.safetensors", + "model.layers.32.self_attn.k_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.32.self_attn.o_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.32.self_attn.q_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.32.self_attn.v_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.33.input_layernorm.weight": "model-00005-of-00005.safetensors", + "model.layers.33.mlp.down_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.33.mlp.gate_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.33.mlp.up_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.33.post_attention_layernorm.weight": "model-00005-of-00005.safetensors", + "model.layers.33.self_attn.k_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.33.self_attn.o_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.33.self_attn.q_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.33.self_attn.v_proj.weight": "model-00004-of-00005.safetensors", + "model.layers.34.input_layernorm.weight": "model-00005-of-00005.safetensors", + "model.layers.34.mlp.down_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.34.mlp.gate_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.34.mlp.up_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.34.post_attention_layernorm.weight": "model-00005-of-00005.safetensors", + "model.layers.34.self_attn.k_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.34.self_attn.o_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.34.self_attn.q_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.34.self_attn.v_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.35.input_layernorm.weight": "model-00005-of-00005.safetensors", + "model.layers.35.mlp.down_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.35.mlp.gate_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.35.mlp.up_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.35.post_attention_layernorm.weight": "model-00005-of-00005.safetensors", + "model.layers.35.self_attn.k_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.35.self_attn.o_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.35.self_attn.q_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.35.self_attn.v_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.36.input_layernorm.weight": "model-00005-of-00005.safetensors", + "model.layers.36.mlp.down_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.36.mlp.gate_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.36.mlp.up_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.36.post_attention_layernorm.weight": "model-00005-of-00005.safetensors", + "model.layers.36.self_attn.k_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.36.self_attn.o_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.36.self_attn.q_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.36.self_attn.v_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.37.input_layernorm.weight": "model-00005-of-00005.safetensors", + "model.layers.37.mlp.down_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.37.mlp.gate_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.37.mlp.up_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.37.post_attention_layernorm.weight": "model-00005-of-00005.safetensors", + "model.layers.37.self_attn.k_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.37.self_attn.o_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.37.self_attn.q_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.37.self_attn.v_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.38.input_layernorm.weight": "model-00005-of-00005.safetensors", + "model.layers.38.mlp.down_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.38.mlp.gate_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.38.mlp.up_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.38.post_attention_layernorm.weight": "model-00005-of-00005.safetensors", + "model.layers.38.self_attn.k_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.38.self_attn.o_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.38.self_attn.q_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.38.self_attn.v_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.39.input_layernorm.weight": "model-00005-of-00005.safetensors", + "model.layers.39.mlp.down_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.39.mlp.gate_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.39.mlp.up_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.39.post_attention_layernorm.weight": "model-00005-of-00005.safetensors", + "model.layers.39.self_attn.k_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.39.self_attn.o_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.39.self_attn.q_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.39.self_attn.v_proj.weight": "model-00005-of-00005.safetensors", + "model.layers.4.input_layernorm.weight": "model-00001-of-00005.safetensors", + "model.layers.4.mlp.down_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.4.mlp.up_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00005.safetensors", + "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.5.input_layernorm.weight": "model-00001-of-00005.safetensors", + "model.layers.5.mlp.down_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.5.mlp.up_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00005.safetensors", + "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.6.input_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.6.mlp.down_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.6.mlp.up_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.6.post_attention_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00005.safetensors", + "model.layers.7.input_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.7.mlp.down_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.7.mlp.gate_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.7.mlp.up_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.7.post_attention_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.7.self_attn.k_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.7.self_attn.o_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.7.self_attn.q_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.7.self_attn.v_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.8.input_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.8.mlp.down_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.8.mlp.gate_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.8.mlp.up_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.8.post_attention_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.8.self_attn.k_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.8.self_attn.o_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.8.self_attn.q_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.8.self_attn.v_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.9.input_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.9.mlp.down_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.9.mlp.gate_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.9.mlp.up_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00005.safetensors", + "model.layers.9.self_attn.k_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.9.self_attn.o_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.9.self_attn.q_proj.weight": "model-00002-of-00005.safetensors", + "model.layers.9.self_attn.v_proj.weight": "model-00002-of-00005.safetensors", + "model.norm.weight": "model-00005-of-00005.safetensors" + } +} diff --git a/special_tokens_map.json b/special_tokens_map.json new file mode 100644 index 0000000..1994416 --- /dev/null +++ b/special_tokens_map.json @@ -0,0 +1,33 @@ +{ + "additional_special_tokens": [ + { + "content": "<|im_start|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false + } + ], + "bos_token": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false + }, + "eos_token": { + "content": "<|im_end|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false + }, + "pad_token": "<|im_end|>", + "unk_token": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false + } +} diff --git a/tokenizer.json b/tokenizer.json new file mode 100644 index 0000000..6a1fc9f --- /dev/null +++ b/tokenizer.json @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4b4c8fcd33487a449c07f423d47adb035bba8347ccf13eb074b4d1fef8acf919 +size 17078288 diff --git a/tokenizer_config.json b/tokenizer_config.json new file mode 100644 index 0000000..e56368c --- /dev/null +++ b/tokenizer_config.json @@ -0,0 +1,8020 @@ +{ + "add_bos_token": true, + "add_eos_token": false, + "add_prefix_space": false, + "added_tokens_decoder": { + "0": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "1": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "2": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "3": { + "content": "[INST]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "4": { + "content": "[/INST]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "5": { + "content": "[AVAILABLE_TOOLS]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "6": { + "content": "[/AVAILABLE_TOOLS]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "7": { + "content": "[TOOL_RESULTS]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "8": { + "content": "[/TOOL_RESULTS]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "9": { + "content": "[TOOL_CALLS]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "10": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "11": { + "content": "[PREFIX]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "12": { + "content": "[MIDDLE]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "13": { + "content": "[SUFFIX]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "14": { + "content": "<|im_start|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "15": { + "content": "<|im_end|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "16": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "17": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "18": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "19": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "20": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "21": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "22": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "23": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "24": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "25": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "26": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "27": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "28": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "29": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "30": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "31": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "32": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "33": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "34": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "35": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "36": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "37": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "38": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "39": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "40": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "41": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "42": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "43": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "44": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "45": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "46": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "47": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "48": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "49": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "50": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "51": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "52": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "53": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "54": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "55": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "56": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "57": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "58": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "59": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "60": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "61": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "62": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "63": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "64": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "65": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "66": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "67": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "68": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "69": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "70": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "71": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "72": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "73": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "74": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "75": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "76": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "77": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "78": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "79": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "80": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "81": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "82": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "83": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "84": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "85": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "86": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "87": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "88": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "89": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "90": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "91": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "92": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "93": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "94": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "95": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "96": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "97": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "98": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "99": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "100": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "101": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "102": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "103": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "104": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "105": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "106": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "107": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "108": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "109": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "110": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "111": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "112": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "113": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "114": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "115": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "116": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "117": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "118": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "119": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "120": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "121": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "122": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "123": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "124": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "125": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "126": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "127": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "129": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "130": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "131": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "132": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "133": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "134": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "135": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "136": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "137": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "138": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "139": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "140": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "141": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "142": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "143": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "144": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "145": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "146": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "147": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "148": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "149": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "150": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "152": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "153": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "154": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "155": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "156": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "157": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "158": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "159": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "160": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "161": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "162": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "163": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "164": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "165": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "166": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "167": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "168": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "169": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "170": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "171": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "172": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "173": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "174": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "175": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "176": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "177": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "178": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "179": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "180": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "181": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "182": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "183": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "184": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "185": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "186": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "187": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "188": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "189": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "190": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "191": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "192": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "193": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "194": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "195": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "196": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "197": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "198": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "199": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "200": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "201": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "202": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "203": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "204": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "205": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "206": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "207": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "208": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "209": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "210": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "211": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "212": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "213": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "214": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "215": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "216": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "217": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "218": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "219": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "220": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "221": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "222": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "223": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "224": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "225": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "226": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "227": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "228": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "229": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "230": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "231": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "232": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "233": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "234": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "235": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "236": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "237": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "238": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "239": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "240": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "241": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "242": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "243": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "244": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "245": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "246": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "247": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "248": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "249": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "250": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "251": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "252": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "253": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "254": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "255": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "256": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "257": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "258": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "259": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "260": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "261": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "262": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "263": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "264": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "265": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "266": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "267": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "268": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "269": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "270": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "271": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "272": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "273": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "274": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "275": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "276": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "277": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "278": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "279": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "280": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "281": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "282": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "283": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "284": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "285": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "286": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "287": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "288": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "289": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "290": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "291": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "292": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "293": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "294": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "295": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "296": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "297": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "298": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "299": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "300": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "301": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "302": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "303": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "304": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "305": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "306": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "307": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "308": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "309": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "310": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "311": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "312": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "313": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "314": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "315": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "316": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "317": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "318": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "319": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "320": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "321": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "322": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "323": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "324": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "325": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "326": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "327": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "328": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "329": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "330": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "331": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "332": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "333": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "334": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "335": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "336": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "337": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "338": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "339": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "340": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "341": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "342": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "343": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "344": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "345": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "346": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "347": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "348": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "349": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "350": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "351": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "352": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "353": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "354": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "355": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "356": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "357": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "358": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "359": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "360": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "361": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "362": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "363": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "364": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "365": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "366": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "367": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "368": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "369": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "370": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "371": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "372": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "373": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "374": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "375": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "376": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "377": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "378": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "379": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "380": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "381": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "382": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "383": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "384": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "385": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "386": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "387": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "388": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "389": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "390": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "391": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "392": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "393": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "394": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "395": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "396": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "397": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "398": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "399": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "400": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "401": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "402": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "403": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "404": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "405": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "406": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "407": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "408": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "409": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "410": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "411": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "412": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "413": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "414": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "415": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "416": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "417": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "418": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "419": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "420": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "421": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "422": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "423": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "424": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "425": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "426": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "427": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "428": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "429": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "430": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "431": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "432": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "433": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "434": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "435": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "436": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "437": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "438": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "439": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "440": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "441": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "442": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "443": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "444": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "445": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "446": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "447": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "448": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "449": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "450": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "451": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "452": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "453": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "454": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "455": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "456": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "457": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "458": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "459": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "460": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "461": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "462": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "463": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "464": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "465": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "466": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "467": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "468": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "469": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "470": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "471": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "472": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "473": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "474": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "475": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "476": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "477": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "478": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "479": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "480": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "481": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "482": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "483": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "484": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "485": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "486": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "487": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "488": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "489": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "490": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "491": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "492": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "493": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "494": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "495": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "496": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "497": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "498": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "499": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "500": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "501": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "502": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "503": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "504": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "505": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "506": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "507": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "508": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "509": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "510": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "511": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "512": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "513": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "514": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "515": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "516": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "517": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "518": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "519": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "520": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "521": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "522": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "523": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "524": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "525": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "526": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "527": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "528": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "529": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "530": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "531": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "532": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "533": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "534": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "535": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "536": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "537": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "538": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "539": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "540": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "541": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "542": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "543": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "544": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "545": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "546": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "547": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "548": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "549": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "550": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "551": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "552": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "553": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "554": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "555": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "556": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "557": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "558": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "559": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "560": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "561": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "562": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "563": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "564": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "565": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "566": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "567": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "568": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "569": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "570": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "571": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "572": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "573": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "574": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "575": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "576": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "577": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "578": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "579": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "580": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "581": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "582": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "583": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "584": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "585": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "586": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "587": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "588": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "589": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "590": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "591": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "592": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "593": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "594": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "595": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "596": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "597": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "598": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "599": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "600": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "601": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "602": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "603": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "604": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "605": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "606": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "607": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "608": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "609": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "610": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "611": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "612": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "613": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "614": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "615": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "616": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "617": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "618": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "619": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "620": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "621": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "622": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "623": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "624": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "625": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "626": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "627": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "628": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "629": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "630": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "631": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "632": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "633": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "634": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "635": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "636": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "637": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "638": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "639": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "640": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "641": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "642": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "643": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "644": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "645": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "646": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "647": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "648": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "649": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "650": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "651": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "652": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "653": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "654": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "655": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "656": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "657": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "658": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "659": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "660": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "661": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "662": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "663": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "664": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "665": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "666": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "667": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "668": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "669": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "670": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "671": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "672": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "673": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "674": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "675": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "676": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "677": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "678": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "679": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "680": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "681": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "682": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "683": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "684": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "685": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "686": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "687": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "688": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "689": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "690": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "691": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "692": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "693": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "694": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "695": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "696": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "697": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "698": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "699": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "700": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "701": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "702": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "703": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "704": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "705": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "706": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "707": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "708": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "709": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "710": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "711": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "712": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "713": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "714": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "715": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "716": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "717": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "718": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "719": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "720": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "721": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "722": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "723": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "724": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "725": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "726": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "727": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "728": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "729": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "730": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "731": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "732": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "733": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "734": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "735": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "736": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "737": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "738": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "739": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "740": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "741": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "742": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "743": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "744": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "745": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "746": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "747": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "748": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "749": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "750": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "751": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "752": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "753": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "754": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "755": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "756": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "757": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "758": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "759": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "760": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "761": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "762": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "763": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "764": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "765": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "766": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "767": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "768": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "769": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "770": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "771": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "772": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "773": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "774": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "775": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "776": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "777": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "778": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "779": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "780": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "781": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "782": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "783": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "784": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "785": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "786": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "787": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "788": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "789": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "790": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "791": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "792": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "793": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "794": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "795": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "796": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "797": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "798": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "799": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "800": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "801": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "802": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "803": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "804": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "805": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "806": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "807": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "808": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "809": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "810": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "811": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "812": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "813": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "814": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "815": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "816": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "817": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "818": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "819": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "820": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "821": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "822": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "823": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "824": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "825": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "826": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "827": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "828": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "829": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "830": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "831": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "832": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "833": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "834": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "835": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "836": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "837": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "838": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "839": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "840": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "841": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "842": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "843": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "844": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "845": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "846": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "847": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "848": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "849": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "850": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "851": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "852": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "853": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "854": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "855": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "856": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "857": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "858": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "859": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "860": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "861": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "862": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "863": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "864": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "865": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "866": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "867": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "868": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "869": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "870": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "871": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "872": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "873": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "874": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "875": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "876": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "877": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "878": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "879": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "880": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "881": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "882": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "883": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "884": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "885": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "886": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "887": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "888": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "889": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "890": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "891": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "892": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "893": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "894": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "895": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "896": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "897": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "898": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "899": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "900": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "901": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "902": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "903": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "904": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "905": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "906": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "907": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "908": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "909": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "910": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "911": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "912": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "913": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "914": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "915": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "916": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "917": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "918": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "919": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "920": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "921": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "922": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "923": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "924": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "925": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "926": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "927": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "928": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "929": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "930": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "931": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "932": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "933": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "934": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "935": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "936": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "937": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "938": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "939": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "940": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "941": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "942": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "943": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "944": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "945": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "946": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "947": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "948": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "949": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "950": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "951": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "952": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "953": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "954": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "955": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "956": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "957": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "958": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "959": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "960": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "961": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "962": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "963": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "964": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "965": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "966": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "967": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "968": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "969": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "970": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "971": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "972": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "973": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "974": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "975": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "976": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "977": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "978": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "979": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "980": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "981": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "982": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "983": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "984": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "985": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "986": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "987": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "988": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "989": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "990": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "991": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "992": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "993": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "994": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "995": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "996": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "997": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "998": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "999": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + } + }, + "additional_special_tokens": [ + "<|im_start|>" + ], + "bos_token": "", + "clean_up_tokenization_spaces": false, + "eos_token": "<|im_end|>", + "extra_special_tokens": {}, + "model_max_length": 1000000000000000019884624838656, + "pad_token": "<|im_end|>", + "padding_side": "right", + "split_special_tokens": false, + "tokenizer_class": "PreTrainedTokenizerFast", + "unk_token": "" +} diff --git a/train_results.json b/train_results.json new file mode 100644 index 0000000..fc6e24e --- /dev/null +++ b/train_results.json @@ -0,0 +1,8 @@ +{ + "epoch": 1.0, + "total_flos": 2.3597531637009613e+17, + "train_loss": 0.4100885201333348, + "train_runtime": 53730.3673, + "train_samples_per_second": 0.354, + "train_steps_per_second": 0.011 +} \ No newline at end of file diff --git a/trainer_log.jsonl b/trainer_log.jsonl new file mode 100644 index 0000000..408c428 --- /dev/null +++ b/trainer_log.jsonl @@ -0,0 +1,596 @@ +{"current_steps": 1, "total_steps": 594, "loss": 0.5, "lr": 0.0, "epoch": 0.0016842105263157896, "percentage": 0.17, "elapsed_time": "0:01:35", "remaining_time": "15:42:14"} +{"current_steps": 2, "total_steps": 594, "loss": 0.5, "lr": 8.333333333333334e-09, "epoch": 0.003368421052631579, "percentage": 0.34, "elapsed_time": "0:02:49", "remaining_time": "13:58:18"} +{"current_steps": 3, "total_steps": 594, "loss": 0.4983, "lr": 1.6666666666666667e-08, "epoch": 0.0050526315789473685, "percentage": 0.51, "elapsed_time": "0:04:10", "remaining_time": "13:43:57"} +{"current_steps": 4, "total_steps": 594, "loss": 0.4997, "lr": 2.5e-08, "epoch": 0.006736842105263158, "percentage": 0.67, "elapsed_time": "0:05:34", "remaining_time": "13:43:12"} +{"current_steps": 5, "total_steps": 594, "loss": 0.4999, "lr": 3.3333333333333334e-08, "epoch": 0.008421052631578947, "percentage": 0.84, "elapsed_time": "0:07:01", "remaining_time": "13:46:46"} +{"current_steps": 6, "total_steps": 594, "loss": 0.499, "lr": 4.166666666666666e-08, "epoch": 0.010105263157894737, "percentage": 1.01, "elapsed_time": "0:08:36", "remaining_time": "14:04:20"} +{"current_steps": 7, "total_steps": 594, "loss": 0.5027, "lr": 5e-08, "epoch": 0.011789473684210527, "percentage": 1.18, "elapsed_time": "0:10:18", "remaining_time": "14:24:01"} +{"current_steps": 8, "total_steps": 594, "loss": 0.4996, "lr": 5.833333333333333e-08, "epoch": 0.013473684210526317, "percentage": 1.35, "elapsed_time": "0:11:51", "remaining_time": "14:28:12"} +{"current_steps": 9, "total_steps": 594, "loss": 0.4984, "lr": 6.666666666666667e-08, "epoch": 0.015157894736842105, "percentage": 1.52, "elapsed_time": "0:13:35", "remaining_time": "14:43:39"} +{"current_steps": 10, "total_steps": 594, "loss": 0.502, "lr": 7.5e-08, "epoch": 0.016842105263157894, "percentage": 1.68, "elapsed_time": "0:14:56", "remaining_time": "14:32:36"} +{"current_steps": 11, "total_steps": 594, "loss": 0.4965, "lr": 8.333333333333333e-08, "epoch": 0.018526315789473686, "percentage": 1.85, "elapsed_time": "0:16:44", "remaining_time": "14:47:13"} +{"current_steps": 12, "total_steps": 594, "loss": 0.5027, "lr": 9.166666666666665e-08, "epoch": 0.020210526315789474, "percentage": 2.02, "elapsed_time": "0:18:22", "remaining_time": "14:51:32"} +{"current_steps": 13, "total_steps": 594, "loss": 0.498, "lr": 1e-07, "epoch": 0.021894736842105262, "percentage": 2.19, "elapsed_time": "0:20:08", "remaining_time": "15:00:31"} +{"current_steps": 14, "total_steps": 594, "loss": 0.4979, "lr": 1.0833333333333334e-07, "epoch": 0.023578947368421053, "percentage": 2.36, "elapsed_time": "0:21:35", "remaining_time": "14:54:27"} +{"current_steps": 15, "total_steps": 594, "loss": 0.4991, "lr": 1.1666666666666667e-07, "epoch": 0.02526315789473684, "percentage": 2.53, "elapsed_time": "0:23:03", "remaining_time": "14:50:09"} +{"current_steps": 16, "total_steps": 594, "loss": 0.5015, "lr": 1.25e-07, "epoch": 0.026947368421052633, "percentage": 2.69, "elapsed_time": "0:24:38", "remaining_time": "14:50:12"} +{"current_steps": 17, "total_steps": 594, "loss": 0.4999, "lr": 1.3333333333333334e-07, "epoch": 0.02863157894736842, "percentage": 2.86, "elapsed_time": "0:26:16", "remaining_time": "14:51:51"} +{"current_steps": 18, "total_steps": 594, "loss": 0.5024, "lr": 1.4166666666666665e-07, "epoch": 0.03031578947368421, "percentage": 3.03, "elapsed_time": "0:27:32", "remaining_time": "14:41:20"} +{"current_steps": 19, "total_steps": 594, "loss": 0.5028, "lr": 1.5e-07, "epoch": 0.032, "percentage": 3.2, "elapsed_time": "0:29:02", "remaining_time": "14:38:48"} +{"current_steps": 20, "total_steps": 594, "loss": 0.4997, "lr": 1.583333333333333e-07, "epoch": 0.03368421052631579, "percentage": 3.37, "elapsed_time": "0:31:01", "remaining_time": "14:50:35"} +{"current_steps": 21, "total_steps": 594, "loss": 0.5039, "lr": 1.6666666666666665e-07, "epoch": 0.03536842105263158, "percentage": 3.54, "elapsed_time": "0:32:29", "remaining_time": "14:46:23"} +{"current_steps": 22, "total_steps": 594, "loss": 0.4971, "lr": 1.75e-07, "epoch": 0.03705263157894737, "percentage": 3.7, "elapsed_time": "0:34:26", "remaining_time": "14:55:28"} +{"current_steps": 23, "total_steps": 594, "loss": 0.5003, "lr": 1.833333333333333e-07, "epoch": 0.03873684210526316, "percentage": 3.87, "elapsed_time": "0:35:40", "remaining_time": "14:45:36"} +{"current_steps": 24, "total_steps": 594, "loss": 0.5045, "lr": 1.9166666666666668e-07, "epoch": 0.04042105263157895, "percentage": 4.04, "elapsed_time": "0:37:25", "remaining_time": "14:49:01"} +{"current_steps": 25, "total_steps": 594, "loss": 0.5038, "lr": 2e-07, "epoch": 0.042105263157894736, "percentage": 4.21, "elapsed_time": "0:38:27", "remaining_time": "14:35:20"} +{"current_steps": 26, "total_steps": 594, "loss": 0.4997, "lr": 2.0833333333333333e-07, "epoch": 0.043789473684210524, "percentage": 4.38, "elapsed_time": "0:39:56", "remaining_time": "14:32:30"} +{"current_steps": 27, "total_steps": 594, "loss": 0.4962, "lr": 2.1666666666666667e-07, "epoch": 0.04547368421052632, "percentage": 4.55, "elapsed_time": "0:41:24", "remaining_time": "14:29:29"} +{"current_steps": 28, "total_steps": 594, "loss": 0.504, "lr": 2.25e-07, "epoch": 0.04715789473684211, "percentage": 4.71, "elapsed_time": "0:43:03", "remaining_time": "14:30:23"} +{"current_steps": 29, "total_steps": 594, "loss": 0.4978, "lr": 2.3333333333333333e-07, "epoch": 0.048842105263157895, "percentage": 4.88, "elapsed_time": "0:44:36", "remaining_time": "14:29:09"} +{"current_steps": 30, "total_steps": 594, "loss": 0.4982, "lr": 2.4166666666666665e-07, "epoch": 0.05052631578947368, "percentage": 5.05, "elapsed_time": "0:45:59", "remaining_time": "14:24:45"} +{"current_steps": 31, "total_steps": 594, "loss": 0.4996, "lr": 2.5e-07, "epoch": 0.05221052631578947, "percentage": 5.22, "elapsed_time": "0:47:10", "remaining_time": "14:16:53"} +{"current_steps": 32, "total_steps": 594, "loss": 0.5005, "lr": 2.5833333333333333e-07, "epoch": 0.053894736842105266, "percentage": 5.39, "elapsed_time": "0:49:05", "remaining_time": "14:22:10"} +{"current_steps": 33, "total_steps": 594, "loss": 0.4947, "lr": 2.6666666666666667e-07, "epoch": 0.055578947368421054, "percentage": 5.56, "elapsed_time": "0:50:38", "remaining_time": "14:20:55"} +{"current_steps": 34, "total_steps": 594, "loss": 0.4936, "lr": 2.75e-07, "epoch": 0.05726315789473684, "percentage": 5.72, "elapsed_time": "0:52:02", "remaining_time": "14:17:05"} +{"current_steps": 35, "total_steps": 594, "loss": 0.4999, "lr": 2.833333333333333e-07, "epoch": 0.05894736842105263, "percentage": 5.89, "elapsed_time": "0:53:29", "remaining_time": "14:14:12"} +{"current_steps": 36, "total_steps": 594, "loss": 0.5006, "lr": 2.916666666666667e-07, "epoch": 0.06063157894736842, "percentage": 6.06, "elapsed_time": "0:54:44", "remaining_time": "14:08:32"} +{"current_steps": 37, "total_steps": 594, "loss": 0.4993, "lr": 3e-07, "epoch": 0.06231578947368421, "percentage": 6.23, "elapsed_time": "0:56:06", "remaining_time": "14:04:38"} +{"current_steps": 38, "total_steps": 594, "loss": 0.4959, "lr": 3.0833333333333333e-07, "epoch": 0.064, "percentage": 6.4, "elapsed_time": "0:57:19", "remaining_time": "13:58:47"} +{"current_steps": 39, "total_steps": 594, "loss": 0.4983, "lr": 3.166666666666666e-07, "epoch": 0.06568421052631579, "percentage": 6.57, "elapsed_time": "0:58:38", "remaining_time": "13:54:36"} +{"current_steps": 40, "total_steps": 594, "loss": 0.4953, "lr": 3.25e-07, "epoch": 0.06736842105263158, "percentage": 6.73, "elapsed_time": "0:59:55", "remaining_time": "13:49:53"} +{"current_steps": 41, "total_steps": 594, "loss": 0.4766, "lr": 3.333333333333333e-07, "epoch": 0.06905263157894737, "percentage": 6.9, "elapsed_time": "1:01:27", "remaining_time": "13:48:50"} +{"current_steps": 42, "total_steps": 594, "loss": 0.4928, "lr": 3.4166666666666664e-07, "epoch": 0.07073684210526315, "percentage": 7.07, "elapsed_time": "1:02:40", "remaining_time": "13:43:38"} +{"current_steps": 43, "total_steps": 594, "loss": 0.4969, "lr": 3.5e-07, "epoch": 0.07242105263157894, "percentage": 7.24, "elapsed_time": "1:03:48", "remaining_time": "13:37:31"} +{"current_steps": 44, "total_steps": 594, "loss": 0.4902, "lr": 3.583333333333333e-07, "epoch": 0.07410526315789474, "percentage": 7.41, "elapsed_time": "1:05:08", "remaining_time": "13:34:20"} +{"current_steps": 45, "total_steps": 594, "loss": 0.4888, "lr": 3.666666666666666e-07, "epoch": 0.07578947368421053, "percentage": 7.58, "elapsed_time": "1:06:55", "remaining_time": "13:36:27"} +{"current_steps": 46, "total_steps": 594, "loss": 0.4941, "lr": 3.75e-07, "epoch": 0.07747368421052632, "percentage": 7.74, "elapsed_time": "1:08:33", "remaining_time": "13:36:42"} +{"current_steps": 47, "total_steps": 594, "loss": 0.4947, "lr": 3.8333333333333335e-07, "epoch": 0.07915789473684211, "percentage": 7.91, "elapsed_time": "1:10:33", "remaining_time": "13:41:13"} +{"current_steps": 48, "total_steps": 594, "loss": 0.4972, "lr": 3.9166666666666664e-07, "epoch": 0.0808421052631579, "percentage": 8.08, "elapsed_time": "1:12:01", "remaining_time": "13:39:17"} +{"current_steps": 49, "total_steps": 594, "loss": 0.4913, "lr": 4e-07, "epoch": 0.08252631578947368, "percentage": 8.25, "elapsed_time": "1:13:08", "remaining_time": "13:33:25"} +{"current_steps": 50, "total_steps": 594, "loss": 0.4921, "lr": 4.083333333333333e-07, "epoch": 0.08421052631578947, "percentage": 8.42, "elapsed_time": "1:14:43", "remaining_time": "13:33:02"} +{"current_steps": 51, "total_steps": 594, "loss": 0.5037, "lr": 4.1666666666666667e-07, "epoch": 0.08589473684210526, "percentage": 8.59, "elapsed_time": "1:16:28", "remaining_time": "13:34:09"} +{"current_steps": 52, "total_steps": 594, "loss": 0.5018, "lr": 4.2499999999999995e-07, "epoch": 0.08757894736842105, "percentage": 8.75, "elapsed_time": "1:18:28", "remaining_time": "13:37:56"} +{"current_steps": 53, "total_steps": 594, "loss": 0.4948, "lr": 4.3333333333333335e-07, "epoch": 0.08926315789473684, "percentage": 8.92, "elapsed_time": "1:19:38", "remaining_time": "13:32:58"} +{"current_steps": 54, "total_steps": 594, "loss": 0.4893, "lr": 4.4166666666666664e-07, "epoch": 0.09094736842105264, "percentage": 9.09, "elapsed_time": "1:20:56", "remaining_time": "13:29:25"} +{"current_steps": 55, "total_steps": 594, "loss": 0.4879, "lr": 4.5e-07, "epoch": 0.09263157894736843, "percentage": 9.26, "elapsed_time": "1:22:05", "remaining_time": "13:24:27"} +{"current_steps": 56, "total_steps": 594, "loss": 0.4921, "lr": 4.5833333333333327e-07, "epoch": 0.09431578947368421, "percentage": 9.43, "elapsed_time": "1:23:47", "remaining_time": "13:25:04"} +{"current_steps": 57, "total_steps": 594, "loss": 0.4867, "lr": 4.6666666666666666e-07, "epoch": 0.096, "percentage": 9.6, "elapsed_time": "1:25:06", "remaining_time": "13:21:49"} +{"current_steps": 58, "total_steps": 594, "loss": 0.4985, "lr": 4.7499999999999995e-07, "epoch": 0.09768421052631579, "percentage": 9.76, "elapsed_time": "1:26:41", "remaining_time": "13:21:04"} +{"current_steps": 59, "total_steps": 594, "loss": 0.4757, "lr": 4.833333333333333e-07, "epoch": 0.09936842105263158, "percentage": 9.93, "elapsed_time": "1:28:09", "remaining_time": "13:19:25"} +{"current_steps": 60, "total_steps": 594, "loss": 0.45, "lr": 4.916666666666666e-07, "epoch": 0.10105263157894737, "percentage": 10.1, "elapsed_time": "1:29:33", "remaining_time": "13:17:06"} +{"current_steps": 61, "total_steps": 594, "loss": 0.5044, "lr": 5e-07, "epoch": 0.10273684210526315, "percentage": 10.27, "elapsed_time": "1:31:30", "remaining_time": "13:19:35"} +{"current_steps": 62, "total_steps": 594, "loss": 0.4983, "lr": 4.999956736067562e-07, "epoch": 0.10442105263157894, "percentage": 10.44, "elapsed_time": "1:32:53", "remaining_time": "13:17:05"} +{"current_steps": 63, "total_steps": 594, "loss": 0.5084, "lr": 4.999826945767664e-07, "epoch": 0.10610526315789473, "percentage": 10.61, "elapsed_time": "1:34:08", "remaining_time": "13:13:26"} +{"current_steps": 64, "total_steps": 594, "loss": 0.4639, "lr": 4.999610633592496e-07, "epoch": 0.10778947368421053, "percentage": 10.77, "elapsed_time": "1:35:24", "remaining_time": "13:10:08"} +{"current_steps": 65, "total_steps": 594, "loss": 0.5016, "lr": 4.99930780702887e-07, "epoch": 0.10947368421052632, "percentage": 10.94, "elapsed_time": "1:37:09", "remaining_time": "13:10:43"} +{"current_steps": 66, "total_steps": 594, "loss": 0.4896, "lr": 4.998918476557962e-07, "epoch": 0.11115789473684211, "percentage": 11.11, "elapsed_time": "1:38:31", "remaining_time": "13:08:09"} +{"current_steps": 67, "total_steps": 594, "loss": 0.4936, "lr": 4.998442655654946e-07, "epoch": 0.1128421052631579, "percentage": 11.28, "elapsed_time": "1:39:51", "remaining_time": "13:05:26"} +{"current_steps": 68, "total_steps": 594, "loss": 0.4908, "lr": 4.997880360788526e-07, "epoch": 0.11452631578947368, "percentage": 11.45, "elapsed_time": "1:41:12", "remaining_time": "13:02:51"} +{"current_steps": 69, "total_steps": 594, "loss": 0.4755, "lr": 4.997231611420373e-07, "epoch": 0.11621052631578947, "percentage": 11.62, "elapsed_time": "1:42:41", "remaining_time": "13:01:21"} +{"current_steps": 70, "total_steps": 594, "loss": 0.4798, "lr": 4.996496430004445e-07, "epoch": 0.11789473684210526, "percentage": 11.78, "elapsed_time": "1:44:07", "remaining_time": "12:59:23"} +{"current_steps": 71, "total_steps": 594, "loss": 0.4523, "lr": 4.995674841986217e-07, "epoch": 0.11957894736842105, "percentage": 11.95, "elapsed_time": "1:45:48", "remaining_time": "12:59:24"} +{"current_steps": 72, "total_steps": 594, "loss": 0.5093, "lr": 4.994766875801788e-07, "epoch": 0.12126315789473684, "percentage": 12.12, "elapsed_time": "1:47:28", "remaining_time": "12:59:12"} +{"current_steps": 73, "total_steps": 594, "loss": 0.4554, "lr": 4.993772562876909e-07, "epoch": 0.12294736842105262, "percentage": 12.29, "elapsed_time": "1:48:58", "remaining_time": "12:57:46"} +{"current_steps": 74, "total_steps": 594, "loss": 0.4493, "lr": 4.992691937625891e-07, "epoch": 0.12463157894736843, "percentage": 12.46, "elapsed_time": "1:50:51", "remaining_time": "12:58:59"} +{"current_steps": 75, "total_steps": 594, "loss": 0.4878, "lr": 4.991525037450412e-07, "epoch": 0.12631578947368421, "percentage": 12.63, "elapsed_time": "1:52:17", "remaining_time": "12:57:03"} +{"current_steps": 76, "total_steps": 594, "loss": 0.45, "lr": 4.990271902738222e-07, "epoch": 0.128, "percentage": 12.79, "elapsed_time": "1:53:37", "remaining_time": "12:54:25"} +{"current_steps": 77, "total_steps": 594, "loss": 0.5148, "lr": 4.988932576861752e-07, "epoch": 0.1296842105263158, "percentage": 12.96, "elapsed_time": "1:54:45", "remaining_time": "12:50:31"} +{"current_steps": 78, "total_steps": 594, "loss": 0.4885, "lr": 4.987507106176606e-07, "epoch": 0.13136842105263158, "percentage": 13.13, "elapsed_time": "1:56:17", "remaining_time": "12:49:16"} +{"current_steps": 79, "total_steps": 594, "loss": 0.4488, "lr": 4.985995540019955e-07, "epoch": 0.13305263157894737, "percentage": 13.3, "elapsed_time": "1:57:50", "remaining_time": "12:48:14"} +{"current_steps": 80, "total_steps": 594, "loss": 0.4547, "lr": 4.984397930708837e-07, "epoch": 0.13473684210526315, "percentage": 13.47, "elapsed_time": "1:59:05", "remaining_time": "12:45:08"} +{"current_steps": 81, "total_steps": 594, "loss": 0.502, "lr": 4.982714333538342e-07, "epoch": 0.13642105263157894, "percentage": 13.64, "elapsed_time": "2:00:30", "remaining_time": "12:43:14"} +{"current_steps": 82, "total_steps": 594, "loss": 0.503, "lr": 4.980944806779698e-07, "epoch": 0.13810526315789473, "percentage": 13.8, "elapsed_time": "2:01:39", "remaining_time": "12:39:38"} +{"current_steps": 83, "total_steps": 594, "loss": 0.4614, "lr": 4.979089411678251e-07, "epoch": 0.13978947368421052, "percentage": 13.97, "elapsed_time": "2:03:14", "remaining_time": "12:38:42"} +{"current_steps": 84, "total_steps": 594, "loss": 0.4784, "lr": 4.977148212451354e-07, "epoch": 0.1414736842105263, "percentage": 14.14, "elapsed_time": "2:04:30", "remaining_time": "12:35:54"} +{"current_steps": 85, "total_steps": 594, "loss": 0.4587, "lr": 4.975121276286136e-07, "epoch": 0.1431578947368421, "percentage": 14.31, "elapsed_time": "2:05:52", "remaining_time": "12:33:49"} +{"current_steps": 86, "total_steps": 594, "loss": 0.4545, "lr": 4.97300867333718e-07, "epoch": 0.14484210526315788, "percentage": 14.48, "elapsed_time": "2:07:11", "remaining_time": "12:31:17"} +{"current_steps": 87, "total_steps": 594, "loss": 0.4915, "lr": 4.970810476724096e-07, "epoch": 0.14652631578947367, "percentage": 14.65, "elapsed_time": "2:08:24", "remaining_time": "12:28:17"} +{"current_steps": 88, "total_steps": 594, "loss": 0.4831, "lr": 4.968526762528988e-07, "epoch": 0.1482105263157895, "percentage": 14.81, "elapsed_time": "2:09:44", "remaining_time": "12:25:59"} +{"current_steps": 89, "total_steps": 594, "loss": 0.4829, "lr": 4.96615760979382e-07, "epoch": 0.14989473684210528, "percentage": 14.98, "elapsed_time": "2:10:58", "remaining_time": "12:23:11"} +{"current_steps": 90, "total_steps": 594, "loss": 0.4542, "lr": 4.963703100517683e-07, "epoch": 0.15157894736842106, "percentage": 15.15, "elapsed_time": "2:12:31", "remaining_time": "12:22:08"} +{"current_steps": 91, "total_steps": 594, "loss": 0.4556, "lr": 4.961163319653958e-07, "epoch": 0.15326315789473685, "percentage": 15.32, "elapsed_time": "2:14:09", "remaining_time": "12:21:31"} +{"current_steps": 92, "total_steps": 594, "loss": 0.4021, "lr": 4.958538355107369e-07, "epoch": 0.15494736842105264, "percentage": 15.49, "elapsed_time": "2:15:25", "remaining_time": "12:18:58"} +{"current_steps": 93, "total_steps": 594, "loss": 0.4575, "lr": 4.955828297730948e-07, "epoch": 0.15663157894736843, "percentage": 15.66, "elapsed_time": "2:17:19", "remaining_time": "12:19:48"} +{"current_steps": 94, "total_steps": 594, "loss": 0.4444, "lr": 4.953033241322886e-07, "epoch": 0.15831578947368422, "percentage": 15.82, "elapsed_time": "2:18:35", "remaining_time": "12:17:10"} +{"current_steps": 95, "total_steps": 594, "loss": 0.4272, "lr": 4.950153282623288e-07, "epoch": 0.16, "percentage": 15.99, "elapsed_time": "2:20:19", "remaining_time": "12:17:06"} +{"current_steps": 96, "total_steps": 594, "loss": 0.468, "lr": 4.947188521310827e-07, "epoch": 0.1616842105263158, "percentage": 16.16, "elapsed_time": "2:21:47", "remaining_time": "12:15:33"} +{"current_steps": 97, "total_steps": 594, "loss": 0.4852, "lr": 4.944139059999286e-07, "epoch": 0.16336842105263158, "percentage": 16.33, "elapsed_time": "2:23:07", "remaining_time": "12:13:20"} +{"current_steps": 98, "total_steps": 594, "loss": 0.4712, "lr": 4.941005004234018e-07, "epoch": 0.16505263157894737, "percentage": 16.5, "elapsed_time": "2:24:15", "remaining_time": "12:10:06"} +{"current_steps": 99, "total_steps": 594, "loss": 0.4397, "lr": 4.937786462488283e-07, "epoch": 0.16673684210526316, "percentage": 16.67, "elapsed_time": "2:25:44", "remaining_time": "12:08:40"} +{"current_steps": 100, "total_steps": 594, "loss": 0.4832, "lr": 4.9344835461595e-07, "epoch": 0.16842105263157894, "percentage": 16.84, "elapsed_time": "2:27:29", "remaining_time": "12:08:37"} +{"current_steps": 101, "total_steps": 594, "loss": 0.4423, "lr": 4.93109636956539e-07, "epoch": 0.17010526315789473, "percentage": 17.0, "elapsed_time": "2:28:53", "remaining_time": "12:06:47"} +{"current_steps": 102, "total_steps": 594, "loss": 0.4554, "lr": 4.927625049940012e-07, "epoch": 0.17178947368421052, "percentage": 17.17, "elapsed_time": "2:30:04", "remaining_time": "12:03:51"} +{"current_steps": 103, "total_steps": 594, "loss": 0.4425, "lr": 4.92406970742972e-07, "epoch": 0.1734736842105263, "percentage": 17.34, "elapsed_time": "2:31:52", "remaining_time": "12:03:57"} +{"current_steps": 104, "total_steps": 594, "loss": 0.4575, "lr": 4.920430465088991e-07, "epoch": 0.1751578947368421, "percentage": 17.51, "elapsed_time": "2:33:05", "remaining_time": "12:01:15"} +{"current_steps": 105, "total_steps": 594, "loss": 0.4254, "lr": 4.916707448876173e-07, "epoch": 0.17684210526315788, "percentage": 17.68, "elapsed_time": "2:34:33", "remaining_time": "11:59:49"} +{"current_steps": 106, "total_steps": 594, "loss": 0.4635, "lr": 4.912900787649123e-07, "epoch": 0.17852631578947367, "percentage": 17.85, "elapsed_time": "2:36:01", "remaining_time": "11:58:16"} +{"current_steps": 107, "total_steps": 594, "loss": 0.4916, "lr": 4.90901061316075e-07, "epoch": 0.18021052631578946, "percentage": 18.01, "elapsed_time": "2:37:17", "remaining_time": "11:55:54"} +{"current_steps": 108, "total_steps": 594, "loss": 0.4426, "lr": 4.905037060054449e-07, "epoch": 0.18189473684210528, "percentage": 18.18, "elapsed_time": "2:38:54", "remaining_time": "11:55:05"} +{"current_steps": 109, "total_steps": 594, "loss": 0.424, "lr": 4.900980265859448e-07, "epoch": 0.18357894736842106, "percentage": 18.35, "elapsed_time": "2:40:19", "remaining_time": "11:53:22"} +{"current_steps": 110, "total_steps": 594, "loss": 0.4595, "lr": 4.896840370986042e-07, "epoch": 0.18526315789473685, "percentage": 18.52, "elapsed_time": "2:41:32", "remaining_time": "11:50:46"} +{"current_steps": 111, "total_steps": 594, "loss": 0.4362, "lr": 4.892617518720737e-07, "epoch": 0.18694736842105264, "percentage": 18.69, "elapsed_time": "2:42:46", "remaining_time": "11:48:18"} +{"current_steps": 112, "total_steps": 594, "loss": 0.5149, "lr": 4.888311855221289e-07, "epoch": 0.18863157894736843, "percentage": 18.86, "elapsed_time": "2:44:04", "remaining_time": "11:46:08"} +{"current_steps": 113, "total_steps": 594, "loss": 0.4584, "lr": 4.883923529511646e-07, "epoch": 0.19031578947368422, "percentage": 19.02, "elapsed_time": "2:45:09", "remaining_time": "11:42:59"} +{"current_steps": 114, "total_steps": 594, "loss": 0.4619, "lr": 4.879452693476789e-07, "epoch": 0.192, "percentage": 19.19, "elapsed_time": "2:46:45", "remaining_time": "11:42:09"} +{"current_steps": 115, "total_steps": 594, "loss": 0.4548, "lr": 4.874899501857477e-07, "epoch": 0.1936842105263158, "percentage": 19.36, "elapsed_time": "2:48:03", "remaining_time": "11:40:00"} +{"current_steps": 116, "total_steps": 594, "loss": 0.405, "lr": 4.87026411224489e-07, "epoch": 0.19536842105263158, "percentage": 19.53, "elapsed_time": "2:49:22", "remaining_time": "11:37:57"} +{"current_steps": 117, "total_steps": 594, "loss": 0.431, "lr": 4.865546685075174e-07, "epoch": 0.19705263157894737, "percentage": 19.7, "elapsed_time": "2:50:42", "remaining_time": "11:35:56"} +{"current_steps": 118, "total_steps": 594, "loss": 0.3927, "lr": 4.860747383623889e-07, "epoch": 0.19873684210526316, "percentage": 19.87, "elapsed_time": "2:52:17", "remaining_time": "11:34:59"} +{"current_steps": 119, "total_steps": 594, "loss": 0.4136, "lr": 4.85586637400036e-07, "epoch": 0.20042105263157894, "percentage": 20.03, "elapsed_time": "2:53:25", "remaining_time": "11:32:12"} +{"current_steps": 120, "total_steps": 594, "loss": 0.4317, "lr": 4.85090382514192e-07, "epoch": 0.20210526315789473, "percentage": 20.2, "elapsed_time": "2:55:01", "remaining_time": "11:31:20"} +{"current_steps": 121, "total_steps": 594, "loss": 0.458, "lr": 4.845859908808073e-07, "epoch": 0.20378947368421052, "percentage": 20.37, "elapsed_time": "2:56:08", "remaining_time": "11:28:32"} +{"current_steps": 122, "total_steps": 594, "loss": 0.4281, "lr": 4.840734799574546e-07, "epoch": 0.2054736842105263, "percentage": 20.54, "elapsed_time": "2:58:14", "remaining_time": "11:29:33"} +{"current_steps": 123, "total_steps": 594, "loss": 0.4352, "lr": 4.835528674827239e-07, "epoch": 0.2071578947368421, "percentage": 20.71, "elapsed_time": "2:59:35", "remaining_time": "11:27:42"} +{"current_steps": 124, "total_steps": 594, "loss": 0.3573, "lr": 4.830241714756098e-07, "epoch": 0.20884210526315788, "percentage": 20.88, "elapsed_time": "3:00:58", "remaining_time": "11:25:58"} +{"current_steps": 125, "total_steps": 594, "loss": 0.5246, "lr": 4.82487410234887e-07, "epoch": 0.21052631578947367, "percentage": 21.04, "elapsed_time": "3:02:19", "remaining_time": "11:24:03"} +{"current_steps": 126, "total_steps": 594, "loss": 0.5059, "lr": 4.819426023384769e-07, "epoch": 0.21221052631578946, "percentage": 21.21, "elapsed_time": "3:03:42", "remaining_time": "11:22:21"} +{"current_steps": 127, "total_steps": 594, "loss": 0.4573, "lr": 4.813897666428053e-07, "epoch": 0.21389473684210528, "percentage": 21.38, "elapsed_time": "3:05:12", "remaining_time": "11:21:01"} +{"current_steps": 128, "total_steps": 594, "loss": 0.4897, "lr": 4.80828922282149e-07, "epoch": 0.21557894736842106, "percentage": 21.55, "elapsed_time": "3:06:25", "remaining_time": "11:18:41"} +{"current_steps": 129, "total_steps": 594, "loss": 0.436, "lr": 4.802600886679741e-07, "epoch": 0.21726315789473685, "percentage": 21.72, "elapsed_time": "3:07:45", "remaining_time": "11:16:47"} +{"current_steps": 130, "total_steps": 594, "loss": 0.3727, "lr": 4.79683285488264e-07, "epoch": 0.21894736842105264, "percentage": 21.89, "elapsed_time": "3:09:01", "remaining_time": "11:14:39"} +{"current_steps": 131, "total_steps": 594, "loss": 0.4189, "lr": 4.790985327068375e-07, "epoch": 0.22063157894736843, "percentage": 22.05, "elapsed_time": "3:10:14", "remaining_time": "11:12:23"} +{"current_steps": 132, "total_steps": 594, "loss": 0.4645, "lr": 4.785058505626587e-07, "epoch": 0.22231578947368422, "percentage": 22.22, "elapsed_time": "3:11:50", "remaining_time": "11:11:25"} +{"current_steps": 133, "total_steps": 594, "loss": 0.4986, "lr": 4.779052595691354e-07, "epoch": 0.224, "percentage": 22.39, "elapsed_time": "3:13:33", "remaining_time": "11:10:55"} +{"current_steps": 134, "total_steps": 594, "loss": 0.4261, "lr": 4.772967805134105e-07, "epoch": 0.2256842105263158, "percentage": 22.56, "elapsed_time": "3:14:57", "remaining_time": "11:09:14"} +{"current_steps": 135, "total_steps": 594, "loss": 0.4773, "lr": 4.766804344556413e-07, "epoch": 0.22736842105263158, "percentage": 22.73, "elapsed_time": "3:16:24", "remaining_time": "11:07:46"} +{"current_steps": 136, "total_steps": 594, "loss": 0.486, "lr": 4.760562427282712e-07, "epoch": 0.22905263157894737, "percentage": 22.9, "elapsed_time": "3:17:58", "remaining_time": "11:06:43"} +{"current_steps": 137, "total_steps": 594, "loss": 0.3863, "lr": 4.754242269352911e-07, "epoch": 0.23073684210526316, "percentage": 23.06, "elapsed_time": "3:19:59", "remaining_time": "11:07:06"} +{"current_steps": 138, "total_steps": 594, "loss": 0.4322, "lr": 4.747844089514919e-07, "epoch": 0.23242105263157894, "percentage": 23.23, "elapsed_time": "3:21:16", "remaining_time": "11:05:04"} +{"current_steps": 139, "total_steps": 594, "loss": 0.4026, "lr": 4.741368109217071e-07, "epoch": 0.23410526315789473, "percentage": 23.4, "elapsed_time": "3:22:37", "remaining_time": "11:03:16"} +{"current_steps": 140, "total_steps": 594, "loss": 0.4317, "lr": 4.734814552600468e-07, "epoch": 0.23578947368421052, "percentage": 23.57, "elapsed_time": "3:23:47", "remaining_time": "11:00:52"} +{"current_steps": 141, "total_steps": 594, "loss": 0.3808, "lr": 4.728183646491214e-07, "epoch": 0.2374736842105263, "percentage": 23.74, "elapsed_time": "3:24:56", "remaining_time": "10:58:26"} +{"current_steps": 142, "total_steps": 594, "loss": 0.4569, "lr": 4.721475620392567e-07, "epoch": 0.2391578947368421, "percentage": 23.91, "elapsed_time": "3:26:17", "remaining_time": "10:56:38"} +{"current_steps": 143, "total_steps": 594, "loss": 0.4128, "lr": 4.7146907064769993e-07, "epoch": 0.24084210526315789, "percentage": 24.07, "elapsed_time": "3:27:32", "remaining_time": "10:54:34"} +{"current_steps": 144, "total_steps": 594, "loss": 0.4001, "lr": 4.707829139578155e-07, "epoch": 0.24252631578947367, "percentage": 24.24, "elapsed_time": "3:29:00", "remaining_time": "10:53:09"} +{"current_steps": 145, "total_steps": 594, "loss": 0.4857, "lr": 4.7008911571827284e-07, "epoch": 0.24421052631578946, "percentage": 24.41, "elapsed_time": "3:30:22", "remaining_time": "10:51:25"} +{"current_steps": 146, "total_steps": 594, "loss": 0.358, "lr": 4.6938769994222406e-07, "epoch": 0.24589473684210525, "percentage": 24.58, "elapsed_time": "3:31:31", "remaining_time": "10:49:02"} +{"current_steps": 147, "total_steps": 594, "loss": 0.421, "lr": 4.686786909064729e-07, "epoch": 0.24757894736842107, "percentage": 24.75, "elapsed_time": "3:32:49", "remaining_time": "10:47:09"} +{"current_steps": 148, "total_steps": 594, "loss": 0.4676, "lr": 4.679621131506346e-07, "epoch": 0.24926315789473685, "percentage": 24.92, "elapsed_time": "3:34:41", "remaining_time": "10:46:57"} +{"current_steps": 149, "total_steps": 594, "loss": 0.4862, "lr": 4.6723799147628664e-07, "epoch": 0.25094736842105264, "percentage": 25.08, "elapsed_time": "3:35:56", "remaining_time": "10:44:54"} +{"current_steps": 150, "total_steps": 594, "loss": 0.4654, "lr": 4.6650635094610966e-07, "epoch": 0.25263157894736843, "percentage": 25.25, "elapsed_time": "3:37:14", "remaining_time": "10:43:02"} +{"current_steps": 151, "total_steps": 594, "loss": 0.4548, "lr": 4.6576721688302103e-07, "epoch": 0.2543157894736842, "percentage": 25.42, "elapsed_time": "3:38:30", "remaining_time": "10:41:03"} +{"current_steps": 152, "total_steps": 594, "loss": 0.4583, "lr": 4.6502061486929765e-07, "epoch": 0.256, "percentage": 25.59, "elapsed_time": "3:40:06", "remaining_time": "10:40:03"} +{"current_steps": 153, "total_steps": 594, "loss": 0.4257, "lr": 4.6426657074569076e-07, "epoch": 0.2576842105263158, "percentage": 25.76, "elapsed_time": "3:41:29", "remaining_time": "10:38:24"} +{"current_steps": 154, "total_steps": 594, "loss": 0.4011, "lr": 4.6350511061053155e-07, "epoch": 0.2593684210526316, "percentage": 25.93, "elapsed_time": "3:42:46", "remaining_time": "10:36:28"} +{"current_steps": 155, "total_steps": 594, "loss": 0.4867, "lr": 4.62736260818828e-07, "epoch": 0.26105263157894737, "percentage": 26.09, "elapsed_time": "3:44:25", "remaining_time": "10:35:37"} +{"current_steps": 156, "total_steps": 594, "loss": 0.4601, "lr": 4.6196004798135236e-07, "epoch": 0.26273684210526316, "percentage": 26.26, "elapsed_time": "3:45:34", "remaining_time": "10:33:22"} +{"current_steps": 157, "total_steps": 594, "loss": 0.5013, "lr": 4.611764989637205e-07, "epoch": 0.26442105263157895, "percentage": 26.43, "elapsed_time": "3:46:56", "remaining_time": "10:31:39"} +{"current_steps": 158, "total_steps": 594, "loss": 0.4991, "lr": 4.603856408854618e-07, "epoch": 0.26610526315789473, "percentage": 26.6, "elapsed_time": "3:48:02", "remaining_time": "10:29:17"} +{"current_steps": 159, "total_steps": 594, "loss": 0.502, "lr": 4.5958750111908065e-07, "epoch": 0.2677894736842105, "percentage": 26.77, "elapsed_time": "3:49:26", "remaining_time": "10:27:43"} +{"current_steps": 160, "total_steps": 594, "loss": 0.4662, "lr": 4.5878210728910886e-07, "epoch": 0.2694736842105263, "percentage": 26.94, "elapsed_time": "3:50:49", "remaining_time": "10:26:07"} +{"current_steps": 161, "total_steps": 594, "loss": 0.3873, "lr": 4.5796948727115e-07, "epoch": 0.2711578947368421, "percentage": 27.1, "elapsed_time": "3:51:57", "remaining_time": "10:23:51"} +{"current_steps": 162, "total_steps": 594, "loss": 0.398, "lr": 4.5714966919091414e-07, "epoch": 0.2728421052631579, "percentage": 27.27, "elapsed_time": "3:53:32", "remaining_time": "10:22:46"} +{"current_steps": 163, "total_steps": 594, "loss": 0.3492, "lr": 4.5632268142324435e-07, "epoch": 0.2745263157894737, "percentage": 27.44, "elapsed_time": "3:55:05", "remaining_time": "10:21:37"} +{"current_steps": 164, "total_steps": 594, "loss": 0.4335, "lr": 4.554885525911351e-07, "epoch": 0.27621052631578946, "percentage": 27.61, "elapsed_time": "3:56:31", "remaining_time": "10:20:10"} +{"current_steps": 165, "total_steps": 594, "loss": 0.42, "lr": 4.546473115647409e-07, "epoch": 0.27789473684210525, "percentage": 27.78, "elapsed_time": "3:57:57", "remaining_time": "10:18:41"} +{"current_steps": 166, "total_steps": 594, "loss": 0.3765, "lr": 4.53798987460378e-07, "epoch": 0.27957894736842104, "percentage": 27.95, "elapsed_time": "3:59:24", "remaining_time": "10:17:16"} +{"current_steps": 167, "total_steps": 594, "loss": 0.4077, "lr": 4.529436096395156e-07, "epoch": 0.2812631578947368, "percentage": 28.11, "elapsed_time": "4:00:37", "remaining_time": "10:15:15"} +{"current_steps": 168, "total_steps": 594, "loss": 0.5229, "lr": 4.520812077077604e-07, "epoch": 0.2829473684210526, "percentage": 28.28, "elapsed_time": "4:01:48", "remaining_time": "10:13:09"} +{"current_steps": 169, "total_steps": 594, "loss": 0.4687, "lr": 4.512118115138314e-07, "epoch": 0.2846315789473684, "percentage": 28.45, "elapsed_time": "4:03:14", "remaining_time": "10:11:42"} +{"current_steps": 170, "total_steps": 594, "loss": 0.4222, "lr": 4.503354511485273e-07, "epoch": 0.2863157894736842, "percentage": 28.62, "elapsed_time": "4:04:58", "remaining_time": "10:10:59"} +{"current_steps": 171, "total_steps": 594, "loss": 0.4366, "lr": 4.4945215694368447e-07, "epoch": 0.288, "percentage": 28.79, "elapsed_time": "4:06:25", "remaining_time": "10:09:35"} +{"current_steps": 172, "total_steps": 594, "loss": 0.4281, "lr": 4.485619594711277e-07, "epoch": 0.28968421052631577, "percentage": 28.96, "elapsed_time": "4:07:27", "remaining_time": "10:07:08"} +{"current_steps": 173, "total_steps": 594, "loss": 0.4359, "lr": 4.4766488954161153e-07, "epoch": 0.29136842105263155, "percentage": 29.12, "elapsed_time": "4:08:45", "remaining_time": "10:05:21"} +{"current_steps": 174, "total_steps": 594, "loss": 0.4254, "lr": 4.4676097820375426e-07, "epoch": 0.29305263157894734, "percentage": 29.29, "elapsed_time": "4:10:23", "remaining_time": "10:04:23"} +{"current_steps": 175, "total_steps": 594, "loss": 0.4344, "lr": 4.458502567429631e-07, "epoch": 0.29473684210526313, "percentage": 29.46, "elapsed_time": "4:11:30", "remaining_time": "10:02:10"} +{"current_steps": 176, "total_steps": 594, "loss": 0.4063, "lr": 4.4493275668035147e-07, "epoch": 0.296421052631579, "percentage": 29.63, "elapsed_time": "4:12:45", "remaining_time": "10:00:18"} +{"current_steps": 177, "total_steps": 594, "loss": 0.4259, "lr": 4.440085097716479e-07, "epoch": 0.29810526315789476, "percentage": 29.8, "elapsed_time": "4:14:01", "remaining_time": "9:58:28"} +{"current_steps": 178, "total_steps": 594, "loss": 0.3982, "lr": 4.430775480060972e-07, "epoch": 0.29978947368421055, "percentage": 29.97, "elapsed_time": "4:15:48", "remaining_time": "9:57:51"} +{"current_steps": 179, "total_steps": 594, "loss": 0.4823, "lr": 4.4213990360535267e-07, "epoch": 0.30147368421052634, "percentage": 30.13, "elapsed_time": "4:17:03", "remaining_time": "9:55:57"} +{"current_steps": 180, "total_steps": 594, "loss": 0.4579, "lr": 4.4119560902236174e-07, "epoch": 0.3031578947368421, "percentage": 30.3, "elapsed_time": "4:18:29", "remaining_time": "9:54:31"} +{"current_steps": 181, "total_steps": 594, "loss": 0.4195, "lr": 4.402446969402419e-07, "epoch": 0.3048421052631579, "percentage": 30.47, "elapsed_time": "4:20:24", "remaining_time": "9:54:10"} +{"current_steps": 182, "total_steps": 594, "loss": 0.4709, "lr": 4.392872002711501e-07, "epoch": 0.3065263157894737, "percentage": 30.64, "elapsed_time": "4:21:43", "remaining_time": "9:52:29"} +{"current_steps": 183, "total_steps": 594, "loss": 0.3825, "lr": 4.3832315215514316e-07, "epoch": 0.3082105263157895, "percentage": 30.81, "elapsed_time": "4:23:30", "remaining_time": "9:51:49"} +{"current_steps": 184, "total_steps": 594, "loss": 0.4091, "lr": 4.3735258595903123e-07, "epoch": 0.3098947368421053, "percentage": 30.98, "elapsed_time": "4:25:02", "remaining_time": "9:50:34"} +{"current_steps": 185, "total_steps": 594, "loss": 0.4377, "lr": 4.3637553527522265e-07, "epoch": 0.31157894736842107, "percentage": 31.14, "elapsed_time": "4:26:41", "remaining_time": "9:49:36"} +{"current_steps": 186, "total_steps": 594, "loss": 0.4404, "lr": 4.353920339205611e-07, "epoch": 0.31326315789473685, "percentage": 31.31, "elapsed_time": "4:28:06", "remaining_time": "9:48:06"} +{"current_steps": 187, "total_steps": 594, "loss": 0.4424, "lr": 4.344021159351555e-07, "epoch": 0.31494736842105264, "percentage": 31.48, "elapsed_time": "4:29:45", "remaining_time": "9:47:07"} +{"current_steps": 188, "total_steps": 594, "loss": 0.3832, "lr": 4.3340581558120195e-07, "epoch": 0.31663157894736843, "percentage": 31.65, "elapsed_time": "4:30:56", "remaining_time": "9:45:06"} +{"current_steps": 189, "total_steps": 594, "loss": 0.4255, "lr": 4.3240316734179705e-07, "epoch": 0.3183157894736842, "percentage": 31.82, "elapsed_time": "4:32:44", "remaining_time": "9:44:26"} +{"current_steps": 190, "total_steps": 594, "loss": 0.4927, "lr": 4.313942059197456e-07, "epoch": 0.32, "percentage": 31.99, "elapsed_time": "4:33:55", "remaining_time": "9:42:27"} +{"current_steps": 191, "total_steps": 594, "loss": 0.367, "lr": 4.3037896623635864e-07, "epoch": 0.3216842105263158, "percentage": 32.15, "elapsed_time": "4:35:21", "remaining_time": "9:40:59"} +{"current_steps": 192, "total_steps": 594, "loss": 0.4826, "lr": 4.29357483430245e-07, "epoch": 0.3233684210526316, "percentage": 32.32, "elapsed_time": "4:36:47", "remaining_time": "9:39:32"} +{"current_steps": 193, "total_steps": 594, "loss": 0.3863, "lr": 4.2832979285609505e-07, "epoch": 0.32505263157894737, "percentage": 32.49, "elapsed_time": "4:38:00", "remaining_time": "9:37:38"} +{"current_steps": 194, "total_steps": 594, "loss": 0.3963, "lr": 4.2729593008345734e-07, "epoch": 0.32673684210526316, "percentage": 32.66, "elapsed_time": "4:39:29", "remaining_time": "9:36:16"} +{"current_steps": 195, "total_steps": 594, "loss": 0.458, "lr": 4.2625593089550717e-07, "epoch": 0.32842105263157895, "percentage": 32.83, "elapsed_time": "4:41:31", "remaining_time": "9:36:02"} +{"current_steps": 196, "total_steps": 594, "loss": 0.3814, "lr": 4.2520983128780825e-07, "epoch": 0.33010526315789473, "percentage": 33.0, "elapsed_time": "4:42:49", "remaining_time": "9:34:17"} +{"current_steps": 197, "total_steps": 594, "loss": 0.3932, "lr": 4.2415766746706674e-07, "epoch": 0.3317894736842105, "percentage": 33.16, "elapsed_time": "4:44:01", "remaining_time": "9:32:22"} +{"current_steps": 198, "total_steps": 594, "loss": 0.4666, "lr": 4.230994758498782e-07, "epoch": 0.3334736842105263, "percentage": 33.33, "elapsed_time": "4:46:20", "remaining_time": "9:32:41"} +{"current_steps": 199, "total_steps": 594, "loss": 0.3773, "lr": 4.220352930614672e-07, "epoch": 0.3351578947368421, "percentage": 33.5, "elapsed_time": "4:48:02", "remaining_time": "9:31:44"} +{"current_steps": 200, "total_steps": 594, "loss": 0.4269, "lr": 4.209651559344194e-07, "epoch": 0.3368421052631579, "percentage": 33.67, "elapsed_time": "4:50:17", "remaining_time": "9:31:52"} +{"current_steps": 201, "total_steps": 594, "loss": 0.4577, "lr": 4.198891015074073e-07, "epoch": 0.3385263157894737, "percentage": 33.84, "elapsed_time": "4:51:51", "remaining_time": "9:30:38"} +{"current_steps": 202, "total_steps": 594, "loss": 0.414, "lr": 4.1880716702390763e-07, "epoch": 0.34021052631578946, "percentage": 34.01, "elapsed_time": "4:53:23", "remaining_time": "9:29:20"} +{"current_steps": 203, "total_steps": 594, "loss": 0.4332, "lr": 4.177193899309126e-07, "epoch": 0.34189473684210525, "percentage": 34.18, "elapsed_time": "4:54:33", "remaining_time": "9:27:20"} +{"current_steps": 204, "total_steps": 594, "loss": 0.4434, "lr": 4.166258078776341e-07, "epoch": 0.34357894736842104, "percentage": 34.34, "elapsed_time": "4:55:47", "remaining_time": "9:25:29"} +{"current_steps": 205, "total_steps": 594, "loss": 0.5528, "lr": 4.155264587142001e-07, "epoch": 0.3452631578947368, "percentage": 34.51, "elapsed_time": "4:57:10", "remaining_time": "9:23:55"} +{"current_steps": 206, "total_steps": 594, "loss": 0.3432, "lr": 4.1442138049034494e-07, "epoch": 0.3469473684210526, "percentage": 34.68, "elapsed_time": "4:59:04", "remaining_time": "9:23:17"} +{"current_steps": 207, "total_steps": 594, "loss": 0.3929, "lr": 4.133106114540923e-07, "epoch": 0.3486315789473684, "percentage": 34.85, "elapsed_time": "5:00:35", "remaining_time": "9:21:58"} +{"current_steps": 208, "total_steps": 594, "loss": 0.5005, "lr": 4.1219419005043154e-07, "epoch": 0.3503157894736842, "percentage": 35.02, "elapsed_time": "5:01:59", "remaining_time": "9:20:26"} +{"current_steps": 209, "total_steps": 594, "loss": 0.3055, "lr": 4.110721549199866e-07, "epoch": 0.352, "percentage": 35.19, "elapsed_time": "5:03:30", "remaining_time": "9:19:05"} +{"current_steps": 210, "total_steps": 594, "loss": 0.3988, "lr": 4.0994454489767927e-07, "epoch": 0.35368421052631577, "percentage": 35.35, "elapsed_time": "5:05:02", "remaining_time": "9:17:47"} +{"current_steps": 211, "total_steps": 594, "loss": 0.3742, "lr": 4.088113990113846e-07, "epoch": 0.35536842105263156, "percentage": 35.52, "elapsed_time": "5:06:36", "remaining_time": "9:16:33"} +{"current_steps": 212, "total_steps": 594, "loss": 0.3696, "lr": 4.076727564805802e-07, "epoch": 0.35705263157894734, "percentage": 35.69, "elapsed_time": "5:08:14", "remaining_time": "9:15:25"} +{"current_steps": 213, "total_steps": 594, "loss": 0.4692, "lr": 4.0652865671498906e-07, "epoch": 0.35873684210526313, "percentage": 35.86, "elapsed_time": "5:09:39", "remaining_time": "9:13:53"} +{"current_steps": 214, "total_steps": 594, "loss": 0.3862, "lr": 4.053791393132149e-07, "epoch": 0.3604210526315789, "percentage": 36.03, "elapsed_time": "5:11:05", "remaining_time": "9:12:24"} +{"current_steps": 215, "total_steps": 594, "loss": 0.378, "lr": 4.0422424406137235e-07, "epoch": 0.36210526315789476, "percentage": 36.2, "elapsed_time": "5:12:27", "remaining_time": "9:10:47"} +{"current_steps": 216, "total_steps": 594, "loss": 0.4346, "lr": 4.0306401093170956e-07, "epoch": 0.36378947368421055, "percentage": 36.36, "elapsed_time": "5:13:51", "remaining_time": "9:09:15"} +{"current_steps": 217, "total_steps": 594, "loss": 0.3487, "lr": 4.0189848008122475e-07, "epoch": 0.36547368421052634, "percentage": 36.53, "elapsed_time": "5:15:20", "remaining_time": "9:07:50"} +{"current_steps": 218, "total_steps": 594, "loss": 0.3927, "lr": 4.007276918502763e-07, "epoch": 0.3671578947368421, "percentage": 36.7, "elapsed_time": "5:16:55", "remaining_time": "9:06:36"} +{"current_steps": 219, "total_steps": 594, "loss": 0.5847, "lr": 3.9955168676118645e-07, "epoch": 0.3688421052631579, "percentage": 36.87, "elapsed_time": "5:18:19", "remaining_time": "9:05:04"} +{"current_steps": 220, "total_steps": 594, "loss": 0.3788, "lr": 3.9837050551683904e-07, "epoch": 0.3705263157894737, "percentage": 37.04, "elapsed_time": "5:20:00", "remaining_time": "9:04:00"} +{"current_steps": 221, "total_steps": 594, "loss": 0.485, "lr": 3.9718418899927056e-07, "epoch": 0.3722105263157895, "percentage": 37.21, "elapsed_time": "5:21:15", "remaining_time": "9:02:12"} +{"current_steps": 222, "total_steps": 594, "loss": 0.4306, "lr": 3.959927782682551e-07, "epoch": 0.3738947368421053, "percentage": 37.37, "elapsed_time": "5:23:00", "remaining_time": "9:01:15"} +{"current_steps": 223, "total_steps": 594, "loss": 0.4078, "lr": 3.947963145598833e-07, "epoch": 0.37557894736842107, "percentage": 37.54, "elapsed_time": "5:24:49", "remaining_time": "9:00:24"} +{"current_steps": 224, "total_steps": 594, "loss": 0.3882, "lr": 3.935948392851353e-07, "epoch": 0.37726315789473686, "percentage": 37.71, "elapsed_time": "5:26:36", "remaining_time": "8:59:28"} +{"current_steps": 225, "total_steps": 594, "loss": 0.4068, "lr": 3.923883940284472e-07, "epoch": 0.37894736842105264, "percentage": 37.88, "elapsed_time": "5:28:09", "remaining_time": "8:58:10"} +{"current_steps": 226, "total_steps": 594, "loss": 0.3531, "lr": 3.9117702054627164e-07, "epoch": 0.38063157894736843, "percentage": 38.05, "elapsed_time": "5:29:33", "remaining_time": "8:56:37"} +{"current_steps": 227, "total_steps": 594, "loss": 0.5188, "lr": 3.8996076076563333e-07, "epoch": 0.3823157894736842, "percentage": 38.22, "elapsed_time": "5:31:15", "remaining_time": "8:55:33"} +{"current_steps": 228, "total_steps": 594, "loss": 0.4398, "lr": 3.8873965678267686e-07, "epoch": 0.384, "percentage": 38.38, "elapsed_time": "5:32:34", "remaining_time": "8:53:52"} +{"current_steps": 229, "total_steps": 594, "loss": 0.4007, "lr": 3.8751375086121027e-07, "epoch": 0.3856842105263158, "percentage": 38.55, "elapsed_time": "5:33:59", "remaining_time": "8:52:21"} +{"current_steps": 230, "total_steps": 594, "loss": 0.4103, "lr": 3.8628308543124264e-07, "epoch": 0.3873684210526316, "percentage": 38.72, "elapsed_time": "5:35:06", "remaining_time": "8:50:20"} +{"current_steps": 231, "total_steps": 594, "loss": 0.3659, "lr": 3.8504770308751465e-07, "epoch": 0.38905263157894737, "percentage": 38.89, "elapsed_time": "5:36:10", "remaining_time": "8:48:17"} +{"current_steps": 232, "total_steps": 594, "loss": 0.3315, "lr": 3.8380764658802476e-07, "epoch": 0.39073684210526316, "percentage": 39.06, "elapsed_time": "5:37:32", "remaining_time": "8:46:40"} +{"current_steps": 233, "total_steps": 594, "loss": 0.3813, "lr": 3.8256295885254977e-07, "epoch": 0.39242105263157895, "percentage": 39.23, "elapsed_time": "5:39:02", "remaining_time": "8:45:17"} +{"current_steps": 234, "total_steps": 594, "loss": 0.2827, "lr": 3.8131368296115823e-07, "epoch": 0.39410526315789474, "percentage": 39.39, "elapsed_time": "5:40:31", "remaining_time": "8:43:53"} +{"current_steps": 235, "total_steps": 594, "loss": 0.3864, "lr": 3.800598621527205e-07, "epoch": 0.3957894736842105, "percentage": 39.56, "elapsed_time": "5:42:07", "remaining_time": "8:42:39"} +{"current_steps": 236, "total_steps": 594, "loss": 0.3463, "lr": 3.7880153982341163e-07, "epoch": 0.3974736842105263, "percentage": 39.73, "elapsed_time": "5:43:17", "remaining_time": "8:40:44"} +{"current_steps": 237, "total_steps": 594, "loss": 0.3967, "lr": 3.7753875952520937e-07, "epoch": 0.3991578947368421, "percentage": 39.9, "elapsed_time": "5:44:31", "remaining_time": "8:38:58"} +{"current_steps": 238, "total_steps": 594, "loss": 0.4355, "lr": 3.7627156496438685e-07, "epoch": 0.4008421052631579, "percentage": 40.07, "elapsed_time": "5:45:42", "remaining_time": "8:37:07"} +{"current_steps": 239, "total_steps": 594, "loss": 0.3694, "lr": 3.75e-07, "epoch": 0.4025263157894737, "percentage": 40.24, "elapsed_time": "5:47:21", "remaining_time": "8:35:56"} +{"current_steps": 240, "total_steps": 594, "loss": 0.3254, "lr": 3.7372410864236947e-07, "epoch": 0.40421052631578946, "percentage": 40.4, "elapsed_time": "5:48:47", "remaining_time": "8:34:28"} +{"current_steps": 241, "total_steps": 594, "loss": 0.3847, "lr": 3.724439350515571e-07, "epoch": 0.40589473684210525, "percentage": 40.57, "elapsed_time": "5:49:59", "remaining_time": "8:32:39"} +{"current_steps": 242, "total_steps": 594, "loss": 0.3904, "lr": 3.7115952353583803e-07, "epoch": 0.40757894736842104, "percentage": 40.74, "elapsed_time": "5:51:27", "remaining_time": "8:31:12"} +{"current_steps": 243, "total_steps": 594, "loss": 0.2951, "lr": 3.698709185501666e-07, "epoch": 0.40926315789473683, "percentage": 40.91, "elapsed_time": "5:52:33", "remaining_time": "8:29:15"} +{"current_steps": 244, "total_steps": 594, "loss": 0.4324, "lr": 3.6857816469463805e-07, "epoch": 0.4109473684210526, "percentage": 41.08, "elapsed_time": "5:53:58", "remaining_time": "8:27:45"} +{"current_steps": 245, "total_steps": 594, "loss": 0.3948, "lr": 3.6728130671294484e-07, "epoch": 0.4126315789473684, "percentage": 41.25, "elapsed_time": "5:55:29", "remaining_time": "8:26:23"} +{"current_steps": 246, "total_steps": 594, "loss": 0.4015, "lr": 3.6598038949082773e-07, "epoch": 0.4143157894736842, "percentage": 41.41, "elapsed_time": "5:56:50", "remaining_time": "8:24:48"} +{"current_steps": 247, "total_steps": 594, "loss": 0.4882, "lr": 3.646754580545226e-07, "epoch": 0.416, "percentage": 41.58, "elapsed_time": "5:58:02", "remaining_time": "8:22:59"} +{"current_steps": 248, "total_steps": 594, "loss": 0.3747, "lr": 3.633665575692019e-07, "epoch": 0.41768421052631577, "percentage": 41.75, "elapsed_time": "5:59:20", "remaining_time": "8:21:20"} +{"current_steps": 249, "total_steps": 594, "loss": 0.4002, "lr": 3.6205373333741137e-07, "epoch": 0.41936842105263156, "percentage": 41.92, "elapsed_time": "6:00:57", "remaining_time": "8:20:07"} +{"current_steps": 250, "total_steps": 594, "loss": 0.4095, "lr": 3.60737030797502e-07, "epoch": 0.42105263157894735, "percentage": 42.09, "elapsed_time": "6:02:21", "remaining_time": "8:18:36"} +{"current_steps": 251, "total_steps": 594, "loss": 0.4228, "lr": 3.5941649552205766e-07, "epoch": 0.42273684210526313, "percentage": 42.26, "elapsed_time": "6:03:56", "remaining_time": "8:17:20"} +{"current_steps": 252, "total_steps": 594, "loss": 0.3853, "lr": 3.580921732163174e-07, "epoch": 0.4244210526315789, "percentage": 42.42, "elapsed_time": "6:05:04", "remaining_time": "8:15:27"} +{"current_steps": 253, "total_steps": 594, "loss": 0.3623, "lr": 3.56764109716594e-07, "epoch": 0.4261052631578947, "percentage": 42.59, "elapsed_time": "6:06:25", "remaining_time": "8:13:52"} +{"current_steps": 254, "total_steps": 594, "loss": 0.4097, "lr": 3.55432350988687e-07, "epoch": 0.42778947368421055, "percentage": 42.76, "elapsed_time": "6:08:01", "remaining_time": "8:12:37"} +{"current_steps": 255, "total_steps": 594, "loss": 0.3034, "lr": 3.5409694312629193e-07, "epoch": 0.42947368421052634, "percentage": 42.93, "elapsed_time": "6:09:32", "remaining_time": "8:11:17"} +{"current_steps": 256, "total_steps": 594, "loss": 0.4546, "lr": 3.5275793234940544e-07, "epoch": 0.43115789473684213, "percentage": 43.1, "elapsed_time": "6:11:04", "remaining_time": "8:09:56"} +{"current_steps": 257, "total_steps": 594, "loss": 0.4514, "lr": 3.514153650027249e-07, "epoch": 0.4328421052631579, "percentage": 43.27, "elapsed_time": "6:12:36", "remaining_time": "8:08:35"} +{"current_steps": 258, "total_steps": 594, "loss": 0.3499, "lr": 3.5006928755404464e-07, "epoch": 0.4345263157894737, "percentage": 43.43, "elapsed_time": "6:13:51", "remaining_time": "8:06:52"} +{"current_steps": 259, "total_steps": 594, "loss": 0.4307, "lr": 3.487197465926478e-07, "epoch": 0.4362105263157895, "percentage": 43.6, "elapsed_time": "6:15:01", "remaining_time": "8:05:04"} +{"current_steps": 260, "total_steps": 594, "loss": 0.3409, "lr": 3.473667888276935e-07, "epoch": 0.4378947368421053, "percentage": 43.77, "elapsed_time": "6:16:10", "remaining_time": "8:03:13"} +{"current_steps": 261, "total_steps": 594, "loss": 0.4231, "lr": 3.460104610866003e-07, "epoch": 0.43957894736842107, "percentage": 43.94, "elapsed_time": "6:18:11", "remaining_time": "8:02:31"} +{"current_steps": 262, "total_steps": 594, "loss": 0.4701, "lr": 3.4465081031342583e-07, "epoch": 0.44126315789473686, "percentage": 44.11, "elapsed_time": "6:19:41", "remaining_time": "8:01:07"} +{"current_steps": 263, "total_steps": 594, "loss": 0.4187, "lr": 3.432878835672413e-07, "epoch": 0.44294736842105265, "percentage": 44.28, "elapsed_time": "6:20:57", "remaining_time": "7:59:27"} +{"current_steps": 264, "total_steps": 594, "loss": 0.4608, "lr": 3.419217280205032e-07, "epoch": 0.44463157894736843, "percentage": 44.44, "elapsed_time": "6:22:27", "remaining_time": "7:58:04"} +{"current_steps": 265, "total_steps": 594, "loss": 0.3501, "lr": 3.405523909574206e-07, "epoch": 0.4463157894736842, "percentage": 44.61, "elapsed_time": "6:23:54", "remaining_time": "7:56:37"} +{"current_steps": 266, "total_steps": 594, "loss": 0.4462, "lr": 3.391799197723185e-07, "epoch": 0.448, "percentage": 44.78, "elapsed_time": "6:25:47", "remaining_time": "7:55:42"} +{"current_steps": 267, "total_steps": 594, "loss": 0.37, "lr": 3.378043619679973e-07, "epoch": 0.4496842105263158, "percentage": 44.95, "elapsed_time": "6:27:13", "remaining_time": "7:54:14"} +{"current_steps": 268, "total_steps": 594, "loss": 0.4873, "lr": 3.3642576515408905e-07, "epoch": 0.4513684210526316, "percentage": 45.12, "elapsed_time": "6:28:44", "remaining_time": "7:52:52"} +{"current_steps": 269, "total_steps": 594, "loss": 0.3324, "lr": 3.350441770454092e-07, "epoch": 0.4530526315789474, "percentage": 45.29, "elapsed_time": "6:30:15", "remaining_time": "7:51:30"} +{"current_steps": 270, "total_steps": 594, "loss": 0.3524, "lr": 3.336596454603054e-07, "epoch": 0.45473684210526316, "percentage": 45.45, "elapsed_time": "6:31:58", "remaining_time": "7:50:22"} +{"current_steps": 271, "total_steps": 594, "loss": 0.445, "lr": 3.322722183190025e-07, "epoch": 0.45642105263157895, "percentage": 45.62, "elapsed_time": "6:33:02", "remaining_time": "7:48:27"} +{"current_steps": 272, "total_steps": 594, "loss": 0.3692, "lr": 3.308819436419437e-07, "epoch": 0.45810526315789474, "percentage": 45.79, "elapsed_time": "6:34:44", "remaining_time": "7:47:18"} +{"current_steps": 273, "total_steps": 594, "loss": 0.4067, "lr": 3.294888695481287e-07, "epoch": 0.4597894736842105, "percentage": 45.96, "elapsed_time": "6:36:52", "remaining_time": "7:46:39"} +{"current_steps": 274, "total_steps": 594, "loss": 0.4646, "lr": 3.2809304425344856e-07, "epoch": 0.4614736842105263, "percentage": 46.13, "elapsed_time": "6:38:31", "remaining_time": "7:45:25"} +{"current_steps": 275, "total_steps": 594, "loss": 0.3497, "lr": 3.266945160690159e-07, "epoch": 0.4631578947368421, "percentage": 46.3, "elapsed_time": "6:39:35", "remaining_time": "7:43:31"} +{"current_steps": 276, "total_steps": 594, "loss": 0.3206, "lr": 3.252933333994942e-07, "epoch": 0.4648421052631579, "percentage": 46.46, "elapsed_time": "6:40:57", "remaining_time": "7:41:57"} +{"current_steps": 277, "total_steps": 594, "loss": 0.2983, "lr": 3.23889544741421e-07, "epoch": 0.4665263157894737, "percentage": 46.63, "elapsed_time": "6:42:14", "remaining_time": "7:40:19"} +{"current_steps": 278, "total_steps": 594, "loss": 0.3122, "lr": 3.224831986815306e-07, "epoch": 0.46821052631578947, "percentage": 46.8, "elapsed_time": "6:43:38", "remaining_time": "7:38:48"} +{"current_steps": 279, "total_steps": 594, "loss": 0.329, "lr": 3.2107434389507177e-07, "epoch": 0.46989473684210525, "percentage": 46.97, "elapsed_time": "6:44:47", "remaining_time": "7:37:01"} +{"current_steps": 280, "total_steps": 594, "loss": 0.3977, "lr": 3.19663029144123e-07, "epoch": 0.47157894736842104, "percentage": 47.14, "elapsed_time": "6:46:05", "remaining_time": "7:35:23"} +{"current_steps": 281, "total_steps": 594, "loss": 0.3496, "lr": 3.1824930327590525e-07, "epoch": 0.47326315789473683, "percentage": 47.31, "elapsed_time": "6:47:28", "remaining_time": "7:33:52"} +{"current_steps": 282, "total_steps": 594, "loss": 0.4291, "lr": 3.168332152210909e-07, "epoch": 0.4749473684210526, "percentage": 47.47, "elapsed_time": "6:48:58", "remaining_time": "7:32:28"} +{"current_steps": 283, "total_steps": 594, "loss": 0.3508, "lr": 3.154148139921102e-07, "epoch": 0.4766315789473684, "percentage": 47.64, "elapsed_time": "6:50:41", "remaining_time": "7:31:20"} +{"current_steps": 284, "total_steps": 594, "loss": 0.4224, "lr": 3.1399414868145503e-07, "epoch": 0.4783157894736842, "percentage": 47.81, "elapsed_time": "6:52:08", "remaining_time": "7:29:52"} +{"current_steps": 285, "total_steps": 594, "loss": 0.2682, "lr": 3.1257126845997996e-07, "epoch": 0.48, "percentage": 47.98, "elapsed_time": "6:53:34", "remaining_time": "7:28:23"} +{"current_steps": 286, "total_steps": 594, "loss": 0.2759, "lr": 3.111462225752e-07, "epoch": 0.48168421052631577, "percentage": 48.15, "elapsed_time": "6:55:21", "remaining_time": "7:27:18"} +{"current_steps": 287, "total_steps": 594, "loss": 0.4725, "lr": 3.097190603495861e-07, "epoch": 0.48336842105263156, "percentage": 48.32, "elapsed_time": "6:56:42", "remaining_time": "7:25:45"} +{"current_steps": 288, "total_steps": 594, "loss": 0.3671, "lr": 3.0828983117885856e-07, "epoch": 0.48505263157894735, "percentage": 48.48, "elapsed_time": "6:58:25", "remaining_time": "7:24:34"} +{"current_steps": 289, "total_steps": 594, "loss": 0.4495, "lr": 3.0685858453027665e-07, "epoch": 0.48673684210526313, "percentage": 48.65, "elapsed_time": "7:00:08", "remaining_time": "7:23:23"} +{"current_steps": 290, "total_steps": 594, "loss": 0.3672, "lr": 3.0542536994092696e-07, "epoch": 0.4884210526315789, "percentage": 48.82, "elapsed_time": "7:01:16", "remaining_time": "7:21:37"} +{"current_steps": 291, "total_steps": 594, "loss": 0.405, "lr": 3.03990237016009e-07, "epoch": 0.4901052631578947, "percentage": 48.99, "elapsed_time": "7:02:44", "remaining_time": "7:20:10"} +{"current_steps": 292, "total_steps": 594, "loss": 0.4201, "lr": 3.025532354271178e-07, "epoch": 0.4917894736842105, "percentage": 49.16, "elapsed_time": "7:04:26", "remaining_time": "7:18:59"} +{"current_steps": 293, "total_steps": 594, "loss": 0.4615, "lr": 3.0111441491052505e-07, "epoch": 0.49347368421052634, "percentage": 49.33, "elapsed_time": "7:05:42", "remaining_time": "7:17:19"} +{"current_steps": 294, "total_steps": 594, "loss": 0.4244, "lr": 2.996738252654577e-07, "epoch": 0.49515789473684213, "percentage": 49.49, "elapsed_time": "7:07:06", "remaining_time": "7:15:49"} +{"current_steps": 295, "total_steps": 594, "loss": 0.4826, "lr": 2.982315163523742e-07, "epoch": 0.4968421052631579, "percentage": 49.66, "elapsed_time": "7:08:30", "remaining_time": "7:14:18"} +{"current_steps": 296, "total_steps": 594, "loss": 0.506, "lr": 2.967875380912388e-07, "epoch": 0.4985263157894737, "percentage": 49.83, "elapsed_time": "7:10:20", "remaining_time": "7:13:14"} +{"current_steps": 297, "total_steps": 594, "loss": 0.372, "lr": 2.9534194045979393e-07, "epoch": 0.5002105263157894, "percentage": 50.0, "elapsed_time": "7:11:33", "remaining_time": "7:11:33"} +{"current_steps": 298, "total_steps": 594, "loss": 0.3792, "lr": 2.9389477349183015e-07, "epoch": 0.5018947368421053, "percentage": 50.17, "elapsed_time": "7:12:53", "remaining_time": "7:09:58"} +{"current_steps": 299, "total_steps": 594, "loss": 0.3672, "lr": 2.924460872754547e-07, "epoch": 0.503578947368421, "percentage": 50.34, "elapsed_time": "7:14:26", "remaining_time": "7:08:37"} +{"current_steps": 300, "total_steps": 594, "loss": 0.3704, "lr": 2.909959319513574e-07, "epoch": 0.5052631578947369, "percentage": 50.51, "elapsed_time": "7:15:54", "remaining_time": "7:07:11"} +{"current_steps": 301, "total_steps": 594, "loss": 0.361, "lr": 2.8954435771107597e-07, "epoch": 0.5069473684210526, "percentage": 50.67, "elapsed_time": "7:17:14", "remaining_time": "7:05:37"} +{"current_steps": 302, "total_steps": 594, "loss": 0.4275, "lr": 2.880914147952584e-07, "epoch": 0.5086315789473684, "percentage": 50.84, "elapsed_time": "7:18:36", "remaining_time": "7:04:05"} +{"current_steps": 303, "total_steps": 594, "loss": 0.3848, "lr": 2.8663715349192383e-07, "epoch": 0.5103157894736842, "percentage": 51.01, "elapsed_time": "7:19:47", "remaining_time": "7:02:22"} +{"current_steps": 304, "total_steps": 594, "loss": 0.4463, "lr": 2.8518162413472263e-07, "epoch": 0.512, "percentage": 51.18, "elapsed_time": "7:21:19", "remaining_time": "7:01:00"} +{"current_steps": 305, "total_steps": 594, "loss": 0.4367, "lr": 2.837248771011937e-07, "epoch": 0.5136842105263157, "percentage": 51.35, "elapsed_time": "7:22:33", "remaining_time": "6:59:20"} +{"current_steps": 306, "total_steps": 594, "loss": 0.3883, "lr": 2.822669628110213e-07, "epoch": 0.5153684210526316, "percentage": 51.52, "elapsed_time": "7:23:58", "remaining_time": "6:57:51"} +{"current_steps": 307, "total_steps": 594, "loss": 0.3848, "lr": 2.808079317242896e-07, "epoch": 0.5170526315789473, "percentage": 51.68, "elapsed_time": "7:25:23", "remaining_time": "6:56:22"} +{"current_steps": 308, "total_steps": 594, "loss": 0.4514, "lr": 2.793478343397367e-07, "epoch": 0.5187368421052632, "percentage": 51.85, "elapsed_time": "7:26:52", "remaining_time": "6:54:57"} +{"current_steps": 309, "total_steps": 594, "loss": 0.3864, "lr": 2.778867211930061e-07, "epoch": 0.5204210526315789, "percentage": 52.02, "elapsed_time": "7:28:05", "remaining_time": "6:53:17"} +{"current_steps": 310, "total_steps": 594, "loss": 0.3706, "lr": 2.7642464285489827e-07, "epoch": 0.5221052631578947, "percentage": 52.19, "elapsed_time": "7:29:18", "remaining_time": "6:51:37"} +{"current_steps": 311, "total_steps": 594, "loss": 0.4233, "lr": 2.749616499296199e-07, "epoch": 0.5237894736842105, "percentage": 52.36, "elapsed_time": "7:30:31", "remaining_time": "6:49:58"} +{"current_steps": 312, "total_steps": 594, "loss": 0.4109, "lr": 2.734977930530326e-07, "epoch": 0.5254736842105263, "percentage": 52.53, "elapsed_time": "7:32:22", "remaining_time": "6:48:52"} +{"current_steps": 313, "total_steps": 594, "loss": 0.4445, "lr": 2.7203312289090044e-07, "epoch": 0.5271578947368422, "percentage": 52.69, "elapsed_time": "7:33:44", "remaining_time": "6:47:21"} +{"current_steps": 314, "total_steps": 594, "loss": 0.3846, "lr": 2.7056769013713623e-07, "epoch": 0.5288421052631579, "percentage": 52.86, "elapsed_time": "7:34:56", "remaining_time": "6:45:40"} +{"current_steps": 315, "total_steps": 594, "loss": 0.4926, "lr": 2.6910154551204676e-07, "epoch": 0.5305263157894737, "percentage": 53.03, "elapsed_time": "7:36:11", "remaining_time": "6:44:03"} +{"current_steps": 316, "total_steps": 594, "loss": 0.351, "lr": 2.6763473976057773e-07, "epoch": 0.5322105263157895, "percentage": 53.2, "elapsed_time": "7:37:34", "remaining_time": "6:42:32"} +{"current_steps": 317, "total_steps": 594, "loss": 0.3784, "lr": 2.6616732365055713e-07, "epoch": 0.5338947368421053, "percentage": 53.37, "elapsed_time": "7:38:53", "remaining_time": "6:40:59"} +{"current_steps": 318, "total_steps": 594, "loss": 0.39, "lr": 2.64699347970938e-07, "epoch": 0.535578947368421, "percentage": 53.54, "elapsed_time": "7:40:08", "remaining_time": "6:39:22"} +{"current_steps": 319, "total_steps": 594, "loss": 0.3992, "lr": 2.632308635300408e-07, "epoch": 0.5372631578947369, "percentage": 53.7, "elapsed_time": "7:41:42", "remaining_time": "6:38:01"} +{"current_steps": 320, "total_steps": 594, "loss": 0.4368, "lr": 2.6176192115379494e-07, "epoch": 0.5389473684210526, "percentage": 53.87, "elapsed_time": "7:42:53", "remaining_time": "6:36:20"} +{"current_steps": 321, "total_steps": 594, "loss": 0.365, "lr": 2.6029257168397944e-07, "epoch": 0.5406315789473685, "percentage": 54.04, "elapsed_time": "7:44:38", "remaining_time": "6:35:09"} +{"current_steps": 322, "total_steps": 594, "loss": 0.5333, "lr": 2.5882286597646313e-07, "epoch": 0.5423157894736842, "percentage": 54.21, "elapsed_time": "7:46:25", "remaining_time": "6:34:00"} +{"current_steps": 323, "total_steps": 594, "loss": 0.4497, "lr": 2.5735285489944485e-07, "epoch": 0.544, "percentage": 54.38, "elapsed_time": "7:47:58", "remaining_time": "6:32:38"} +{"current_steps": 324, "total_steps": 594, "loss": 0.4313, "lr": 2.5588258933169244e-07, "epoch": 0.5456842105263158, "percentage": 54.55, "elapsed_time": "7:49:07", "remaining_time": "6:30:55"} +{"current_steps": 325, "total_steps": 594, "loss": 0.4731, "lr": 2.544121201607822e-07, "epoch": 0.5473684210526316, "percentage": 54.71, "elapsed_time": "7:50:31", "remaining_time": "6:29:27"} +{"current_steps": 326, "total_steps": 594, "loss": 0.2975, "lr": 2.5294149828133704e-07, "epoch": 0.5490526315789473, "percentage": 54.88, "elapsed_time": "7:51:39", "remaining_time": "6:27:44"} +{"current_steps": 327, "total_steps": 594, "loss": 0.3018, "lr": 2.5147077459326555e-07, "epoch": 0.5507368421052632, "percentage": 55.05, "elapsed_time": "7:53:32", "remaining_time": "6:26:39"} +{"current_steps": 328, "total_steps": 594, "loss": 0.3669, "lr": 2.5e-07, "epoch": 0.5524210526315789, "percentage": 55.22, "elapsed_time": "7:55:18", "remaining_time": "6:25:28"} +{"current_steps": 329, "total_steps": 594, "loss": 0.3849, "lr": 2.485292254067345e-07, "epoch": 0.5541052631578948, "percentage": 55.39, "elapsed_time": "7:56:38", "remaining_time": "6:23:55"} +{"current_steps": 330, "total_steps": 594, "loss": 0.3178, "lr": 2.4705850171866294e-07, "epoch": 0.5557894736842105, "percentage": 55.56, "elapsed_time": "7:58:10", "remaining_time": "6:22:32"} +{"current_steps": 331, "total_steps": 594, "loss": 0.354, "lr": 2.4558787983921785e-07, "epoch": 0.5574736842105263, "percentage": 55.72, "elapsed_time": "7:59:41", "remaining_time": "6:21:08"} +{"current_steps": 332, "total_steps": 594, "loss": 0.4314, "lr": 2.441174106683076e-07, "epoch": 0.5591578947368421, "percentage": 55.89, "elapsed_time": "8:01:12", "remaining_time": "6:19:44"} +{"current_steps": 333, "total_steps": 594, "loss": 0.379, "lr": 2.4264714510055513e-07, "epoch": 0.5608421052631579, "percentage": 56.06, "elapsed_time": "8:02:35", "remaining_time": "6:18:14"} +{"current_steps": 334, "total_steps": 594, "loss": 0.4805, "lr": 2.4117713402353685e-07, "epoch": 0.5625263157894737, "percentage": 56.23, "elapsed_time": "8:03:55", "remaining_time": "6:16:42"} +{"current_steps": 335, "total_steps": 594, "loss": 0.3514, "lr": 2.397074283160206e-07, "epoch": 0.5642105263157895, "percentage": 56.4, "elapsed_time": "8:05:07", "remaining_time": "6:15:03"} +{"current_steps": 336, "total_steps": 594, "loss": 0.4203, "lr": 2.38238078846205e-07, "epoch": 0.5658947368421052, "percentage": 56.57, "elapsed_time": "8:06:26", "remaining_time": "6:13:31"} +{"current_steps": 337, "total_steps": 594, "loss": 0.4159, "lr": 2.367691364699592e-07, "epoch": 0.5675789473684211, "percentage": 56.73, "elapsed_time": "8:08:03", "remaining_time": "6:12:11"} +{"current_steps": 338, "total_steps": 594, "loss": 0.365, "lr": 2.3530065202906208e-07, "epoch": 0.5692631578947368, "percentage": 56.9, "elapsed_time": "8:09:16", "remaining_time": "6:10:34"} +{"current_steps": 339, "total_steps": 594, "loss": 0.3328, "lr": 2.3383267634944288e-07, "epoch": 0.5709473684210526, "percentage": 57.07, "elapsed_time": "8:10:54", "remaining_time": "6:09:16"} +{"current_steps": 340, "total_steps": 594, "loss": 0.3583, "lr": 2.3236526023942222e-07, "epoch": 0.5726315789473684, "percentage": 57.24, "elapsed_time": "8:12:15", "remaining_time": "6:07:45"} +{"current_steps": 341, "total_steps": 594, "loss": 0.3367, "lr": 2.3089845448795325e-07, "epoch": 0.5743157894736842, "percentage": 57.41, "elapsed_time": "8:13:56", "remaining_time": "6:06:28"} +{"current_steps": 342, "total_steps": 594, "loss": 0.5505, "lr": 2.2943230986286386e-07, "epoch": 0.576, "percentage": 57.58, "elapsed_time": "8:15:02", "remaining_time": "6:04:46"} +{"current_steps": 343, "total_steps": 594, "loss": 0.4058, "lr": 2.2796687710909961e-07, "epoch": 0.5776842105263158, "percentage": 57.74, "elapsed_time": "8:16:19", "remaining_time": "6:03:11"} +{"current_steps": 344, "total_steps": 594, "loss": 0.3634, "lr": 2.2650220694696746e-07, "epoch": 0.5793684210526315, "percentage": 57.91, "elapsed_time": "8:17:26", "remaining_time": "6:01:30"} +{"current_steps": 345, "total_steps": 594, "loss": 0.3775, "lr": 2.2503835007038018e-07, "epoch": 0.5810526315789474, "percentage": 58.08, "elapsed_time": "8:18:54", "remaining_time": "6:00:04"} +{"current_steps": 346, "total_steps": 594, "loss": 0.3373, "lr": 2.2357535714510179e-07, "epoch": 0.5827368421052631, "percentage": 58.25, "elapsed_time": "8:20:16", "remaining_time": "5:58:35"} +{"current_steps": 347, "total_steps": 594, "loss": 0.4641, "lr": 2.2211327880699389e-07, "epoch": 0.584421052631579, "percentage": 58.42, "elapsed_time": "8:21:49", "remaining_time": "5:57:12"} +{"current_steps": 348, "total_steps": 594, "loss": 0.3671, "lr": 2.206521656602633e-07, "epoch": 0.5861052631578947, "percentage": 58.59, "elapsed_time": "8:23:38", "remaining_time": "5:56:01"} +{"current_steps": 349, "total_steps": 594, "loss": 0.4117, "lr": 2.1919206827571034e-07, "epoch": 0.5877894736842105, "percentage": 58.75, "elapsed_time": "8:25:08", "remaining_time": "5:54:36"} +{"current_steps": 350, "total_steps": 594, "loss": 0.4165, "lr": 2.177330371889787e-07, "epoch": 0.5894736842105263, "percentage": 58.92, "elapsed_time": "8:26:40", "remaining_time": "5:53:13"} +{"current_steps": 351, "total_steps": 594, "loss": 0.3839, "lr": 2.162751228988063e-07, "epoch": 0.5911578947368421, "percentage": 59.09, "elapsed_time": "8:27:54", "remaining_time": "5:51:37"} +{"current_steps": 352, "total_steps": 594, "loss": 0.4139, "lr": 2.148183758652774e-07, "epoch": 0.592842105263158, "percentage": 59.26, "elapsed_time": "8:28:59", "remaining_time": "5:49:55"} +{"current_steps": 353, "total_steps": 594, "loss": 0.3704, "lr": 2.1336284650807612e-07, "epoch": 0.5945263157894737, "percentage": 59.43, "elapsed_time": "8:30:32", "remaining_time": "5:48:33"} +{"current_steps": 354, "total_steps": 594, "loss": 0.3452, "lr": 2.1190858520474163e-07, "epoch": 0.5962105263157895, "percentage": 59.6, "elapsed_time": "8:32:16", "remaining_time": "5:47:18"} +{"current_steps": 355, "total_steps": 594, "loss": 0.338, "lr": 2.10455642288924e-07, "epoch": 0.5978947368421053, "percentage": 59.76, "elapsed_time": "8:33:53", "remaining_time": "5:45:58"} +{"current_steps": 356, "total_steps": 594, "loss": 0.4356, "lr": 2.090040680486426e-07, "epoch": 0.5995789473684211, "percentage": 59.93, "elapsed_time": "8:35:10", "remaining_time": "5:44:24"} +{"current_steps": 357, "total_steps": 594, "loss": 0.3444, "lr": 2.0755391272454537e-07, "epoch": 0.6012631578947368, "percentage": 60.1, "elapsed_time": "8:36:54", "remaining_time": "5:43:09"} +{"current_steps": 358, "total_steps": 594, "loss": 0.3766, "lr": 2.0610522650816982e-07, "epoch": 0.6029473684210527, "percentage": 60.27, "elapsed_time": "8:38:41", "remaining_time": "5:41:55"} +{"current_steps": 359, "total_steps": 594, "loss": 0.3669, "lr": 2.04658059540206e-07, "epoch": 0.6046315789473684, "percentage": 60.44, "elapsed_time": "8:40:04", "remaining_time": "5:40:26"} +{"current_steps": 360, "total_steps": 594, "loss": 0.2421, "lr": 2.0321246190876112e-07, "epoch": 0.6063157894736843, "percentage": 60.61, "elapsed_time": "8:41:16", "remaining_time": "5:38:49"} +{"current_steps": 361, "total_steps": 594, "loss": 0.4451, "lr": 2.0176848364762576e-07, "epoch": 0.608, "percentage": 60.77, "elapsed_time": "8:42:39", "remaining_time": "5:37:20"} +{"current_steps": 362, "total_steps": 594, "loss": 0.373, "lr": 2.0032617473454225e-07, "epoch": 0.6096842105263158, "percentage": 60.94, "elapsed_time": "8:44:17", "remaining_time": "5:36:00"} +{"current_steps": 363, "total_steps": 594, "loss": 0.3177, "lr": 1.9888558508947492e-07, "epoch": 0.6113684210526316, "percentage": 61.11, "elapsed_time": "8:45:54", "remaining_time": "5:34:40"} +{"current_steps": 364, "total_steps": 594, "loss": 0.3765, "lr": 1.974467645728822e-07, "epoch": 0.6130526315789474, "percentage": 61.28, "elapsed_time": "8:47:33", "remaining_time": "5:33:20"} +{"current_steps": 365, "total_steps": 594, "loss": 0.3531, "lr": 1.9600976298399108e-07, "epoch": 0.6147368421052631, "percentage": 61.45, "elapsed_time": "8:49:04", "remaining_time": "5:31:56"} +{"current_steps": 366, "total_steps": 594, "loss": 0.3618, "lr": 1.945746300590731e-07, "epoch": 0.616421052631579, "percentage": 61.62, "elapsed_time": "8:50:17", "remaining_time": "5:30:20"} +{"current_steps": 367, "total_steps": 594, "loss": 0.3547, "lr": 1.9314141546972344e-07, "epoch": 0.6181052631578947, "percentage": 61.78, "elapsed_time": "8:51:59", "remaining_time": "5:29:03"} +{"current_steps": 368, "total_steps": 594, "loss": 0.38, "lr": 1.9171016882114152e-07, "epoch": 0.6197894736842106, "percentage": 61.95, "elapsed_time": "8:53:18", "remaining_time": "5:27:30"} +{"current_steps": 369, "total_steps": 594, "loss": 0.3751, "lr": 1.9028093965041391e-07, "epoch": 0.6214736842105263, "percentage": 62.12, "elapsed_time": "8:54:48", "remaining_time": "5:26:06"} +{"current_steps": 370, "total_steps": 594, "loss": 0.418, "lr": 1.888537774248e-07, "epoch": 0.6231578947368421, "percentage": 62.29, "elapsed_time": "8:55:55", "remaining_time": "5:24:26"} +{"current_steps": 371, "total_steps": 594, "loss": 0.4701, "lr": 1.8742873154002004e-07, "epoch": 0.6248421052631579, "percentage": 62.46, "elapsed_time": "8:57:08", "remaining_time": "5:22:51"} +{"current_steps": 372, "total_steps": 594, "loss": 0.3828, "lr": 1.86005851318545e-07, "epoch": 0.6265263157894737, "percentage": 62.63, "elapsed_time": "8:58:43", "remaining_time": "5:21:29"} +{"current_steps": 373, "total_steps": 594, "loss": 0.3776, "lr": 1.8458518600788987e-07, "epoch": 0.6282105263157894, "percentage": 62.79, "elapsed_time": "9:00:37", "remaining_time": "5:20:18"} +{"current_steps": 374, "total_steps": 594, "loss": 0.422, "lr": 1.8316678477890913e-07, "epoch": 0.6298947368421053, "percentage": 62.96, "elapsed_time": "9:01:57", "remaining_time": "5:18:47"} +{"current_steps": 375, "total_steps": 594, "loss": 0.4461, "lr": 1.8175069672409475e-07, "epoch": 0.631578947368421, "percentage": 63.13, "elapsed_time": "9:03:30", "remaining_time": "5:17:24"} +{"current_steps": 376, "total_steps": 594, "loss": 0.4749, "lr": 1.8033697085587696e-07, "epoch": 0.6332631578947369, "percentage": 63.3, "elapsed_time": "9:05:09", "remaining_time": "5:16:04"} +{"current_steps": 377, "total_steps": 594, "loss": 0.4272, "lr": 1.7892565610492826e-07, "epoch": 0.6349473684210526, "percentage": 63.47, "elapsed_time": "9:06:38", "remaining_time": "5:14:38"} +{"current_steps": 378, "total_steps": 594, "loss": 0.3069, "lr": 1.775168013184694e-07, "epoch": 0.6366315789473684, "percentage": 63.64, "elapsed_time": "9:08:16", "remaining_time": "5:13:18"} +{"current_steps": 379, "total_steps": 594, "loss": 0.3319, "lr": 1.7611045525857898e-07, "epoch": 0.6383157894736842, "percentage": 63.8, "elapsed_time": "9:09:55", "remaining_time": "5:11:58"} +{"current_steps": 380, "total_steps": 594, "loss": 0.3082, "lr": 1.7470666660050584e-07, "epoch": 0.64, "percentage": 63.97, "elapsed_time": "9:11:08", "remaining_time": "5:10:22"} +{"current_steps": 381, "total_steps": 594, "loss": 0.3465, "lr": 1.7330548393098403e-07, "epoch": 0.6416842105263157, "percentage": 64.14, "elapsed_time": "9:12:32", "remaining_time": "5:08:53"} +{"current_steps": 382, "total_steps": 594, "loss": 0.4055, "lr": 1.7190695574655144e-07, "epoch": 0.6433684210526316, "percentage": 64.31, "elapsed_time": "9:14:12", "remaining_time": "5:07:34"} +{"current_steps": 383, "total_steps": 594, "loss": 0.2469, "lr": 1.705111304518712e-07, "epoch": 0.6450526315789473, "percentage": 64.48, "elapsed_time": "9:15:50", "remaining_time": "5:06:13"} +{"current_steps": 384, "total_steps": 594, "loss": 0.3401, "lr": 1.6911805635805632e-07, "epoch": 0.6467368421052632, "percentage": 64.65, "elapsed_time": "9:17:32", "remaining_time": "5:04:54"} +{"current_steps": 385, "total_steps": 594, "loss": 0.3767, "lr": 1.677277816809975e-07, "epoch": 0.6484210526315789, "percentage": 64.81, "elapsed_time": "9:18:50", "remaining_time": "5:03:22"} +{"current_steps": 386, "total_steps": 594, "loss": 0.3759, "lr": 1.6634035453969457e-07, "epoch": 0.6501052631578947, "percentage": 64.98, "elapsed_time": "9:20:00", "remaining_time": "5:01:45"} +{"current_steps": 387, "total_steps": 594, "loss": 0.3717, "lr": 1.6495582295459078e-07, "epoch": 0.6517894736842105, "percentage": 65.15, "elapsed_time": "9:21:19", "remaining_time": "5:00:14"} +{"current_steps": 388, "total_steps": 594, "loss": 0.312, "lr": 1.6357423484591087e-07, "epoch": 0.6534736842105263, "percentage": 65.32, "elapsed_time": "9:22:58", "remaining_time": "4:58:53"} +{"current_steps": 389, "total_steps": 594, "loss": 0.4427, "lr": 1.621956380320027e-07, "epoch": 0.655157894736842, "percentage": 65.49, "elapsed_time": "9:24:22", "remaining_time": "4:57:25"} +{"current_steps": 390, "total_steps": 594, "loss": 0.3828, "lr": 1.608200802276815e-07, "epoch": 0.6568421052631579, "percentage": 65.66, "elapsed_time": "9:25:44", "remaining_time": "4:55:55"} +{"current_steps": 391, "total_steps": 594, "loss": 0.3784, "lr": 1.5944760904257942e-07, "epoch": 0.6585263157894737, "percentage": 65.82, "elapsed_time": "9:27:16", "remaining_time": "4:54:30"} +{"current_steps": 392, "total_steps": 594, "loss": 0.4446, "lr": 1.5807827197949685e-07, "epoch": 0.6602105263157895, "percentage": 65.99, "elapsed_time": "9:28:44", "remaining_time": "4:53:04"} +{"current_steps": 393, "total_steps": 594, "loss": 0.4374, "lr": 1.5671211643275875e-07, "epoch": 0.6618947368421053, "percentage": 66.16, "elapsed_time": "9:30:01", "remaining_time": "4:51:32"} +{"current_steps": 394, "total_steps": 594, "loss": 0.3553, "lr": 1.553491896865742e-07, "epoch": 0.663578947368421, "percentage": 66.33, "elapsed_time": "9:31:26", "remaining_time": "4:50:04"} +{"current_steps": 395, "total_steps": 594, "loss": 0.3877, "lr": 1.5398953891339968e-07, "epoch": 0.6652631578947369, "percentage": 66.5, "elapsed_time": "9:32:57", "remaining_time": "4:48:39"} +{"current_steps": 396, "total_steps": 594, "loss": 0.2986, "lr": 1.5263321117230655e-07, "epoch": 0.6669473684210526, "percentage": 66.67, "elapsed_time": "9:34:18", "remaining_time": "4:47:09"} +{"current_steps": 397, "total_steps": 594, "loss": 0.367, "lr": 1.512802534073522e-07, "epoch": 0.6686315789473685, "percentage": 66.84, "elapsed_time": "9:35:58", "remaining_time": "4:45:48"} +{"current_steps": 398, "total_steps": 594, "loss": 0.4321, "lr": 1.4993071244595534e-07, "epoch": 0.6703157894736842, "percentage": 67.0, "elapsed_time": "9:37:36", "remaining_time": "4:44:26"} +{"current_steps": 399, "total_steps": 594, "loss": 0.3035, "lr": 1.4858463499727507e-07, "epoch": 0.672, "percentage": 67.17, "elapsed_time": "9:38:57", "remaining_time": "4:42:57"} +{"current_steps": 400, "total_steps": 594, "loss": 0.4627, "lr": 1.4724206765059454e-07, "epoch": 0.6736842105263158, "percentage": 67.34, "elapsed_time": "9:40:17", "remaining_time": "4:41:26"} +{"current_steps": 401, "total_steps": 594, "loss": 0.3821, "lr": 1.459030568737081e-07, "epoch": 0.6753684210526316, "percentage": 67.51, "elapsed_time": "9:41:23", "remaining_time": "4:39:49"} +{"current_steps": 402, "total_steps": 594, "loss": 0.4017, "lr": 1.4456764901131308e-07, "epoch": 0.6770526315789474, "percentage": 67.68, "elapsed_time": "9:42:44", "remaining_time": "4:38:19"} +{"current_steps": 403, "total_steps": 594, "loss": 0.3506, "lr": 1.4323589028340597e-07, "epoch": 0.6787368421052632, "percentage": 67.85, "elapsed_time": "9:44:05", "remaining_time": "4:36:49"} +{"current_steps": 404, "total_steps": 594, "loss": 0.3981, "lr": 1.4190782678368257e-07, "epoch": 0.6804210526315789, "percentage": 68.01, "elapsed_time": "9:45:24", "remaining_time": "4:35:18"} +{"current_steps": 405, "total_steps": 594, "loss": 0.4264, "lr": 1.4058350447794235e-07, "epoch": 0.6821052631578948, "percentage": 68.18, "elapsed_time": "9:46:44", "remaining_time": "4:33:48"} +{"current_steps": 406, "total_steps": 594, "loss": 0.3206, "lr": 1.3926296920249793e-07, "epoch": 0.6837894736842105, "percentage": 68.35, "elapsed_time": "9:48:08", "remaining_time": "4:32:20"} +{"current_steps": 407, "total_steps": 594, "loss": 0.4434, "lr": 1.3794626666258866e-07, "epoch": 0.6854736842105263, "percentage": 68.52, "elapsed_time": "9:49:55", "remaining_time": "4:31:02"} +{"current_steps": 408, "total_steps": 594, "loss": 0.3924, "lr": 1.3663344243079806e-07, "epoch": 0.6871578947368421, "percentage": 68.69, "elapsed_time": "9:51:34", "remaining_time": "4:29:41"} +{"current_steps": 409, "total_steps": 594, "loss": 0.2897, "lr": 1.3532454194547732e-07, "epoch": 0.6888421052631579, "percentage": 68.86, "elapsed_time": "9:52:48", "remaining_time": "4:28:08"} +{"current_steps": 410, "total_steps": 594, "loss": 0.3623, "lr": 1.3401961050917228e-07, "epoch": 0.6905263157894737, "percentage": 69.02, "elapsed_time": "9:54:41", "remaining_time": "4:26:52"} +{"current_steps": 411, "total_steps": 594, "loss": 0.3754, "lr": 1.3271869328705516e-07, "epoch": 0.6922105263157895, "percentage": 69.19, "elapsed_time": "9:56:25", "remaining_time": "4:25:33"} +{"current_steps": 412, "total_steps": 594, "loss": 0.3954, "lr": 1.314218353053619e-07, "epoch": 0.6938947368421052, "percentage": 69.36, "elapsed_time": "9:57:40", "remaining_time": "4:24:01"} +{"current_steps": 413, "total_steps": 594, "loss": 0.3873, "lr": 1.301290814498335e-07, "epoch": 0.6955789473684211, "percentage": 69.53, "elapsed_time": "9:59:08", "remaining_time": "4:22:34"} +{"current_steps": 414, "total_steps": 594, "loss": 0.2895, "lr": 1.2884047646416205e-07, "epoch": 0.6972631578947368, "percentage": 69.7, "elapsed_time": "10:00:28", "remaining_time": "4:21:04"} +{"current_steps": 415, "total_steps": 594, "loss": 0.3066, "lr": 1.2755606494844294e-07, "epoch": 0.6989473684210527, "percentage": 69.87, "elapsed_time": "10:01:49", "remaining_time": "4:19:35"} +{"current_steps": 416, "total_steps": 594, "loss": 0.3805, "lr": 1.2627589135763066e-07, "epoch": 0.7006315789473684, "percentage": 70.03, "elapsed_time": "10:03:27", "remaining_time": "4:18:12"} +{"current_steps": 417, "total_steps": 594, "loss": 0.3545, "lr": 1.2500000000000005e-07, "epoch": 0.7023157894736842, "percentage": 70.2, "elapsed_time": "10:04:51", "remaining_time": "4:16:44"} +{"current_steps": 418, "total_steps": 594, "loss": 0.3653, "lr": 1.2372843503561318e-07, "epoch": 0.704, "percentage": 70.37, "elapsed_time": "10:06:42", "remaining_time": "4:15:27"} +{"current_steps": 419, "total_steps": 594, "loss": 0.2715, "lr": 1.2246124047479074e-07, "epoch": 0.7056842105263158, "percentage": 70.54, "elapsed_time": "10:08:27", "remaining_time": "4:14:07"} +{"current_steps": 420, "total_steps": 594, "loss": 0.4162, "lr": 1.211984601765884e-07, "epoch": 0.7073684210526315, "percentage": 70.71, "elapsed_time": "10:09:52", "remaining_time": "4:12:39"} +{"current_steps": 421, "total_steps": 594, "loss": 0.3481, "lr": 1.1994013784727946e-07, "epoch": 0.7090526315789474, "percentage": 70.88, "elapsed_time": "10:11:13", "remaining_time": "4:11:10"} +{"current_steps": 422, "total_steps": 594, "loss": 0.3706, "lr": 1.1868631703884183e-07, "epoch": 0.7107368421052631, "percentage": 71.04, "elapsed_time": "10:12:24", "remaining_time": "4:09:36"} +{"current_steps": 423, "total_steps": 594, "loss": 0.3508, "lr": 1.1743704114745029e-07, "epoch": 0.712421052631579, "percentage": 71.21, "elapsed_time": "10:13:51", "remaining_time": "4:08:09"} +{"current_steps": 424, "total_steps": 594, "loss": 0.4409, "lr": 1.1619235341197517e-07, "epoch": 0.7141052631578947, "percentage": 71.38, "elapsed_time": "10:15:17", "remaining_time": "4:06:41"} +{"current_steps": 425, "total_steps": 594, "loss": 0.3797, "lr": 1.1495229691248543e-07, "epoch": 0.7157894736842105, "percentage": 71.55, "elapsed_time": "10:16:47", "remaining_time": "4:05:15"} +{"current_steps": 426, "total_steps": 594, "loss": 0.4071, "lr": 1.1371691456875735e-07, "epoch": 0.7174736842105263, "percentage": 71.72, "elapsed_time": "10:18:05", "remaining_time": "4:03:45"} +{"current_steps": 427, "total_steps": 594, "loss": 0.4088, "lr": 1.1248624913878965e-07, "epoch": 0.7191578947368421, "percentage": 71.89, "elapsed_time": "10:19:21", "remaining_time": "4:02:13"} +{"current_steps": 428, "total_steps": 594, "loss": 0.3987, "lr": 1.1126034321732325e-07, "epoch": 0.7208421052631578, "percentage": 72.05, "elapsed_time": "10:20:53", "remaining_time": "4:00:48"} +{"current_steps": 429, "total_steps": 594, "loss": 0.3453, "lr": 1.100392392343667e-07, "epoch": 0.7225263157894737, "percentage": 72.22, "elapsed_time": "10:22:21", "remaining_time": "3:59:22"} +{"current_steps": 430, "total_steps": 594, "loss": 0.2549, "lr": 1.0882297945372828e-07, "epoch": 0.7242105263157895, "percentage": 72.39, "elapsed_time": "10:23:50", "remaining_time": "3:57:55"} +{"current_steps": 431, "total_steps": 594, "loss": 0.4655, "lr": 1.0761160597155286e-07, "epoch": 0.7258947368421053, "percentage": 72.56, "elapsed_time": "10:25:08", "remaining_time": "3:56:25"} +{"current_steps": 432, "total_steps": 594, "loss": 0.484, "lr": 1.0640516071486466e-07, "epoch": 0.7275789473684211, "percentage": 72.73, "elapsed_time": "10:26:07", "remaining_time": "3:54:47"} +{"current_steps": 433, "total_steps": 594, "loss": 0.3733, "lr": 1.052036854401166e-07, "epoch": 0.7292631578947368, "percentage": 72.9, "elapsed_time": "10:27:31", "remaining_time": "3:53:19"} +{"current_steps": 434, "total_steps": 594, "loss": 0.3971, "lr": 1.0400722173174489e-07, "epoch": 0.7309473684210527, "percentage": 73.06, "elapsed_time": "10:28:40", "remaining_time": "3:51:46"} +{"current_steps": 435, "total_steps": 594, "loss": 0.3316, "lr": 1.0281581100072939e-07, "epoch": 0.7326315789473684, "percentage": 73.23, "elapsed_time": "10:29:55", "remaining_time": "3:50:15"} +{"current_steps": 436, "total_steps": 594, "loss": 0.4252, "lr": 1.0162949448316088e-07, "epoch": 0.7343157894736843, "percentage": 73.4, "elapsed_time": "10:31:20", "remaining_time": "3:48:47"} +{"current_steps": 437, "total_steps": 594, "loss": 0.3626, "lr": 1.0044831323881357e-07, "epoch": 0.736, "percentage": 73.57, "elapsed_time": "10:32:23", "remaining_time": "3:47:11"} +{"current_steps": 438, "total_steps": 594, "loss": 0.4456, "lr": 9.927230814972382e-08, "epoch": 0.7376842105263158, "percentage": 73.74, "elapsed_time": "10:34:04", "remaining_time": "3:45:50"} +{"current_steps": 439, "total_steps": 594, "loss": 0.2965, "lr": 9.81015199187753e-08, "epoch": 0.7393684210526316, "percentage": 73.91, "elapsed_time": "10:35:39", "remaining_time": "3:44:26"} +{"current_steps": 440, "total_steps": 594, "loss": 0.4066, "lr": 9.693598906829046e-08, "epoch": 0.7410526315789474, "percentage": 74.07, "elapsed_time": "10:37:19", "remaining_time": "3:43:03"} +{"current_steps": 441, "total_steps": 594, "loss": 0.3972, "lr": 9.577575593862775e-08, "epoch": 0.7427368421052631, "percentage": 74.24, "elapsed_time": "10:38:45", "remaining_time": "3:41:36"} +{"current_steps": 442, "total_steps": 594, "loss": 0.401, "lr": 9.462086068678519e-08, "epoch": 0.744421052631579, "percentage": 74.41, "elapsed_time": "10:40:13", "remaining_time": "3:40:09"} +{"current_steps": 443, "total_steps": 594, "loss": 0.3954, "lr": 9.347134328501097e-08, "epoch": 0.7461052631578947, "percentage": 74.58, "elapsed_time": "10:41:31", "remaining_time": "3:38:40"} +{"current_steps": 444, "total_steps": 594, "loss": 0.3794, "lr": 9.232724351941978e-08, "epoch": 0.7477894736842106, "percentage": 74.75, "elapsed_time": "10:42:49", "remaining_time": "3:37:10"} +{"current_steps": 445, "total_steps": 594, "loss": 0.4162, "lr": 9.118860098861537e-08, "epoch": 0.7494736842105263, "percentage": 74.92, "elapsed_time": "10:43:58", "remaining_time": "3:35:37"} +{"current_steps": 446, "total_steps": 594, "loss": 0.2992, "lr": 9.005545510232068e-08, "epoch": 0.7511578947368421, "percentage": 75.08, "elapsed_time": "10:45:59", "remaining_time": "3:34:22"} +{"current_steps": 447, "total_steps": 594, "loss": 0.3587, "lr": 8.892784508001341e-08, "epoch": 0.7528421052631579, "percentage": 75.25, "elapsed_time": "10:47:44", "remaining_time": "3:33:00"} +{"current_steps": 448, "total_steps": 594, "loss": 0.3923, "lr": 8.780580994956849e-08, "epoch": 0.7545263157894737, "percentage": 75.42, "elapsed_time": "10:49:08", "remaining_time": "3:31:33"} +{"current_steps": 449, "total_steps": 594, "loss": 0.4064, "lr": 8.668938854590763e-08, "epoch": 0.7562105263157894, "percentage": 75.59, "elapsed_time": "10:50:31", "remaining_time": "3:30:04"} +{"current_steps": 450, "total_steps": 594, "loss": 0.3724, "lr": 8.557861950965509e-08, "epoch": 0.7578947368421053, "percentage": 75.76, "elapsed_time": "10:52:00", "remaining_time": "3:28:38"} +{"current_steps": 451, "total_steps": 594, "loss": 0.3763, "lr": 8.44735412857999e-08, "epoch": 0.759578947368421, "percentage": 75.93, "elapsed_time": "10:53:40", "remaining_time": "3:27:15"} +{"current_steps": 452, "total_steps": 594, "loss": 0.3628, "lr": 8.337419212236584e-08, "epoch": 0.7612631578947369, "percentage": 76.09, "elapsed_time": "10:55:04", "remaining_time": "3:25:47"} +{"current_steps": 453, "total_steps": 594, "loss": 0.3558, "lr": 8.228061006908738e-08, "epoch": 0.7629473684210526, "percentage": 76.26, "elapsed_time": "10:56:06", "remaining_time": "3:24:13"} +{"current_steps": 454, "total_steps": 594, "loss": 0.3183, "lr": 8.119283297609236e-08, "epoch": 0.7646315789473684, "percentage": 76.43, "elapsed_time": "10:57:38", "remaining_time": "3:22:47"} +{"current_steps": 455, "total_steps": 594, "loss": 0.3446, "lr": 8.011089849259262e-08, "epoch": 0.7663157894736842, "percentage": 76.6, "elapsed_time": "10:59:03", "remaining_time": "3:21:20"} +{"current_steps": 456, "total_steps": 594, "loss": 0.43, "lr": 7.903484406558053e-08, "epoch": 0.768, "percentage": 76.77, "elapsed_time": "11:00:17", "remaining_time": "3:19:49"} +{"current_steps": 457, "total_steps": 594, "loss": 0.4204, "lr": 7.79647069385328e-08, "epoch": 0.7696842105263157, "percentage": 76.94, "elapsed_time": "11:01:52", "remaining_time": "3:18:24"} +{"current_steps": 458, "total_steps": 594, "loss": 0.2499, "lr": 7.690052415012174e-08, "epoch": 0.7713684210526316, "percentage": 77.1, "elapsed_time": "11:03:16", "remaining_time": "3:16:57"} +{"current_steps": 459, "total_steps": 594, "loss": 0.2746, "lr": 7.584233253293326e-08, "epoch": 0.7730526315789473, "percentage": 77.27, "elapsed_time": "11:04:54", "remaining_time": "3:15:33"} +{"current_steps": 460, "total_steps": 594, "loss": 0.4074, "lr": 7.479016871219173e-08, "epoch": 0.7747368421052632, "percentage": 77.44, "elapsed_time": "11:06:15", "remaining_time": "3:14:04"} +{"current_steps": 461, "total_steps": 594, "loss": 0.3537, "lr": 7.374406910449276e-08, "epoch": 0.7764210526315789, "percentage": 77.61, "elapsed_time": "11:07:42", "remaining_time": "3:12:38"} +{"current_steps": 462, "total_steps": 594, "loss": 0.425, "lr": 7.270406991654273e-08, "epoch": 0.7781052631578947, "percentage": 77.78, "elapsed_time": "11:09:12", "remaining_time": "3:11:12"} +{"current_steps": 463, "total_steps": 594, "loss": 0.436, "lr": 7.167020714390501e-08, "epoch": 0.7797894736842105, "percentage": 77.95, "elapsed_time": "11:10:30", "remaining_time": "3:09:42"} +{"current_steps": 464, "total_steps": 594, "loss": 0.361, "lr": 7.064251656975503e-08, "epoch": 0.7814736842105263, "percentage": 78.11, "elapsed_time": "11:12:08", "remaining_time": "3:08:18"} +{"current_steps": 465, "total_steps": 594, "loss": 0.3961, "lr": 6.96210337636414e-08, "epoch": 0.783157894736842, "percentage": 78.28, "elapsed_time": "11:13:34", "remaining_time": "3:06:51"} +{"current_steps": 466, "total_steps": 594, "loss": 0.3267, "lr": 6.860579408025435e-08, "epoch": 0.7848421052631579, "percentage": 78.45, "elapsed_time": "11:15:02", "remaining_time": "3:05:25"} +{"current_steps": 467, "total_steps": 594, "loss": 0.4184, "lr": 6.759683265820293e-08, "epoch": 0.7865263157894736, "percentage": 78.62, "elapsed_time": "11:16:24", "remaining_time": "3:03:56"} +{"current_steps": 468, "total_steps": 594, "loss": 0.4054, "lr": 6.659418441879816e-08, "epoch": 0.7882105263157895, "percentage": 78.79, "elapsed_time": "11:17:42", "remaining_time": "3:02:27"} +{"current_steps": 469, "total_steps": 594, "loss": 0.3713, "lr": 6.559788406484445e-08, "epoch": 0.7898947368421053, "percentage": 78.96, "elapsed_time": "11:19:16", "remaining_time": "3:01:02"} +{"current_steps": 470, "total_steps": 594, "loss": 0.3572, "lr": 6.460796607943888e-08, "epoch": 0.791578947368421, "percentage": 79.12, "elapsed_time": "11:20:28", "remaining_time": "2:59:31"} +{"current_steps": 471, "total_steps": 594, "loss": 0.3431, "lr": 6.362446472477739e-08, "epoch": 0.7932631578947369, "percentage": 79.29, "elapsed_time": "11:21:38", "remaining_time": "2:58:00"} +{"current_steps": 472, "total_steps": 594, "loss": 0.4382, "lr": 6.264741404096873e-08, "epoch": 0.7949473684210526, "percentage": 79.46, "elapsed_time": "11:22:59", "remaining_time": "2:56:32"} +{"current_steps": 473, "total_steps": 594, "loss": 0.4316, "lr": 6.16768478448568e-08, "epoch": 0.7966315789473685, "percentage": 79.63, "elapsed_time": "11:24:13", "remaining_time": "2:55:02"} +{"current_steps": 474, "total_steps": 594, "loss": 0.374, "lr": 6.071279972884996e-08, "epoch": 0.7983157894736842, "percentage": 79.8, "elapsed_time": "11:25:43", "remaining_time": "2:53:36"} +{"current_steps": 475, "total_steps": 594, "loss": 0.3032, "lr": 5.975530305975806e-08, "epoch": 0.8, "percentage": 79.97, "elapsed_time": "11:27:08", "remaining_time": "2:52:08"} +{"current_steps": 476, "total_steps": 594, "loss": 0.3109, "lr": 5.88043909776382e-08, "epoch": 0.8016842105263158, "percentage": 80.13, "elapsed_time": "11:28:52", "remaining_time": "2:50:46"} +{"current_steps": 477, "total_steps": 594, "loss": 0.3819, "lr": 5.786009639464728e-08, "epoch": 0.8033684210526316, "percentage": 80.3, "elapsed_time": "11:30:35", "remaining_time": "2:49:23"} +{"current_steps": 478, "total_steps": 594, "loss": 0.4186, "lr": 5.69224519939028e-08, "epoch": 0.8050526315789474, "percentage": 80.47, "elapsed_time": "11:32:21", "remaining_time": "2:48:01"} +{"current_steps": 479, "total_steps": 594, "loss": 0.3371, "lr": 5.599149022835201e-08, "epoch": 0.8067368421052632, "percentage": 80.64, "elapsed_time": "11:33:45", "remaining_time": "2:46:33"} +{"current_steps": 480, "total_steps": 594, "loss": 0.3951, "lr": 5.5067243319648517e-08, "epoch": 0.8084210526315789, "percentage": 80.81, "elapsed_time": "11:35:03", "remaining_time": "2:45:04"} +{"current_steps": 481, "total_steps": 594, "loss": 0.4195, "lr": 5.414974325703686e-08, "epoch": 0.8101052631578948, "percentage": 80.98, "elapsed_time": "11:36:30", "remaining_time": "2:43:37"} +{"current_steps": 482, "total_steps": 594, "loss": 0.3827, "lr": 5.3239021796245707e-08, "epoch": 0.8117894736842105, "percentage": 81.14, "elapsed_time": "11:37:53", "remaining_time": "2:42:09"} +{"current_steps": 483, "total_steps": 594, "loss": 0.3977, "lr": 5.2335110458388457e-08, "epoch": 0.8134736842105263, "percentage": 81.31, "elapsed_time": "11:39:27", "remaining_time": "2:40:44"} +{"current_steps": 484, "total_steps": 594, "loss": 0.5002, "lr": 5.1438040528872265e-08, "epoch": 0.8151578947368421, "percentage": 81.48, "elapsed_time": "11:40:44", "remaining_time": "2:39:15"} +{"current_steps": 485, "total_steps": 594, "loss": 0.2893, "lr": 5.054784305631546e-08, "epoch": 0.8168421052631579, "percentage": 81.65, "elapsed_time": "11:42:07", "remaining_time": "2:37:47"} +{"current_steps": 486, "total_steps": 594, "loss": 0.4482, "lr": 4.96645488514727e-08, "epoch": 0.8185263157894737, "percentage": 81.82, "elapsed_time": "11:43:32", "remaining_time": "2:36:20"} +{"current_steps": 487, "total_steps": 594, "loss": 0.3179, "lr": 4.8788188486168616e-08, "epoch": 0.8202105263157895, "percentage": 81.99, "elapsed_time": "11:44:59", "remaining_time": "2:34:53"} +{"current_steps": 488, "total_steps": 594, "loss": 0.4311, "lr": 4.7918792292239635e-08, "epoch": 0.8218947368421052, "percentage": 82.15, "elapsed_time": "11:46:10", "remaining_time": "2:33:23"} +{"current_steps": 489, "total_steps": 594, "loss": 0.3739, "lr": 4.705639036048439e-08, "epoch": 0.8235789473684211, "percentage": 82.32, "elapsed_time": "11:47:43", "remaining_time": "2:31:57"} +{"current_steps": 490, "total_steps": 594, "loss": 0.3748, "lr": 4.6201012539622054e-08, "epoch": 0.8252631578947368, "percentage": 82.49, "elapsed_time": "11:49:14", "remaining_time": "2:30:32"} +{"current_steps": 491, "total_steps": 594, "loss": 0.4187, "lr": 4.535268843525908e-08, "epoch": 0.8269473684210527, "percentage": 82.66, "elapsed_time": "11:50:44", "remaining_time": "2:29:05"} +{"current_steps": 492, "total_steps": 594, "loss": 0.3157, "lr": 4.4511447408864975e-08, "epoch": 0.8286315789473684, "percentage": 82.83, "elapsed_time": "11:52:18", "remaining_time": "2:27:40"} +{"current_steps": 493, "total_steps": 594, "loss": 0.4666, "lr": 4.367731857675569e-08, "epoch": 0.8303157894736842, "percentage": 83.0, "elapsed_time": "11:53:55", "remaining_time": "2:26:15"} +{"current_steps": 494, "total_steps": 594, "loss": 0.3156, "lr": 4.2850330809085874e-08, "epoch": 0.832, "percentage": 83.16, "elapsed_time": "11:55:25", "remaining_time": "2:24:49"} +{"current_steps": 495, "total_steps": 594, "loss": 0.4533, "lr": 4.2030512728849945e-08, "epoch": 0.8336842105263158, "percentage": 83.33, "elapsed_time": "11:56:44", "remaining_time": "2:23:20"} +{"current_steps": 496, "total_steps": 594, "loss": 0.3971, "lr": 4.1217892710891126e-08, "epoch": 0.8353684210526315, "percentage": 83.5, "elapsed_time": "11:58:02", "remaining_time": "2:21:52"} +{"current_steps": 497, "total_steps": 594, "loss": 0.3902, "lr": 4.041249888091941e-08, "epoch": 0.8370526315789474, "percentage": 83.67, "elapsed_time": "11:59:44", "remaining_time": "2:20:28"} +{"current_steps": 498, "total_steps": 594, "loss": 0.4429, "lr": 3.9614359114538196e-08, "epoch": 0.8387368421052631, "percentage": 83.84, "elapsed_time": "12:01:11", "remaining_time": "2:19:01"} +{"current_steps": 499, "total_steps": 594, "loss": 0.3504, "lr": 3.8823501036279514e-08, "epoch": 0.840421052631579, "percentage": 84.01, "elapsed_time": "12:02:24", "remaining_time": "2:17:31"} +{"current_steps": 500, "total_steps": 594, "loss": 0.3079, "lr": 3.803995201864762e-08, "epoch": 0.8421052631578947, "percentage": 84.18, "elapsed_time": "12:03:57", "remaining_time": "2:16:06"} +{"current_steps": 500, "total_steps": 594, "eval_loss": 0.378118097782135, "epoch": 0.8421052631578947, "percentage": 84.18, "elapsed_time": "12:29:52", "remaining_time": "2:20:58"} +{"current_steps": 501, "total_steps": 594, "loss": 0.4355, "lr": 3.726373918117196e-08, "epoch": 0.8437894736842105, "percentage": 84.34, "elapsed_time": "12:31:24", "remaining_time": "2:19:28"} +{"current_steps": 502, "total_steps": 594, "loss": 0.3343, "lr": 3.6494889389468434e-08, "epoch": 0.8454736842105263, "percentage": 84.51, "elapsed_time": "12:33:07", "remaining_time": "2:18:01"} +{"current_steps": 503, "total_steps": 594, "loss": 0.373, "lr": 3.573342925430925e-08, "epoch": 0.8471578947368421, "percentage": 84.68, "elapsed_time": "12:34:25", "remaining_time": "2:16:29"} +{"current_steps": 504, "total_steps": 594, "loss": 0.3714, "lr": 3.497938513070234e-08, "epoch": 0.8488421052631578, "percentage": 84.85, "elapsed_time": "12:35:35", "remaining_time": "2:14:55"} +{"current_steps": 505, "total_steps": 594, "loss": 0.3555, "lr": 3.423278311697897e-08, "epoch": 0.8505263157894737, "percentage": 85.02, "elapsed_time": "12:37:01", "remaining_time": "2:13:24"} +{"current_steps": 506, "total_steps": 594, "loss": 0.3319, "lr": 3.349364905389032e-08, "epoch": 0.8522105263157894, "percentage": 85.19, "elapsed_time": "12:38:18", "remaining_time": "2:11:52"} +{"current_steps": 507, "total_steps": 594, "loss": 0.4044, "lr": 3.2762008523713386e-08, "epoch": 0.8538947368421053, "percentage": 85.35, "elapsed_time": "12:40:02", "remaining_time": "2:10:25"} +{"current_steps": 508, "total_steps": 594, "loss": 0.4473, "lr": 3.203788684936534e-08, "epoch": 0.8555789473684211, "percentage": 85.52, "elapsed_time": "12:41:33", "remaining_time": "2:08:55"} +{"current_steps": 509, "total_steps": 594, "loss": 0.4578, "lr": 3.132130909352709e-08, "epoch": 0.8572631578947368, "percentage": 85.69, "elapsed_time": "12:42:49", "remaining_time": "2:07:23"} +{"current_steps": 510, "total_steps": 594, "loss": 0.3928, "lr": 3.061230005777593e-08, "epoch": 0.8589473684210527, "percentage": 85.86, "elapsed_time": "12:44:03", "remaining_time": "2:05:50"} +{"current_steps": 511, "total_steps": 594, "loss": 0.3588, "lr": 2.991088428172722e-08, "epoch": 0.8606315789473684, "percentage": 86.03, "elapsed_time": "12:45:35", "remaining_time": "2:04:21"} +{"current_steps": 512, "total_steps": 594, "loss": 0.3516, "lr": 2.9217086042184534e-08, "epoch": 0.8623157894736843, "percentage": 86.2, "elapsed_time": "12:47:03", "remaining_time": "2:02:50"} +{"current_steps": 513, "total_steps": 594, "loss": 0.3888, "lr": 2.8530929352300087e-08, "epoch": 0.864, "percentage": 86.36, "elapsed_time": "12:48:36", "remaining_time": "2:01:21"} +{"current_steps": 514, "total_steps": 594, "loss": 0.3541, "lr": 2.7852437960743326e-08, "epoch": 0.8656842105263158, "percentage": 86.53, "elapsed_time": "12:50:02", "remaining_time": "1:59:50"} +{"current_steps": 515, "total_steps": 594, "loss": 0.3397, "lr": 2.718163535087864e-08, "epoch": 0.8673684210526316, "percentage": 86.7, "elapsed_time": "12:51:36", "remaining_time": "1:58:21"} +{"current_steps": 516, "total_steps": 594, "loss": 0.3362, "lr": 2.6518544739953187e-08, "epoch": 0.8690526315789474, "percentage": 86.87, "elapsed_time": "12:52:55", "remaining_time": "1:56:50"} +{"current_steps": 517, "total_steps": 594, "loss": 0.3661, "lr": 2.586318907829291e-08, "epoch": 0.8707368421052631, "percentage": 87.04, "elapsed_time": "12:54:27", "remaining_time": "1:55:20"} +{"current_steps": 518, "total_steps": 594, "loss": 0.4201, "lr": 2.5215591048508152e-08, "epoch": 0.872421052631579, "percentage": 87.21, "elapsed_time": "12:55:48", "remaining_time": "1:53:49"} +{"current_steps": 519, "total_steps": 594, "loss": 0.3464, "lr": 2.4575773064708898e-08, "epoch": 0.8741052631578947, "percentage": 87.37, "elapsed_time": "12:56:56", "remaining_time": "1:52:16"} +{"current_steps": 520, "total_steps": 594, "loss": 0.2565, "lr": 2.3943757271728816e-08, "epoch": 0.8757894736842106, "percentage": 87.54, "elapsed_time": "12:58:14", "remaining_time": "1:50:45"} +{"current_steps": 521, "total_steps": 594, "loss": 0.4034, "lr": 2.3319565544358628e-08, "epoch": 0.8774736842105263, "percentage": 87.71, "elapsed_time": "12:59:35", "remaining_time": "1:49:13"} +{"current_steps": 522, "total_steps": 594, "loss": 0.4473, "lr": 2.270321948658943e-08, "epoch": 0.8791578947368421, "percentage": 87.88, "elapsed_time": "13:01:28", "remaining_time": "1:47:47"} +{"current_steps": 523, "total_steps": 594, "loss": 0.4046, "lr": 2.2094740430864567e-08, "epoch": 0.8808421052631579, "percentage": 88.05, "elapsed_time": "13:02:40", "remaining_time": "1:46:15"} +{"current_steps": 524, "total_steps": 594, "loss": 0.3585, "lr": 2.1494149437341373e-08, "epoch": 0.8825263157894737, "percentage": 88.22, "elapsed_time": "13:04:25", "remaining_time": "1:44:47"} +{"current_steps": 525, "total_steps": 594, "loss": 0.4087, "lr": 2.0901467293162444e-08, "epoch": 0.8842105263157894, "percentage": 88.38, "elapsed_time": "13:05:38", "remaining_time": "1:43:15"} +{"current_steps": 526, "total_steps": 594, "loss": 0.259, "lr": 2.0316714511735998e-08, "epoch": 0.8858947368421053, "percentage": 88.55, "elapsed_time": "13:06:40", "remaining_time": "1:41:42"} +{"current_steps": 527, "total_steps": 594, "loss": 0.34, "lr": 1.9739911332025793e-08, "epoch": 0.887578947368421, "percentage": 88.72, "elapsed_time": "13:07:55", "remaining_time": "1:40:10"} +{"current_steps": 528, "total_steps": 594, "loss": 0.3229, "lr": 1.9171077717850955e-08, "epoch": 0.8892631578947369, "percentage": 88.89, "elapsed_time": "13:09:25", "remaining_time": "1:38:40"} +{"current_steps": 529, "total_steps": 594, "loss": 0.3711, "lr": 1.8610233357194747e-08, "epoch": 0.8909473684210526, "percentage": 89.06, "elapsed_time": "13:10:53", "remaining_time": "1:37:10"} +{"current_steps": 530, "total_steps": 594, "loss": 0.3876, "lr": 1.805739766152309e-08, "epoch": 0.8926315789473684, "percentage": 89.23, "elapsed_time": "13:12:50", "remaining_time": "1:35:44"} +{"current_steps": 531, "total_steps": 594, "loss": 0.3033, "lr": 1.7512589765112994e-08, "epoch": 0.8943157894736842, "percentage": 89.39, "elapsed_time": "13:14:37", "remaining_time": "1:34:16"} +{"current_steps": 532, "total_steps": 594, "loss": 0.3423, "lr": 1.697582852439011e-08, "epoch": 0.896, "percentage": 89.56, "elapsed_time": "13:16:22", "remaining_time": "1:32:48"} +{"current_steps": 533, "total_steps": 594, "loss": 0.3967, "lr": 1.6447132517276004e-08, "epoch": 0.8976842105263158, "percentage": 89.73, "elapsed_time": "13:18:00", "remaining_time": "1:31:19"} +{"current_steps": 534, "total_steps": 594, "loss": 0.3006, "lr": 1.5926520042545383e-08, "epoch": 0.8993684210526316, "percentage": 89.9, "elapsed_time": "13:19:40", "remaining_time": "1:29:51"} +{"current_steps": 535, "total_steps": 594, "loss": 0.4779, "lr": 1.5414009119192635e-08, "epoch": 0.9010526315789473, "percentage": 90.07, "elapsed_time": "13:21:03", "remaining_time": "1:28:20"} +{"current_steps": 536, "total_steps": 594, "loss": 0.3913, "lr": 1.4909617485808073e-08, "epoch": 0.9027368421052632, "percentage": 90.24, "elapsed_time": "13:22:46", "remaining_time": "1:26:52"} +{"current_steps": 537, "total_steps": 594, "loss": 0.4247, "lr": 1.4413362599964118e-08, "epoch": 0.9044210526315789, "percentage": 90.4, "elapsed_time": "13:24:08", "remaining_time": "1:25:21"} +{"current_steps": 538, "total_steps": 594, "loss": 0.4245, "lr": 1.3925261637611068e-08, "epoch": 0.9061052631578947, "percentage": 90.57, "elapsed_time": "13:26:14", "remaining_time": "1:23:55"} +{"current_steps": 539, "total_steps": 594, "loss": 0.3553, "lr": 1.3445331492482614e-08, "epoch": 0.9077894736842105, "percentage": 90.74, "elapsed_time": "13:27:58", "remaining_time": "1:22:26"} +{"current_steps": 540, "total_steps": 594, "loss": 0.325, "lr": 1.2973588775511024e-08, "epoch": 0.9094736842105263, "percentage": 90.91, "elapsed_time": "13:29:30", "remaining_time": "1:20:57"} +{"current_steps": 541, "total_steps": 594, "loss": 0.4062, "lr": 1.2510049814252299e-08, "epoch": 0.9111578947368421, "percentage": 91.08, "elapsed_time": "13:30:38", "remaining_time": "1:19:24"} +{"current_steps": 542, "total_steps": 594, "loss": 0.4345, "lr": 1.2054730652321127e-08, "epoch": 0.9128421052631579, "percentage": 91.25, "elapsed_time": "13:31:48", "remaining_time": "1:17:53"} +{"current_steps": 543, "total_steps": 594, "loss": 0.3224, "lr": 1.1607647048835462e-08, "epoch": 0.9145263157894736, "percentage": 91.41, "elapsed_time": "13:32:50", "remaining_time": "1:16:20"} +{"current_steps": 544, "total_steps": 594, "loss": 0.357, "lr": 1.116881447787113e-08, "epoch": 0.9162105263157895, "percentage": 91.58, "elapsed_time": "13:34:11", "remaining_time": "1:14:50"} +{"current_steps": 545, "total_steps": 594, "loss": 0.3392, "lr": 1.0738248127926342e-08, "epoch": 0.9178947368421052, "percentage": 91.75, "elapsed_time": "13:35:49", "remaining_time": "1:13:20"} +{"current_steps": 546, "total_steps": 594, "loss": 0.3334, "lr": 1.0315962901395802e-08, "epoch": 0.919578947368421, "percentage": 91.92, "elapsed_time": "13:37:09", "remaining_time": "1:11:50"} +{"current_steps": 547, "total_steps": 594, "loss": 0.4516, "lr": 9.901973414055186e-09, "epoch": 0.9212631578947369, "percentage": 92.09, "elapsed_time": "13:38:23", "remaining_time": "1:10:19"} +{"current_steps": 548, "total_steps": 594, "loss": 0.4624, "lr": 9.496293994555066e-09, "epoch": 0.9229473684210526, "percentage": 92.26, "elapsed_time": "13:40:11", "remaining_time": "1:08:50"} +{"current_steps": 549, "total_steps": 594, "loss": 0.4655, "lr": 9.098938683924972e-09, "epoch": 0.9246315789473685, "percentage": 92.42, "elapsed_time": "13:41:32", "remaining_time": "1:07:20"} +{"current_steps": 550, "total_steps": 594, "loss": 0.3937, "lr": 8.709921235087597e-09, "epoch": 0.9263157894736842, "percentage": 92.59, "elapsed_time": "13:43:05", "remaining_time": "1:05:50"} +{"current_steps": 551, "total_steps": 594, "loss": 0.3363, "lr": 8.329255112382665e-09, "epoch": 0.928, "percentage": 92.76, "elapsed_time": "13:44:28", "remaining_time": "1:04:20"} +{"current_steps": 552, "total_steps": 594, "loss": 0.3715, "lr": 7.956953491100871e-09, "epoch": 0.9296842105263158, "percentage": 92.93, "elapsed_time": "13:45:39", "remaining_time": "1:02:49"} +{"current_steps": 553, "total_steps": 594, "loss": 0.3602, "lr": 7.593029257027956e-09, "epoch": 0.9313684210526316, "percentage": 93.1, "elapsed_time": "13:47:01", "remaining_time": "1:01:18"} +{"current_steps": 554, "total_steps": 594, "loss": 0.3429, "lr": 7.23749500599874e-09, "epoch": 0.9330526315789474, "percentage": 93.27, "elapsed_time": "13:48:30", "remaining_time": "0:59:49"} +{"current_steps": 555, "total_steps": 594, "loss": 0.3214, "lr": 6.89036304346105e-09, "epoch": 0.9347368421052632, "percentage": 93.43, "elapsed_time": "13:49:52", "remaining_time": "0:58:18"} +{"current_steps": 556, "total_steps": 594, "loss": 0.3524, "lr": 6.551645384049897e-09, "epoch": 0.9364210526315789, "percentage": 93.6, "elapsed_time": "13:51:28", "remaining_time": "0:56:49"} +{"current_steps": 557, "total_steps": 594, "loss": 0.3274, "lr": 6.221353751171665e-09, "epoch": 0.9381052631578948, "percentage": 93.77, "elapsed_time": "13:52:40", "remaining_time": "0:55:18"} +{"current_steps": 558, "total_steps": 594, "loss": 0.36, "lr": 5.899499576598216e-09, "epoch": 0.9397894736842105, "percentage": 93.94, "elapsed_time": "13:53:44", "remaining_time": "0:53:47"} +{"current_steps": 559, "total_steps": 594, "loss": 0.4097, "lr": 5.586094000071401e-09, "epoch": 0.9414736842105264, "percentage": 94.11, "elapsed_time": "13:54:49", "remaining_time": "0:52:16"} +{"current_steps": 560, "total_steps": 594, "loss": 0.3742, "lr": 5.2811478689173686e-09, "epoch": 0.9431578947368421, "percentage": 94.28, "elapsed_time": "13:56:04", "remaining_time": "0:50:45"} +{"current_steps": 561, "total_steps": 594, "loss": 0.4038, "lr": 4.984671737671142e-09, "epoch": 0.9448421052631579, "percentage": 94.44, "elapsed_time": "13:57:06", "remaining_time": "0:49:14"} +{"current_steps": 562, "total_steps": 594, "loss": 0.4127, "lr": 4.696675867711386e-09, "epoch": 0.9465263157894737, "percentage": 94.61, "elapsed_time": "13:58:15", "remaining_time": "0:47:43"} +{"current_steps": 563, "total_steps": 594, "loss": 0.3022, "lr": 4.417170226905187e-09, "epoch": 0.9482105263157895, "percentage": 94.78, "elapsed_time": "13:59:57", "remaining_time": "0:46:15"} +{"current_steps": 564, "total_steps": 594, "loss": 0.4415, "lr": 4.146164489263054e-09, "epoch": 0.9498947368421052, "percentage": 94.95, "elapsed_time": "14:01:15", "remaining_time": "0:44:44"} +{"current_steps": 565, "total_steps": 594, "loss": 0.3607, "lr": 3.88366803460416e-09, "epoch": 0.9515789473684211, "percentage": 95.12, "elapsed_time": "14:02:37", "remaining_time": "0:43:14"} +{"current_steps": 566, "total_steps": 594, "loss": 0.4462, "lr": 3.6296899482316236e-09, "epoch": 0.9532631578947368, "percentage": 95.29, "elapsed_time": "14:03:48", "remaining_time": "0:41:44"} +{"current_steps": 567, "total_steps": 594, "loss": 0.3247, "lr": 3.384239020618018e-09, "epoch": 0.9549473684210527, "percentage": 95.45, "elapsed_time": "14:05:34", "remaining_time": "0:40:15"} +{"current_steps": 568, "total_steps": 594, "loss": 0.3644, "lr": 3.1473237471012214e-09, "epoch": 0.9566315789473684, "percentage": 95.62, "elapsed_time": "14:06:56", "remaining_time": "0:38:46"} +{"current_steps": 569, "total_steps": 594, "loss": 0.3284, "lr": 2.9189523275903736e-09, "epoch": 0.9583157894736842, "percentage": 95.79, "elapsed_time": "14:08:27", "remaining_time": "0:37:16"} +{"current_steps": 570, "total_steps": 594, "loss": 0.3478, "lr": 2.6991326662819667e-09, "epoch": 0.96, "percentage": 95.96, "elapsed_time": "14:09:56", "remaining_time": "0:35:47"} +{"current_steps": 571, "total_steps": 594, "loss": 0.3783, "lr": 2.4878723713864234e-09, "epoch": 0.9616842105263158, "percentage": 96.13, "elapsed_time": "14:11:21", "remaining_time": "0:34:17"} +{"current_steps": 572, "total_steps": 594, "loss": 0.4376, "lr": 2.285178754864614e-09, "epoch": 0.9633684210526315, "percentage": 96.3, "elapsed_time": "14:12:52", "remaining_time": "0:32:48"} +{"current_steps": 573, "total_steps": 594, "loss": 0.317, "lr": 2.091058832174891e-09, "epoch": 0.9650526315789474, "percentage": 96.46, "elapsed_time": "14:14:12", "remaining_time": "0:31:18"} +{"current_steps": 574, "total_steps": 594, "loss": 0.428, "lr": 1.905519322030258e-09, "epoch": 0.9667368421052631, "percentage": 96.63, "elapsed_time": "14:15:28", "remaining_time": "0:29:48"} +{"current_steps": 575, "total_steps": 594, "loss": 0.4116, "lr": 1.7285666461657467e-09, "epoch": 0.968421052631579, "percentage": 96.8, "elapsed_time": "14:16:45", "remaining_time": "0:28:18"} +{"current_steps": 576, "total_steps": 594, "loss": 0.2972, "lr": 1.5602069291162368e-09, "epoch": 0.9701052631578947, "percentage": 96.97, "elapsed_time": "14:18:45", "remaining_time": "0:26:50"} +{"current_steps": 577, "total_steps": 594, "loss": 0.4455, "lr": 1.4004459980045124e-09, "epoch": 0.9717894736842105, "percentage": 97.14, "elapsed_time": "14:20:03", "remaining_time": "0:25:20"} +{"current_steps": 578, "total_steps": 594, "loss": 0.3929, "lr": 1.2492893823394246e-09, "epoch": 0.9734736842105263, "percentage": 97.31, "elapsed_time": "14:21:18", "remaining_time": "0:23:50"} +{"current_steps": 579, "total_steps": 594, "loss": 0.3157, "lr": 1.10674231382471e-09, "epoch": 0.9751578947368421, "percentage": 97.47, "elapsed_time": "14:22:20", "remaining_time": "0:22:20"} +{"current_steps": 580, "total_steps": 594, "loss": 0.3864, "lr": 9.7280972617772e-10, "epoch": 0.9768421052631578, "percentage": 97.64, "elapsed_time": "14:24:19", "remaining_time": "0:20:51"} +{"current_steps": 581, "total_steps": 594, "loss": 0.3613, "lr": 8.474962549588349e-10, "epoch": 0.9785263157894737, "percentage": 97.81, "elapsed_time": "14:25:52", "remaining_time": "0:19:22"} +{"current_steps": 582, "total_steps": 594, "loss": 0.3856, "lr": 7.308062374108692e-10, "epoch": 0.9802105263157894, "percentage": 97.98, "elapsed_time": "14:27:29", "remaining_time": "0:17:53"} +{"current_steps": 583, "total_steps": 594, "loss": 0.2775, "lr": 6.227437123090539e-10, "epoch": 0.9818947368421053, "percentage": 98.15, "elapsed_time": "14:28:54", "remaining_time": "0:16:23"} +{"current_steps": 584, "total_steps": 594, "loss": 0.3426, "lr": 5.233124198212035e-10, "epoch": 0.983578947368421, "percentage": 98.32, "elapsed_time": "14:30:11", "remaining_time": "0:14:54"} +{"current_steps": 585, "total_steps": 594, "loss": 0.2408, "lr": 4.3251580137831924e-10, "epoch": 0.9852631578947368, "percentage": 98.48, "elapsed_time": "14:31:21", "remaining_time": "0:13:24"} +{"current_steps": 586, "total_steps": 594, "loss": 0.2932, "lr": 3.5035699955540675e-10, "epoch": 0.9869473684210527, "percentage": 98.65, "elapsed_time": "14:32:37", "remaining_time": "0:11:54"} +{"current_steps": 587, "total_steps": 594, "loss": 0.3505, "lr": 2.768388579627301e-10, "epoch": 0.9886315789473684, "percentage": 98.82, "elapsed_time": "14:34:09", "remaining_time": "0:10:25"} +{"current_steps": 588, "total_steps": 594, "loss": 0.3191, "lr": 2.1196392114744556e-10, "epoch": 0.9903157894736843, "percentage": 98.99, "elapsed_time": "14:35:24", "remaining_time": "0:08:55"} +{"current_steps": 589, "total_steps": 594, "loss": 0.3878, "lr": 1.5573443450545009e-10, "epoch": 0.992, "percentage": 99.16, "elapsed_time": "14:36:55", "remaining_time": "0:07:26"} +{"current_steps": 590, "total_steps": 594, "loss": 0.3653, "lr": 1.0815234420369357e-10, "epoch": 0.9936842105263158, "percentage": 99.33, "elapsed_time": "14:38:19", "remaining_time": "0:05:57"} +{"current_steps": 591, "total_steps": 594, "loss": 0.4221, "lr": 6.921929711287134e-11, "epoch": 0.9953684210526316, "percentage": 99.49, "elapsed_time": "14:39:41", "remaining_time": "0:04:27"} +{"current_steps": 592, "total_steps": 594, "loss": 0.4209, "lr": 3.893664075035885e-11, "epoch": 0.9970526315789474, "percentage": 99.66, "elapsed_time": "14:41:13", "remaining_time": "0:02:58"} +{"current_steps": 593, "total_steps": 594, "loss": 0.3481, "lr": 1.730542323355455e-11, "epoch": 0.9987368421052631, "percentage": 99.83, "elapsed_time": "14:43:23", "remaining_time": "0:01:29"} +{"current_steps": 594, "total_steps": 594, "loss": 0.342, "lr": 4.326393243742066e-12, "epoch": 1.0, "percentage": 100.0, "elapsed_time": "14:44:47", "remaining_time": "0:00:00"} +{"current_steps": 594, "total_steps": 594, "epoch": 1.0, "percentage": 100.0, "elapsed_time": "14:55:27", "remaining_time": "0:00:00"} diff --git a/trainer_state.json b/trainer_state.json new file mode 100644 index 0000000..da55463 --- /dev/null +++ b/trainer_state.json @@ -0,0 +1,8953 @@ +{ + "best_global_step": null, + "best_metric": null, + "best_model_checkpoint": null, + "epoch": 1.0, + "eval_steps": 500, + "global_step": 594, + "is_hyper_param_search": false, + "is_local_process_zero": true, + "is_world_process_zero": true, + "log_history": [ + { + "epoch": 0.0016842105263157896, + "grad_norm": 756.2248967148687, + "kl": 0.0, + "learning_rate": 0.0, + "logits/chosen": -1430707968.0, + "logits/rejected": -627221650.2857143, + "logps/chosen": -274.7185974121094, + "logps/rejected": -244.01890345982142, + "loss": 0.5, + "rewards/chosen": 0.0, + "rewards/margins": 0.0, + "rewards/rejected": 0.0, + "step": 1 + }, + { + "epoch": 0.003368421052631579, + "grad_norm": 127.71874425517983, + "kl": 0.0, + "learning_rate": 8.333333333333334e-09, + "logits/chosen": -625243200.0, + "logits/rejected": -765075712.0, + "logps/chosen": -214.222900390625, + "logps/rejected": -240.65220642089844, + "loss": 0.5, + "rewards/chosen": 0.0, + "rewards/margins": 0.0, + "rewards/rejected": 0.0, + "step": 2 + }, + { + "epoch": 0.0050526315789473685, + "grad_norm": 181.90697566158104, + "kl": 0.4360809326171875, + "learning_rate": 1.6666666666666667e-08, + "logits/chosen": -1081217792.0, + "logits/rejected": -543856768.0, + "logps/chosen": -203.947265625, + "logps/rejected": -286.96112060546875, + "loss": 0.4983, + "rewards/chosen": -0.00811920315027237, + "rewards/margins": -0.013331988826394081, + "rewards/rejected": 0.005212785676121712, + "step": 3 + }, + { + "epoch": 0.006736842105263158, + "grad_norm": 157.50669772082293, + "kl": 0.939788818359375, + "learning_rate": 2.5e-08, + "logits/chosen": -665212416.0, + "logits/rejected": -989478912.0, + "logps/chosen": -212.04036458333334, + "logps/rejected": -268.3646484375, + "loss": 0.4997, + "rewards/chosen": -0.022487640380859375, + "rewards/margins": -0.013136444240808487, + "rewards/rejected": -0.009351196140050888, + "step": 4 + }, + { + "epoch": 0.008421052631578947, + "grad_norm": 323.0915477325882, + "kl": 0.5852317810058594, + "learning_rate": 3.3333333333333334e-08, + "logits/chosen": -1499776170.6666667, + "logits/rejected": -823883468.8, + "logps/chosen": -206.73421223958334, + "logps/rejected": -249.1157470703125, + "loss": 0.4999, + "rewards/chosen": 0.03809356689453125, + "rewards/margins": 0.03527191206812859, + "rewards/rejected": 0.002821654826402664, + "step": 5 + }, + { + "epoch": 0.010105263157894737, + "grad_norm": 985.1109364386158, + "kl": 1.2429885864257812, + "learning_rate": 4.166666666666666e-08, + "logits/chosen": -830737578.6666666, + "logits/rejected": -1135501516.8, + "logps/chosen": -179.890869140625, + "logps/rejected": -192.8841796875, + "loss": 0.499, + "rewards/chosen": 0.012117513765891394, + "rewards/margins": -0.012128396580616633, + "rewards/rejected": 0.024245910346508026, + "step": 6 + }, + { + "epoch": 0.011789473684210527, + "grad_norm": 122.52722890356792, + "kl": 2.1596755981445312, + "learning_rate": 5e-08, + "logits/chosen": -917885610.6666666, + "logits/rejected": -1120814080.0, + "logps/chosen": -223.6461181640625, + "logps/rejected": -218.6181396484375, + "loss": 0.5027, + "rewards/chosen": -0.0010091145522892475, + "rewards/margins": -0.017569171357899906, + "rewards/rejected": 0.016560056805610658, + "step": 7 + }, + { + "epoch": 0.013473684210526317, + "grad_norm": 95.64200187206991, + "kl": 1.8461112976074219, + "learning_rate": 5.833333333333333e-08, + "logits/chosen": -963475648.0, + "logits/rejected": -777407829.3333334, + "logps/chosen": -194.63409423828125, + "logps/rejected": -209.066162109375, + "loss": 0.4996, + "rewards/chosen": 0.036714937537908554, + "rewards/margins": 0.05407130097349484, + "rewards/rejected": -0.017356363435586292, + "step": 8 + }, + { + "epoch": 0.015157894736842105, + "grad_norm": 230.4658286109559, + "kl": 0.5257720947265625, + "learning_rate": 6.666666666666667e-08, + "logits/chosen": -474723840.0, + "logits/rejected": -627376426.6666666, + "logps/chosen": -223.980908203125, + "logps/rejected": -260.8198649088542, + "loss": 0.4984, + "rewards/chosen": 0.005419158190488815, + "rewards/margins": 0.021833036839962006, + "rewards/rejected": -0.01641387864947319, + "step": 9 + }, + { + "epoch": 0.016842105263157894, + "grad_norm": 144.31561283203507, + "kl": 2.7238082885742188, + "learning_rate": 7.5e-08, + "logits/chosen": -834623385.6, + "logits/rejected": -1033011200.0, + "logps/chosen": -257.28134765625, + "logps/rejected": -233.40643310546875, + "loss": 0.502, + "rewards/chosen": 0.04759369194507599, + "rewards/margins": 0.024074305097262067, + "rewards/rejected": 0.023519386847813923, + "step": 10 + }, + { + "epoch": 0.018526315789473686, + "grad_norm": 246.23166775055776, + "kl": 0.989532470703125, + "learning_rate": 8.333333333333333e-08, + "logits/chosen": -933497548.8, + "logits/rejected": -1285298858.6666667, + "logps/chosen": -221.352392578125, + "logps/rejected": -204.99224853515625, + "loss": 0.4965, + "rewards/chosen": 0.0369128406047821, + "rewards/margins": 0.07419260541598002, + "rewards/rejected": -0.037279764811197914, + "step": 11 + }, + { + "epoch": 0.020210526315789474, + "grad_norm": 173.9408541060697, + "kl": 0.8486251831054688, + "learning_rate": 9.166666666666665e-08, + "logits/chosen": -960234432.0, + "logits/rejected": -981980800.0, + "logps/chosen": -243.4825439453125, + "logps/rejected": -170.26206970214844, + "loss": 0.5027, + "rewards/chosen": -0.004507828503847122, + "rewards/margins": -0.03447256237268448, + "rewards/rejected": 0.029964733868837357, + "step": 12 + }, + { + "epoch": 0.021894736842105262, + "grad_norm": 175.56996418245757, + "kl": 0.223480224609375, + "learning_rate": 1e-07, + "logits/chosen": -1123834752.0, + "logits/rejected": -974051264.0, + "logps/chosen": -217.30453491210938, + "logps/rejected": -285.2402038574219, + "loss": 0.498, + "rewards/chosen": 0.011188508942723274, + "rewards/margins": 0.04091682657599449, + "rewards/rejected": -0.029728317633271217, + "step": 13 + }, + { + "epoch": 0.023578947368421053, + "grad_norm": 109.85035410014211, + "kl": 1.2569313049316406, + "learning_rate": 1.0833333333333334e-07, + "logits/chosen": -752794521.6, + "logits/rejected": -981168981.3333334, + "logps/chosen": -259.7815673828125, + "logps/rejected": -145.32327270507812, + "loss": 0.4979, + "rewards/chosen": 0.027963563799858093, + "rewards/margins": 0.05449656397104263, + "rewards/rejected": -0.02653300017118454, + "step": 14 + }, + { + "epoch": 0.02526315789473684, + "grad_norm": 112.13711806138755, + "kl": 0.5673065185546875, + "learning_rate": 1.1666666666666667e-07, + "logits/chosen": -726509824.0, + "logits/rejected": -840169088.0, + "logps/chosen": -206.02989196777344, + "logps/rejected": -283.2633972167969, + "loss": 0.4991, + "rewards/chosen": 0.005207442678511143, + "rewards/margins": 0.007555770222097635, + "rewards/rejected": -0.0023483275435864925, + "step": 15 + }, + { + "epoch": 0.026947368421052633, + "grad_norm": 103.03187365170976, + "kl": 0.21651458740234375, + "learning_rate": 1.25e-07, + "logits/chosen": -1134580838.4, + "logits/rejected": -816441002.6666666, + "logps/chosen": -255.2339111328125, + "logps/rejected": -226.8826904296875, + "loss": 0.5015, + "rewards/chosen": -0.0194052129983902, + "rewards/margins": -0.014941000193357468, + "rewards/rejected": -0.00446421280503273, + "step": 16 + }, + { + "epoch": 0.02863157894736842, + "grad_norm": 775.9066138659358, + "kl": 1.3112754821777344, + "learning_rate": 1.3333333333333334e-07, + "logits/chosen": -1642847744.0, + "logits/rejected": -1082994346.6666667, + "logps/chosen": -223.9878692626953, + "logps/rejected": -277.746337890625, + "loss": 0.4999, + "rewards/chosen": 0.01275329664349556, + "rewards/margins": 0.02697169159849485, + "rewards/rejected": -0.014218394954999289, + "step": 17 + }, + { + "epoch": 0.03031578947368421, + "grad_norm": 168.9285804872629, + "kl": 0.6275863647460938, + "learning_rate": 1.4166666666666665e-07, + "logits/chosen": -667281792.0, + "logits/rejected": -773589248.0, + "logps/chosen": -258.7420654296875, + "logps/rejected": -229.06549072265625, + "loss": 0.5024, + "rewards/chosen": 8.049095049500465e-05, + "rewards/margins": -0.011579894926398993, + "rewards/rejected": 0.011660385876893997, + "step": 18 + }, + { + "epoch": 0.032, + "grad_norm": 96.35367098860345, + "kl": 0.8299026489257812, + "learning_rate": 1.5e-07, + "logits/chosen": -1054523520.0, + "logits/rejected": -1119684096.0, + "logps/chosen": -227.5456085205078, + "logps/rejected": -175.5878448486328, + "loss": 0.5028, + "rewards/chosen": -0.006691360846161842, + "rewards/margins": -0.007221984677016735, + "rewards/rejected": 0.0005306238308548927, + "step": 19 + }, + { + "epoch": 0.03368421052631579, + "grad_norm": 212.06858439091923, + "kl": 0.49672698974609375, + "learning_rate": 1.583333333333333e-07, + "logits/chosen": -1220449484.8, + "logits/rejected": -1335795029.3333333, + "logps/chosen": -192.7220947265625, + "logps/rejected": -213.80257161458334, + "loss": 0.4997, + "rewards/chosen": 0.028179934620857237, + "rewards/margins": 0.10689647098382313, + "rewards/rejected": -0.0787165363629659, + "step": 20 + }, + { + "epoch": 0.03536842105263158, + "grad_norm": 142.05363612622702, + "kl": 0.5618019104003906, + "learning_rate": 1.6666666666666665e-07, + "logits/chosen": -865751616.0, + "logits/rejected": -749211562.6666666, + "logps/chosen": -274.3310852050781, + "logps/rejected": -161.32567342122397, + "loss": 0.5039, + "rewards/chosen": 0.006252288818359375, + "rewards/margins": -0.04492963353792826, + "rewards/rejected": 0.051181922356287636, + "step": 21 + }, + { + "epoch": 0.03705263157894737, + "grad_norm": 202.39733674967437, + "kl": 2.1550140380859375, + "learning_rate": 1.75e-07, + "logits/chosen": -852115285.3333334, + "logits/rejected": -1103316787.2, + "logps/chosen": -221.8100382486979, + "logps/rejected": -217.199462890625, + "loss": 0.4971, + "rewards/chosen": 0.03007812798023224, + "rewards/margins": 0.013606876134872437, + "rewards/rejected": 0.016471251845359802, + "step": 22 + }, + { + "epoch": 0.03873684210526316, + "grad_norm": 205.96536510497384, + "kl": 0.9358940124511719, + "learning_rate": 1.833333333333333e-07, + "logits/chosen": -1277413580.8, + "logits/rejected": -971898112.0, + "logps/chosen": -217.6686767578125, + "logps/rejected": -276.12188720703125, + "loss": 0.5003, + "rewards/chosen": 0.022262269258499147, + "rewards/margins": 0.0022859687606493657, + "rewards/rejected": 0.01997630049784978, + "step": 23 + }, + { + "epoch": 0.04042105263157895, + "grad_norm": 97.86200798922602, + "kl": 1.6557350158691406, + "learning_rate": 1.9166666666666668e-07, + "logits/chosen": -837048012.8, + "logits/rejected": -1441815552.0, + "logps/chosen": -212.37763671875, + "logps/rejected": -136.08146158854166, + "loss": 0.5045, + "rewards/chosen": -0.0468679815530777, + "rewards/margins": -0.06955506652593613, + "rewards/rejected": 0.02268708497285843, + "step": 24 + }, + { + "epoch": 0.042105263157894736, + "grad_norm": 146.9171587034632, + "kl": 1.9702682495117188, + "learning_rate": 2e-07, + "logits/chosen": -692624256.0, + "logits/rejected": -764641536.0, + "logps/chosen": -240.31827799479166, + "logps/rejected": -236.6541015625, + "loss": 0.5038, + "rewards/chosen": -0.0033594767252604165, + "rewards/margins": 0.008408406128485998, + "rewards/rejected": -0.011767882853746414, + "step": 25 + }, + { + "epoch": 0.043789473684210524, + "grad_norm": 372.015222658044, + "kl": 2.9479598999023438, + "learning_rate": 2.0833333333333333e-07, + "logits/chosen": -836751744.0, + "logits/rejected": -1142913280.0, + "logps/chosen": -212.76544189453125, + "logps/rejected": -166.8655548095703, + "loss": 0.4997, + "rewards/chosen": -0.01788177713751793, + "rewards/margins": -0.06701736897230148, + "rewards/rejected": 0.049135591834783554, + "step": 26 + }, + { + "epoch": 0.04547368421052632, + "grad_norm": 173.67968690260923, + "kl": 0.8008079528808594, + "learning_rate": 2.1666666666666667e-07, + "logits/chosen": -796426240.0, + "logits/rejected": -697571200.0, + "logps/chosen": -200.24410400390624, + "logps/rejected": -241.0162556966146, + "loss": 0.4962, + "rewards/chosen": 0.07569732666015624, + "rewards/margins": 0.07425994885464508, + "rewards/rejected": 0.0014373778055111568, + "step": 27 + }, + { + "epoch": 0.04715789473684211, + "grad_norm": 179.48473623203154, + "kl": 1.4168357849121094, + "learning_rate": 2.25e-07, + "logits/chosen": -744388736.0, + "logits/rejected": -863063040.0, + "logps/chosen": -290.7626953125, + "logps/rejected": -231.60797119140625, + "loss": 0.504, + "rewards/chosen": -0.019368745386600494, + "rewards/margins": -0.058126453310251236, + "rewards/rejected": 0.03875770792365074, + "step": 28 + }, + { + "epoch": 0.048842105263157895, + "grad_norm": 130.22057137941954, + "kl": 1.3458938598632812, + "learning_rate": 2.3333333333333333e-07, + "logits/chosen": -691689472.0, + "logits/rejected": -918764885.3333334, + "logps/chosen": -209.3045654296875, + "logps/rejected": -241.4656982421875, + "loss": 0.4978, + "rewards/chosen": 0.07106705009937286, + "rewards/margins": 0.06067454318205515, + "rewards/rejected": 0.010392506917317709, + "step": 29 + }, + { + "epoch": 0.05052631578947368, + "grad_norm": 130.9807773433376, + "kl": 0.4033660888671875, + "learning_rate": 2.4166666666666665e-07, + "logits/chosen": -920996224.0, + "logits/rejected": -841631424.0, + "logps/chosen": -190.30531311035156, + "logps/rejected": -162.928466796875, + "loss": 0.4982, + "rewards/chosen": 0.035247232764959335, + "rewards/margins": 0.06811809912323952, + "rewards/rejected": -0.03287086635828018, + "step": 30 + }, + { + "epoch": 0.05221052631578947, + "grad_norm": 76.30157221981797, + "kl": 0.47165679931640625, + "learning_rate": 2.5e-07, + "logits/chosen": -1194017920.0, + "logits/rejected": -877919317.3333334, + "logps/chosen": -306.70257568359375, + "logps/rejected": -252.0715535481771, + "loss": 0.4996, + "rewards/chosen": 0.04151611402630806, + "rewards/margins": 0.03829981635014216, + "rewards/rejected": 0.003216297676165899, + "step": 31 + }, + { + "epoch": 0.053894736842105266, + "grad_norm": 102.32572244854322, + "kl": 2.0101585388183594, + "learning_rate": 2.5833333333333333e-07, + "logits/chosen": -944446336.0, + "logits/rejected": -1106841088.0, + "logps/chosen": -199.4850311279297, + "logps/rejected": -183.14755249023438, + "loss": 0.5005, + "rewards/chosen": 0.046735379844903946, + "rewards/margins": -0.004496384412050247, + "rewards/rejected": 0.05123176425695419, + "step": 32 + }, + { + "epoch": 0.055578947368421054, + "grad_norm": 99.04470417971368, + "kl": 2.4130401611328125, + "learning_rate": 2.6666666666666667e-07, + "logits/chosen": -825663078.4, + "logits/rejected": -991676245.3333334, + "logps/chosen": -220.846435546875, + "logps/rejected": -356.6270751953125, + "loss": 0.4947, + "rewards/chosen": 0.052228087186813356, + "rewards/margins": 0.03795552278558413, + "rewards/rejected": 0.014272564401229223, + "step": 33 + }, + { + "epoch": 0.05726315789473684, + "grad_norm": 246.96656404040365, + "kl": 1.233936071395874, + "learning_rate": 2.75e-07, + "logits/chosen": -1095287296.0, + "logits/rejected": -1060861504.0, + "logps/chosen": -182.49984741210938, + "logps/rejected": -234.69015502929688, + "loss": 0.4936, + "rewards/chosen": 0.06470909714698792, + "rewards/margins": 0.04259873181581497, + "rewards/rejected": 0.022110365331172943, + "step": 34 + }, + { + "epoch": 0.05894736842105263, + "grad_norm": 152.73517140204396, + "kl": 2.7289276123046875, + "learning_rate": 2.833333333333333e-07, + "logits/chosen": -1097634133.3333333, + "logits/rejected": -1259796582.4, + "logps/chosen": -202.9609375, + "logps/rejected": -189.42052001953124, + "loss": 0.4999, + "rewards/chosen": 0.01765136917432149, + "rewards/margins": -0.034761963287989306, + "rewards/rejected": 0.052413332462310794, + "step": 35 + }, + { + "epoch": 0.06063157894736842, + "grad_norm": 215.293974300091, + "kl": 2.45306396484375, + "learning_rate": 2.916666666666667e-07, + "logits/chosen": -631062869.3333334, + "logits/rejected": -987482112.0, + "logps/chosen": -213.41678873697916, + "logps/rejected": -221.72783203125, + "loss": 0.5006, + "rewards/chosen": 0.04089965671300888, + "rewards/margins": -0.024423225224018102, + "rewards/rejected": 0.06532288193702698, + "step": 36 + }, + { + "epoch": 0.06231578947368421, + "grad_norm": 195.90220199948243, + "kl": 2.9997177124023438, + "learning_rate": 3e-07, + "logits/chosen": -848359219.2, + "logits/rejected": -1426793557.3333333, + "logps/chosen": -195.2117431640625, + "logps/rejected": -180.9798787434896, + "loss": 0.4993, + "rewards/chosen": 0.08168579339981079, + "rewards/margins": 0.04379872928063074, + "rewards/rejected": 0.037887064119180046, + "step": 37 + }, + { + "epoch": 0.064, + "grad_norm": 121.58903894060508, + "kl": 2.0522327423095703, + "learning_rate": 3.0833333333333333e-07, + "logits/chosen": -1282800435.2, + "logits/rejected": -435072682.6666667, + "logps/chosen": -226.767138671875, + "logps/rejected": -249.10555013020834, + "loss": 0.4959, + "rewards/chosen": 0.11044464111328126, + "rewards/margins": 0.0951441449423631, + "rewards/rejected": 0.015300496170918146, + "step": 38 + }, + { + "epoch": 0.06568421052631579, + "grad_norm": 116.69257759601491, + "kl": 2.5439231395721436, + "learning_rate": 3.166666666666666e-07, + "logits/chosen": -557800192.0, + "logits/rejected": -930213376.0, + "logps/chosen": -211.4619140625, + "logps/rejected": -234.09031677246094, + "loss": 0.4983, + "rewards/chosen": 0.09951868653297424, + "rewards/margins": 0.06241312623023987, + "rewards/rejected": 0.037105560302734375, + "step": 39 + }, + { + "epoch": 0.06736842105263158, + "grad_norm": 365.02933086523586, + "kl": 2.2659378051757812, + "learning_rate": 3.25e-07, + "logits/chosen": -965182668.8, + "logits/rejected": -642834560.0, + "logps/chosen": -220.280224609375, + "logps/rejected": -222.31856282552084, + "loss": 0.4953, + "rewards/chosen": 0.09896668195724487, + "rewards/margins": 0.024154870708783457, + "rewards/rejected": 0.07481181124846141, + "step": 40 + }, + { + "epoch": 0.06905263157894737, + "grad_norm": 582.132852175163, + "kl": 2.304011344909668, + "learning_rate": 3.333333333333333e-07, + "logits/chosen": -885514444.8, + "logits/rejected": -748998314.6666666, + "logps/chosen": -227.1810302734375, + "logps/rejected": -196.22355143229166, + "loss": 0.4766, + "rewards/chosen": 0.18464019298553466, + "rewards/margins": 0.5814312537511189, + "rewards/rejected": -0.3967910607655843, + "step": 41 + }, + { + "epoch": 0.07073684210526315, + "grad_norm": 168.55393612476047, + "kl": 2.7851181030273438, + "learning_rate": 3.4166666666666664e-07, + "logits/chosen": -1012587946.6666666, + "logits/rejected": -900292300.8, + "logps/chosen": -227.2638956705729, + "logps/rejected": -187.71868896484375, + "loss": 0.4928, + "rewards/chosen": 0.1678065061569214, + "rewards/margins": 0.08132548332214355, + "rewards/rejected": 0.08648102283477783, + "step": 42 + }, + { + "epoch": 0.07242105263157894, + "grad_norm": 236.97744686773765, + "kl": 3.527801513671875, + "learning_rate": 3.5e-07, + "logits/chosen": -970476885.3333334, + "logits/rejected": -398392224.0, + "logps/chosen": -196.3362019856771, + "logps/rejected": -353.429443359375, + "loss": 0.4969, + "rewards/chosen": 0.08341166377067566, + "rewards/margins": 0.04898173362016678, + "rewards/rejected": 0.03442993015050888, + "step": 43 + }, + { + "epoch": 0.07410526315789474, + "grad_norm": 251.79089726814883, + "kl": 4.6118011474609375, + "learning_rate": 3.583333333333333e-07, + "logits/chosen": -944620885.3333334, + "logits/rejected": -546614579.2, + "logps/chosen": -229.73470052083334, + "logps/rejected": -211.629931640625, + "loss": 0.4902, + "rewards/chosen": 0.1472813884417216, + "rewards/margins": 0.1069905529419581, + "rewards/rejected": 0.04029083549976349, + "step": 44 + }, + { + "epoch": 0.07578947368421053, + "grad_norm": 115.12623939626701, + "kl": 4.802654266357422, + "learning_rate": 3.666666666666666e-07, + "logits/chosen": -1199768166.4, + "logits/rejected": -1026606506.6666666, + "logps/chosen": -250.584716796875, + "logps/rejected": -256.4547932942708, + "loss": 0.4888, + "rewards/chosen": 0.15660583972930908, + "rewards/margins": 0.056487321853637695, + "rewards/rejected": 0.10011851787567139, + "step": 45 + }, + { + "epoch": 0.07747368421052632, + "grad_norm": 147.48000207541645, + "kl": 2.1971359252929688, + "learning_rate": 3.75e-07, + "logits/chosen": -779579904.0, + "logits/rejected": -777894058.6666666, + "logps/chosen": -234.33892822265625, + "logps/rejected": -279.74733479817706, + "loss": 0.4941, + "rewards/chosen": 0.2183128446340561, + "rewards/margins": 0.09147873024145761, + "rewards/rejected": 0.12683411439259848, + "step": 46 + }, + { + "epoch": 0.07915789473684211, + "grad_norm": 172.83190668649902, + "kl": 5.049167633056641, + "learning_rate": 3.8333333333333335e-07, + "logits/chosen": -911320192.0, + "logits/rejected": -1069644800.0, + "logps/chosen": -215.0435791015625, + "logps/rejected": -261.28668212890625, + "loss": 0.4947, + "rewards/chosen": 0.17158202826976776, + "rewards/margins": 0.06581267714500427, + "rewards/rejected": 0.10576935112476349, + "step": 47 + }, + { + "epoch": 0.0808421052631579, + "grad_norm": 135.23703398521903, + "kl": 3.5211753845214844, + "learning_rate": 3.9166666666666664e-07, + "logits/chosen": -1736729472.0, + "logits/rejected": -939244617.1428572, + "logps/chosen": -226.9305877685547, + "logps/rejected": -232.20169503348214, + "loss": 0.4972, + "rewards/chosen": 0.16886138916015625, + "rewards/margins": 0.05592117990766253, + "rewards/rejected": 0.11294020925249372, + "step": 48 + }, + { + "epoch": 0.08252631578947368, + "grad_norm": 87.76901964257267, + "kl": 5.832710266113281, + "learning_rate": 4e-07, + "logits/chosen": -794667622.4, + "logits/rejected": -622882474.6666666, + "logps/chosen": -230.16806640625, + "logps/rejected": -224.6505330403646, + "loss": 0.4913, + "rewards/chosen": 0.2949032783508301, + "rewards/margins": 0.20258251825968426, + "rewards/rejected": 0.09232076009114583, + "step": 49 + }, + { + "epoch": 0.08421052631578947, + "grad_norm": 131.7748262627241, + "kl": 4.04681396484375, + "learning_rate": 4.083333333333333e-07, + "logits/chosen": -1042156714.6666666, + "logits/rejected": -956070912.0, + "logps/chosen": -220.65376790364584, + "logps/rejected": -351.86298828125, + "loss": 0.4921, + "rewards/chosen": 0.1149485210577647, + "rewards/margins": -0.044381918509801235, + "rewards/rejected": 0.15933043956756593, + "step": 50 + }, + { + "epoch": 0.08589473684210526, + "grad_norm": 93.7193003978435, + "kl": 5.549095153808594, + "learning_rate": 4.1666666666666667e-07, + "logits/chosen": -913337088.0, + "logits/rejected": -1463801514.6666667, + "logps/chosen": -234.42970275878906, + "logps/rejected": -256.32749430338544, + "loss": 0.5037, + "rewards/chosen": 0.12055359780788422, + "rewards/margins": 0.05089595417181651, + "rewards/rejected": 0.0696576436360677, + "step": 51 + }, + { + "epoch": 0.08757894736842105, + "grad_norm": 124.37458019612187, + "kl": 5.329154968261719, + "learning_rate": 4.2499999999999995e-07, + "logits/chosen": -769787840.0, + "logits/rejected": -873242752.0, + "logps/chosen": -228.21946716308594, + "logps/rejected": -178.02896118164062, + "loss": 0.5018, + "rewards/chosen": 0.1875530183315277, + "rewards/margins": -0.029434025287628174, + "rewards/rejected": 0.21698704361915588, + "step": 52 + }, + { + "epoch": 0.08926315789473684, + "grad_norm": 151.7193063462552, + "kl": 2.5696582794189453, + "learning_rate": 4.3333333333333335e-07, + "logits/chosen": -966253760.0, + "logits/rejected": -818051840.0, + "logps/chosen": -210.7382354736328, + "logps/rejected": -210.70034790039062, + "loss": 0.4948, + "rewards/chosen": 0.09387054294347763, + "rewards/margins": -0.11254673451185226, + "rewards/rejected": 0.2064172774553299, + "step": 53 + }, + { + "epoch": 0.09094736842105264, + "grad_norm": 113.57346599809466, + "kl": 2.8759994506835938, + "learning_rate": 4.4166666666666664e-07, + "logits/chosen": -936665600.0, + "logits/rejected": -744050602.6666666, + "logps/chosen": -210.92718505859375, + "logps/rejected": -213.8590087890625, + "loss": 0.4893, + "rewards/chosen": 0.17248383164405823, + "rewards/margins": -0.09370574355125427, + "rewards/rejected": 0.2661895751953125, + "step": 54 + }, + { + "epoch": 0.09263157894736843, + "grad_norm": 174.61803322959847, + "kl": 4.2318267822265625, + "learning_rate": 4.5e-07, + "logits/chosen": -677702400.0, + "logits/rejected": -692814116.5714285, + "logps/chosen": -218.57711791992188, + "logps/rejected": -213.56180245535714, + "loss": 0.4879, + "rewards/chosen": 0.4497177302837372, + "rewards/margins": 0.41578697732516695, + "rewards/rejected": 0.03393075295857021, + "step": 55 + }, + { + "epoch": 0.09431578947368421, + "grad_norm": 147.27747224286998, + "kl": 3.260711669921875, + "learning_rate": 4.5833333333333327e-07, + "logits/chosen": -1289244544.0, + "logits/rejected": -916406464.0, + "logps/chosen": -246.1941375732422, + "logps/rejected": -253.98768615722656, + "loss": 0.4921, + "rewards/chosen": 0.16882018744945526, + "rewards/margins": -0.14416618645191193, + "rewards/rejected": 0.3129863739013672, + "step": 56 + }, + { + "epoch": 0.096, + "grad_norm": 95.08985748052072, + "kl": 1.6493759155273438, + "learning_rate": 4.6666666666666666e-07, + "logits/chosen": -733326400.0, + "logits/rejected": -922427392.0, + "logps/chosen": -178.46246337890625, + "logps/rejected": -258.2501627604167, + "loss": 0.4867, + "rewards/chosen": 0.3112235963344574, + "rewards/margins": 0.17294743657112122, + "rewards/rejected": 0.13827615976333618, + "step": 57 + }, + { + "epoch": 0.09768421052631579, + "grad_norm": 133.57090255911547, + "kl": 3.168426513671875, + "learning_rate": 4.7499999999999995e-07, + "logits/chosen": -767859712.0, + "logits/rejected": -977725248.0, + "logps/chosen": -228.76634216308594, + "logps/rejected": -284.31427001953125, + "loss": 0.4985, + "rewards/chosen": 0.1971721649169922, + "rewards/margins": 0.03166121244430542, + "rewards/rejected": 0.16551095247268677, + "step": 58 + }, + { + "epoch": 0.09936842105263158, + "grad_norm": 108.0916864306588, + "kl": 0.6577796936035156, + "learning_rate": 4.833333333333333e-07, + "logits/chosen": -1173459148.8, + "logits/rejected": -655583232.0, + "logps/chosen": -196.4993408203125, + "logps/rejected": -219.53653971354166, + "loss": 0.4757, + "rewards/chosen": 0.23870911598205566, + "rewards/margins": 0.28514466087023416, + "rewards/rejected": -0.046435544888178505, + "step": 59 + }, + { + "epoch": 0.10105263157894737, + "grad_norm": 87.62449313729059, + "kl": 2.194793701171875, + "learning_rate": 4.916666666666666e-07, + "logits/chosen": -1028928256.0, + "logits/rejected": -540835200.0, + "logps/chosen": -227.88580322265625, + "logps/rejected": -234.1096649169922, + "loss": 0.45, + "rewards/chosen": 0.5302422046661377, + "rewards/margins": 1.1665672659873962, + "rewards/rejected": -0.6363250613212585, + "step": 60 + }, + { + "epoch": 0.10273684210526315, + "grad_norm": 119.47945666482947, + "kl": 1.5802383422851562, + "learning_rate": 5e-07, + "logits/chosen": -1018257536.0, + "logits/rejected": -1083207040.0, + "logps/chosen": -222.7873992919922, + "logps/rejected": -275.23138427734375, + "loss": 0.5044, + "rewards/chosen": 0.32063254714012146, + "rewards/margins": 0.10322417318820953, + "rewards/rejected": 0.21740837395191193, + "step": 61 + }, + { + "epoch": 0.10442105263157894, + "grad_norm": 114.74202024192041, + "kl": 1.9775314331054688, + "learning_rate": 4.999956736067562e-07, + "logits/chosen": -720981299.2, + "logits/rejected": -543601152.0, + "logps/chosen": -240.6268798828125, + "logps/rejected": -207.02860514322916, + "loss": 0.4983, + "rewards/chosen": 0.25333800315856936, + "rewards/margins": -0.036342994372049964, + "rewards/rejected": 0.2896809975306193, + "step": 62 + }, + { + "epoch": 0.10610526315789473, + "grad_norm": 77.65993832754032, + "kl": 0.7235641479492188, + "learning_rate": 4.999826945767664e-07, + "logits/chosen": -692212443.4285715, + "logits/rejected": -1071096576.0, + "logps/chosen": -255.70343889508928, + "logps/rejected": -259.9676208496094, + "loss": 0.5084, + "rewards/chosen": 0.2564025947025844, + "rewards/margins": -0.06277771506990704, + "rewards/rejected": 0.31918030977249146, + "step": 63 + }, + { + "epoch": 0.10778947368421053, + "grad_norm": 101.9412698769531, + "kl": 2.6254653930664062, + "learning_rate": 4.999610633592496e-07, + "logits/chosen": -1115974656.0, + "logits/rejected": -880023961.6, + "logps/chosen": -227.8680419921875, + "logps/rejected": -210.0842041015625, + "loss": 0.4639, + "rewards/chosen": 0.1862970987955729, + "rewards/margins": 0.7373786608378092, + "rewards/rejected": -0.5510815620422364, + "step": 64 + }, + { + "epoch": 0.10947368421052632, + "grad_norm": 83.26215203358481, + "kl": 0.0, + "learning_rate": 4.99930780702887e-07, + "logits/chosen": -2037875712.0, + "logits/rejected": -891881472.0, + "logps/chosen": -298.9163818359375, + "logps/rejected": -212.7870076497396, + "loss": 0.5016, + "rewards/chosen": 0.0009033232927322388, + "rewards/margins": -0.28533248603343964, + "rewards/rejected": 0.2862358093261719, + "step": 65 + }, + { + "epoch": 0.11115789473684211, + "grad_norm": 65.37492119822153, + "kl": 4.6044464111328125, + "learning_rate": 4.998918476557962e-07, + "logits/chosen": -1163713365.3333333, + "logits/rejected": -717799936.0, + "logps/chosen": -291.79477945963544, + "logps/rejected": -137.73323974609374, + "loss": 0.4896, + "rewards/chosen": 0.2572901447614034, + "rewards/margins": 0.1486562927563985, + "rewards/rejected": 0.10863385200500489, + "step": 66 + }, + { + "epoch": 0.1128421052631579, + "grad_norm": 108.123906294101, + "kl": 0.8921241760253906, + "learning_rate": 4.998442655654946e-07, + "logits/chosen": -1018941312.0, + "logits/rejected": -723264512.0, + "logps/chosen": -227.9298553466797, + "logps/rejected": -196.53343200683594, + "loss": 0.4936, + "rewards/chosen": 0.1972118318080902, + "rewards/margins": -0.028878778219223022, + "rewards/rejected": 0.22609061002731323, + "step": 67 + }, + { + "epoch": 0.11452631578947368, + "grad_norm": 154.7633558965442, + "kl": 3.6258544921875, + "learning_rate": 4.997880360788526e-07, + "logits/chosen": -1207795302.4, + "logits/rejected": -640148437.3333334, + "logps/chosen": -207.35986328125, + "logps/rejected": -239.59635416666666, + "loss": 0.4908, + "rewards/chosen": 0.24745573997497558, + "rewards/margins": 0.09161568085352578, + "rewards/rejected": 0.1558400591214498, + "step": 68 + }, + { + "epoch": 0.11621052631578947, + "grad_norm": 189.59978273083877, + "kl": 1.4420166015625, + "learning_rate": 4.997231611420373e-07, + "logits/chosen": -722635929.6, + "logits/rejected": -1086283520.0, + "logps/chosen": -205.415087890625, + "logps/rejected": -223.599853515625, + "loss": 0.4755, + "rewards/chosen": 0.2955571174621582, + "rewards/margins": 0.23686978320280708, + "rewards/rejected": 0.0586873342593511, + "step": 69 + }, + { + "epoch": 0.11789473684210526, + "grad_norm": 82.88808037398137, + "kl": 0.0, + "learning_rate": 4.996496430004445e-07, + "logits/chosen": -894456704.0, + "logits/rejected": -1008478336.0, + "logps/chosen": -197.12306213378906, + "logps/rejected": -189.72410583496094, + "loss": 0.4798, + "rewards/chosen": 0.5647777318954468, + "rewards/margins": 0.4188384860754013, + "rewards/rejected": 0.14593924582004547, + "step": 70 + }, + { + "epoch": 0.11957894736842105, + "grad_norm": 116.22452277931659, + "kl": 0.0, + "learning_rate": 4.995674841986217e-07, + "logits/chosen": -1016096426.6666666, + "logits/rejected": -340471296.0, + "logps/chosen": -235.82279459635416, + "logps/rejected": -181.5946807861328, + "loss": 0.4523, + "rewards/chosen": 0.3477465311686198, + "rewards/margins": 0.14004083971182507, + "rewards/rejected": 0.20770569145679474, + "step": 71 + }, + { + "epoch": 0.12126315789473684, + "grad_norm": 169.23180648930278, + "kl": 2.728069305419922, + "learning_rate": 4.994766875801788e-07, + "logits/chosen": -802654592.0, + "logits/rejected": -754744960.0, + "logps/chosen": -195.60926818847656, + "logps/rejected": -193.22048950195312, + "loss": 0.5093, + "rewards/chosen": 0.20827637612819672, + "rewards/margins": 0.17099057137966156, + "rewards/rejected": 0.037285804748535156, + "step": 72 + }, + { + "epoch": 0.12294736842105262, + "grad_norm": 97.86580017412098, + "kl": 1.1262283325195312, + "learning_rate": 4.993772562876909e-07, + "logits/chosen": -791456000.0, + "logits/rejected": -912904704.0, + "logps/chosen": -172.02268981933594, + "logps/rejected": -196.89213053385416, + "loss": 0.4554, + "rewards/chosen": 0.7986927032470703, + "rewards/margins": 1.128045916557312, + "rewards/rejected": -0.3293532133102417, + "step": 73 + }, + { + "epoch": 0.12463157894736843, + "grad_norm": 87.12336042753508, + "kl": 0.322601318359375, + "learning_rate": 4.992691937625891e-07, + "logits/chosen": -579524224.0, + "logits/rejected": -731365120.0, + "logps/chosen": -232.52737426757812, + "logps/rejected": -234.31417846679688, + "loss": 0.4493, + "rewards/chosen": 0.24241486191749573, + "rewards/margins": 1.9988974630832672, + "rewards/rejected": -1.7564826011657715, + "step": 74 + }, + { + "epoch": 0.12631578947368421, + "grad_norm": 72.27499509744034, + "kl": 0.0, + "learning_rate": 4.991525037450412e-07, + "logits/chosen": -883592192.0, + "logits/rejected": -728281088.0, + "logps/chosen": -225.2747599283854, + "logps/rejected": -199.8900146484375, + "loss": 0.4878, + "rewards/chosen": 0.1509552001953125, + "rewards/margins": 0.05537720024585724, + "rewards/rejected": 0.09557799994945526, + "step": 75 + }, + { + "epoch": 0.128, + "grad_norm": 107.49035218888741, + "kl": 0.0, + "learning_rate": 4.990271902738222e-07, + "logits/chosen": -625081190.4, + "logits/rejected": -646702250.6666666, + "logps/chosen": -232.6982421875, + "logps/rejected": -326.85638427734375, + "loss": 0.45, + "rewards/chosen": 0.23982207775115966, + "rewards/margins": 0.3932990511258443, + "rewards/rejected": -0.15347697337468466, + "step": 76 + }, + { + "epoch": 0.1296842105263158, + "grad_norm": 101.17758060728043, + "kl": 0.0, + "learning_rate": 4.988932576861752e-07, + "logits/chosen": -1040184640.0, + "logits/rejected": -690037376.0, + "logps/chosen": -184.14572143554688, + "logps/rejected": -228.58084106445312, + "loss": 0.5148, + "rewards/chosen": 0.24517688155174255, + "rewards/margins": 0.009785279631614685, + "rewards/rejected": 0.23539160192012787, + "step": 77 + }, + { + "epoch": 0.13136842105263158, + "grad_norm": 84.42483041363506, + "kl": 0.3793144226074219, + "learning_rate": 4.987507106176606e-07, + "logits/chosen": -611603046.4, + "logits/rejected": -684819114.6666666, + "logps/chosen": -241.2910400390625, + "logps/rejected": -251.3832804361979, + "loss": 0.4885, + "rewards/chosen": 0.5874441623687744, + "rewards/margins": -0.2702706495920817, + "rewards/rejected": 0.8577148119608561, + "step": 78 + }, + { + "epoch": 0.13305263157894737, + "grad_norm": 123.42412793246983, + "kl": 0.0, + "learning_rate": 4.985995540019955e-07, + "logits/chosen": -902578752.0, + "logits/rejected": -1213336917.3333333, + "logps/chosen": -271.75140380859375, + "logps/rejected": -266.7210286458333, + "loss": 0.4488, + "rewards/chosen": 0.22760315239429474, + "rewards/margins": -0.06182275712490082, + "rewards/rejected": 0.28942590951919556, + "step": 79 + }, + { + "epoch": 0.13473684210526315, + "grad_norm": 70.07470086670996, + "kl": 0.5335845947265625, + "learning_rate": 4.984397930708837e-07, + "logits/chosen": -737288960.0, + "logits/rejected": -944431616.0, + "logps/chosen": -198.37362670898438, + "logps/rejected": -224.55250549316406, + "loss": 0.4547, + "rewards/chosen": 0.5928070545196533, + "rewards/margins": 0.4272579848766327, + "rewards/rejected": 0.16554906964302063, + "step": 80 + }, + { + "epoch": 0.13642105263157894, + "grad_norm": 66.8382583080533, + "kl": 0.0, + "learning_rate": 4.982714333538342e-07, + "logits/chosen": -604147353.6, + "logits/rejected": -792211114.6666666, + "logps/chosen": -264.7359375, + "logps/rejected": -207.922119140625, + "loss": 0.502, + "rewards/chosen": 0.11271027326583863, + "rewards/margins": -0.04983551700909933, + "rewards/rejected": 0.16254579027493796, + "step": 81 + }, + { + "epoch": 0.13810526315789473, + "grad_norm": 90.56573316497496, + "kl": 1.1843795776367188, + "learning_rate": 4.980944806779698e-07, + "logits/chosen": -994012032.0, + "logits/rejected": -756074154.6666666, + "logps/chosen": -211.22325134277344, + "logps/rejected": -273.8466389973958, + "loss": 0.503, + "rewards/chosen": 0.21327057480812073, + "rewards/margins": 0.3816507955392202, + "rewards/rejected": -0.16838022073109946, + "step": 82 + }, + { + "epoch": 0.13978947368421052, + "grad_norm": 95.89588780633147, + "kl": 0.0, + "learning_rate": 4.979089411678251e-07, + "logits/chosen": -1088499200.0, + "logits/rejected": -580556672.0, + "logps/chosen": -201.80905151367188, + "logps/rejected": -149.05142211914062, + "loss": 0.4614, + "rewards/chosen": 0.5321415066719055, + "rewards/margins": 0.3093837797641754, + "rewards/rejected": 0.2227577269077301, + "step": 83 + }, + { + "epoch": 0.1414736842105263, + "grad_norm": 84.70589504268058, + "kl": 0.0, + "learning_rate": 4.977148212451354e-07, + "logits/chosen": -1006119296.0, + "logits/rejected": -702025600.0, + "logps/chosen": -220.37423706054688, + "logps/rejected": -213.92950439453125, + "loss": 0.4784, + "rewards/chosen": 0.37330934405326843, + "rewards/margins": -0.14478299021720886, + "rewards/rejected": 0.5180923342704773, + "step": 84 + }, + { + "epoch": 0.1431578947368421, + "grad_norm": 80.24131984783912, + "kl": 0.18048095703125, + "learning_rate": 4.975121276286136e-07, + "logits/chosen": -574469888.0, + "logits/rejected": -1207006720.0, + "logps/chosen": -224.60697428385416, + "logps/rejected": -230.05050659179688, + "loss": 0.4587, + "rewards/chosen": 0.3602706988652547, + "rewards/margins": 0.06148912509282428, + "rewards/rejected": 0.2987815737724304, + "step": 85 + }, + { + "epoch": 0.14484210526315788, + "grad_norm": 119.4817380814399, + "kl": 0.0, + "learning_rate": 4.97300867333718e-07, + "logits/chosen": -559665344.0, + "logits/rejected": -1335321600.0, + "logps/chosen": -246.3226318359375, + "logps/rejected": -240.66729736328125, + "loss": 0.4545, + "rewards/chosen": 0.27413901686668396, + "rewards/margins": 0.11862906813621521, + "rewards/rejected": 0.15550994873046875, + "step": 86 + }, + { + "epoch": 0.14652631578947367, + "grad_norm": 76.72262570711749, + "kl": 0.0, + "learning_rate": 4.970810476724096e-07, + "logits/chosen": -794261632.0, + "logits/rejected": -888321194.6666666, + "logps/chosen": -223.9755096435547, + "logps/rejected": -230.53841145833334, + "loss": 0.4915, + "rewards/chosen": 0.6259682178497314, + "rewards/margins": 0.16484379768371582, + "rewards/rejected": 0.4611244201660156, + "step": 87 + }, + { + "epoch": 0.1482105263157895, + "grad_norm": 103.45011997542589, + "kl": 0.6499404907226562, + "learning_rate": 4.968526762528988e-07, + "logits/chosen": -780058624.0, + "logits/rejected": -916248371.2, + "logps/chosen": -256.363525390625, + "logps/rejected": -207.6240234375, + "loss": 0.4831, + "rewards/chosen": 0.28617451588312787, + "rewards/margins": 0.07391071716944378, + "rewards/rejected": 0.2122637987136841, + "step": 88 + }, + { + "epoch": 0.14989473684210528, + "grad_norm": 215.23670048541007, + "kl": 0.0, + "learning_rate": 4.96615760979382e-07, + "logits/chosen": -637054976.0, + "logits/rejected": -924139712.0, + "logps/chosen": -238.05352783203125, + "logps/rejected": -168.34219360351562, + "loss": 0.4829, + "rewards/chosen": 0.5608859062194824, + "rewards/margins": -0.13643085956573486, + "rewards/rejected": 0.6973167657852173, + "step": 89 + }, + { + "epoch": 0.15157894736842106, + "grad_norm": 76.57882355799092, + "kl": 0.0, + "learning_rate": 4.963703100517683e-07, + "logits/chosen": -1086607488.0, + "logits/rejected": -987515264.0, + "logps/chosen": -204.48553466796875, + "logps/rejected": -251.81475830078125, + "loss": 0.4542, + "rewards/chosen": 0.7157703638076782, + "rewards/margins": 1.3066982626914978, + "rewards/rejected": -0.5909278988838196, + "step": 90 + }, + { + "epoch": 0.15326315789473685, + "grad_norm": 85.3247850434762, + "kl": 0.0, + "learning_rate": 4.961163319653958e-07, + "logits/chosen": -1008856704.0, + "logits/rejected": -618826112.0, + "logps/chosen": -187.7606201171875, + "logps/rejected": -191.36654663085938, + "loss": 0.4556, + "rewards/chosen": 0.2826835811138153, + "rewards/margins": -0.048234134912490845, + "rewards/rejected": 0.33091771602630615, + "step": 91 + }, + { + "epoch": 0.15494736842105264, + "grad_norm": 172.74293893285517, + "kl": 0.0, + "learning_rate": 4.958538355107369e-07, + "logits/chosen": -771198272.0, + "logits/rejected": -911799936.0, + "logps/chosen": -198.0313262939453, + "logps/rejected": -243.49655151367188, + "loss": 0.4021, + "rewards/chosen": 0.537731945514679, + "rewards/margins": 1.0454849600791931, + "rewards/rejected": -0.5077530145645142, + "step": 92 + }, + { + "epoch": 0.15663157894736843, + "grad_norm": 115.09597490195979, + "kl": 0.0, + "learning_rate": 4.955828297730948e-07, + "logits/chosen": -578800576.0, + "logits/rejected": -940042752.0, + "logps/chosen": -166.36688232421875, + "logps/rejected": -236.39175415039062, + "loss": 0.4575, + "rewards/chosen": 0.2515438199043274, + "rewards/margins": 0.63800048828125, + "rewards/rejected": -0.3864566683769226, + "step": 93 + }, + { + "epoch": 0.15831578947368422, + "grad_norm": 189.22015271094992, + "kl": 0.0, + "learning_rate": 4.953033241322886e-07, + "logits/chosen": -971949875.2, + "logits/rejected": -642591701.3333334, + "logps/chosen": -208.137646484375, + "logps/rejected": -267.102294921875, + "loss": 0.4444, + "rewards/chosen": 0.30209686756134035, + "rewards/margins": 0.15472238659858706, + "rewards/rejected": 0.1473744809627533, + "step": 94 + }, + { + "epoch": 0.16, + "grad_norm": 151.69924056862672, + "kl": 0.8690567016601562, + "learning_rate": 4.950153282623288e-07, + "logits/chosen": -929138029.7142857, + "logits/rejected": -305715200.0, + "logps/chosen": -243.89243861607142, + "logps/rejected": -234.21505737304688, + "loss": 0.4272, + "rewards/chosen": 1.003729956490653, + "rewards/margins": 1.042243140084403, + "rewards/rejected": -0.03851318359375, + "step": 95 + }, + { + "epoch": 0.1616842105263158, + "grad_norm": 117.75406936780551, + "kl": 0.0, + "learning_rate": 4.947188521310827e-07, + "logits/chosen": -1078521088.0, + "logits/rejected": -754925696.0, + "logps/chosen": -241.37603759765625, + "logps/rejected": -246.62840270996094, + "loss": 0.468, + "rewards/chosen": 0.06325645744800568, + "rewards/margins": 2.5708944350481033, + "rewards/rejected": -2.5076379776000977, + "step": 96 + }, + { + "epoch": 0.16336842105263158, + "grad_norm": 92.90737005905727, + "kl": 0.0, + "learning_rate": 4.944139059999286e-07, + "logits/chosen": -939986624.0, + "logits/rejected": -757044608.0, + "logps/chosen": -255.14584350585938, + "logps/rejected": -217.84385681152344, + "loss": 0.4852, + "rewards/chosen": 0.14831504225730896, + "rewards/margins": -0.12935563921928406, + "rewards/rejected": 0.277670681476593, + "step": 97 + }, + { + "epoch": 0.16505263157894737, + "grad_norm": 75.87889683822247, + "kl": 2.0634384155273438, + "learning_rate": 4.941005004234018e-07, + "logits/chosen": -674378444.8, + "logits/rejected": -790328490.6666666, + "logps/chosen": -222.7276123046875, + "logps/rejected": -319.3123372395833, + "loss": 0.4712, + "rewards/chosen": 0.9145062446594239, + "rewards/margins": 1.130675482749939, + "rewards/rejected": -0.21616923809051514, + "step": 98 + }, + { + "epoch": 0.16673684210526316, + "grad_norm": 83.71279388164065, + "kl": 0.06458663940429688, + "learning_rate": 4.937786462488283e-07, + "logits/chosen": -715301376.0, + "logits/rejected": -695437458.2857143, + "logps/chosen": -187.44473266601562, + "logps/rejected": -221.67951311383928, + "loss": 0.4397, + "rewards/chosen": 0.4498535096645355, + "rewards/margins": 0.21583446860313416, + "rewards/rejected": 0.23401904106140137, + "step": 99 + }, + { + "epoch": 0.16842105263157894, + "grad_norm": 105.87757666440052, + "kl": 4.265495300292969, + "learning_rate": 4.9344835461595e-07, + "logits/chosen": -985528320.0, + "logits/rejected": -841993301.3333334, + "logps/chosen": -187.4853759765625, + "logps/rejected": -244.225830078125, + "loss": 0.4832, + "rewards/chosen": 0.04033417701721191, + "rewards/margins": -0.09548329114913941, + "rewards/rejected": 0.13581746816635132, + "step": 100 + }, + { + "epoch": 0.17010526315789473, + "grad_norm": 99.19589292443813, + "kl": 0.0, + "learning_rate": 4.93109636956539e-07, + "logits/chosen": -1094334464.0, + "logits/rejected": -712445226.6666666, + "logps/chosen": -216.4138671875, + "logps/rejected": -227.96439615885416, + "loss": 0.4423, + "rewards/chosen": 0.8524996757507324, + "rewards/margins": 1.196986190478007, + "rewards/rejected": -0.3444865147272746, + "step": 101 + }, + { + "epoch": 0.17178947368421052, + "grad_norm": 74.15099841568721, + "kl": 2.151416778564453, + "learning_rate": 4.927625049940012e-07, + "logits/chosen": -766361152.0, + "logits/rejected": -333142048.0, + "logps/chosen": -199.77578735351562, + "logps/rejected": -168.93804931640625, + "loss": 0.4554, + "rewards/chosen": 0.7172653079032898, + "rewards/margins": 0.35114437341690063, + "rewards/rejected": 0.36612093448638916, + "step": 102 + }, + { + "epoch": 0.1734736842105263, + "grad_norm": 107.51116166109057, + "kl": 1.1529388427734375, + "learning_rate": 4.92406970742972e-07, + "logits/chosen": -971440256.0, + "logits/rejected": -828623232.0, + "logps/chosen": -195.1447296142578, + "logps/rejected": -280.0885009765625, + "loss": 0.4425, + "rewards/chosen": 0.5913848876953125, + "rewards/margins": 1.5447425842285156, + "rewards/rejected": -0.9533576965332031, + "step": 103 + }, + { + "epoch": 0.1751578947368421, + "grad_norm": 74.47709378212676, + "kl": 4.383308410644531, + "learning_rate": 4.920430465088991e-07, + "logits/chosen": -670352640.0, + "logits/rejected": -671316608.0, + "logps/chosen": -255.62147521972656, + "logps/rejected": -200.2668914794922, + "loss": 0.4575, + "rewards/chosen": 0.5768207311630249, + "rewards/margins": 0.4924035966396332, + "rewards/rejected": 0.08441713452339172, + "step": 104 + }, + { + "epoch": 0.17684210526315788, + "grad_norm": 132.0792640659443, + "kl": 1.2592926025390625, + "learning_rate": 4.916707448876173e-07, + "logits/chosen": -832779328.0, + "logits/rejected": -1296543232.0, + "logps/chosen": -227.14276123046875, + "logps/rejected": -218.0616912841797, + "loss": 0.4254, + "rewards/chosen": 1.1795654296875, + "rewards/margins": 1.3467857092618942, + "rewards/rejected": -0.16722027957439423, + "step": 105 + }, + { + "epoch": 0.17852631578947367, + "grad_norm": 145.72514277447476, + "kl": 0.0, + "learning_rate": 4.912900787649123e-07, + "logits/chosen": -898515675.4285715, + "logits/rejected": -1168541696.0, + "logps/chosen": -201.853515625, + "logps/rejected": -228.6392822265625, + "loss": 0.4635, + "rewards/chosen": 0.14519500732421875, + "rewards/margins": -0.7220779657363892, + "rewards/rejected": 0.8672729730606079, + "step": 106 + }, + { + "epoch": 0.18021052631578946, + "grad_norm": 96.96868580065858, + "kl": 0.47565460205078125, + "learning_rate": 4.90901061316075e-07, + "logits/chosen": -979152704.0, + "logits/rejected": -788988416.0, + "logps/chosen": -262.84649658203125, + "logps/rejected": -219.4362589518229, + "loss": 0.4916, + "rewards/chosen": -0.06726683676242828, + "rewards/margins": 1.062198743224144, + "rewards/rejected": -1.1294655799865723, + "step": 107 + }, + { + "epoch": 0.18189473684210528, + "grad_norm": 131.83686757875427, + "kl": 0.0, + "learning_rate": 4.905037060054449e-07, + "logits/chosen": -1066169344.0, + "logits/rejected": -1416755712.0, + "logps/chosen": -177.3980255126953, + "logps/rejected": -229.30007934570312, + "loss": 0.4426, + "rewards/chosen": 1.1091067790985107, + "rewards/margins": 0.9519591778516769, + "rewards/rejected": 0.1571476012468338, + "step": 108 + }, + { + "epoch": 0.18357894736842106, + "grad_norm": 127.9527788904727, + "kl": 0.0, + "learning_rate": 4.900980265859448e-07, + "logits/chosen": -559809066.6666666, + "logits/rejected": -611907481.6, + "logps/chosen": -164.3425089518229, + "logps/rejected": -240.176318359375, + "loss": 0.424, + "rewards/chosen": 1.06194003423055, + "rewards/margins": 0.5945489724477131, + "rewards/rejected": 0.4673910617828369, + "step": 109 + }, + { + "epoch": 0.18526315789473685, + "grad_norm": 84.90732586257637, + "kl": 0.0, + "learning_rate": 4.896840370986042e-07, + "logits/chosen": -976780032.0, + "logits/rejected": -716965632.0, + "logps/chosen": -253.70814514160156, + "logps/rejected": -167.56753540039062, + "loss": 0.4595, + "rewards/chosen": 0.47311899065971375, + "rewards/margins": 0.09406355023384094, + "rewards/rejected": 0.3790554404258728, + "step": 110 + }, + { + "epoch": 0.18694736842105264, + "grad_norm": 89.53866936328932, + "kl": 0.0, + "learning_rate": 4.892617518720737e-07, + "logits/chosen": -829139285.3333334, + "logits/rejected": -455405536.0, + "logps/chosen": -206.80936686197916, + "logps/rejected": -257.3726806640625, + "loss": 0.4362, + "rewards/chosen": 0.4422444502512614, + "rewards/margins": 2.1721797386805215, + "rewards/rejected": -1.7299352884292603, + "step": 111 + }, + { + "epoch": 0.18863157894736843, + "grad_norm": 192.37189364365034, + "kl": 0.0, + "learning_rate": 4.888311855221289e-07, + "logits/chosen": -770445312.0, + "logits/rejected": -951467392.0, + "logps/chosen": -199.4364471435547, + "logps/rejected": -215.43002319335938, + "loss": 0.5149, + "rewards/chosen": -1.172553300857544, + "rewards/margins": -1.1502388417720795, + "rewards/rejected": -0.022314459085464478, + "step": 112 + }, + { + "epoch": 0.19031578947368422, + "grad_norm": 95.28956568265271, + "kl": 0.0, + "learning_rate": 4.883923529511646e-07, + "logits/chosen": -535924416.0, + "logits/rejected": -786336960.0, + "logps/chosen": -247.45355224609375, + "logps/rejected": -203.68768310546875, + "loss": 0.4584, + "rewards/chosen": 0.5863098502159119, + "rewards/margins": -0.6281996369361877, + "rewards/rejected": 1.2145094871520996, + "step": 113 + }, + { + "epoch": 0.192, + "grad_norm": 141.25750368118935, + "kl": 0.0, + "learning_rate": 4.879452693476789e-07, + "logits/chosen": -1001671372.8, + "logits/rejected": -836295936.0, + "logps/chosen": -201.454052734375, + "logps/rejected": -205.16349283854166, + "loss": 0.4619, + "rewards/chosen": 0.1915292501449585, + "rewards/margins": 1.2447933912277223, + "rewards/rejected": -1.0532641410827637, + "step": 114 + }, + { + "epoch": 0.1936842105263158, + "grad_norm": 90.01647424789367, + "kl": 1.8182296752929688, + "learning_rate": 4.874899501857477e-07, + "logits/chosen": -642349952.0, + "logits/rejected": -997530176.0, + "logps/chosen": -269.48760986328125, + "logps/rejected": -281.49591064453125, + "loss": 0.4548, + "rewards/chosen": -0.8407798409461975, + "rewards/margins": -0.7594596445560455, + "rewards/rejected": -0.08132019639015198, + "step": 115 + }, + { + "epoch": 0.19536842105263158, + "grad_norm": 142.4095270366161, + "kl": 0.0, + "learning_rate": 4.87026411224489e-07, + "logits/chosen": -909213798.4, + "logits/rejected": -513591637.3333333, + "logps/chosen": -240.9342041015625, + "logps/rejected": -284.7578531901042, + "loss": 0.405, + "rewards/chosen": 0.716831636428833, + "rewards/margins": 1.0737988074620564, + "rewards/rejected": -0.35696717103322345, + "step": 116 + }, + { + "epoch": 0.19705263157894737, + "grad_norm": 313.6380304727835, + "kl": 0.0, + "learning_rate": 4.865546685075174e-07, + "logits/chosen": -904379648.0, + "logits/rejected": -820042547.2, + "logps/chosen": -220.26141357421875, + "logps/rejected": -245.7005859375, + "loss": 0.431, + "rewards/chosen": 1.131263256072998, + "rewards/margins": 1.3263026356697083, + "rewards/rejected": -0.1950393795967102, + "step": 117 + }, + { + "epoch": 0.19873684210526316, + "grad_norm": 108.28624508912246, + "kl": 0.0, + "learning_rate": 4.860747383623889e-07, + "logits/chosen": -723659136.0, + "logits/rejected": -772578752.0, + "logps/chosen": -184.33941650390625, + "logps/rejected": -185.81124877929688, + "loss": 0.3927, + "rewards/chosen": 1.3619561195373535, + "rewards/margins": 1.463188648223877, + "rewards/rejected": -0.10123252868652344, + "step": 118 + }, + { + "epoch": 0.20042105263157894, + "grad_norm": 314.88581870057607, + "kl": 0.0, + "learning_rate": 4.85586637400036e-07, + "logits/chosen": -458558624.0, + "logits/rejected": -640709376.0, + "logps/chosen": -257.4935302734375, + "logps/rejected": -263.326416015625, + "loss": 0.4136, + "rewards/chosen": 0.4196751117706299, + "rewards/margins": 1.0923622846603394, + "rewards/rejected": -0.6726871728897095, + "step": 119 + }, + { + "epoch": 0.20210526315789473, + "grad_norm": 138.4365785651591, + "kl": 0.0, + "learning_rate": 4.85090382514192e-07, + "logits/chosen": -712995328.0, + "logits/rejected": -820921941.3333334, + "logps/chosen": -222.21136474609375, + "logps/rejected": -301.34344482421875, + "loss": 0.4317, + "rewards/chosen": 0.45389634370803833, + "rewards/margins": 0.7454813917477925, + "rewards/rejected": -0.2915850480397542, + "step": 120 + }, + { + "epoch": 0.20378947368421052, + "grad_norm": 92.36506768933428, + "kl": 0.0, + "learning_rate": 4.845859908808073e-07, + "logits/chosen": -695455232.0, + "logits/rejected": -842530406.4, + "logps/chosen": -220.80411783854166, + "logps/rejected": -278.7416015625, + "loss": 0.458, + "rewards/chosen": 0.664484699567159, + "rewards/margins": 1.1589130957921348, + "rewards/rejected": -0.4944283962249756, + "step": 121 + }, + { + "epoch": 0.2054736842105263, + "grad_norm": 73.63166789116582, + "kl": 0.4639434814453125, + "learning_rate": 4.840734799574546e-07, + "logits/chosen": -1259681877.3333333, + "logits/rejected": -1012162560.0, + "logps/chosen": -189.5477091471354, + "logps/rejected": -160.91326904296875, + "loss": 0.4281, + "rewards/chosen": 0.8242993354797363, + "rewards/margins": 0.5254170298576355, + "rewards/rejected": 0.29888230562210083, + "step": 122 + }, + { + "epoch": 0.2071578947368421, + "grad_norm": 97.28869763760606, + "kl": 0.318084716796875, + "learning_rate": 4.835528674827239e-07, + "logits/chosen": -732118954.6666666, + "logits/rejected": -770999680.0, + "logps/chosen": -228.71061197916666, + "logps/rejected": -203.45211791992188, + "loss": 0.4352, + "rewards/chosen": 0.7461237907409668, + "rewards/margins": 0.08015847206115723, + "rewards/rejected": 0.6659653186798096, + "step": 123 + }, + { + "epoch": 0.20884210526315788, + "grad_norm": 103.28487237641944, + "kl": 0.0, + "learning_rate": 4.830241714756098e-07, + "logits/chosen": -580801901.7142857, + "logits/rejected": -687661312.0, + "logps/chosen": -200.93122209821428, + "logps/rejected": -259.4430847167969, + "loss": 0.3573, + "rewards/chosen": 1.0478062629699707, + "rewards/margins": -0.5003474950790405, + "rewards/rejected": 1.5481537580490112, + "step": 124 + }, + { + "epoch": 0.21052631578947367, + "grad_norm": 80.62146513928802, + "kl": 1.8628311157226562, + "learning_rate": 4.82487410234887e-07, + "logits/chosen": -867455707.4285715, + "logits/rejected": -523073568.0, + "logps/chosen": -173.80622209821428, + "logps/rejected": -181.6681671142578, + "loss": 0.5246, + "rewards/chosen": -0.30137692178998676, + "rewards/margins": -1.384874565260751, + "rewards/rejected": 1.0834976434707642, + "step": 125 + }, + { + "epoch": 0.21221052631578946, + "grad_norm": 82.55380217967634, + "kl": 2.27947998046875, + "learning_rate": 4.819426023384769e-07, + "logits/chosen": -958819942.4, + "logits/rejected": -718372181.3333334, + "logps/chosen": -212.732763671875, + "logps/rejected": -163.0384724934896, + "loss": 0.5059, + "rewards/chosen": 0.950899314880371, + "rewards/margins": 0.2773471991221109, + "rewards/rejected": 0.6735521157582601, + "step": 126 + }, + { + "epoch": 0.21389473684210528, + "grad_norm": 134.57523570837543, + "kl": 0.0, + "learning_rate": 4.813897666428053e-07, + "logits/chosen": -582362880.0, + "logits/rejected": -902837043.2, + "logps/chosen": -258.3628743489583, + "logps/rejected": -191.00611572265626, + "loss": 0.4573, + "rewards/chosen": 0.8414382934570312, + "rewards/margins": 0.5884557962417603, + "rewards/rejected": 0.252982497215271, + "step": 127 + }, + { + "epoch": 0.21557894736842106, + "grad_norm": 77.91215003119481, + "kl": 0.0, + "learning_rate": 4.80828922282149e-07, + "logits/chosen": -715421593.6, + "logits/rejected": -996736597.3333334, + "logps/chosen": -221.0246337890625, + "logps/rejected": -193.3650105794271, + "loss": 0.4897, + "rewards/chosen": 0.5778164863586426, + "rewards/margins": -0.47630953788757324, + "rewards/rejected": 1.0541260242462158, + "step": 128 + }, + { + "epoch": 0.21726315789473685, + "grad_norm": 84.25313070423411, + "kl": 2.1194534301757812, + "learning_rate": 4.802600886679741e-07, + "logits/chosen": -852968704.0, + "logits/rejected": -692032000.0, + "logps/chosen": -224.84365844726562, + "logps/rejected": -224.15106201171875, + "loss": 0.436, + "rewards/chosen": 0.7637878656387329, + "rewards/margins": 0.6827945858240128, + "rewards/rejected": 0.08099327981472015, + "step": 129 + }, + { + "epoch": 0.21894736842105264, + "grad_norm": 76.6604526230067, + "kl": 0.0, + "learning_rate": 4.79683285488264e-07, + "logits/chosen": -768312064.0, + "logits/rejected": -553834240.0, + "logps/chosen": -209.3781982421875, + "logps/rejected": -275.8822428385417, + "loss": 0.3727, + "rewards/chosen": 1.0357406616210938, + "rewards/margins": 10.20118408203125, + "rewards/rejected": -9.165443420410156, + "step": 130 + }, + { + "epoch": 0.22063157894736843, + "grad_norm": 58.96966604279425, + "kl": 0.0, + "learning_rate": 4.790985327068375e-07, + "logits/chosen": -1307087360.0, + "logits/rejected": -688753612.8, + "logps/chosen": -197.68208821614584, + "logps/rejected": -239.55947265625, + "loss": 0.4189, + "rewards/chosen": 1.5207815170288086, + "rewards/margins": 1.0784048080444335, + "rewards/rejected": 0.442376708984375, + "step": 131 + }, + { + "epoch": 0.22231578947368422, + "grad_norm": 50.487962918238146, + "kl": 0.0, + "learning_rate": 4.785058505626587e-07, + "logits/chosen": -903821824.0, + "logits/rejected": -479294592.0, + "logps/chosen": -188.7058563232422, + "logps/rejected": -270.5711364746094, + "loss": 0.4645, + "rewards/chosen": 0.9261478781700134, + "rewards/margins": 0.3811134696006775, + "rewards/rejected": 0.5450344085693359, + "step": 132 + }, + { + "epoch": 0.224, + "grad_norm": 73.6874876394, + "kl": 0.84033203125, + "learning_rate": 4.779052595691354e-07, + "logits/chosen": -728201152.0, + "logits/rejected": -1323703552.0, + "logps/chosen": -219.38204956054688, + "logps/rejected": -226.48712158203125, + "loss": 0.4986, + "rewards/chosen": 0.8902133703231812, + "rewards/margins": -0.04759258031845093, + "rewards/rejected": 0.9378059506416321, + "step": 133 + }, + { + "epoch": 0.2256842105263158, + "grad_norm": 95.59678991510799, + "kl": 0.3926353454589844, + "learning_rate": 4.772967805134105e-07, + "logits/chosen": -1039820992.0, + "logits/rejected": -1321091328.0, + "logps/chosen": -226.0445556640625, + "logps/rejected": -204.591064453125, + "loss": 0.4261, + "rewards/chosen": 1.1810604333877563, + "rewards/margins": 0.4814262390136719, + "rewards/rejected": 0.6996341943740845, + "step": 134 + }, + { + "epoch": 0.22736842105263158, + "grad_norm": 188.3750547944905, + "kl": 0.0, + "learning_rate": 4.766804344556413e-07, + "logits/chosen": -612920320.0, + "logits/rejected": -678868736.0, + "logps/chosen": -246.20358276367188, + "logps/rejected": -229.70596313476562, + "loss": 0.4773, + "rewards/chosen": 0.6662670373916626, + "rewards/margins": 1.006878674030304, + "rewards/rejected": -0.34061163663864136, + "step": 135 + }, + { + "epoch": 0.22905263157894737, + "grad_norm": 68.4177176261016, + "kl": 0.0, + "learning_rate": 4.760562427282712e-07, + "logits/chosen": -829823573.3333334, + "logits/rejected": -959791168.0, + "logps/chosen": -276.5669352213542, + "logps/rejected": -458.56427001953125, + "loss": 0.486, + "rewards/chosen": -0.15483548243840536, + "rewards/margins": 0.1716126004854838, + "rewards/rejected": -0.32644808292388916, + "step": 136 + }, + { + "epoch": 0.23073684210526316, + "grad_norm": 170.91909658805818, + "kl": 0.0, + "learning_rate": 4.754242269352911e-07, + "logits/chosen": -975248237.7142857, + "logits/rejected": -1017260032.0, + "logps/chosen": -223.60107421875, + "logps/rejected": -248.3441162109375, + "loss": 0.3863, + "rewards/chosen": 1.2192728860037667, + "rewards/margins": 1.134688836123262, + "rewards/rejected": 0.08458404988050461, + "step": 137 + }, + { + "epoch": 0.23242105263157894, + "grad_norm": 49.863175356012206, + "kl": 0.0, + "learning_rate": 4.747844089514919e-07, + "logits/chosen": -1015484825.6, + "logits/rejected": -859580074.6666666, + "logps/chosen": -238.688232421875, + "logps/rejected": -222.31477864583334, + "loss": 0.4322, + "rewards/chosen": 1.2349286079406738, + "rewards/margins": 1.1146054069201152, + "rewards/rejected": 0.12032320102055867, + "step": 138 + }, + { + "epoch": 0.23410526315789473, + "grad_norm": 253.28180299915874, + "kl": 2.1038894653320312, + "learning_rate": 4.741368109217071e-07, + "logits/chosen": -1175646293.3333333, + "logits/rejected": -688029491.2, + "logps/chosen": -291.4467366536458, + "logps/rejected": -250.909619140625, + "loss": 0.4026, + "rewards/chosen": 0.8213460445404053, + "rewards/margins": 1.0762538671493531, + "rewards/rejected": -0.25490782260894773, + "step": 139 + }, + { + "epoch": 0.23578947368421052, + "grad_norm": 80.42815443807443, + "kl": 0.0, + "learning_rate": 4.734814552600468e-07, + "logits/chosen": -934606677.3333334, + "logits/rejected": -1007555904.0, + "logps/chosen": -220.41634114583334, + "logps/rejected": -240.6993865966797, + "loss": 0.4317, + "rewards/chosen": 0.8413275877634684, + "rewards/margins": 0.3180929819742838, + "rewards/rejected": 0.5232346057891846, + "step": 140 + }, + { + "epoch": 0.2374736842105263, + "grad_norm": 466.33070749527764, + "kl": 0.0, + "learning_rate": 4.728183646491214e-07, + "logits/chosen": -753246873.6, + "logits/rejected": -974831786.6666666, + "logps/chosen": -238.600927734375, + "logps/rejected": -243.89959716796875, + "loss": 0.3808, + "rewards/chosen": -0.32029788494110106, + "rewards/margins": -0.08840243816375731, + "rewards/rejected": -0.23189544677734375, + "step": 141 + }, + { + "epoch": 0.2391578947368421, + "grad_norm": 176.73411471688468, + "kl": 0.0, + "learning_rate": 4.721475620392567e-07, + "logits/chosen": -908911445.3333334, + "logits/rejected": -713944217.6, + "logps/chosen": -227.62554931640625, + "logps/rejected": -191.32474365234376, + "loss": 0.4569, + "rewards/chosen": 1.7199311256408691, + "rewards/margins": 2.082160711288452, + "rewards/rejected": -0.362229585647583, + "step": 142 + }, + { + "epoch": 0.24084210526315789, + "grad_norm": 566.7557950897859, + "kl": 0.0, + "learning_rate": 4.7146907064769993e-07, + "logits/chosen": -1072672170.6666666, + "logits/rejected": -1189377228.8, + "logps/chosen": -252.31046549479166, + "logps/rejected": -199.03504638671876, + "loss": 0.4128, + "rewards/chosen": 0.5313369830449423, + "rewards/margins": 1.2499615271886189, + "rewards/rejected": -0.7186245441436767, + "step": 143 + }, + { + "epoch": 0.24252631578947367, + "grad_norm": 4243.997152918305, + "kl": 0.0, + "learning_rate": 4.707829139578155e-07, + "logits/chosen": -873702195.2, + "logits/rejected": -1221907285.3333333, + "logps/chosen": -200.1794921875, + "logps/rejected": -223.1510009765625, + "loss": 0.4001, + "rewards/chosen": 0.7977983474731445, + "rewards/margins": 2.0752524852752687, + "rewards/rejected": -1.277454137802124, + "step": 144 + }, + { + "epoch": 0.24421052631578946, + "grad_norm": 314.5163932036576, + "kl": 4.798194885253906, + "learning_rate": 4.7008911571827284e-07, + "logits/chosen": -978259712.0, + "logits/rejected": -640961024.0, + "logps/chosen": -256.1488342285156, + "logps/rejected": -242.27796936035156, + "loss": 0.4857, + "rewards/chosen": 0.40538519620895386, + "rewards/margins": 0.4533770680427551, + "rewards/rejected": -0.04799187183380127, + "step": 145 + }, + { + "epoch": 0.24589473684210525, + "grad_norm": 360.7422844202394, + "kl": 0.0, + "learning_rate": 4.6938769994222406e-07, + "logits/chosen": -825879756.8, + "logits/rejected": -888731989.3333334, + "logps/chosen": -198.52425537109374, + "logps/rejected": -280.38706461588544, + "loss": 0.358, + "rewards/chosen": 0.7168326377868652, + "rewards/margins": 1.5658982594807944, + "rewards/rejected": -0.8490656216939291, + "step": 146 + }, + { + "epoch": 0.24757894736842107, + "grad_norm": 2156.8379086317163, + "kl": 0.0, + "learning_rate": 4.686786909064729e-07, + "logits/chosen": -410741248.0, + "logits/rejected": -832765440.0, + "logps/chosen": -249.67108154296875, + "logps/rejected": -203.24400329589844, + "loss": 0.421, + "rewards/chosen": -0.19027066230773926, + "rewards/margins": -0.8781961798667908, + "rewards/rejected": 0.6879255175590515, + "step": 147 + }, + { + "epoch": 0.24926315789473685, + "grad_norm": 286.56368982318264, + "kl": 0.0, + "learning_rate": 4.679621131506346e-07, + "logits/chosen": -1148589421.7142856, + "logits/rejected": -688305152.0, + "logps/chosen": -267.63023158482144, + "logps/rejected": -254.83946228027344, + "loss": 0.4676, + "rewards/chosen": 0.4722416400909424, + "rewards/margins": 0.7534733414649963, + "rewards/rejected": -0.28123170137405396, + "step": 148 + }, + { + "epoch": 0.25094736842105264, + "grad_norm": 222.97278644560413, + "kl": 0.0, + "learning_rate": 4.6723799147628664e-07, + "logits/chosen": -779184896.0, + "logits/rejected": -826172416.0, + "logps/chosen": -190.68980407714844, + "logps/rejected": -227.09054565429688, + "loss": 0.4862, + "rewards/chosen": 1.420555830001831, + "rewards/margins": 1.2711764872074127, + "rewards/rejected": 0.14937934279441833, + "step": 149 + }, + { + "epoch": 0.25263157894736843, + "grad_norm": 145.46733652305187, + "kl": 0.0, + "learning_rate": 4.6650635094610966e-07, + "logits/chosen": -831112106.6666666, + "logits/rejected": -727667648.0, + "logps/chosen": -205.18501790364584, + "logps/rejected": -203.62704467773438, + "loss": 0.4654, + "rewards/chosen": 0.4877975384394328, + "rewards/margins": -0.5469383796056111, + "rewards/rejected": 1.034735918045044, + "step": 150 + }, + { + "epoch": 0.2543157894736842, + "grad_norm": 148.86465225969857, + "kl": 0.0, + "learning_rate": 4.6576721688302103e-07, + "logits/chosen": -616928128.0, + "logits/rejected": -506641365.3333333, + "logps/chosen": -269.32086181640625, + "logps/rejected": -279.0386962890625, + "loss": 0.4548, + "rewards/chosen": -0.4652397334575653, + "rewards/margins": -0.7438713212807972, + "rewards/rejected": 0.278631587823232, + "step": 151 + }, + { + "epoch": 0.256, + "grad_norm": 80.8642282340819, + "kl": 0.0, + "learning_rate": 4.6502061486929765e-07, + "logits/chosen": -1172402585.6, + "logits/rejected": -611508778.6666666, + "logps/chosen": -199.3252197265625, + "logps/rejected": -200.5145263671875, + "loss": 0.4583, + "rewards/chosen": 0.7260040283203125, + "rewards/margins": 0.8308873613675436, + "rewards/rejected": -0.10488333304723103, + "step": 152 + }, + { + "epoch": 0.2576842105263158, + "grad_norm": 59.01380093137268, + "kl": 0.0, + "learning_rate": 4.6426657074569076e-07, + "logits/chosen": -679793049.6, + "logits/rejected": -954248618.6666666, + "logps/chosen": -219.611181640625, + "logps/rejected": -154.1990966796875, + "loss": 0.4257, + "rewards/chosen": 0.9013955116271972, + "rewards/margins": 1.4685372829437255, + "rewards/rejected": -0.5671417713165283, + "step": 153 + }, + { + "epoch": 0.2593684210526316, + "grad_norm": 165.5135211510967, + "kl": 0.0, + "learning_rate": 4.6350511061053155e-07, + "logits/chosen": -996774144.0, + "logits/rejected": -484326496.0, + "logps/chosen": -222.6114501953125, + "logps/rejected": -187.74560546875, + "loss": 0.4011, + "rewards/chosen": 0.8318793773651123, + "rewards/margins": 1.6771708726882935, + "rewards/rejected": -0.8452914953231812, + "step": 154 + }, + { + "epoch": 0.26105263157894737, + "grad_norm": 62.878815512005815, + "kl": 0.0, + "learning_rate": 4.62736260818828e-07, + "logits/chosen": -1305949610.6666667, + "logits/rejected": -473263104.0, + "logps/chosen": -258.23980712890625, + "logps/rejected": -231.45439453125, + "loss": 0.4867, + "rewards/chosen": -0.5735707680384318, + "rewards/margins": -0.266325561205546, + "rewards/rejected": -0.30724520683288575, + "step": 155 + }, + { + "epoch": 0.26273684210526316, + "grad_norm": 68.5671614498506, + "kl": 0.0, + "learning_rate": 4.6196004798135236e-07, + "logits/chosen": -818773568.0, + "logits/rejected": -855193408.0, + "logps/chosen": -297.919677734375, + "logps/rejected": -230.89285278320312, + "loss": 0.4601, + "rewards/chosen": -0.12369692325592041, + "rewards/margins": -0.5421707332134247, + "rewards/rejected": 0.4184738099575043, + "step": 156 + }, + { + "epoch": 0.26442105263157895, + "grad_norm": 49.45697259448593, + "kl": 0.0, + "learning_rate": 4.611764989637205e-07, + "logits/chosen": -671601715.2, + "logits/rejected": -1087509162.6666667, + "logps/chosen": -242.408056640625, + "logps/rejected": -313.653564453125, + "loss": 0.5013, + "rewards/chosen": 1.0554190635681153, + "rewards/margins": 3.382936731974284, + "rewards/rejected": -2.3275176684061685, + "step": 157 + }, + { + "epoch": 0.26610526315789473, + "grad_norm": 106.47797133432495, + "kl": 0.0, + "learning_rate": 4.603856408854618e-07, + "logits/chosen": -757610240.0, + "logits/rejected": -791967232.0, + "logps/chosen": -249.16958618164062, + "logps/rejected": -192.21998596191406, + "loss": 0.4991, + "rewards/chosen": 1.1682857275009155, + "rewards/margins": 0.8241084814071655, + "rewards/rejected": 0.34417724609375, + "step": 158 + }, + { + "epoch": 0.2677894736842105, + "grad_norm": 49.18374091004554, + "kl": 0.0, + "learning_rate": 4.5958750111908065e-07, + "logits/chosen": -1117810432.0, + "logits/rejected": -668232345.6, + "logps/chosen": -165.53057861328125, + "logps/rejected": -253.265234375, + "loss": 0.502, + "rewards/chosen": 0.5383493900299072, + "rewards/margins": 0.5533985137939453, + "rewards/rejected": -0.015049123764038086, + "step": 159 + }, + { + "epoch": 0.2694736842105263, + "grad_norm": 44.5074703062048, + "kl": 0.0, + "learning_rate": 4.5878210728910886e-07, + "logits/chosen": -953598976.0, + "logits/rejected": -1030626918.4, + "logps/chosen": -242.2763875325521, + "logps/rejected": -223.04658203125, + "loss": 0.4662, + "rewards/chosen": 0.856818675994873, + "rewards/margins": 0.2501978874206543, + "rewards/rejected": 0.6066207885742188, + "step": 160 + }, + { + "epoch": 0.2711578947368421, + "grad_norm": 75.43811234031128, + "kl": 0.0, + "learning_rate": 4.5796948727115e-07, + "logits/chosen": -793493299.2, + "logits/rejected": -714943658.6666666, + "logps/chosen": -278.5044189453125, + "logps/rejected": -209.54473876953125, + "loss": 0.3873, + "rewards/chosen": 0.2761260747909546, + "rewards/margins": -0.06068220933278401, + "rewards/rejected": 0.3368082841237386, + "step": 161 + }, + { + "epoch": 0.2728421052631579, + "grad_norm": 63.90407966356232, + "kl": 0.0, + "learning_rate": 4.5714966919091414e-07, + "logits/chosen": -482634784.0, + "logits/rejected": -1263959466.6666667, + "logps/chosen": -282.9561767578125, + "logps/rejected": -233.3080037434896, + "loss": 0.398, + "rewards/chosen": 0.31816405057907104, + "rewards/margins": 0.5229944586753845, + "rewards/rejected": -0.20483040809631348, + "step": 162 + }, + { + "epoch": 0.2745263157894737, + "grad_norm": 121.01434181632379, + "kl": 0.0, + "learning_rate": 4.5632268142324435e-07, + "logits/chosen": -697656746.6666666, + "logits/rejected": -517255552.0, + "logps/chosen": -242.203369140625, + "logps/rejected": -196.41952514648438, + "loss": 0.3492, + "rewards/chosen": 1.0457820097605388, + "rewards/margins": 1.9965357581774394, + "rewards/rejected": -0.9507537484169006, + "step": 163 + }, + { + "epoch": 0.27621052631578946, + "grad_norm": 59.7684852435887, + "kl": 0.0, + "learning_rate": 4.554885525911351e-07, + "logits/chosen": -932756352.0, + "logits/rejected": -709267626.6666666, + "logps/chosen": -220.3507537841797, + "logps/rejected": -211.04886881510416, + "loss": 0.4335, + "rewards/chosen": 0.9487717151641846, + "rewards/margins": 2.4863777955373125, + "rewards/rejected": -1.5376060803731282, + "step": 164 + }, + { + "epoch": 0.27789473684210525, + "grad_norm": 84.16432681430886, + "kl": 0.0, + "learning_rate": 4.546473115647409e-07, + "logits/chosen": -594779008.0, + "logits/rejected": -749184102.4, + "logps/chosen": -238.7244873046875, + "logps/rejected": -268.174755859375, + "loss": 0.42, + "rewards/chosen": -1.2170298099517822, + "rewards/margins": 0.033264207839965865, + "rewards/rejected": -1.250294017791748, + "step": 165 + }, + { + "epoch": 0.27957894736842104, + "grad_norm": 72.08410795046584, + "kl": 0.0, + "learning_rate": 4.53798987460378e-07, + "logits/chosen": -850566144.0, + "logits/rejected": -682344243.2, + "logps/chosen": -284.1813557942708, + "logps/rejected": -257.943896484375, + "loss": 0.3765, + "rewards/chosen": -0.22490237156550089, + "rewards/margins": 2.2622447689374288, + "rewards/rejected": -2.4871471405029295, + "step": 166 + }, + { + "epoch": 0.2812631578947368, + "grad_norm": 226.29378772227912, + "kl": 0.0, + "learning_rate": 4.529436096395156e-07, + "logits/chosen": -800250816.0, + "logits/rejected": -1130786816.0, + "logps/chosen": -183.79263305664062, + "logps/rejected": -233.85260009765625, + "loss": 0.4077, + "rewards/chosen": 0.2685844302177429, + "rewards/margins": 0.8706353306770325, + "rewards/rejected": -0.6020509004592896, + "step": 167 + }, + { + "epoch": 0.2829473684210526, + "grad_norm": 69.313319440612, + "kl": 0.0, + "learning_rate": 4.520812077077604e-07, + "logits/chosen": -840975616.0, + "logits/rejected": -762263040.0, + "logps/chosen": -194.9156005859375, + "logps/rejected": -208.27591959635416, + "loss": 0.5229, + "rewards/chosen": -0.07560654282569886, + "rewards/margins": 0.054520648717880246, + "rewards/rejected": -0.1301271915435791, + "step": 168 + }, + { + "epoch": 0.2846315789473684, + "grad_norm": 93.08308180288073, + "kl": 0.0, + "learning_rate": 4.512118115138314e-07, + "logits/chosen": -760346965.3333334, + "logits/rejected": -865922432.0, + "logps/chosen": -213.150634765625, + "logps/rejected": -177.36080932617188, + "loss": 0.4687, + "rewards/chosen": 0.10082270701726277, + "rewards/margins": 0.7567354639371237, + "rewards/rejected": -0.6559127569198608, + "step": 169 + }, + { + "epoch": 0.2863157894736842, + "grad_norm": 91.57290262396198, + "kl": 0.0, + "learning_rate": 4.503354511485273e-07, + "logits/chosen": -858243276.8, + "logits/rejected": -1284581632.0, + "logps/chosen": -279.31884765625, + "logps/rejected": -272.38559977213544, + "loss": 0.4222, + "rewards/chosen": -0.1363077163696289, + "rewards/margins": 0.4786560535430908, + "rewards/rejected": -0.6149637699127197, + "step": 170 + }, + { + "epoch": 0.288, + "grad_norm": 99.04964646632929, + "kl": 0.0, + "learning_rate": 4.4945215694368447e-07, + "logits/chosen": -328858944.0, + "logits/rejected": -713192234.6666666, + "logps/chosen": -172.78839111328125, + "logps/rejected": -212.23014322916666, + "loss": 0.4366, + "rewards/chosen": 0.2652008533477783, + "rewards/margins": 1.0063419342041016, + "rewards/rejected": -0.7411410808563232, + "step": 171 + }, + { + "epoch": 0.28968421052631577, + "grad_norm": 89.56438737059334, + "kl": 0.0, + "learning_rate": 4.485619594711277e-07, + "logits/chosen": -598744512.0, + "logits/rejected": -598062378.6666666, + "logps/chosen": -262.1839599609375, + "logps/rejected": -212.77608235677084, + "loss": 0.4281, + "rewards/chosen": 0.616127073764801, + "rewards/margins": 0.38657637437184655, + "rewards/rejected": 0.2295506993929545, + "step": 172 + }, + { + "epoch": 0.29136842105263155, + "grad_norm": 55.09758282232377, + "kl": 0.0, + "learning_rate": 4.4766488954161153e-07, + "logits/chosen": -804758186.6666666, + "logits/rejected": -721973760.0, + "logps/chosen": -261.65061442057294, + "logps/rejected": -265.42978515625, + "loss": 0.4359, + "rewards/chosen": -0.7668237686157227, + "rewards/margins": 0.3059118270874024, + "rewards/rejected": -1.072735595703125, + "step": 173 + }, + { + "epoch": 0.29305263157894734, + "grad_norm": 69.82107320148485, + "kl": 0.0, + "learning_rate": 4.4676097820375426e-07, + "logits/chosen": -515698304.0, + "logits/rejected": -1235355648.0, + "logps/chosen": -205.48640950520834, + "logps/rejected": -245.5269287109375, + "loss": 0.4254, + "rewards/chosen": 0.9509898026784261, + "rewards/margins": 0.5845218737920126, + "rewards/rejected": 0.36646792888641355, + "step": 174 + }, + { + "epoch": 0.29473684210526313, + "grad_norm": 78.35333198019593, + "kl": 0.0, + "learning_rate": 4.458502567429631e-07, + "logits/chosen": -498773248.0, + "logits/rejected": -519963648.0, + "logps/chosen": -194.12052408854166, + "logps/rejected": -172.4505615234375, + "loss": 0.4344, + "rewards/chosen": 0.4248570998509725, + "rewards/margins": 1.5527800162633258, + "rewards/rejected": -1.1279229164123534, + "step": 175 + }, + { + "epoch": 0.296421052631579, + "grad_norm": 169.14882087581336, + "kl": 0.0, + "learning_rate": 4.4493275668035147e-07, + "logits/chosen": -721955242.6666666, + "logits/rejected": -710825932.8, + "logps/chosen": -167.6915079752604, + "logps/rejected": -147.37952880859376, + "loss": 0.4063, + "rewards/chosen": 0.4422839085261027, + "rewards/margins": 2.1021666447321574, + "rewards/rejected": -1.6598827362060546, + "step": 176 + }, + { + "epoch": 0.29810526315789476, + "grad_norm": 67.3786479497002, + "kl": 0.0, + "learning_rate": 4.440085097716479e-07, + "logits/chosen": -663990016.0, + "logits/rejected": -555922304.0, + "logps/chosen": -259.0157470703125, + "logps/rejected": -192.321533203125, + "loss": 0.4259, + "rewards/chosen": 0.24189604818820953, + "rewards/margins": 0.6399521082639694, + "rewards/rejected": -0.3980560600757599, + "step": 177 + }, + { + "epoch": 0.29978947368421055, + "grad_norm": 80.2865086271179, + "kl": 0.0, + "learning_rate": 4.430775480060972e-07, + "logits/chosen": -851462912.0, + "logits/rejected": -1203325952.0, + "logps/chosen": -203.14907836914062, + "logps/rejected": -263.4740905761719, + "loss": 0.3982, + "rewards/chosen": -1.356869101524353, + "rewards/margins": 0.5466030836105347, + "rewards/rejected": -1.9034721851348877, + "step": 178 + }, + { + "epoch": 0.30147368421052634, + "grad_norm": 84.09627558392572, + "kl": 0.0, + "learning_rate": 4.4213990360535267e-07, + "logits/chosen": -656672042.6666666, + "logits/rejected": -944876748.8, + "logps/chosen": -294.62876383463544, + "logps/rejected": -214.0318115234375, + "loss": 0.4823, + "rewards/chosen": -1.5061303774515789, + "rewards/margins": -1.1466392676035564, + "rewards/rejected": -0.35949110984802246, + "step": 179 + }, + { + "epoch": 0.3031578947368421, + "grad_norm": 50.8032139364716, + "kl": 0.0, + "learning_rate": 4.4119560902236174e-07, + "logits/chosen": -764519808.0, + "logits/rejected": -615565184.0, + "logps/chosen": -261.0996398925781, + "logps/rejected": -222.42861938476562, + "loss": 0.4579, + "rewards/chosen": -0.5169571042060852, + "rewards/margins": 1.855327069759369, + "rewards/rejected": -2.372284173965454, + "step": 180 + }, + { + "epoch": 0.3048421052631579, + "grad_norm": 60.14921204443068, + "kl": 0.0, + "learning_rate": 4.402446969402419e-07, + "logits/chosen": -1501441194.6666667, + "logits/rejected": -1463862681.6, + "logps/chosen": -328.10398356119794, + "logps/rejected": -221.4558349609375, + "loss": 0.4195, + "rewards/chosen": -3.202397664388021, + "rewards/margins": -1.721674950917562, + "rewards/rejected": -1.480722713470459, + "step": 181 + }, + { + "epoch": 0.3065263157894737, + "grad_norm": 57.771134136758484, + "kl": 0.0, + "learning_rate": 4.392872002711501e-07, + "logits/chosen": -459086745.6, + "logits/rejected": -1257842090.6666667, + "logps/chosen": -290.8982421875, + "logps/rejected": -208.20979817708334, + "loss": 0.4709, + "rewards/chosen": -1.0606164932250977, + "rewards/margins": 0.5609116554260254, + "rewards/rejected": -1.621528148651123, + "step": 182 + }, + { + "epoch": 0.3082105263157895, + "grad_norm": 237.395695237356, + "kl": 0.0, + "learning_rate": 4.3832315215514316e-07, + "logits/chosen": -752865587.2, + "logits/rejected": -800709973.3333334, + "logps/chosen": -222.4587646484375, + "logps/rejected": -164.1585693359375, + "loss": 0.3825, + "rewards/chosen": 0.5814694404602051, + "rewards/margins": 0.6762328982353211, + "rewards/rejected": -0.09476345777511597, + "step": 183 + }, + { + "epoch": 0.3098947368421053, + "grad_norm": 68.49160205355011, + "kl": 0.0, + "learning_rate": 4.3735258595903123e-07, + "logits/chosen": -1131358412.8, + "logits/rejected": -615695616.0, + "logps/chosen": -263.4253173828125, + "logps/rejected": -251.53666178385416, + "loss": 0.4091, + "rewards/chosen": -0.1255630373954773, + "rewards/margins": 2.537329304218292, + "rewards/rejected": -2.6628923416137695, + "step": 184 + }, + { + "epoch": 0.31157894736842107, + "grad_norm": 56.555913296788326, + "kl": 0.0, + "learning_rate": 4.3637553527522265e-07, + "logits/chosen": -537920128.0, + "logits/rejected": -931403434.6666666, + "logps/chosen": -261.276611328125, + "logps/rejected": -258.6404622395833, + "loss": 0.4377, + "rewards/chosen": -0.8533248901367188, + "rewards/margins": -0.1802154382069906, + "rewards/rejected": -0.6731094519297282, + "step": 185 + }, + { + "epoch": 0.31326315789473685, + "grad_norm": 62.41772660361089, + "kl": 0.0, + "learning_rate": 4.353920339205611e-07, + "logits/chosen": -722902016.0, + "logits/rejected": -948598144.0, + "logps/chosen": -224.0010223388672, + "logps/rejected": -272.22760009765625, + "loss": 0.4404, + "rewards/chosen": 0.367422878742218, + "rewards/margins": 0.9069034457206726, + "rewards/rejected": -0.5394805669784546, + "step": 186 + }, + { + "epoch": 0.31494736842105264, + "grad_norm": 64.75287382395764, + "kl": 0.0, + "learning_rate": 4.344021159351555e-07, + "logits/chosen": -1008326553.6, + "logits/rejected": -771843413.3333334, + "logps/chosen": -227.4381591796875, + "logps/rejected": -186.0328369140625, + "loss": 0.4424, + "rewards/chosen": -0.36620848178863524, + "rewards/margins": -0.2139639139175415, + "rewards/rejected": -0.15224456787109375, + "step": 187 + }, + { + "epoch": 0.31663157894736843, + "grad_norm": 70.9290525757904, + "kl": 0.0, + "learning_rate": 4.3340581558120195e-07, + "logits/chosen": -618546688.0, + "logits/rejected": -912823125.3333334, + "logps/chosen": -246.94599609375, + "logps/rejected": -296.3729654947917, + "loss": 0.3832, + "rewards/chosen": -0.18909727334976195, + "rewards/margins": 3.131844015916189, + "rewards/rejected": -3.3209412892659507, + "step": 188 + }, + { + "epoch": 0.3183157894736842, + "grad_norm": 70.41433062876469, + "kl": 0.0, + "learning_rate": 4.3240316734179705e-07, + "logits/chosen": -761420928.0, + "logits/rejected": -672102144.0, + "logps/chosen": -227.72821044921875, + "logps/rejected": -258.29815673828125, + "loss": 0.4255, + "rewards/chosen": 0.08762133121490479, + "rewards/margins": 3.5601266622543335, + "rewards/rejected": -3.4725053310394287, + "step": 189 + }, + { + "epoch": 0.32, + "grad_norm": 66.55630414028238, + "kl": 0.0, + "learning_rate": 4.313942059197456e-07, + "logits/chosen": -618079808.0, + "logits/rejected": -658955968.0, + "logps/chosen": -252.71575927734375, + "logps/rejected": -140.86489868164062, + "loss": 0.4927, + "rewards/chosen": -0.4330390989780426, + "rewards/margins": -0.2638881802558899, + "rewards/rejected": -0.1691509187221527, + "step": 190 + }, + { + "epoch": 0.3216842105263158, + "grad_norm": 94.67677667797632, + "kl": 0.0, + "learning_rate": 4.3037896623635864e-07, + "logits/chosen": -923324928.0, + "logits/rejected": -594331520.0, + "logps/chosen": -247.37379455566406, + "logps/rejected": -226.16342163085938, + "loss": 0.367, + "rewards/chosen": 0.9253654479980469, + "rewards/margins": 2.8820842504501343, + "rewards/rejected": -1.9567188024520874, + "step": 191 + }, + { + "epoch": 0.3233684210526316, + "grad_norm": 54.17392420976664, + "kl": 0.0, + "learning_rate": 4.29357483430245e-07, + "logits/chosen": -664615808.0, + "logits/rejected": -569389952.0, + "logps/chosen": -216.7957763671875, + "logps/rejected": -197.62344360351562, + "loss": 0.4826, + "rewards/chosen": -0.4946579138437907, + "rewards/margins": 0.1053409179051717, + "rewards/rejected": -0.5999988317489624, + "step": 192 + }, + { + "epoch": 0.32505263157894737, + "grad_norm": 75.00984735078153, + "kl": 1.3663177490234375, + "learning_rate": 4.2832979285609505e-07, + "logits/chosen": -614758997.3333334, + "logits/rejected": -654426368.0, + "logps/chosen": -235.02339680989584, + "logps/rejected": -149.56541442871094, + "loss": 0.3863, + "rewards/chosen": 0.690934975941976, + "rewards/margins": 0.510087807973226, + "rewards/rejected": 0.18084716796875, + "step": 193 + }, + { + "epoch": 0.32673684210526316, + "grad_norm": 60.51774121025042, + "kl": 0.0, + "learning_rate": 4.2729593008345734e-07, + "logits/chosen": -1201705642.6666667, + "logits/rejected": -741241139.2, + "logps/chosen": -172.75325520833334, + "logps/rejected": -184.46663818359374, + "loss": 0.3963, + "rewards/chosen": 1.0127630233764648, + "rewards/margins": 2.2689538955688477, + "rewards/rejected": -1.256190872192383, + "step": 194 + }, + { + "epoch": 0.32842105263157895, + "grad_norm": 90.59367998785133, + "kl": 0.0, + "learning_rate": 4.2625593089550717e-07, + "logits/chosen": -541644697.6, + "logits/rejected": -1247126954.6666667, + "logps/chosen": -258.279150390625, + "logps/rejected": -216.1505126953125, + "loss": 0.458, + "rewards/chosen": 0.21086244583129882, + "rewards/margins": 0.3597394009431203, + "rewards/rejected": -0.1488769551118215, + "step": 195 + }, + { + "epoch": 0.33010526315789473, + "grad_norm": 95.59189487331257, + "kl": 0.0, + "learning_rate": 4.2520983128780825e-07, + "logits/chosen": -635985920.0, + "logits/rejected": -922811136.0, + "logps/chosen": -279.5119323730469, + "logps/rejected": -208.4548543294271, + "loss": 0.3814, + "rewards/chosen": -0.1562713384628296, + "rewards/margins": 1.136791984240214, + "rewards/rejected": -1.2930633227030437, + "step": 196 + }, + { + "epoch": 0.3317894736842105, + "grad_norm": 54.38017510634436, + "kl": 0.0, + "learning_rate": 4.2415766746706674e-07, + "logits/chosen": -605963136.0, + "logits/rejected": -808883507.2, + "logps/chosen": -224.5291544596354, + "logps/rejected": -254.458984375, + "loss": 0.3932, + "rewards/chosen": -1.42087189356486, + "rewards/margins": 1.0473615964253742, + "rewards/rejected": -2.4682334899902343, + "step": 197 + }, + { + "epoch": 0.3334736842105263, + "grad_norm": 77.91234986865499, + "kl": 0.0, + "learning_rate": 4.230994758498782e-07, + "logits/chosen": -977752576.0, + "logits/rejected": -573459456.0, + "logps/chosen": -287.57672119140625, + "logps/rejected": -200.0487823486328, + "loss": 0.4666, + "rewards/chosen": -0.592963457107544, + "rewards/margins": -0.5604517459869385, + "rewards/rejected": -0.03251171112060547, + "step": 198 + }, + { + "epoch": 0.3351578947368421, + "grad_norm": 217.16301375592573, + "kl": 0.0, + "learning_rate": 4.220352930614672e-07, + "logits/chosen": -1347623082.6666667, + "logits/rejected": -1223722905.6, + "logps/chosen": -250.11665852864584, + "logps/rejected": -210.2994140625, + "loss": 0.3773, + "rewards/chosen": 0.5189147790273031, + "rewards/margins": 0.7931217749913534, + "rewards/rejected": -0.2742069959640503, + "step": 199 + }, + { + "epoch": 0.3368421052631579, + "grad_norm": 58.97358660347629, + "kl": 0.0, + "learning_rate": 4.209651559344194e-07, + "logits/chosen": -1134724096.0, + "logits/rejected": -713243093.3333334, + "logps/chosen": -202.700537109375, + "logps/rejected": -273.8854573567708, + "loss": 0.4269, + "rewards/chosen": 0.44961090087890626, + "rewards/margins": 0.7851913293202718, + "rewards/rejected": -0.33558042844136554, + "step": 200 + }, + { + "epoch": 0.3385263157894737, + "grad_norm": 72.17373452754133, + "kl": 0.0, + "learning_rate": 4.198891015074073e-07, + "logits/chosen": -646841984.0, + "logits/rejected": -952984896.0, + "logps/chosen": -190.56002807617188, + "logps/rejected": -252.1724090576172, + "loss": 0.4577, + "rewards/chosen": -0.3583839535713196, + "rewards/margins": -1.000853717327118, + "rewards/rejected": 0.6424697637557983, + "step": 201 + }, + { + "epoch": 0.34021052631578946, + "grad_norm": 80.45111424098052, + "kl": 0.0, + "learning_rate": 4.1880716702390763e-07, + "logits/chosen": -1190700544.0, + "logits/rejected": -969268480.0, + "logps/chosen": -192.81265258789062, + "logps/rejected": -200.2042236328125, + "loss": 0.414, + "rewards/chosen": 1.0531678199768066, + "rewards/margins": 3.0669215520222983, + "rewards/rejected": -2.0137537320454917, + "step": 202 + }, + { + "epoch": 0.34189473684210525, + "grad_norm": 94.02730380206307, + "kl": 0.0, + "learning_rate": 4.177193899309126e-07, + "logits/chosen": -503011993.6, + "logits/rejected": -704219050.6666666, + "logps/chosen": -218.402734375, + "logps/rejected": -179.79007975260416, + "loss": 0.4332, + "rewards/chosen": 0.8202567100524902, + "rewards/margins": 1.5902131398518882, + "rewards/rejected": -0.7699564297993978, + "step": 203 + }, + { + "epoch": 0.34357894736842104, + "grad_norm": 54.77503186544864, + "kl": 0.0, + "learning_rate": 4.166258078776341e-07, + "logits/chosen": -832879786.6666666, + "logits/rejected": -515199590.4, + "logps/chosen": -202.35528564453125, + "logps/rejected": -200.24393310546876, + "loss": 0.4434, + "rewards/chosen": 0.5717076857884725, + "rewards/margins": 2.901677473386129, + "rewards/rejected": -2.329969787597656, + "step": 204 + }, + { + "epoch": 0.3452631578947368, + "grad_norm": 148.39721947330267, + "kl": 0.0, + "learning_rate": 4.155264587142001e-07, + "logits/chosen": -1255011840.0, + "logits/rejected": -1037022464.0, + "logps/chosen": -242.1149139404297, + "logps/rejected": -220.04427083333334, + "loss": 0.5528, + "rewards/chosen": -0.22197723388671875, + "rewards/margins": -0.06859992941220602, + "rewards/rejected": -0.15337730447451273, + "step": 205 + }, + { + "epoch": 0.3469473684210526, + "grad_norm": 49.784955690964, + "kl": 0.0, + "learning_rate": 4.1442138049034494e-07, + "logits/chosen": -775684266.6666666, + "logits/rejected": -880259379.2, + "logps/chosen": -191.68634033203125, + "logps/rejected": -225.9909912109375, + "loss": 0.3432, + "rewards/chosen": 1.5534348487854004, + "rewards/margins": 3.249916934967041, + "rewards/rejected": -1.6964820861816405, + "step": 206 + }, + { + "epoch": 0.3486315789473684, + "grad_norm": 91.02392105011648, + "kl": 0.0, + "learning_rate": 4.133106114540923e-07, + "logits/chosen": -971414336.0, + "logits/rejected": -1156522368.0, + "logps/chosen": -213.8065185546875, + "logps/rejected": -225.66744995117188, + "loss": 0.3929, + "rewards/chosen": -0.12490347027778625, + "rewards/margins": 0.5315063297748566, + "rewards/rejected": -0.6564098000526428, + "step": 207 + }, + { + "epoch": 0.3503157894736842, + "grad_norm": 68.74268293327242, + "kl": 0.0, + "learning_rate": 4.1219419005043154e-07, + "logits/chosen": -916664064.0, + "logits/rejected": -622102208.0, + "logps/chosen": -237.611083984375, + "logps/rejected": -284.42645263671875, + "loss": 0.5005, + "rewards/chosen": 0.0357242226600647, + "rewards/margins": 0.9489158391952515, + "rewards/rejected": -0.9131916165351868, + "step": 208 + }, + { + "epoch": 0.352, + "grad_norm": 76.77975476469975, + "kl": 0.0, + "learning_rate": 4.110721549199866e-07, + "logits/chosen": -1072110400.0, + "logits/rejected": -753468608.0, + "logps/chosen": -184.95042419433594, + "logps/rejected": -249.1209716796875, + "loss": 0.3055, + "rewards/chosen": 0.7373764514923096, + "rewards/margins": 2.8142330646514893, + "rewards/rejected": -2.0768566131591797, + "step": 209 + }, + { + "epoch": 0.35368421052631577, + "grad_norm": 80.64382792874225, + "kl": 0.0, + "learning_rate": 4.0994454489767927e-07, + "logits/chosen": -689984256.0, + "logits/rejected": -1297480448.0, + "logps/chosen": -210.7186279296875, + "logps/rejected": -249.07969665527344, + "loss": 0.3988, + "rewards/chosen": 0.8691381613413492, + "rewards/margins": 4.103166739145915, + "rewards/rejected": -3.2340285778045654, + "step": 210 + }, + { + "epoch": 0.35536842105263156, + "grad_norm": 149.68736456789574, + "kl": 0.0, + "learning_rate": 4.088113990113846e-07, + "logits/chosen": -731458048.0, + "logits/rejected": -345419264.0, + "logps/chosen": -280.026025390625, + "logps/rejected": -250.71551513671875, + "loss": 0.3742, + "rewards/chosen": 0.011942720413208008, + "rewards/margins": 0.32606066862742106, + "rewards/rejected": -0.31411794821421307, + "step": 211 + }, + { + "epoch": 0.35705263157894734, + "grad_norm": 88.38448491891266, + "kl": 0.0, + "learning_rate": 4.076727564805802e-07, + "logits/chosen": -1213323571.2, + "logits/rejected": -650571477.3333334, + "logps/chosen": -225.9, + "logps/rejected": -215.1610107421875, + "loss": 0.3696, + "rewards/chosen": 1.0948437690734862, + "rewards/margins": 1.9371202627817787, + "rewards/rejected": -0.8422764937082926, + "step": 212 + }, + { + "epoch": 0.35873684210526313, + "grad_norm": 65.08222255296863, + "kl": 0.0, + "learning_rate": 4.0652865671498906e-07, + "logits/chosen": -1109334869.3333333, + "logits/rejected": -673303040.0, + "logps/chosen": -209.34431966145834, + "logps/rejected": -264.689892578125, + "loss": 0.4692, + "rewards/chosen": 0.025906383991241455, + "rewards/margins": 0.10383939743041992, + "rewards/rejected": -0.07793301343917847, + "step": 213 + }, + { + "epoch": 0.3604210526315789, + "grad_norm": 95.42507509421407, + "kl": 0.0, + "learning_rate": 4.053791393132149e-07, + "logits/chosen": -2192135936.0, + "logits/rejected": -679301717.3333334, + "logps/chosen": -280.882080078125, + "logps/rejected": -235.8901570638021, + "loss": 0.3862, + "rewards/chosen": 1.1142441034317017, + "rewards/margins": 1.286988377571106, + "rewards/rejected": -0.1727442741394043, + "step": 214 + }, + { + "epoch": 0.36210526315789476, + "grad_norm": 57.980133275698655, + "kl": 0.0, + "learning_rate": 4.0422424406137235e-07, + "logits/chosen": -533541728.0, + "logits/rejected": -664400128.0, + "logps/chosen": -241.3803253173828, + "logps/rejected": -255.393310546875, + "loss": 0.378, + "rewards/chosen": 0.4608551263809204, + "rewards/margins": 2.0191084146499634, + "rewards/rejected": -1.558253288269043, + "step": 215 + }, + { + "epoch": 0.36378947368421055, + "grad_norm": 62.774785038898976, + "kl": 0.0, + "learning_rate": 4.0306401093170956e-07, + "logits/chosen": -863296819.2, + "logits/rejected": -931824469.3333334, + "logps/chosen": -209.0201171875, + "logps/rejected": -258.80812581380206, + "loss": 0.4346, + "rewards/chosen": -0.07239959239959717, + "rewards/margins": -0.04090743859608969, + "rewards/rejected": -0.031492153803507485, + "step": 216 + }, + { + "epoch": 0.36547368421052634, + "grad_norm": 43.09358868475917, + "kl": 0.0, + "learning_rate": 4.0189848008122475e-07, + "logits/chosen": -832388096.0, + "logits/rejected": -767827328.0, + "logps/chosen": -231.43902587890625, + "logps/rejected": -311.52874755859375, + "loss": 0.3487, + "rewards/chosen": 0.1406116485595703, + "rewards/margins": 1.9062778949737549, + "rewards/rejected": -1.7656662464141846, + "step": 217 + }, + { + "epoch": 0.3671578947368421, + "grad_norm": 65.51992296550779, + "kl": 0.0, + "learning_rate": 4.007276918502763e-07, + "logits/chosen": -1086964224.0, + "logits/rejected": -893988352.0, + "logps/chosen": -246.42593383789062, + "logps/rejected": -244.95123291015625, + "loss": 0.3927, + "rewards/chosen": 0.23493842780590057, + "rewards/margins": 1.4225323647260666, + "rewards/rejected": -1.187593936920166, + "step": 218 + }, + { + "epoch": 0.3688421052631579, + "grad_norm": 88.60132651868206, + "kl": 0.0, + "learning_rate": 3.9955168676118645e-07, + "logits/chosen": -698185813.3333334, + "logits/rejected": -1118987673.6, + "logps/chosen": -195.71712239583334, + "logps/rejected": -205.612646484375, + "loss": 0.5847, + "rewards/chosen": 0.2993367513020833, + "rewards/margins": -0.6073127428690592, + "rewards/rejected": 0.9066494941711426, + "step": 219 + }, + { + "epoch": 0.3705263157894737, + "grad_norm": 49.2393342246424, + "kl": 0.0, + "learning_rate": 3.9837050551683904e-07, + "logits/chosen": -615184170.6666666, + "logits/rejected": -701940121.6, + "logps/chosen": -227.86104329427084, + "logps/rejected": -263.94287109375, + "loss": 0.3788, + "rewards/chosen": 0.2989359696706136, + "rewards/margins": 2.076623805363973, + "rewards/rejected": -1.7776878356933594, + "step": 220 + }, + { + "epoch": 0.3722105263157895, + "grad_norm": 186.40613554967234, + "kl": 0.0, + "learning_rate": 3.9718418899927056e-07, + "logits/chosen": -326911872.0, + "logits/rejected": -801610093.7142857, + "logps/chosen": -450.5743408203125, + "logps/rejected": -212.93069893973214, + "loss": 0.485, + "rewards/chosen": -2.387774705886841, + "rewards/margins": -2.170388392039708, + "rewards/rejected": -0.21738631384713308, + "step": 221 + }, + { + "epoch": 0.3738947368421053, + "grad_norm": 64.26673569575549, + "kl": 0.0, + "learning_rate": 3.959927782682551e-07, + "logits/chosen": -726571008.0, + "logits/rejected": -706536896.0, + "logps/chosen": -230.51875813802084, + "logps/rejected": -126.30707550048828, + "loss": 0.4306, + "rewards/chosen": -0.7994547684987386, + "rewards/margins": -1.1165289779504142, + "rewards/rejected": 0.3170742094516754, + "step": 222 + }, + { + "epoch": 0.37557894736842107, + "grad_norm": 73.20938961913613, + "kl": 0.0, + "learning_rate": 3.947963145598833e-07, + "logits/chosen": -1290946048.0, + "logits/rejected": -1101487957.3333333, + "logps/chosen": -192.78887939453125, + "logps/rejected": -246.4966023763021, + "loss": 0.4078, + "rewards/chosen": 1.7199859619140625, + "rewards/margins": 2.088035305341085, + "rewards/rejected": -0.3680493434270223, + "step": 223 + }, + { + "epoch": 0.37726315789473686, + "grad_norm": 47.763563197737376, + "kl": 0.0, + "learning_rate": 3.935948392851353e-07, + "logits/chosen": -2707193088.0, + "logits/rejected": -1093253997.7142856, + "logps/chosen": -202.72845458984375, + "logps/rejected": -189.14756556919642, + "loss": 0.3882, + "rewards/chosen": -2.7178618907928467, + "rewards/margins": -2.8144114358084544, + "rewards/rejected": 0.09654954501560756, + "step": 224 + }, + { + "epoch": 0.37894736842105264, + "grad_norm": 67.91419872933243, + "kl": 0.0, + "learning_rate": 3.923883940284472e-07, + "logits/chosen": -493479210.6666667, + "logits/rejected": -663955968.0, + "logps/chosen": -208.96756998697916, + "logps/rejected": -219.612744140625, + "loss": 0.4068, + "rewards/chosen": 0.5567906697591146, + "rewards/margins": 2.475810750325521, + "rewards/rejected": -1.9190200805664062, + "step": 225 + }, + { + "epoch": 0.38063157894736843, + "grad_norm": 45.896671613894135, + "kl": 0.0, + "learning_rate": 3.9117702054627164e-07, + "logits/chosen": -843394901.3333334, + "logits/rejected": -696314688.0, + "logps/chosen": -184.59212239583334, + "logps/rejected": -289.22589111328125, + "loss": 0.3531, + "rewards/chosen": 1.452480634053548, + "rewards/margins": 4.889726003011067, + "rewards/rejected": -3.4372453689575195, + "step": 226 + }, + { + "epoch": 0.3823157894736842, + "grad_norm": 117.9741866381254, + "kl": 0.0, + "learning_rate": 3.8996076076563333e-07, + "logits/chosen": -633168768.0, + "logits/rejected": -1228658005.3333333, + "logps/chosen": -255.6255645751953, + "logps/rejected": -198.0909423828125, + "loss": 0.5188, + "rewards/chosen": -0.49002379179000854, + "rewards/margins": -1.5519655744234722, + "rewards/rejected": 1.0619417826334636, + "step": 227 + }, + { + "epoch": 0.384, + "grad_norm": 59.87930468202007, + "kl": 0.0, + "learning_rate": 3.8873965678267686e-07, + "logits/chosen": -748600405.3333334, + "logits/rejected": -564317888.0, + "logps/chosen": -205.5136515299479, + "logps/rejected": -190.98033142089844, + "loss": 0.4398, + "rewards/chosen": 0.013506333033243815, + "rewards/margins": 0.8160205880800883, + "rewards/rejected": -0.8025142550468445, + "step": 228 + }, + { + "epoch": 0.3856842105263158, + "grad_norm": 55.75368595740401, + "kl": 0.0, + "learning_rate": 3.8751375086121027e-07, + "logits/chosen": -689110741.3333334, + "logits/rejected": -534848768.0, + "logps/chosen": -284.23940022786456, + "logps/rejected": -163.17276000976562, + "loss": 0.4007, + "rewards/chosen": 0.30055542786916095, + "rewards/margins": 0.026084144910176577, + "rewards/rejected": 0.2744712829589844, + "step": 229 + }, + { + "epoch": 0.3873684210526316, + "grad_norm": 85.00209812207287, + "kl": 0.0, + "learning_rate": 3.8628308543124264e-07, + "logits/chosen": -733420748.8, + "logits/rejected": -613972394.6666666, + "logps/chosen": -240.112109375, + "logps/rejected": -277.9342854817708, + "loss": 0.4103, + "rewards/chosen": 0.4192187309265137, + "rewards/margins": 0.19571206172307334, + "rewards/rejected": 0.22350666920344034, + "step": 230 + }, + { + "epoch": 0.38905263157894737, + "grad_norm": 67.82002108621575, + "kl": 0.0, + "learning_rate": 3.8504770308751465e-07, + "logits/chosen": -815699712.0, + "logits/rejected": -538265600.0, + "logps/chosen": -195.72711181640625, + "logps/rejected": -168.96177673339844, + "loss": 0.3659, + "rewards/chosen": 0.9801532030105591, + "rewards/margins": 0.6151527762413025, + "rewards/rejected": 0.3650004267692566, + "step": 231 + }, + { + "epoch": 0.39073684210526316, + "grad_norm": 77.76306759423939, + "kl": 0.0, + "learning_rate": 3.8380764658802476e-07, + "logits/chosen": -863703961.6, + "logits/rejected": -500137898.6666667, + "logps/chosen": -190.1537353515625, + "logps/rejected": -248.5384318033854, + "loss": 0.3315, + "rewards/chosen": 0.9144930839538574, + "rewards/margins": 1.550446311632792, + "rewards/rejected": -0.6359532276789347, + "step": 232 + }, + { + "epoch": 0.39242105263157895, + "grad_norm": 66.37493353593294, + "kl": 0.0, + "learning_rate": 3.8256295885254977e-07, + "logits/chosen": -929981747.2, + "logits/rejected": -953465258.6666666, + "logps/chosen": -216.4085205078125, + "logps/rejected": -182.04376220703125, + "loss": 0.3813, + "rewards/chosen": 1.277543067932129, + "rewards/margins": 1.0003930727640789, + "rewards/rejected": 0.2771499951680501, + "step": 233 + }, + { + "epoch": 0.39410526315789474, + "grad_norm": 75.1964810655982, + "kl": 0.0, + "learning_rate": 3.8131368296115823e-07, + "logits/chosen": -896362752.0, + "logits/rejected": -561620096.0, + "logps/chosen": -249.6129353841146, + "logps/rejected": -264.5303955078125, + "loss": 0.2827, + "rewards/chosen": 1.139868974685669, + "rewards/margins": 2.838852047920227, + "rewards/rejected": -1.698983073234558, + "step": 234 + }, + { + "epoch": 0.3957894736842105, + "grad_norm": 122.44554306763347, + "kl": 0.0, + "learning_rate": 3.800598621527205e-07, + "logits/chosen": -969984146.2857143, + "logits/rejected": -854562432.0, + "logps/chosen": -182.46547154017858, + "logps/rejected": -194.51100158691406, + "loss": 0.3864, + "rewards/chosen": 1.404153551374163, + "rewards/margins": 1.3077897163374084, + "rewards/rejected": 0.09636383503675461, + "step": 235 + }, + { + "epoch": 0.3974736842105263, + "grad_norm": 60.34196841703653, + "kl": 0.0, + "learning_rate": 3.7880153982341163e-07, + "logits/chosen": -823134464.0, + "logits/rejected": -873622101.3333334, + "logps/chosen": -235.61932373046875, + "logps/rejected": -271.49196370442706, + "loss": 0.3463, + "rewards/chosen": 2.0248725414276123, + "rewards/margins": 3.3654048442840576, + "rewards/rejected": -1.3405323028564453, + "step": 236 + }, + { + "epoch": 0.3991578947368421, + "grad_norm": 73.61126053660914, + "kl": 0.0, + "learning_rate": 3.7753875952520937e-07, + "logits/chosen": -791607424.0, + "logits/rejected": -736724992.0, + "logps/chosen": -260.60198974609375, + "logps/rejected": -270.4560852050781, + "loss": 0.3967, + "rewards/chosen": 0.33294641971588135, + "rewards/margins": 2.238220691680908, + "rewards/rejected": -1.9052742719650269, + "step": 237 + }, + { + "epoch": 0.4008421052631579, + "grad_norm": 44.26653587553288, + "kl": 0.0, + "learning_rate": 3.7627156496438685e-07, + "logits/chosen": -824610218.6666666, + "logits/rejected": -856471756.8, + "logps/chosen": -174.15934244791666, + "logps/rejected": -221.6721435546875, + "loss": 0.4355, + "rewards/chosen": 1.1449518998463948, + "rewards/margins": 0.593308385213216, + "rewards/rejected": 0.5516435146331787, + "step": 238 + }, + { + "epoch": 0.4025263157894737, + "grad_norm": 47.69085670063618, + "kl": 0.0, + "learning_rate": 3.75e-07, + "logits/chosen": -820081216.0, + "logits/rejected": -955681408.0, + "logps/chosen": -224.08152770996094, + "logps/rejected": -240.1038818359375, + "loss": 0.3694, + "rewards/chosen": 1.8589047193527222, + "rewards/margins": 3.0458600521087646, + "rewards/rejected": -1.1869553327560425, + "step": 239 + }, + { + "epoch": 0.40421052631578946, + "grad_norm": 48.281804889717954, + "kl": 0.0, + "learning_rate": 3.7372410864236947e-07, + "logits/chosen": -717564245.3333334, + "logits/rejected": -679164364.8, + "logps/chosen": -152.29527791341147, + "logps/rejected": -278.3427490234375, + "loss": 0.3254, + "rewards/chosen": 1.1597716808319092, + "rewards/margins": 3.844050931930542, + "rewards/rejected": -2.6842792510986326, + "step": 240 + }, + { + "epoch": 0.40589473684210525, + "grad_norm": 75.76751371273427, + "kl": 0.0, + "learning_rate": 3.724439350515571e-07, + "logits/chosen": -629566144.0, + "logits/rejected": -622408640.0, + "logps/chosen": -201.63623046875, + "logps/rejected": -248.00782775878906, + "loss": 0.3847, + "rewards/chosen": 0.5420559048652649, + "rewards/margins": 1.8232646584510803, + "rewards/rejected": -1.2812087535858154, + "step": 241 + }, + { + "epoch": 0.40757894736842104, + "grad_norm": 95.46822610468628, + "kl": 0.0, + "learning_rate": 3.7115952353583803e-07, + "logits/chosen": -702194048.0, + "logps/chosen": -212.0802459716797, + "loss": 0.3904, + "rewards/chosen": 1.2338132858276367, + "step": 242 + }, + { + "epoch": 0.40926315789473683, + "grad_norm": 86.1205385319129, + "kl": 0.0, + "learning_rate": 3.698709185501666e-07, + "logits/chosen": -567989094.4, + "logits/rejected": -984176554.6666666, + "logps/chosen": -259.487060546875, + "logps/rejected": -214.1506144205729, + "loss": 0.2951, + "rewards/chosen": 0.584656047821045, + "rewards/margins": 1.1261462450027466, + "rewards/rejected": -0.5414901971817017, + "step": 243 + }, + { + "epoch": 0.4109473684210526, + "grad_norm": 94.11033116313197, + "kl": 0.0, + "learning_rate": 3.6857816469463805e-07, + "logits/chosen": -414785728.0, + "logits/rejected": -834902308.5714285, + "logps/chosen": -117.85265350341797, + "logps/rejected": -237.00685337611608, + "loss": 0.4324, + "rewards/chosen": 1.1912513971328735, + "rewards/margins": 1.5301621811730521, + "rewards/rejected": -0.33891078404017855, + "step": 244 + }, + { + "epoch": 0.4126315789473684, + "grad_norm": 73.01801229445103, + "kl": 0.0, + "learning_rate": 3.6728130671294484e-07, + "logits/chosen": -480209600.0, + "logits/rejected": -1261644629.3333333, + "logps/chosen": -242.870361328125, + "logps/rejected": -297.1340738932292, + "loss": 0.3948, + "rewards/chosen": 1.906396508216858, + "rewards/margins": 3.281412720680237, + "rewards/rejected": -1.375016212463379, + "step": 245 + }, + { + "epoch": 0.4143157894736842, + "grad_norm": 50.21406501502345, + "kl": 0.0, + "learning_rate": 3.6598038949082773e-07, + "logits/chosen": -1082578688.0, + "logits/rejected": -1054340505.6, + "logps/chosen": -202.77239990234375, + "logps/rejected": -241.3439697265625, + "loss": 0.4015, + "rewards/chosen": 0.08610687653223674, + "rewards/margins": -0.06312040885289509, + "rewards/rejected": 0.14922728538513183, + "step": 246 + }, + { + "epoch": 0.416, + "grad_norm": 69.90136809562398, + "kl": 0.0, + "learning_rate": 3.646754580545226e-07, + "logits/chosen": -551080704.0, + "logits/rejected": -721525162.6666666, + "logps/chosen": -253.119091796875, + "logps/rejected": -209.84647623697916, + "loss": 0.4882, + "rewards/chosen": -0.7751388549804688, + "rewards/margins": -1.7328248023986816, + "rewards/rejected": 0.9576859474182129, + "step": 247 + }, + { + "epoch": 0.41768421052631577, + "grad_norm": 66.55136067755517, + "kl": 0.0, + "learning_rate": 3.633665575692019e-07, + "logits/chosen": -469412010.6666667, + "logits/rejected": -688726988.8, + "logps/chosen": -186.66743977864584, + "logps/rejected": -300.4759765625, + "loss": 0.3747, + "rewards/chosen": 0.8189966678619385, + "rewards/margins": 2.057415819168091, + "rewards/rejected": -1.2384191513061524, + "step": 248 + }, + { + "epoch": 0.41936842105263156, + "grad_norm": 91.34875948495586, + "kl": 0.0, + "learning_rate": 3.6205373333741137e-07, + "logits/chosen": -695206528.0, + "logits/rejected": -909713856.0, + "logps/chosen": -220.77886962890625, + "logps/rejected": -275.035400390625, + "loss": 0.4002, + "rewards/chosen": 0.9514415860176086, + "rewards/margins": 2.3362786173820496, + "rewards/rejected": -1.384837031364441, + "step": 249 + }, + { + "epoch": 0.42105263157894735, + "grad_norm": 41.18603961196617, + "kl": 0.0, + "learning_rate": 3.60737030797502e-07, + "logits/chosen": -578035840.0, + "logits/rejected": -939070378.6666666, + "logps/chosen": -261.39300537109375, + "logps/rejected": -181.3475341796875, + "loss": 0.4095, + "rewards/chosen": -1.6891616582870483, + "rewards/margins": -1.0875247716903687, + "rewards/rejected": -0.6016368865966797, + "step": 250 + }, + { + "epoch": 0.42273684210526313, + "grad_norm": 116.0424680075244, + "kl": 0.0, + "learning_rate": 3.5941649552205766e-07, + "logits/chosen": -1035228672.0, + "logits/rejected": -634597068.8, + "logps/chosen": -246.8405558268229, + "logps/rejected": -214.5775390625, + "loss": 0.4228, + "rewards/chosen": -0.7042236328125, + "rewards/margins": -1.805875873565674, + "rewards/rejected": 1.101652240753174, + "step": 251 + }, + { + "epoch": 0.4244210526315789, + "grad_norm": 57.738431491671584, + "kl": 0.0, + "learning_rate": 3.580921732163174e-07, + "logits/chosen": -844649472.0, + "logits/rejected": -548579840.0, + "logps/chosen": -214.6180419921875, + "logps/rejected": -231.5613250732422, + "loss": 0.3853, + "rewards/chosen": 0.8030085563659668, + "rewards/margins": 1.5388163328170776, + "rewards/rejected": -0.7358077764511108, + "step": 252 + }, + { + "epoch": 0.4261052631578947, + "grad_norm": 346.5318458002952, + "kl": 0.0, + "learning_rate": 3.56764109716594e-07, + "logits/chosen": -551624384.0, + "logits/rejected": -1136507611.4285715, + "logps/chosen": -178.53115844726562, + "logps/rejected": -271.80191476004467, + "loss": 0.3623, + "rewards/chosen": 3.3293793201446533, + "rewards/margins": 8.204052346093313, + "rewards/rejected": -4.874673025948661, + "step": 253 + }, + { + "epoch": 0.42778947368421055, + "grad_norm": 115.23971460689606, + "kl": 0.0, + "learning_rate": 3.55432350988687e-07, + "logits/chosen": -830976576.0, + "logits/rejected": -1067142144.0, + "logps/chosen": -153.7244415283203, + "logps/rejected": -251.48733520507812, + "loss": 0.4097, + "rewards/chosen": 0.4606727957725525, + "rewards/margins": 1.2488128542900085, + "rewards/rejected": -0.788140058517456, + "step": 254 + }, + { + "epoch": 0.42947368421052634, + "grad_norm": 54.92704059463097, + "kl": 0.0, + "learning_rate": 3.5409694312629193e-07, + "logits/chosen": -630669141.3333334, + "logits/rejected": -987313868.8, + "logps/chosen": -209.82574462890625, + "logps/rejected": -287.5651611328125, + "loss": 0.3034, + "rewards/chosen": 1.022467056910197, + "rewards/margins": 2.030047623316447, + "rewards/rejected": -1.00758056640625, + "step": 255 + }, + { + "epoch": 0.43115789473684213, + "grad_norm": 310.7398715383573, + "kl": 0.0, + "learning_rate": 3.5275793234940544e-07, + "logits/chosen": -917445760.0, + "logits/rejected": -839287808.0, + "logps/chosen": -376.3251953125, + "logps/rejected": -219.46243722098214, + "loss": 0.4546, + "rewards/chosen": -1.2787322998046875, + "rewards/margins": -1.270533732005528, + "rewards/rejected": -0.00819856779915946, + "step": 256 + }, + { + "epoch": 0.4328421052631579, + "grad_norm": 92.05663310439607, + "kl": 1.0852890014648438, + "learning_rate": 3.514153650027249e-07, + "logits/chosen": -917584298.6666666, + "logits/rejected": -720754227.2, + "logps/chosen": -277.08099365234375, + "logps/rejected": -215.7774169921875, + "loss": 0.4514, + "rewards/chosen": -1.6589380900065105, + "rewards/margins": -1.4071196715037029, + "rewards/rejected": -0.2518184185028076, + "step": 257 + }, + { + "epoch": 0.4345263157894737, + "grad_norm": 46.29681896211451, + "kl": 0.0, + "learning_rate": 3.5006928755404464e-07, + "logits/chosen": -601805260.8, + "logits/rejected": -705630848.0, + "logps/chosen": -302.690771484375, + "logps/rejected": -220.0159912109375, + "loss": 0.3499, + "rewards/chosen": 0.9885660171508789, + "rewards/margins": -0.45607048670450856, + "rewards/rejected": 1.4446365038553874, + "step": 258 + }, + { + "epoch": 0.4362105263157895, + "grad_norm": 60.3839696290393, + "kl": 0.0, + "learning_rate": 3.487197465926478e-07, + "logits/chosen": -877942016.0, + "logits/rejected": -762154956.8, + "logps/chosen": -241.57857259114584, + "logps/rejected": -244.444482421875, + "loss": 0.4307, + "rewards/chosen": 1.5284678141276042, + "rewards/margins": 1.983873716990153, + "rewards/rejected": -0.45540590286254884, + "step": 259 + }, + { + "epoch": 0.4378947368421053, + "grad_norm": 47.33258383634144, + "kl": 0.0, + "learning_rate": 3.473667888276935e-07, + "logits/chosen": -555102617.6, + "logits/rejected": -590850773.3333334, + "logps/chosen": -220.5026123046875, + "logps/rejected": -312.41845703125, + "loss": 0.3409, + "rewards/chosen": 1.8922588348388671, + "rewards/margins": 2.5513052860895793, + "rewards/rejected": -0.659046451250712, + "step": 260 + }, + { + "epoch": 0.43957894736842107, + "grad_norm": 108.21153319687332, + "kl": 0.0, + "learning_rate": 3.460104610866003e-07, + "logits/chosen": -1036087466.6666666, + "logits/rejected": -1177354444.8, + "logps/chosen": -258.5961100260417, + "logps/rejected": -187.52177734375, + "loss": 0.4231, + "rewards/chosen": 1.5323643684387207, + "rewards/margins": 2.1087326049804687, + "rewards/rejected": -0.5763682365417481, + "step": 261 + }, + { + "epoch": 0.44126315789473686, + "grad_norm": 119.88936087714019, + "kl": 0.0, + "learning_rate": 3.4465081031342583e-07, + "logits/chosen": -810871347.2, + "logits/rejected": -1137264128.0, + "logps/chosen": -274.263916015625, + "logps/rejected": -234.40445963541666, + "loss": 0.4701, + "rewards/chosen": 0.387792706489563, + "rewards/margins": -0.040941635767618834, + "rewards/rejected": 0.4287343422571818, + "step": 262 + }, + { + "epoch": 0.44294736842105265, + "grad_norm": 61.18523248696411, + "kl": 0.0, + "learning_rate": 3.432878835672413e-07, + "logits/chosen": -875720789.3333334, + "logits/rejected": -787683264.0, + "logps/chosen": -223.71048990885416, + "logps/rejected": -237.80686950683594, + "loss": 0.4187, + "rewards/chosen": 1.0855443477630615, + "rewards/margins": 0.4147740602493286, + "rewards/rejected": 0.6707702875137329, + "step": 263 + }, + { + "epoch": 0.44463157894736843, + "grad_norm": 85.74861272190243, + "kl": 0.0, + "learning_rate": 3.419217280205032e-07, + "logits/chosen": -493305770.6666667, + "logits/rejected": -738251980.8, + "logps/chosen": -250.9864501953125, + "logps/rejected": -200.50716552734374, + "loss": 0.4608, + "rewards/chosen": 0.6460779507954916, + "rewards/margins": 0.17245400746663414, + "rewards/rejected": 0.47362394332885743, + "step": 264 + }, + { + "epoch": 0.4463157894736842, + "grad_norm": 54.77540128655694, + "kl": 0.0, + "learning_rate": 3.405523909574206e-07, + "logits/chosen": -651585450.6666666, + "logits/rejected": -683791001.6, + "logps/chosen": -242.39778645833334, + "logps/rejected": -298.1034912109375, + "loss": 0.3501, + "rewards/chosen": 0.5337386926015218, + "rewards/margins": 0.8820349057515462, + "rewards/rejected": -0.3482962131500244, + "step": 265 + }, + { + "epoch": 0.448, + "grad_norm": 89.19503556391335, + "kl": 0.0, + "learning_rate": 3.391799197723185e-07, + "logits/chosen": -1162056499.2, + "logits/rejected": -1215143509.3333333, + "logps/chosen": -259.603076171875, + "logps/rejected": -309.9653727213542, + "loss": 0.4462, + "rewards/chosen": 0.42551660537719727, + "rewards/margins": 2.0183765093485517, + "rewards/rejected": -1.5928599039713542, + "step": 266 + }, + { + "epoch": 0.4496842105263158, + "grad_norm": 50.9461938511508, + "kl": 0.0, + "learning_rate": 3.378043619679973e-07, + "logits/chosen": -1112641536.0, + "logits/rejected": -554899541.3333334, + "logps/chosen": -208.4407470703125, + "logps/rejected": -223.9110107421875, + "loss": 0.37, + "rewards/chosen": 1.009368324279785, + "rewards/margins": 1.0311517755190531, + "rewards/rejected": -0.021783451239267986, + "step": 267 + }, + { + "epoch": 0.4513684210526316, + "grad_norm": 127.83348671346022, + "kl": 0.0, + "learning_rate": 3.3642576515408905e-07, + "logits/chosen": -642838357.3333334, + "logits/rejected": -1174076211.2, + "logps/chosen": -164.12750244140625, + "logps/rejected": -177.136376953125, + "loss": 0.4873, + "rewards/chosen": 0.642901619275411, + "rewards/margins": 1.1527833541234336, + "rewards/rejected": -0.5098817348480225, + "step": 268 + }, + { + "epoch": 0.4530526315789474, + "grad_norm": 106.6924383104548, + "kl": 0.0, + "learning_rate": 3.350441770454092e-07, + "logits/chosen": -881408614.4, + "logits/rejected": -748522496.0, + "logps/chosen": -271.259326171875, + "logps/rejected": -234.607421875, + "loss": 0.3324, + "rewards/chosen": 0.8681741714477539, + "rewards/margins": 1.9296697139739991, + "rewards/rejected": -1.0614955425262451, + "step": 269 + }, + { + "epoch": 0.45473684210526316, + "grad_norm": 51.51923388471137, + "kl": 0.0, + "learning_rate": 3.336596454603054e-07, + "logits/chosen": -1130812074.6666667, + "logits/rejected": -598989926.4, + "logps/chosen": -204.42683919270834, + "logps/rejected": -234.242333984375, + "loss": 0.3524, + "rewards/chosen": 0.7063151200612386, + "rewards/margins": 1.668326743443807, + "rewards/rejected": -0.9620116233825684, + "step": 270 + }, + { + "epoch": 0.45642105263157895, + "grad_norm": 56.40900184314831, + "kl": 0.0, + "learning_rate": 3.322722183190025e-07, + "logits/chosen": -734314154.6666666, + "logits/rejected": -522018406.4, + "logps/chosen": -165.00169881184897, + "logps/rejected": -248.694873046875, + "loss": 0.445, + "rewards/chosen": -1.2975707054138184, + "rewards/margins": -1.0448772430419921, + "rewards/rejected": -0.25269346237182616, + "step": 271 + }, + { + "epoch": 0.45810526315789474, + "grad_norm": 68.53879212164166, + "kl": 0.0, + "learning_rate": 3.308819436419437e-07, + "logits/chosen": -1095972181.3333333, + "logits/rejected": -1827424384.0, + "logps/chosen": -254.98893229166666, + "logps/rejected": -206.98135375976562, + "loss": 0.3692, + "rewards/chosen": 0.8086059888203939, + "rewards/margins": 4.705807526906331, + "rewards/rejected": -3.8972015380859375, + "step": 272 + }, + { + "epoch": 0.4597894736842105, + "grad_norm": 62.413627457878825, + "kl": 0.0, + "learning_rate": 3.294888695481287e-07, + "logits/chosen": -1661173418.6666667, + "logits/rejected": -1076040294.4, + "logps/chosen": -195.62664794921875, + "logps/rejected": -220.1724609375, + "loss": 0.4067, + "rewards/chosen": 1.7726329167683919, + "rewards/margins": 2.3173933347066242, + "rewards/rejected": -0.5447604179382324, + "step": 273 + }, + { + "epoch": 0.4614736842105263, + "grad_norm": 41.769964940057946, + "kl": 0.0, + "learning_rate": 3.2809304425344856e-07, + "logits/chosen": -1145147264.0, + "logits/rejected": -744639780.5714285, + "logps/chosen": -178.32012939453125, + "logps/rejected": -222.49560546875, + "loss": 0.4646, + "rewards/chosen": 1.3849213123321533, + "rewards/margins": 1.6690550361360823, + "rewards/rejected": -0.28413372380392893, + "step": 274 + }, + { + "epoch": 0.4631578947368421, + "grad_norm": 51.7666775396847, + "kl": 0.0, + "learning_rate": 3.266945160690159e-07, + "logits/chosen": -718938112.0, + "logits/rejected": -701548672.0, + "logps/chosen": -168.946533203125, + "logps/rejected": -342.5791015625, + "loss": 0.3497, + "rewards/chosen": 1.088832139968872, + "rewards/margins": 3.3126580715179443, + "rewards/rejected": -2.2238259315490723, + "step": 275 + }, + { + "epoch": 0.4648421052631579, + "grad_norm": 40.27881909450745, + "kl": 0.0, + "learning_rate": 3.252933333994942e-07, + "logits/chosen": -618843904.0, + "logits/rejected": -992992768.0, + "logps/chosen": -176.893798828125, + "logps/rejected": -240.34130859375, + "loss": 0.3206, + "rewards/chosen": 1.2537689208984375, + "rewards/margins": 2.309921582539876, + "rewards/rejected": -1.0561526616414387, + "step": 276 + }, + { + "epoch": 0.4665263157894737, + "grad_norm": 37.41871849188705, + "kl": 0.0, + "learning_rate": 3.23889544741421e-07, + "logits/chosen": -787857536.0, + "logits/rejected": -838022016.0, + "logps/chosen": -214.87179565429688, + "logps/rejected": -229.31048583984375, + "loss": 0.2983, + "rewards/chosen": 2.574376106262207, + "rewards/margins": 4.175034284591675, + "rewards/rejected": -1.6006581783294678, + "step": 277 + }, + { + "epoch": 0.46821052631578947, + "grad_norm": 46.77822204181921, + "kl": 0.0, + "learning_rate": 3.224831986815306e-07, + "logits/chosen": -1206424576.0, + "logits/rejected": -531875840.0, + "logps/chosen": -218.09373474121094, + "logps/rejected": -287.2392578125, + "loss": 0.3122, + "rewards/chosen": 1.3858611583709717, + "rewards/margins": 7.854579210281372, + "rewards/rejected": -6.4687180519104, + "step": 278 + }, + { + "epoch": 0.46989473684210525, + "grad_norm": 48.645481075648526, + "kl": 0.0, + "learning_rate": 3.2107434389507177e-07, + "logits/chosen": -829400217.6, + "logits/rejected": -402655232.0, + "logps/chosen": -180.144287109375, + "logps/rejected": -130.1499226888021, + "loss": 0.329, + "rewards/chosen": 1.9622688293457031, + "rewards/margins": 1.4281412760416665, + "rewards/rejected": 0.5341275533040365, + "step": 279 + }, + { + "epoch": 0.47157894736842104, + "grad_norm": 41.28195566807422, + "kl": 0.0, + "learning_rate": 3.19663029144123e-07, + "logits/chosen": -841884723.2, + "logits/rejected": -862919680.0, + "logps/chosen": -200.8014892578125, + "logps/rejected": -234.42822265625, + "loss": 0.3977, + "rewards/chosen": 0.43205790519714354, + "rewards/margins": 0.7687125682830811, + "rewards/rejected": -0.3366546630859375, + "step": 280 + }, + { + "epoch": 0.47326315789473683, + "grad_norm": 84.07688846755318, + "kl": 0.0, + "learning_rate": 3.1824930327590525e-07, + "logits/chosen": -710278314.6666666, + "logits/rejected": -995849472.0, + "logps/chosen": -224.30419921875, + "logps/rejected": -217.58340454101562, + "loss": 0.3496, + "rewards/chosen": 0.7394833564758301, + "rewards/margins": 0.8354802131652832, + "rewards/rejected": -0.09599685668945312, + "step": 281 + }, + { + "epoch": 0.4749473684210526, + "grad_norm": 112.7314053842234, + "kl": 0.0, + "learning_rate": 3.168332152210909e-07, + "logits/chosen": -497686954.6666667, + "logits/rejected": -851313049.6, + "logps/chosen": -303.6035970052083, + "logps/rejected": -268.5775390625, + "loss": 0.4291, + "rewards/chosen": -0.6700571378072103, + "rewards/margins": -1.136230500539144, + "rewards/rejected": 0.4661733627319336, + "step": 282 + }, + { + "epoch": 0.4766315789473684, + "grad_norm": 47.718340024126896, + "kl": 0.0, + "learning_rate": 3.154148139921102e-07, + "logits/chosen": -1520765056.0, + "logits/rejected": -917776018.2857143, + "logps/chosen": -182.46583557128906, + "logps/rejected": -224.23322405133928, + "loss": 0.3508, + "rewards/chosen": -1.4963241815567017, + "rewards/margins": -0.8272254637309483, + "rewards/rejected": -0.6690987178257534, + "step": 283 + }, + { + "epoch": 0.4783157894736842, + "grad_norm": 51.50174863063673, + "kl": 0.0, + "learning_rate": 3.1399414868145503e-07, + "logits/chosen": -879817557.3333334, + "logits/rejected": -509886771.2, + "logps/chosen": -196.01708984375, + "logps/rejected": -247.4649169921875, + "loss": 0.4224, + "rewards/chosen": -1.2518666585286458, + "rewards/margins": 2.6389341990152992, + "rewards/rejected": -3.8908008575439452, + "step": 284 + }, + { + "epoch": 0.48, + "grad_norm": 46.979767312290676, + "kl": 0.0, + "learning_rate": 3.1257126845997996e-07, + "logits/chosen": -605518720.0, + "logits/rejected": -689517056.0, + "logps/chosen": -217.33444213867188, + "logps/rejected": -329.7290954589844, + "loss": 0.2682, + "rewards/chosen": 0.8716354370117188, + "rewards/margins": 11.700860023498535, + "rewards/rejected": -10.829224586486816, + "step": 285 + }, + { + "epoch": 0.48168421052631577, + "grad_norm": 53.078466818339, + "kl": 0.0, + "learning_rate": 3.111462225752e-07, + "logits/chosen": -1177064704.0, + "logits/rejected": -1062161664.0, + "logps/chosen": -172.35658264160156, + "logps/rejected": -229.10531616210938, + "loss": 0.2759, + "rewards/chosen": 1.4827051162719727, + "rewards/margins": 3.3628339767456055, + "rewards/rejected": -1.8801288604736328, + "step": 286 + }, + { + "epoch": 0.48336842105263156, + "grad_norm": 73.05034651400905, + "kl": 0.0, + "learning_rate": 3.097190603495861e-07, + "logits/chosen": -1032015040.0, + "logits/rejected": -431953184.0, + "logps/chosen": -204.94126892089844, + "logps/rejected": -273.42724609375, + "loss": 0.4725, + "rewards/chosen": 0.42430874705314636, + "rewards/margins": 0.12238076329231262, + "rewards/rejected": 0.30192798376083374, + "step": 287 + }, + { + "epoch": 0.48505263157894735, + "grad_norm": 58.199485015649564, + "kl": 0.0, + "learning_rate": 3.0828983117885856e-07, + "logits/chosen": -947543360.0, + "logits/rejected": -1011533738.6666666, + "logps/chosen": -225.79100036621094, + "logps/rejected": -238.0352579752604, + "loss": 0.3671, + "rewards/chosen": 0.18504944443702698, + "rewards/margins": 2.1747958560784655, + "rewards/rejected": -1.9897464116414387, + "step": 288 + }, + { + "epoch": 0.48673684210526313, + "grad_norm": 38.613615018070185, + "kl": 0.0, + "learning_rate": 3.0685858453027665e-07, + "logits/chosen": -1476833689.6, + "logits/rejected": -866345984.0, + "logps/chosen": -210.87587890625, + "logps/rejected": -175.0899861653646, + "loss": 0.4495, + "rewards/chosen": 1.3699981689453125, + "rewards/margins": 0.4098469734191894, + "rewards/rejected": 0.960151195526123, + "step": 289 + }, + { + "epoch": 0.4884210526315789, + "grad_norm": 62.47965185776049, + "kl": 0.0, + "learning_rate": 3.0542536994092696e-07, + "logits/chosen": -1110704128.0, + "logits/rejected": -717989222.4, + "logps/chosen": -271.7138671875, + "logps/rejected": -226.588330078125, + "loss": 0.3672, + "rewards/chosen": 0.2757614056269328, + "rewards/margins": 2.1927677075068157, + "rewards/rejected": -1.9170063018798829, + "step": 290 + }, + { + "epoch": 0.4901052631578947, + "grad_norm": 44.32623223762354, + "kl": 0.0, + "learning_rate": 3.03990237016009e-07, + "logits/chosen": -798940876.8, + "logits/rejected": -814150997.3333334, + "logps/chosen": -253.6781005859375, + "logps/rejected": -307.87721761067706, + "loss": 0.405, + "rewards/chosen": 0.5707196235656739, + "rewards/margins": 3.1684674263000487, + "rewards/rejected": -2.597747802734375, + "step": 291 + }, + { + "epoch": 0.4917894736842105, + "grad_norm": 55.91294023967487, + "kl": 0.0, + "learning_rate": 3.025532354271178e-07, + "logits/chosen": -961314816.0, + "logits/rejected": -835593676.8, + "logps/chosen": -208.382568359375, + "logps/rejected": -190.52183837890624, + "loss": 0.4201, + "rewards/chosen": 0.48334558804829914, + "rewards/margins": 0.11008816560109452, + "rewards/rejected": 0.3732574224472046, + "step": 292 + }, + { + "epoch": 0.49347368421052634, + "grad_norm": 39.083753233538296, + "kl": 0.0, + "learning_rate": 3.0111441491052505e-07, + "logits/chosen": -898919116.8, + "logits/rejected": -784928000.0, + "logps/chosen": -233.350927734375, + "logps/rejected": -234.80265299479166, + "loss": 0.4615, + "rewards/chosen": 0.10661284923553467, + "rewards/margins": 1.890099851290385, + "rewards/rejected": -1.7834870020548503, + "step": 293 + }, + { + "epoch": 0.49515789473684213, + "grad_norm": 51.14880984410606, + "kl": 0.0, + "learning_rate": 2.996738252654577e-07, + "logits/chosen": -821468160.0, + "logits/rejected": -1598704128.0, + "logps/chosen": -196.221533203125, + "logps/rejected": -359.334228515625, + "loss": 0.4244, + "rewards/chosen": -0.5685586929321289, + "rewards/margins": 3.059499422709147, + "rewards/rejected": -3.628058115641276, + "step": 294 + }, + { + "epoch": 0.4968421052631579, + "grad_norm": 106.86865404878358, + "kl": 0.0, + "learning_rate": 2.982315163523742e-07, + "logits/chosen": -661637952.0, + "logits/rejected": -624379264.0, + "logps/chosen": -208.84210205078125, + "logps/rejected": -236.47945149739584, + "loss": 0.4826, + "rewards/chosen": 0.8578643798828125, + "rewards/margins": 1.4199414253234863, + "rewards/rejected": -0.5620770454406738, + "step": 295 + }, + { + "epoch": 0.4985263157894737, + "grad_norm": 60.74058573931991, + "kl": 0.0, + "learning_rate": 2.967875380912388e-07, + "logits/chosen": -1204205824.0, + "logits/rejected": -1308154026.6666667, + "logps/chosen": -226.80226135253906, + "logps/rejected": -224.9236857096354, + "loss": 0.506, + "rewards/chosen": 0.23896867036819458, + "rewards/margins": 0.5748037298520405, + "rewards/rejected": -0.335835059483846, + "step": 296 + }, + { + "epoch": 0.5002105263157894, + "grad_norm": 42.435672467475534, + "kl": 0.0, + "learning_rate": 2.9534194045979393e-07, + "logits/chosen": -706206506.6666666, + "logits/rejected": -569180224.0, + "logps/chosen": -252.4887898763021, + "logps/rejected": -193.00033569335938, + "loss": 0.372, + "rewards/chosen": 0.9058823585510254, + "rewards/margins": 3.202226400375366, + "rewards/rejected": -2.296344041824341, + "step": 297 + }, + { + "epoch": 0.5018947368421053, + "grad_norm": 56.50443377225829, + "kl": 0.0, + "learning_rate": 2.9389477349183015e-07, + "logits/chosen": -490414496.0, + "logits/rejected": -853274048.0, + "logps/chosen": -217.90597534179688, + "logps/rejected": -235.65103149414062, + "loss": 0.3792, + "rewards/chosen": 1.547549843788147, + "rewards/margins": 3.4790706634521484, + "rewards/rejected": -1.9315208196640015, + "step": 298 + }, + { + "epoch": 0.503578947368421, + "grad_norm": 61.19734777336964, + "kl": 0.0, + "learning_rate": 2.924460872754547e-07, + "logits/chosen": -481068714.6666667, + "logits/rejected": -665770188.8, + "logps/chosen": -227.10294596354166, + "logps/rejected": -185.52493896484376, + "loss": 0.3672, + "rewards/chosen": 1.1002014478047688, + "rewards/margins": 2.9621237119038897, + "rewards/rejected": -1.861922264099121, + "step": 299 + }, + { + "epoch": 0.5052631578947369, + "grad_norm": 104.48557270061485, + "kl": 0.0, + "learning_rate": 2.909959319513574e-07, + "logits/chosen": -709741760.0, + "logits/rejected": -1217407146.6666667, + "logps/chosen": -191.64903259277344, + "logps/rejected": -252.3373819986979, + "loss": 0.3704, + "rewards/chosen": -0.1919853240251541, + "rewards/margins": 0.7327715208133062, + "rewards/rejected": -0.9247568448384603, + "step": 300 + }, + { + "epoch": 0.5069473684210526, + "grad_norm": 43.12147455849188, + "kl": 0.0, + "learning_rate": 2.8954435771107597e-07, + "logits/chosen": -765106368.0, + "logits/rejected": -901597440.0, + "logps/chosen": -226.85153198242188, + "logps/rejected": -243.94345092773438, + "loss": 0.361, + "rewards/chosen": 1.015770435333252, + "rewards/margins": 2.2279579639434814, + "rewards/rejected": -1.2121875286102295, + "step": 301 + }, + { + "epoch": 0.5086315789473684, + "grad_norm": 68.53182250423035, + "kl": 0.0, + "learning_rate": 2.880914147952584e-07, + "logits/chosen": -892777728.0, + "logits/rejected": -893605683.2, + "logps/chosen": -231.4443359375, + "logps/rejected": -212.4324462890625, + "loss": 0.4275, + "rewards/chosen": 1.4770619074503581, + "rewards/margins": 0.6548136393229167, + "rewards/rejected": 0.8222482681274415, + "step": 302 + }, + { + "epoch": 0.5103157894736842, + "grad_norm": 57.399630263093485, + "kl": 0.0, + "learning_rate": 2.8663715349192383e-07, + "logits/chosen": -980971605.3333334, + "logits/rejected": -1283480320.0, + "logps/chosen": -271.35976155598956, + "logps/rejected": -338.93670654296875, + "loss": 0.3848, + "rewards/chosen": 1.7701203028361003, + "rewards/margins": 6.533517519632976, + "rewards/rejected": -4.763397216796875, + "step": 303 + }, + { + "epoch": 0.512, + "grad_norm": 50.30087172775045, + "kl": 0.0, + "learning_rate": 2.8518162413472263e-07, + "logits/chosen": -876101017.6, + "logits/rejected": -833215488.0, + "logps/chosen": -223.075244140625, + "logps/rejected": -205.84307861328125, + "loss": 0.4463, + "rewards/chosen": 1.0503494262695312, + "rewards/margins": 2.2924479166666667, + "rewards/rejected": -1.2420984903971355, + "step": 304 + }, + { + "epoch": 0.5136842105263157, + "grad_norm": 56.83106728944569, + "kl": 0.0, + "learning_rate": 2.837248771011937e-07, + "logits/chosen": -778440089.6, + "logits/rejected": -998400341.3333334, + "logps/chosen": -258.7751953125, + "logps/rejected": -218.8525594075521, + "loss": 0.4367, + "rewards/chosen": -0.8302111625671387, + "rewards/margins": -1.7733329931894937, + "rewards/rejected": 0.9431218306223551, + "step": 305 + }, + { + "epoch": 0.5153684210526316, + "grad_norm": 37.14827382589415, + "kl": 0.0, + "learning_rate": 2.822669628110213e-07, + "logits/chosen": -949675349.3333334, + "logits/rejected": -1104420966.4, + "logps/chosen": -197.35005696614584, + "logps/rejected": -236.111083984375, + "loss": 0.3883, + "rewards/chosen": 1.4387264251708984, + "rewards/margins": 1.295378041267395, + "rewards/rejected": 0.1433483839035034, + "step": 306 + }, + { + "epoch": 0.5170526315789473, + "grad_norm": 42.48400320185914, + "kl": 0.0, + "learning_rate": 2.808079317242896e-07, + "logits/chosen": -733285376.0, + "logits/rejected": -290018304.0, + "logps/chosen": -230.39531598772322, + "logps/rejected": -196.5876922607422, + "loss": 0.3848, + "rewards/chosen": -0.14083341189793178, + "rewards/margins": -1.530245338167463, + "rewards/rejected": 1.3894119262695312, + "step": 307 + }, + { + "epoch": 0.5187368421052632, + "grad_norm": 49.30251176881769, + "kl": 0.0, + "learning_rate": 2.793478343397367e-07, + "logits/chosen": -637947801.6, + "logits/rejected": -1100627200.0, + "logps/chosen": -236.856103515625, + "logps/rejected": -254.74955240885416, + "loss": 0.4514, + "rewards/chosen": 0.8377789497375489, + "rewards/margins": 0.31805851459503176, + "rewards/rejected": 0.5197204351425171, + "step": 308 + }, + { + "epoch": 0.5204210526315789, + "grad_norm": 38.15594290546514, + "kl": 0.0, + "learning_rate": 2.778867211930061e-07, + "logits/chosen": -946033956.5714285, + "logits/rejected": -322719744.0, + "logps/chosen": -188.54940359933036, + "logps/rejected": -337.9103088378906, + "loss": 0.3864, + "rewards/chosen": 0.2591233934674944, + "rewards/margins": 4.263719626835415, + "rewards/rejected": -4.00459623336792, + "step": 309 + }, + { + "epoch": 0.5221052631578947, + "grad_norm": 36.73206711146749, + "kl": 0.0, + "learning_rate": 2.7642464285489827e-07, + "logits/chosen": -1031071232.0, + "logits/rejected": -701534515.2, + "logps/chosen": -285.6693115234375, + "logps/rejected": -270.429541015625, + "loss": 0.3706, + "rewards/chosen": -0.7057943344116211, + "rewards/margins": 0.5104178428649901, + "rewards/rejected": -1.2162121772766112, + "step": 310 + }, + { + "epoch": 0.5237894736842105, + "grad_norm": 35.15174071748067, + "kl": 0.0, + "learning_rate": 2.749616499296199e-07, + "logits/chosen": -999771648.0, + "logits/rejected": -866819712.0, + "logps/chosen": -261.1120910644531, + "logps/rejected": -233.74642944335938, + "loss": 0.4233, + "rewards/chosen": -0.03837205469608307, + "rewards/margins": 2.5518483966588974, + "rewards/rejected": -2.5902204513549805, + "step": 311 + }, + { + "epoch": 0.5254736842105263, + "grad_norm": 40.26805134000718, + "kl": 0.0, + "learning_rate": 2.734977930530326e-07, + "logits/chosen": -521609728.0, + "logits/rejected": -1214733482.6666667, + "logps/chosen": -222.58474731445312, + "logps/rejected": -238.75984700520834, + "loss": 0.4109, + "rewards/chosen": 0.5281776785850525, + "rewards/margins": -0.3983253439267477, + "rewards/rejected": 0.9265030225118002, + "step": 312 + }, + { + "epoch": 0.5271578947368422, + "grad_norm": 41.427977504816596, + "kl": 0.0, + "learning_rate": 2.7203312289090044e-07, + "logits/chosen": -484565952.0, + "logits/rejected": -1246305024.0, + "logps/chosen": -273.0993347167969, + "logps/rejected": -276.9410400390625, + "loss": 0.4445, + "rewards/chosen": -0.8267297148704529, + "rewards/margins": 2.6424904465675354, + "rewards/rejected": -3.4692201614379883, + "step": 313 + }, + { + "epoch": 0.5288421052631579, + "grad_norm": 68.994308446122, + "kl": 0.0, + "learning_rate": 2.7056769013713623e-07, + "logits/chosen": -822423360.0, + "logits/rejected": -650067328.0, + "logps/chosen": -216.25306701660156, + "logps/rejected": -217.991943359375, + "loss": 0.3846, + "rewards/chosen": 1.5515110492706299, + "rewards/margins": 1.647828370332718, + "rewards/rejected": -0.09631732106208801, + "step": 314 + }, + { + "epoch": 0.5305263157894737, + "grad_norm": 57.35653485170902, + "kl": 0.0, + "learning_rate": 2.6910154551204676e-07, + "logits/chosen": -745939541.3333334, + "logits/rejected": -736545408.0, + "logps/chosen": -232.83365885416666, + "logps/rejected": -210.52879333496094, + "loss": 0.4926, + "rewards/chosen": 0.7454870541890463, + "rewards/margins": 0.6709303458531698, + "rewards/rejected": 0.07455670833587646, + "step": 315 + }, + { + "epoch": 0.5322105263157895, + "grad_norm": 81.62832673291715, + "kl": 0.0, + "learning_rate": 2.6763473976057773e-07, + "logits/chosen": -1231821397.3333333, + "logits/rejected": -768418918.4, + "logps/chosen": -211.95243326822916, + "logps/rejected": -236.2597900390625, + "loss": 0.351, + "rewards/chosen": 0.9875667095184326, + "rewards/margins": 3.251889371871948, + "rewards/rejected": -2.2643226623535155, + "step": 316 + }, + { + "epoch": 0.5338947368421053, + "grad_norm": 40.40366631438029, + "kl": 0.0, + "learning_rate": 2.6616732365055713e-07, + "logits/chosen": -878860083.2, + "logits/rejected": -1151675733.3333333, + "logps/chosen": -263.190625, + "logps/rejected": -241.0025634765625, + "loss": 0.3784, + "rewards/chosen": 1.9023317337036132, + "rewards/margins": 2.3599199295043944, + "rewards/rejected": -0.45758819580078125, + "step": 317 + }, + { + "epoch": 0.535578947368421, + "grad_norm": 37.906423958216756, + "kl": 0.0, + "learning_rate": 2.64699347970938e-07, + "logits/chosen": -643882432.0, + "logits/rejected": -725854293.3333334, + "logps/chosen": -210.35226440429688, + "logps/rejected": -264.7013346354167, + "loss": 0.39, + "rewards/chosen": 0.8370101451873779, + "rewards/margins": 0.3739887475967407, + "rewards/rejected": 0.4630213975906372, + "step": 318 + }, + { + "epoch": 0.5372631578947369, + "grad_norm": 41.62282940753045, + "kl": 0.0, + "learning_rate": 2.632308635300408e-07, + "logits/chosen": -1076104704.0, + "logits/rejected": -821822592.0, + "logps/chosen": -270.3343505859375, + "logps/rejected": -172.8352508544922, + "loss": 0.3992, + "rewards/chosen": -0.8739106059074402, + "rewards/margins": -0.6177101731300354, + "rewards/rejected": -0.2562004327774048, + "step": 319 + }, + { + "epoch": 0.5389473684210526, + "grad_norm": 32.33388694240573, + "kl": 0.0, + "learning_rate": 2.6176192115379494e-07, + "logits/chosen": -1163147776.0, + "logits/rejected": -671515520.0, + "logps/chosen": -232.58616638183594, + "logps/rejected": -390.2035319010417, + "loss": 0.4368, + "rewards/chosen": 3.005178928375244, + "rewards/margins": 4.992446422576904, + "rewards/rejected": -1.9872674942016602, + "step": 320 + }, + { + "epoch": 0.5406315789473685, + "grad_norm": 99.7767501397208, + "kl": 0.0, + "learning_rate": 2.6029257168397944e-07, + "logits/chosen": -1067986124.8, + "logits/rejected": -520208341.3333333, + "logps/chosen": -205.670068359375, + "logps/rejected": -209.88446044921875, + "loss": 0.365, + "rewards/chosen": 0.7348861694335938, + "rewards/margins": 0.3829381465911865, + "rewards/rejected": 0.3519480228424072, + "step": 321 + }, + { + "epoch": 0.5423157894736842, + "grad_norm": 43.79004322458768, + "kl": 0.0, + "learning_rate": 2.5882286597646313e-07, + "logits/chosen": -837222016.0, + "logits/rejected": -825369429.3333334, + "logps/chosen": -224.58160400390625, + "logps/rejected": -298.0353597005208, + "loss": 0.5333, + "rewards/chosen": 1.3384246826171875, + "rewards/margins": 2.8695327440897627, + "rewards/rejected": -1.531108061472575, + "step": 322 + }, + { + "epoch": 0.544, + "grad_norm": 45.949254583233454, + "kl": 0.0, + "learning_rate": 2.5735285489944485e-07, + "logits/chosen": -728969216.0, + "logits/rejected": -885923225.6, + "logps/chosen": -220.39337158203125, + "logps/rejected": -217.1091552734375, + "loss": 0.4497, + "rewards/chosen": 3.17012882232666, + "rewards/margins": 2.6761786460876467, + "rewards/rejected": 0.49395017623901366, + "step": 323 + }, + { + "epoch": 0.5456842105263158, + "grad_norm": 51.89181096647278, + "kl": 0.0, + "learning_rate": 2.5588258933169244e-07, + "logits/chosen": -918494272.0, + "logits/rejected": -728522112.0, + "logps/chosen": -144.35769653320312, + "logps/rejected": -252.4864501953125, + "loss": 0.4313, + "rewards/chosen": 0.6073247194290161, + "rewards/margins": 1.5144293308258057, + "rewards/rejected": -0.9071046113967896, + "step": 324 + }, + { + "epoch": 0.5473684210526316, + "grad_norm": 42.4363414069206, + "kl": 0.0, + "learning_rate": 2.544121201607822e-07, + "logits/chosen": -741229977.6, + "logits/rejected": -902070272.0, + "logps/chosen": -216.93974609375, + "logps/rejected": -230.6597900390625, + "loss": 0.4731, + "rewards/chosen": -0.7157364368438721, + "rewards/margins": -0.09154821236928312, + "rewards/rejected": -0.624188224474589, + "step": 325 + }, + { + "epoch": 0.5490526315789473, + "grad_norm": 49.977660612765106, + "kl": 0.0, + "learning_rate": 2.5294149828133704e-07, + "logits/chosen": -785820672.0, + "logits/rejected": -410074240.0, + "logps/chosen": -192.42119489397322, + "logps/rejected": -291.78887939453125, + "loss": 0.2975, + "rewards/chosen": 1.6415718623570033, + "rewards/margins": 2.1497139760426114, + "rewards/rejected": -0.5081421136856079, + "step": 326 + }, + { + "epoch": 0.5507368421052632, + "grad_norm": 40.20254402613094, + "kl": 0.0, + "learning_rate": 2.5147077459326555e-07, + "logits/chosen": -540746086.4, + "logits/rejected": -600018858.6666666, + "logps/chosen": -271.22294921875, + "logps/rejected": -190.82244873046875, + "loss": 0.3018, + "rewards/chosen": 1.12197265625, + "rewards/margins": 4.917408243815104, + "rewards/rejected": -3.795435587565104, + "step": 327 + }, + { + "epoch": 0.5524210526315789, + "grad_norm": 36.746566177202766, + "kl": 0.0, + "learning_rate": 2.5e-07, + "logits/chosen": -960942811.4285715, + "logits/rejected": -569875712.0, + "logps/chosen": -220.05186244419642, + "logps/rejected": -294.9991149902344, + "loss": 0.3669, + "rewards/chosen": 1.3416931969778878, + "rewards/margins": 3.6566162449972968, + "rewards/rejected": -2.314923048019409, + "step": 328 + }, + { + "epoch": 0.5541052631578948, + "grad_norm": 92.10516677186112, + "kl": 0.0, + "learning_rate": 2.485292254067345e-07, + "logits/chosen": -911412736.0, + "logits/rejected": -994976153.6, + "logps/chosen": -234.53729248046875, + "logps/rejected": -238.84951171875, + "loss": 0.3849, + "rewards/chosen": 0.3486948808034261, + "rewards/margins": 1.069074519475301, + "rewards/rejected": -0.720379638671875, + "step": 329 + }, + { + "epoch": 0.5557894736842105, + "grad_norm": 40.68592836090874, + "kl": 0.0, + "learning_rate": 2.4705850171866294e-07, + "logits/chosen": -916755840.0, + "logits/rejected": -721056704.0, + "logps/chosen": -226.9412841796875, + "logps/rejected": -197.04571533203125, + "loss": 0.3178, + "rewards/chosen": 1.0302788019180298, + "rewards/margins": 2.7032233476638794, + "rewards/rejected": -1.6729445457458496, + "step": 330 + }, + { + "epoch": 0.5574736842105263, + "grad_norm": 64.00175361910134, + "kl": 0.0, + "learning_rate": 2.4558787983921785e-07, + "logits/chosen": -1136037478.4, + "logits/rejected": -776420096.0, + "logps/chosen": -184.01240234375, + "logps/rejected": -270.2752278645833, + "loss": 0.354, + "rewards/chosen": 1.9217365264892579, + "rewards/margins": 2.225330440203349, + "rewards/rejected": -0.303593913714091, + "step": 331 + }, + { + "epoch": 0.5591578947368421, + "grad_norm": 53.20843592872294, + "kl": 0.0, + "learning_rate": 2.441174106683076e-07, + "logits/chosen": -987133542.4, + "logits/rejected": -757408768.0, + "logps/chosen": -270.960498046875, + "logps/rejected": -269.0520426432292, + "loss": 0.4314, + "rewards/chosen": 0.6490658760070801, + "rewards/margins": 1.0015769640604655, + "rewards/rejected": -0.35251108805338544, + "step": 332 + }, + { + "epoch": 0.5608421052631579, + "grad_norm": 39.62692923007265, + "kl": 0.0, + "learning_rate": 2.4264714510055513e-07, + "logits/chosen": -779204096.0, + "logits/rejected": -775985749.3333334, + "logps/chosen": -172.64938354492188, + "logps/rejected": -189.1490275065104, + "loss": 0.379, + "rewards/chosen": 0.8577301502227783, + "rewards/margins": 0.2588157653808594, + "rewards/rejected": 0.598914384841919, + "step": 333 + }, + { + "epoch": 0.5625263157894737, + "grad_norm": 189.00573139104904, + "kl": 0.0, + "learning_rate": 2.4117713402353685e-07, + "logits/chosen": -717026304.0, + "logits/rejected": -589375616.0, + "logps/chosen": -255.56656901041666, + "logps/rejected": -222.212158203125, + "loss": 0.4805, + "rewards/chosen": -0.3913642962773641, + "rewards/margins": -0.5911636402209599, + "rewards/rejected": 0.1997993439435959, + "step": 334 + }, + { + "epoch": 0.5642105263157895, + "grad_norm": 45.7662847906049, + "kl": 0.0, + "learning_rate": 2.397074283160206e-07, + "logits/chosen": -753268096.0, + "logits/rejected": -917319168.0, + "logps/chosen": -198.811279296875, + "logps/rejected": -218.18515014648438, + "loss": 0.3514, + "rewards/chosen": 1.425872802734375, + "rewards/margins": 2.8109121322631836, + "rewards/rejected": -1.3850393295288086, + "step": 335 + }, + { + "epoch": 0.5658947368421052, + "grad_norm": 39.80557628734281, + "kl": 0.0, + "learning_rate": 2.38238078846205e-07, + "logits/chosen": -806648490.6666666, + "logits/rejected": -1043137280.0, + "logps/chosen": -197.84334309895834, + "logps/rejected": -166.3507080078125, + "loss": 0.4203, + "rewards/chosen": 2.0990654627482095, + "rewards/margins": 0.3083709875742593, + "rewards/rejected": 1.7906944751739502, + "step": 336 + }, + { + "epoch": 0.5675789473684211, + "grad_norm": 55.26598793514629, + "kl": 0.0, + "learning_rate": 2.367691364699592e-07, + "logits/chosen": -990476117.3333334, + "logits/rejected": -843749478.4, + "logps/chosen": -211.03731282552084, + "logps/rejected": -200.39736328125, + "loss": 0.4159, + "rewards/chosen": 1.5658411979675293, + "rewards/margins": 2.0694480895996095, + "rewards/rejected": -0.5036068916320801, + "step": 337 + }, + { + "epoch": 0.5692631578947368, + "grad_norm": 43.55127827223215, + "kl": 0.0, + "learning_rate": 2.3530065202906208e-07, + "logits/chosen": -677904725.3333334, + "logits/rejected": -739208768.0, + "logps/chosen": -215.2685546875, + "logps/rejected": -168.70806884765625, + "loss": 0.365, + "rewards/chosen": -0.36894532044728595, + "rewards/margins": 3.7609441677729287, + "rewards/rejected": -4.129889488220215, + "step": 338 + }, + { + "epoch": 0.5709473684210526, + "grad_norm": 44.273467218054, + "kl": 0.0, + "learning_rate": 2.3383267634944288e-07, + "logits/chosen": -708441152.0, + "logits/rejected": -909734720.0, + "logps/chosen": -236.1248016357422, + "logps/rejected": -329.7475280761719, + "loss": 0.3328, + "rewards/chosen": 0.44537657499313354, + "rewards/margins": 2.3071479201316833, + "rewards/rejected": -1.8617713451385498, + "step": 339 + }, + { + "epoch": 0.5726315789473684, + "grad_norm": 44.104623030836734, + "kl": 0.0, + "learning_rate": 2.3236526023942222e-07, + "logits/chosen": -1229630873.6, + "logits/rejected": -615721045.3333334, + "logps/chosen": -201.258935546875, + "logps/rejected": -160.00018310546875, + "loss": 0.3583, + "rewards/chosen": 0.4522857666015625, + "rewards/margins": 3.613059679667155, + "rewards/rejected": -3.1607739130655923, + "step": 340 + }, + { + "epoch": 0.5743157894736842, + "grad_norm": 37.46747269379094, + "kl": 0.0, + "learning_rate": 2.3089845448795325e-07, + "logits/chosen": -972004778.6666666, + "logits/rejected": -684686336.0, + "logps/chosen": -186.35986328125, + "logps/rejected": -252.18388671875, + "loss": 0.3367, + "rewards/chosen": 2.4348222414652505, + "rewards/margins": 3.513483206431071, + "rewards/rejected": -1.0786609649658203, + "step": 341 + }, + { + "epoch": 0.576, + "grad_norm": 41.066699309390806, + "kl": 0.0, + "learning_rate": 2.2943230986286386e-07, + "logits/chosen": -776381866.6666666, + "logits/rejected": -411293952.0, + "logps/chosen": -237.33272298177084, + "logps/rejected": -230.616357421875, + "loss": 0.5505, + "rewards/chosen": -1.2874414920806885, + "rewards/margins": -2.0171528339385985, + "rewards/rejected": 0.7297113418579102, + "step": 342 + }, + { + "epoch": 0.5776842105263158, + "grad_norm": 34.78601277894388, + "kl": 0.0, + "learning_rate": 2.2796687710909961e-07, + "logits/chosen": -894285824.0, + "logits/rejected": -1057857365.3333334, + "logps/chosen": -193.6359619140625, + "logps/rejected": -198.5640665690104, + "loss": 0.4058, + "rewards/chosen": 0.9409942626953125, + "rewards/margins": -0.3375056584676106, + "rewards/rejected": 1.278499921162923, + "step": 343 + }, + { + "epoch": 0.5793684210526315, + "grad_norm": 43.656984836332974, + "kl": 0.0, + "learning_rate": 2.2650220694696746e-07, + "logits/chosen": -936853913.6, + "logits/rejected": -637026218.6666666, + "logps/chosen": -258.152392578125, + "logps/rejected": -172.32596842447916, + "loss": 0.3634, + "rewards/chosen": 0.7583493709564209, + "rewards/margins": 0.6318836331367492, + "rewards/rejected": 0.12646573781967163, + "step": 344 + }, + { + "epoch": 0.5810526315789474, + "grad_norm": 43.09840300795585, + "kl": 0.0, + "learning_rate": 2.2503835007038018e-07, + "logits/chosen": -1292558336.0, + "logits/rejected": -511415200.0, + "logps/chosen": -219.50491333007812, + "logps/rejected": -236.9748077392578, + "loss": 0.3775, + "rewards/chosen": 1.3266575336456299, + "rewards/margins": 2.1448956727981567, + "rewards/rejected": -0.8182381391525269, + "step": 345 + }, + { + "epoch": 0.5827368421052631, + "grad_norm": 38.706929628825115, + "kl": 0.0, + "learning_rate": 2.2357535714510179e-07, + "logits/chosen": -1250603776.0, + "logits/rejected": -919112908.8, + "logps/chosen": -281.291748046875, + "logps/rejected": -183.513916015625, + "loss": 0.3373, + "rewards/chosen": -0.5119430224100748, + "rewards/margins": 1.2942089398701988, + "rewards/rejected": -1.8061519622802735, + "step": 346 + }, + { + "epoch": 0.584421052631579, + "grad_norm": 54.06441877859246, + "kl": 0.0, + "learning_rate": 2.2211327880699389e-07, + "logits/chosen": -983545036.8, + "logits/rejected": -1222936832.0, + "logps/chosen": -196.76480712890626, + "logps/rejected": -226.22184244791666, + "loss": 0.4641, + "rewards/chosen": 0.9738927841186523, + "rewards/margins": -0.2919163544972737, + "rewards/rejected": 1.265809138615926, + "step": 347 + }, + { + "epoch": 0.5861052631578947, + "grad_norm": 31.758781014479883, + "kl": 0.0, + "learning_rate": 2.206521656602633e-07, + "logits/chosen": -929091264.0, + "logits/rejected": -505945888.0, + "logps/chosen": -176.41741943359375, + "logps/rejected": -188.78915405273438, + "loss": 0.3671, + "rewards/chosen": 1.3973584175109863, + "rewards/margins": 1.813145935535431, + "rewards/rejected": -0.4157875180244446, + "step": 348 + }, + { + "epoch": 0.5877894736842105, + "grad_norm": 37.86314064294317, + "kl": 0.0, + "learning_rate": 2.1919206827571034e-07, + "logits/chosen": -801264640.0, + "logits/rejected": -464600064.0, + "logps/chosen": -198.34344482421875, + "logps/rejected": -244.7729034423828, + "loss": 0.4117, + "rewards/chosen": 1.5484859943389893, + "rewards/margins": 3.387136936187744, + "rewards/rejected": -1.8386509418487549, + "step": 349 + }, + { + "epoch": 0.5894736842105263, + "grad_norm": 50.13743066285001, + "kl": 0.0, + "learning_rate": 2.177330371889787e-07, + "logits/chosen": -899666124.8, + "logits/rejected": -916757248.0, + "logps/chosen": -219.9362060546875, + "logps/rejected": -315.0555419921875, + "loss": 0.4165, + "rewards/chosen": 0.3476431369781494, + "rewards/margins": 0.9587682882944742, + "rewards/rejected": -0.6111251513163248, + "step": 350 + }, + { + "epoch": 0.5911578947368421, + "grad_norm": 51.02552672848151, + "kl": 0.0, + "learning_rate": 2.162751228988063e-07, + "logits/chosen": -853029760.0, + "logits/rejected": -762731136.0, + "logps/chosen": -260.21746826171875, + "logps/rejected": -254.34788513183594, + "loss": 0.3839, + "rewards/chosen": 0.05103302001953125, + "rewards/margins": 1.4993259906768799, + "rewards/rejected": -1.4482929706573486, + "step": 351 + }, + { + "epoch": 0.592842105263158, + "grad_norm": 34.82384239421157, + "kl": 0.0, + "learning_rate": 2.148183758652774e-07, + "logits/chosen": -740140032.0, + "logits/rejected": -741718784.0, + "logps/chosen": -147.6769561767578, + "logps/rejected": -276.81591796875, + "loss": 0.4139, + "rewards/chosen": 2.366483211517334, + "rewards/margins": 3.0736358165740967, + "rewards/rejected": -0.7071526050567627, + "step": 352 + }, + { + "epoch": 0.5945263157894737, + "grad_norm": 34.883343445970695, + "kl": 0.0, + "learning_rate": 2.1336284650807612e-07, + "logits/chosen": -1254429781.3333333, + "logits/rejected": -611748761.6, + "logps/chosen": -215.1737060546875, + "logps/rejected": -214.90478515625, + "loss": 0.3704, + "rewards/chosen": 2.0823793411254883, + "rewards/margins": 2.4012239456176756, + "rewards/rejected": -0.3188446044921875, + "step": 353 + }, + { + "epoch": 0.5962105263157895, + "grad_norm": 55.9131613356441, + "kl": 0.0, + "learning_rate": 2.1190858520474163e-07, + "logits/chosen": -1588147200.0, + "logits/rejected": -903177318.4, + "logps/chosen": -219.27388509114584, + "logps/rejected": -283.88779296875, + "loss": 0.3452, + "rewards/chosen": 2.384967645009359, + "rewards/margins": 2.509509845574697, + "rewards/rejected": -0.12454220056533813, + "step": 354 + }, + { + "epoch": 0.5978947368421053, + "grad_norm": 43.395183494537164, + "kl": 0.0, + "learning_rate": 2.10455642288924e-07, + "logits/chosen": -1037725632.0, + "logits/rejected": -652462464.0, + "logps/chosen": -192.9014892578125, + "logps/rejected": -218.373779296875, + "loss": 0.338, + "rewards/chosen": 1.9063019752502441, + "rewards/margins": 2.8689812421798706, + "rewards/rejected": -0.9626792669296265, + "step": 355 + }, + { + "epoch": 0.5995789473684211, + "grad_norm": 72.0621578330362, + "kl": 0.0, + "learning_rate": 2.090040680486426e-07, + "logits/chosen": -1054155008.0, + "logits/rejected": -724854912.0, + "logps/chosen": -225.2272491455078, + "logps/rejected": -268.515869140625, + "loss": 0.4356, + "rewards/chosen": -0.31277698278427124, + "rewards/margins": 1.6200557351112366, + "rewards/rejected": -1.9328327178955078, + "step": 356 + }, + { + "epoch": 0.6012631578947368, + "grad_norm": 46.80458240411995, + "kl": 0.0, + "learning_rate": 2.0755391272454537e-07, + "logits/chosen": -1309161600.0, + "logits/rejected": -773884096.0, + "logps/chosen": -221.47581481933594, + "logps/rejected": -243.53134155273438, + "loss": 0.3444, + "rewards/chosen": 0.6755388379096985, + "rewards/margins": 0.5105028748512268, + "rewards/rejected": 0.16503596305847168, + "step": 357 + }, + { + "epoch": 0.6029473684210527, + "grad_norm": 44.25731693592669, + "kl": 0.0, + "learning_rate": 2.0610522650816982e-07, + "logits/chosen": -781600384.0, + "logits/rejected": -991481728.0, + "logps/chosen": -269.00433349609375, + "logps/rejected": -233.53921508789062, + "loss": 0.3766, + "rewards/chosen": -0.19508057832717896, + "rewards/margins": 1.9026996493339539, + "rewards/rejected": -2.097780227661133, + "step": 358 + }, + { + "epoch": 0.6046315789473684, + "grad_norm": 51.072468854632184, + "kl": 0.0, + "learning_rate": 2.04658059540206e-07, + "logits/chosen": -955063210.6666666, + "logits/rejected": -842874060.8, + "logps/chosen": -206.79142252604166, + "logps/rejected": -263.4759521484375, + "loss": 0.3669, + "rewards/chosen": 1.126490831375122, + "rewards/margins": 1.8041922569274902, + "rewards/rejected": -0.6777014255523681, + "step": 359 + }, + { + "epoch": 0.6063157894736843, + "grad_norm": 43.797014090675, + "kl": 0.0, + "learning_rate": 2.0321246190876112e-07, + "logits/chosen": -813810773.3333334, + "logits/rejected": -735887360.0, + "logps/chosen": -223.61555989583334, + "logps/rejected": -310.8141357421875, + "loss": 0.2421, + "rewards/chosen": 2.7599121729532876, + "rewards/margins": 4.69793898264567, + "rewards/rejected": -1.9380268096923827, + "step": 360 + }, + { + "epoch": 0.608, + "grad_norm": 44.19945884028008, + "kl": 0.0, + "learning_rate": 2.0176848364762576e-07, + "logits/chosen": -1044594995.2, + "logits/rejected": -838090410.6666666, + "logps/chosen": -241.623193359375, + "logps/rejected": -180.17657470703125, + "loss": 0.4451, + "rewards/chosen": 0.6463582992553711, + "rewards/margins": 3.4257356007893884, + "rewards/rejected": -2.779377301534017, + "step": 361 + }, + { + "epoch": 0.6096842105263158, + "grad_norm": 84.65829093491772, + "kl": 0.0, + "learning_rate": 2.0032617473454225e-07, + "logits/chosen": -993763620.5714285, + "logits/rejected": -1168001152.0, + "logps/chosen": -222.610107421875, + "logps/rejected": -359.9516296386719, + "loss": 0.373, + "rewards/chosen": 0.029357058661324636, + "rewards/margins": 5.195747954504831, + "rewards/rejected": -5.166390895843506, + "step": 362 + }, + { + "epoch": 0.6113684210526316, + "grad_norm": 37.454198883632955, + "kl": 0.0, + "learning_rate": 1.9888558508947492e-07, + "logits/chosen": -656499840.0, + "logits/rejected": -800222016.0, + "logps/chosen": -248.09259033203125, + "logps/rejected": -252.30076599121094, + "loss": 0.3177, + "rewards/chosen": 1.0152060985565186, + "rewards/margins": 3.4270780086517334, + "rewards/rejected": -2.411871910095215, + "step": 363 + }, + { + "epoch": 0.6130526315789474, + "grad_norm": 44.05381109880352, + "kl": 0.0, + "learning_rate": 1.974467645728822e-07, + "logits/chosen": -734617600.0, + "logits/rejected": -1037793280.0, + "logps/chosen": -210.473876953125, + "logps/rejected": -227.2939453125, + "loss": 0.3765, + "rewards/chosen": 0.8343788385391235, + "rewards/margins": 2.4099492629369097, + "rewards/rejected": -1.5755704243977864, + "step": 364 + }, + { + "epoch": 0.6147368421052631, + "grad_norm": 48.57136273490337, + "kl": 0.0, + "learning_rate": 1.9600976298399108e-07, + "logits/chosen": -942329036.8, + "logits/rejected": -660537429.3333334, + "logps/chosen": -285.062890625, + "logps/rejected": -235.09102376302084, + "loss": 0.3531, + "rewards/chosen": -0.9990038871765137, + "rewards/margins": 0.40311670303344727, + "rewards/rejected": -1.402120590209961, + "step": 365 + }, + { + "epoch": 0.616421052631579, + "grad_norm": 37.488732058461395, + "kl": 0.0, + "learning_rate": 1.945746300590731e-07, + "logits/chosen": -483911520.0, + "logits/rejected": -1166225408.0, + "logps/chosen": -247.5726776123047, + "logps/rejected": -214.5377197265625, + "loss": 0.3618, + "rewards/chosen": 0.26900553703308105, + "rewards/margins": 0.6737111012140911, + "rewards/rejected": -0.40470556418100995, + "step": 366 + }, + { + "epoch": 0.6181052631578947, + "grad_norm": 44.38980472905698, + "kl": 0.0, + "learning_rate": 1.9314141546972344e-07, + "logits/chosen": -783111424.0, + "logits/rejected": -941412928.0, + "logps/chosen": -177.5583953857422, + "logps/rejected": -260.1361083984375, + "loss": 0.3547, + "rewards/chosen": -0.2703380584716797, + "rewards/margins": -0.19543498754501343, + "rewards/rejected": -0.07490307092666626, + "step": 367 + }, + { + "epoch": 0.6197894736842106, + "grad_norm": 37.93187426865055, + "kl": 0.0, + "learning_rate": 1.9171016882114152e-07, + "logits/chosen": -849338432.0, + "logits/rejected": -782459200.0, + "logps/chosen": -195.59857177734375, + "logps/rejected": -232.708984375, + "loss": 0.38, + "rewards/chosen": 1.189550757408142, + "rewards/margins": 3.122536540031433, + "rewards/rejected": -1.932985782623291, + "step": 368 + }, + { + "epoch": 0.6214736842105263, + "grad_norm": 38.196373434045015, + "kl": 0.0, + "learning_rate": 1.9028093965041391e-07, + "logits/chosen": -1023574976.0, + "logits/rejected": -983692288.0, + "logps/chosen": -205.63710021972656, + "logps/rejected": -268.6028747558594, + "loss": 0.3751, + "rewards/chosen": 0.3406059741973877, + "rewards/margins": 1.6527996063232422, + "rewards/rejected": -1.3121936321258545, + "step": 369 + }, + { + "epoch": 0.6231578947368421, + "grad_norm": 55.12173721267581, + "kl": 0.0, + "learning_rate": 1.888537774248e-07, + "logits/chosen": -741024597.3333334, + "logits/rejected": -600171980.8, + "logps/chosen": -179.9103800455729, + "logps/rejected": -237.39990234375, + "loss": 0.418, + "rewards/chosen": 1.0137186845143635, + "rewards/margins": 2.055362590154012, + "rewards/rejected": -1.0416439056396485, + "step": 370 + }, + { + "epoch": 0.6248421052631579, + "grad_norm": 34.16987497139576, + "kl": 0.0, + "learning_rate": 1.8742873154002004e-07, + "logits/chosen": -837830826.6666666, + "logits/rejected": -1010286720.0, + "logps/chosen": -276.2460530598958, + "logps/rejected": -183.48431396484375, + "loss": 0.4701, + "rewards/chosen": 0.2395154039065043, + "rewards/margins": -0.8625567158063253, + "rewards/rejected": 1.1020721197128296, + "step": 371 + }, + { + "epoch": 0.6265263157894737, + "grad_norm": 46.66359237058551, + "kl": 0.0, + "learning_rate": 1.86005851318545e-07, + "logits/chosen": -669656012.8, + "logits/rejected": -1303058517.3333333, + "logps/chosen": -244.3946044921875, + "logps/rejected": -313.6065266927083, + "loss": 0.3828, + "rewards/chosen": -0.9342806816101075, + "rewards/margins": 2.6151256879170734, + "rewards/rejected": -3.549406369527181, + "step": 372 + }, + { + "epoch": 0.6282105263157894, + "grad_norm": 35.82420878860377, + "kl": 0.0, + "learning_rate": 1.8458518600788987e-07, + "logits/chosen": -1138871722.6666667, + "logits/rejected": -761401753.6, + "logps/chosen": -230.2353312174479, + "logps/rejected": -246.2571044921875, + "loss": 0.3776, + "rewards/chosen": 0.059048473834991455, + "rewards/margins": 2.80713711977005, + "rewards/rejected": -2.7480886459350584, + "step": 373 + }, + { + "epoch": 0.6298947368421053, + "grad_norm": 47.50923482875504, + "kl": 0.0, + "learning_rate": 1.8316678477890913e-07, + "logits/chosen": -1124751530.6666667, + "logits/rejected": -855729868.8, + "logps/chosen": -216.44978841145834, + "logps/rejected": -233.7421875, + "loss": 0.422, + "rewards/chosen": 0.27029621601104736, + "rewards/margins": 3.8977864503860475, + "rewards/rejected": -3.627490234375, + "step": 374 + }, + { + "epoch": 0.631578947368421, + "grad_norm": 59.37971841261376, + "kl": 0.0, + "learning_rate": 1.8175069672409475e-07, + "logits/chosen": -564849280.0, + "logits/rejected": -890142080.0, + "logps/chosen": -229.4367218017578, + "logps/rejected": -251.96417236328125, + "loss": 0.4461, + "rewards/chosen": -2.096214532852173, + "rewards/margins": -0.3813058137893677, + "rewards/rejected": -1.7149087190628052, + "step": 375 + }, + { + "epoch": 0.6332631578947369, + "grad_norm": 42.11784105230497, + "kl": 0.0, + "learning_rate": 1.8033697085587696e-07, + "logits/chosen": -910276096.0, + "logits/rejected": -714325845.3333334, + "logps/chosen": -231.393359375, + "logps/rejected": -271.1864420572917, + "loss": 0.4749, + "rewards/chosen": -0.2165008544921875, + "rewards/margins": 1.6030135790506999, + "rewards/rejected": -1.8195144335428874, + "step": 376 + }, + { + "epoch": 0.6349473684210526, + "grad_norm": 38.23749676627054, + "kl": 0.0, + "learning_rate": 1.7892565610492826e-07, + "logits/chosen": -592622421.3333334, + "logits/rejected": -968836416.0, + "logps/chosen": -251.1493937174479, + "logps/rejected": -316.63690185546875, + "loss": 0.4272, + "rewards/chosen": 0.3538096348444621, + "rewards/margins": 1.5449076096216838, + "rewards/rejected": -1.1910979747772217, + "step": 377 + }, + { + "epoch": 0.6366315789473684, + "grad_norm": 33.65033547018444, + "kl": 0.0, + "learning_rate": 1.775168013184694e-07, + "logits/chosen": -1084484352.0, + "logits/rejected": -1348099840.0, + "logps/chosen": -294.9368896484375, + "logps/rejected": -199.22079467773438, + "loss": 0.3069, + "rewards/chosen": -0.5554192066192627, + "rewards/margins": 2.1953020095825195, + "rewards/rejected": -2.7507212162017822, + "step": 378 + }, + { + "epoch": 0.6383157894736842, + "grad_norm": 57.13484688248307, + "kl": 0.0, + "learning_rate": 1.7611045525857898e-07, + "logits/chosen": -878904466.2857143, + "logits/rejected": -939131328.0, + "logps/chosen": -230.76688058035714, + "logps/rejected": -233.318603515625, + "loss": 0.3319, + "rewards/chosen": 0.6585009438650948, + "rewards/margins": 0.9025790137904031, + "rewards/rejected": -0.24407806992530823, + "step": 379 + }, + { + "epoch": 0.64, + "grad_norm": 75.99248678640355, + "kl": 0.0, + "learning_rate": 1.7470666660050584e-07, + "logits/chosen": -838683050.6666666, + "logits/rejected": -667947072.0, + "logps/chosen": -205.16400146484375, + "logps/rejected": -321.7478332519531, + "loss": 0.3082, + "rewards/chosen": 1.483425776163737, + "rewards/margins": 1.0787352124849956, + "rewards/rejected": 0.40469056367874146, + "step": 380 + }, + { + "epoch": 0.6416842105263157, + "grad_norm": 41.23482970655455, + "kl": 0.0, + "learning_rate": 1.7330548393098403e-07, + "logits/chosen": -790441728.0, + "logits/rejected": -1023563200.0, + "logps/chosen": -277.3321533203125, + "logps/rejected": -204.98135375976562, + "loss": 0.3465, + "rewards/chosen": 0.8123489618301392, + "rewards/margins": 4.149082779884338, + "rewards/rejected": -3.336733818054199, + "step": 381 + }, + { + "epoch": 0.6433684210526316, + "grad_norm": 61.65012784187018, + "kl": 0.0, + "learning_rate": 1.7190695574655144e-07, + "logits/chosen": -850949056.0, + "logits/rejected": -890390656.0, + "logps/chosen": -191.81503295898438, + "logps/rejected": -262.1980285644531, + "loss": 0.4055, + "rewards/chosen": 2.1015682220458984, + "rewards/margins": 2.900312066078186, + "rewards/rejected": -0.7987438440322876, + "step": 382 + }, + { + "epoch": 0.6450526315789473, + "grad_norm": 40.46551574043127, + "kl": 0.0, + "learning_rate": 1.705111304518712e-07, + "logits/chosen": -800098986.6666666, + "logits/rejected": -1169969920.0, + "logps/chosen": -194.7923787434896, + "logps/rejected": -145.28848266601562, + "loss": 0.2469, + "rewards/chosen": 2.726773262023926, + "rewards/margins": 4.934856414794922, + "rewards/rejected": -2.208083152770996, + "step": 383 + }, + { + "epoch": 0.6467368421052632, + "grad_norm": 43.106097272592606, + "kl": 0.0, + "learning_rate": 1.6911805635805632e-07, + "logits/chosen": -827328576.0, + "logits/rejected": -977397696.0, + "logps/chosen": -231.96633911132812, + "logps/rejected": -252.32371520996094, + "loss": 0.3401, + "rewards/chosen": 0.41304701566696167, + "rewards/margins": 1.2065090537071228, + "rewards/rejected": -0.7934620380401611, + "step": 384 + }, + { + "epoch": 0.6484210526315789, + "grad_norm": 42.75933123532293, + "kl": 0.0, + "learning_rate": 1.677277816809975e-07, + "logits/chosen": -1216948864.0, + "logits/rejected": -623093888.0, + "logps/chosen": -240.3336181640625, + "logps/rejected": -257.3775939941406, + "loss": 0.3767, + "rewards/chosen": 1.1069988012313843, + "rewards/margins": 4.4674311876297, + "rewards/rejected": -3.3604323863983154, + "step": 385 + }, + { + "epoch": 0.6501052631578947, + "grad_norm": 41.09667706031157, + "kl": 0.0, + "learning_rate": 1.6634035453969457e-07, + "logits/chosen": -689569740.8, + "logits/rejected": -782888533.3333334, + "logps/chosen": -218.35732421875, + "logps/rejected": -258.15380859375, + "loss": 0.3759, + "rewards/chosen": 1.433096218109131, + "rewards/margins": 1.4153085629145306, + "rewards/rejected": 0.017787655194600422, + "step": 386 + }, + { + "epoch": 0.6517894736842105, + "grad_norm": 54.55329906409335, + "kl": 0.0, + "learning_rate": 1.6495582295459078e-07, + "logits/chosen": -605408320.0, + "logits/rejected": -774927232.0, + "logps/chosen": -204.93194580078125, + "logps/rejected": -248.58714294433594, + "loss": 0.3717, + "rewards/chosen": 0.8562226891517639, + "rewards/margins": 2.2260130047798157, + "rewards/rejected": -1.3697903156280518, + "step": 387 + }, + { + "epoch": 0.6534736842105263, + "grad_norm": 29.797849697928793, + "kl": 0.0, + "learning_rate": 1.6357423484591087e-07, + "logits/chosen": -766515712.0, + "logits/rejected": -736909653.3333334, + "logps/chosen": -210.6973388671875, + "logps/rejected": -224.88907877604166, + "loss": 0.312, + "rewards/chosen": 0.03613958358764648, + "rewards/margins": 0.5231950441996257, + "rewards/rejected": -0.4870554606119792, + "step": 388 + }, + { + "epoch": 0.655157894736842, + "grad_norm": 64.33167682170762, + "kl": 0.0, + "learning_rate": 1.621956380320027e-07, + "logits/rejected": -840579712.0, + "logps/rejected": -260.83721923828125, + "loss": 0.4427, + "rewards/rejected": -0.597730278968811, + "step": 389 + }, + { + "epoch": 0.6568421052631579, + "grad_norm": 42.22779651027316, + "kl": 0.0, + "learning_rate": 1.608200802276815e-07, + "logits/chosen": -596097536.0, + "logits/rejected": -820432298.6666666, + "logps/chosen": -243.89404296875, + "logps/rejected": -213.07832845052084, + "loss": 0.3828, + "rewards/chosen": 1.3395811080932618, + "rewards/margins": 0.6359276135762534, + "rewards/rejected": 0.7036534945170084, + "step": 390 + }, + { + "epoch": 0.6585263157894737, + "grad_norm": 38.06012726917251, + "kl": 0.0, + "learning_rate": 1.5944760904257942e-07, + "logits/chosen": -941375040.0, + "logits/rejected": -882885802.6666666, + "logps/chosen": -157.46560668945312, + "logps/rejected": -215.0748291015625, + "loss": 0.3784, + "rewards/chosen": 0.9262268543243408, + "rewards/margins": 0.7893735567728678, + "rewards/rejected": 0.136853297551473, + "step": 391 + }, + { + "epoch": 0.6602105263157895, + "grad_norm": 40.676015795529814, + "kl": 0.0, + "learning_rate": 1.5807827197949685e-07, + "logits/chosen": -708315968.0, + "logits/rejected": -1063172181.3333334, + "logps/chosen": -245.2415771484375, + "logps/rejected": -261.03733317057294, + "loss": 0.4446, + "rewards/chosen": 1.1207358837127686, + "rewards/margins": 0.9106475313504537, + "rewards/rejected": 0.21008835236231485, + "step": 392 + }, + { + "epoch": 0.6618947368421053, + "grad_norm": 120.64719239721539, + "kl": 0.0, + "learning_rate": 1.5671211643275875e-07, + "logits/chosen": -633163648.0, + "logits/rejected": -805495872.0, + "logps/chosen": -236.74386596679688, + "logps/rejected": -252.68606567382812, + "loss": 0.4374, + "rewards/chosen": 0.5052295923233032, + "rewards/margins": -0.2789260149002075, + "rewards/rejected": 0.7841556072235107, + "step": 393 + }, + { + "epoch": 0.663578947368421, + "grad_norm": 34.633986243971194, + "kl": 0.0, + "learning_rate": 1.553491896865742e-07, + "logits/chosen": -1026384000.0, + "logits/rejected": -597304448.0, + "logps/chosen": -198.23861694335938, + "logps/rejected": -220.9433135986328, + "loss": 0.3553, + "rewards/chosen": 2.9143166542053223, + "rewards/margins": 1.0535552501678467, + "rewards/rejected": 1.8607614040374756, + "step": 394 + }, + { + "epoch": 0.6652631578947369, + "grad_norm": 45.21200734298712, + "kl": 0.0, + "learning_rate": 1.5398953891339968e-07, + "logits/chosen": -1087951360.0, + "logits/rejected": -763728640.0, + "logps/chosen": -215.31341552734375, + "logps/rejected": -249.522705078125, + "loss": 0.3877, + "rewards/chosen": 0.6071091294288635, + "rewards/margins": 0.3096790115038554, + "rewards/rejected": 0.2974301179250081, + "step": 395 + }, + { + "epoch": 0.6669473684210526, + "grad_norm": 53.142787819676414, + "kl": 0.0, + "learning_rate": 1.5263321117230655e-07, + "logits/chosen": -674901376.0, + "logits/rejected": -1003873728.0, + "logps/chosen": -160.89675903320312, + "logps/rejected": -298.0811767578125, + "loss": 0.2986, + "rewards/chosen": 0.6914758682250977, + "rewards/margins": 3.5794568061828613, + "rewards/rejected": -2.8879809379577637, + "step": 396 + }, + { + "epoch": 0.6686315789473685, + "grad_norm": 48.05043349132663, + "kl": 2.46405029296875, + "learning_rate": 1.512802534073522e-07, + "logits/chosen": -578873856.0, + "logits/rejected": -697750656.0, + "logps/chosen": -232.189501953125, + "logps/rejected": -211.70320638020834, + "loss": 0.367, + "rewards/chosen": 1.3567285537719727, + "rewards/margins": 1.517380674680074, + "rewards/rejected": -0.1606521209081014, + "step": 397 + }, + { + "epoch": 0.6703157894736842, + "grad_norm": 62.66713109876527, + "kl": 0.0, + "learning_rate": 1.4993071244595534e-07, + "logits/chosen": -1050684544.0, + "logits/rejected": -831061440.0, + "logps/chosen": -178.69546508789062, + "logps/rejected": -267.62298583984375, + "loss": 0.4321, + "rewards/chosen": -0.0750776082277298, + "rewards/margins": 1.4986597746610641, + "rewards/rejected": -1.573737382888794, + "step": 398 + }, + { + "epoch": 0.672, + "grad_norm": 31.500731382305798, + "kl": 0.0, + "learning_rate": 1.4858463499727507e-07, + "logits/chosen": -985857280.0, + "logits/rejected": -838302720.0, + "logps/chosen": -201.73216247558594, + "logps/rejected": -263.0357666015625, + "loss": 0.3035, + "rewards/chosen": 1.0928525924682617, + "rewards/margins": 3.4981284141540527, + "rewards/rejected": -2.405275821685791, + "step": 399 + }, + { + "epoch": 0.6736842105263158, + "grad_norm": 56.975473997069834, + "kl": 0.0, + "learning_rate": 1.4724206765059454e-07, + "logits/chosen": -706666624.0, + "logits/rejected": -940830976.0, + "logps/chosen": -226.00015258789062, + "logps/rejected": -203.8423868815104, + "loss": 0.4627, + "rewards/chosen": 0.5999778509140015, + "rewards/margins": -0.3421023289362589, + "rewards/rejected": 0.9420801798502604, + "step": 400 + }, + { + "epoch": 0.6753684210526316, + "grad_norm": 38.51367993065121, + "kl": 0.0, + "learning_rate": 1.459030568737081e-07, + "logits/chosen": -688410496.0, + "logits/rejected": -619393088.0, + "logps/chosen": -228.07568359375, + "logps/rejected": -262.47265625, + "loss": 0.3821, + "rewards/chosen": -0.5633602738380432, + "rewards/margins": -0.32700619101524353, + "rewards/rejected": -0.23635408282279968, + "step": 401 + }, + { + "epoch": 0.6770526315789474, + "grad_norm": 48.63664615883306, + "kl": 0.0, + "learning_rate": 1.4456764901131308e-07, + "logits/chosen": -785745664.0, + "logits/rejected": -527159765.3333333, + "logps/chosen": -361.95355224609375, + "logps/rejected": -176.71563720703125, + "loss": 0.4017, + "rewards/chosen": 1.7527008056640625, + "rewards/margins": 5.218010902404785, + "rewards/rejected": -3.4653100967407227, + "step": 402 + }, + { + "epoch": 0.6787368421052632, + "grad_norm": 42.63233832130583, + "kl": 0.0, + "learning_rate": 1.4323589028340597e-07, + "logits/chosen": -542379605.3333334, + "logits/rejected": -1274020224.0, + "logps/chosen": -244.51863606770834, + "logps/rejected": -148.19590759277344, + "loss": 0.3506, + "rewards/chosen": 1.1265826225280762, + "rewards/margins": 0.32805508375167847, + "rewards/rejected": 0.7985275387763977, + "step": 403 + }, + { + "epoch": 0.6804210526315789, + "grad_norm": 61.18182583064187, + "kl": 0.0, + "learning_rate": 1.4190782678368257e-07, + "logits/chosen": -793320960.0, + "logits/rejected": -901524684.8, + "logps/chosen": -240.1772664388021, + "logps/rejected": -349.7687255859375, + "loss": 0.3981, + "rewards/chosen": 0.828582763671875, + "rewards/margins": 1.3413378715515136, + "rewards/rejected": -0.5127551078796386, + "step": 404 + }, + { + "epoch": 0.6821052631578948, + "grad_norm": 44.427082645231216, + "kl": 0.0, + "learning_rate": 1.4058350447794235e-07, + "logits/chosen": -734803200.0, + "logits/rejected": -947461184.0, + "logps/chosen": -209.5147705078125, + "logps/rejected": -400.060302734375, + "loss": 0.4264, + "rewards/chosen": 0.879941463470459, + "rewards/margins": 15.826271533966064, + "rewards/rejected": -14.946330070495605, + "step": 405 + }, + { + "epoch": 0.6837894736842105, + "grad_norm": 33.804563029238984, + "kl": 0.0, + "learning_rate": 1.3926296920249793e-07, + "logits/chosen": -536416288.0, + "logits/rejected": -671906624.0, + "logps/chosen": -211.46499633789062, + "logps/rejected": -246.7613067626953, + "loss": 0.3206, + "rewards/chosen": 1.6434662342071533, + "rewards/margins": 5.484874725341797, + "rewards/rejected": -3.8414084911346436, + "step": 406 + }, + { + "epoch": 0.6854736842105263, + "grad_norm": 41.68975540862706, + "kl": 0.0, + "learning_rate": 1.3794626666258866e-07, + "logits/chosen": -946841600.0, + "logits/rejected": -811282005.3333334, + "logps/chosen": -278.705859375, + "logps/rejected": -203.1152547200521, + "loss": 0.4434, + "rewards/chosen": -1.5871347427368163, + "rewards/margins": -2.073893650372823, + "rewards/rejected": 0.48675890763600665, + "step": 407 + }, + { + "epoch": 0.6871578947368421, + "grad_norm": 37.54233790104561, + "kl": 0.0, + "learning_rate": 1.3663344243079806e-07, + "logits/chosen": -1188021120.0, + "logits/rejected": -819954395.4285715, + "logps/chosen": -194.07516479492188, + "logps/rejected": -175.0321044921875, + "loss": 0.3924, + "rewards/chosen": 2.3552887439727783, + "rewards/margins": 1.9132578032357352, + "rewards/rejected": 0.44203094073704313, + "step": 408 + }, + { + "epoch": 0.6888421052631579, + "grad_norm": 49.55293923346674, + "kl": 0.0, + "learning_rate": 1.3532454194547732e-07, + "logits/chosen": -869699072.0, + "logits/rejected": -755703552.0, + "logps/chosen": -211.62156677246094, + "logps/rejected": -255.9964141845703, + "loss": 0.2897, + "rewards/chosen": 2.3169875144958496, + "rewards/margins": 3.9210219383239746, + "rewards/rejected": -1.604034423828125, + "step": 409 + }, + { + "epoch": 0.6905263157894737, + "grad_norm": 45.07517151760579, + "kl": 0.0, + "learning_rate": 1.3401961050917228e-07, + "logits/chosen": -1723337728.0, + "logits/rejected": -646925004.8, + "logps/chosen": -234.47102864583334, + "logps/rejected": -205.66240234375, + "loss": 0.3623, + "rewards/chosen": 0.6976583003997803, + "rewards/margins": 0.9717099666595459, + "rewards/rejected": -0.2740516662597656, + "step": 410 + }, + { + "epoch": 0.6922105263157895, + "grad_norm": 41.62482913947959, + "kl": 0.0, + "learning_rate": 1.3271869328705516e-07, + "logits/chosen": -539952554.6666666, + "logits/rejected": -892416409.6, + "logps/chosen": -198.6785888671875, + "logps/rejected": -201.31024169921875, + "loss": 0.3754, + "rewards/chosen": 1.5827067693074544, + "rewards/margins": 3.3067630132039385, + "rewards/rejected": -1.7240562438964844, + "step": 411 + }, + { + "epoch": 0.6938947368421052, + "grad_norm": 42.26855513911886, + "kl": 0.0, + "learning_rate": 1.314218353053619e-07, + "logits/chosen": -660784768.0, + "logits/rejected": -657162342.4, + "logps/chosen": -219.67671712239584, + "logps/rejected": -248.3271484375, + "loss": 0.3954, + "rewards/chosen": 0.8856130441029867, + "rewards/margins": 2.3070303757985435, + "rewards/rejected": -1.4214173316955567, + "step": 412 + }, + { + "epoch": 0.6955789473684211, + "grad_norm": 63.372635671964915, + "kl": 0.0, + "learning_rate": 1.301290814498335e-07, + "logits/chosen": -682650026.6666666, + "logits/rejected": -750546227.2, + "logps/chosen": -216.4493408203125, + "logps/rejected": -230.68525390625, + "loss": 0.3873, + "rewards/chosen": 2.892109235127767, + "rewards/margins": 3.49957750638326, + "rewards/rejected": -0.6074682712554932, + "step": 413 + }, + { + "epoch": 0.6972631578947368, + "grad_norm": 31.557969726141994, + "kl": 0.0, + "learning_rate": 1.2884047646416205e-07, + "logits/chosen": -455988778.6666667, + "logits/rejected": -1020206796.8, + "logps/chosen": -256.14182535807294, + "logps/rejected": -181.9263916015625, + "loss": 0.2895, + "rewards/chosen": 0.852165699005127, + "rewards/margins": 0.872660094499588, + "rewards/rejected": -0.02049439549446106, + "step": 414 + }, + { + "epoch": 0.6989473684210527, + "grad_norm": 44.99525550241327, + "kl": 0.0, + "learning_rate": 1.2755606494844294e-07, + "logits/chosen": -619836032.0, + "logits/rejected": -689233024.0, + "logps/chosen": -243.01300048828125, + "logps/rejected": -235.70297241210938, + "loss": 0.3066, + "rewards/chosen": 1.5389785766601562, + "rewards/margins": 2.8424285650253296, + "rewards/rejected": -1.3034499883651733, + "step": 415 + }, + { + "epoch": 0.7006315789473684, + "grad_norm": 46.608977034452934, + "kl": 0.0, + "learning_rate": 1.2627589135763066e-07, + "logits/chosen": -622393664.0, + "logits/rejected": -588082816.0, + "logps/chosen": -263.4278869628906, + "logps/rejected": -237.45460510253906, + "loss": 0.3805, + "rewards/chosen": 0.9929741621017456, + "rewards/margins": 2.5076760053634644, + "rewards/rejected": -1.5147018432617188, + "step": 416 + }, + { + "epoch": 0.7023157894736842, + "grad_norm": 45.15173225020037, + "kl": 0.0, + "learning_rate": 1.2500000000000005e-07, + "logits/chosen": -290351680.0, + "logits/rejected": -570943590.4, + "logps/chosen": -210.52325439453125, + "logps/rejected": -230.669921875, + "loss": 0.3545, + "rewards/chosen": 1.303624947865804, + "rewards/margins": 1.4630508740743, + "rewards/rejected": -0.1594259262084961, + "step": 417 + }, + { + "epoch": 0.704, + "grad_norm": 49.827214101384186, + "kl": 0.0, + "learning_rate": 1.2372843503561318e-07, + "logits/chosen": -860238028.8, + "logits/rejected": -1133508778.6666667, + "logps/chosen": -212.53994140625, + "logps/rejected": -191.24481201171875, + "loss": 0.3653, + "rewards/chosen": 1.4028396606445312, + "rewards/margins": 1.0293253660202026, + "rewards/rejected": 0.3735142946243286, + "step": 418 + }, + { + "epoch": 0.7056842105263158, + "grad_norm": 41.39161745285879, + "kl": 0.0, + "learning_rate": 1.2246124047479074e-07, + "logits/chosen": -1557299200.0, + "logits/rejected": -1170218752.0, + "logps/chosen": -264.32098388671875, + "logps/rejected": -288.19140625, + "loss": 0.2715, + "rewards/chosen": -0.8450756072998047, + "rewards/margins": 5.187082767486572, + "rewards/rejected": -6.032158374786377, + "step": 419 + }, + { + "epoch": 0.7073684210526315, + "grad_norm": 55.33615495444503, + "kl": 0.0, + "learning_rate": 1.211984601765884e-07, + "logits/chosen": -1000780970.6666666, + "logits/rejected": -1060321177.6, + "logps/chosen": -244.44415283203125, + "logps/rejected": -210.060546875, + "loss": 0.4162, + "rewards/chosen": 0.8905111153920492, + "rewards/margins": 1.1921974500020345, + "rewards/rejected": -0.30168633460998534, + "step": 420 + }, + { + "epoch": 0.7090526315789474, + "grad_norm": 48.50467903033437, + "kl": 0.0, + "learning_rate": 1.1994013784727946e-07, + "logits/chosen": -798935961.6, + "logits/rejected": -556915968.0, + "logps/chosen": -193.3767333984375, + "logps/rejected": -204.73103841145834, + "loss": 0.3481, + "rewards/chosen": 1.7989086151123046, + "rewards/margins": 0.6901429335276286, + "rewards/rejected": 1.108765681584676, + "step": 421 + }, + { + "epoch": 0.7107368421052631, + "grad_norm": 43.771933039570335, + "kl": 0.0, + "learning_rate": 1.1868631703884183e-07, + "logits/chosen": -770181836.8, + "logits/rejected": -566833578.6666666, + "logps/chosen": -239.477392578125, + "logps/rejected": -194.196533203125, + "loss": 0.3706, + "rewards/chosen": 0.2928351402282715, + "rewards/margins": -1.4519967397054034, + "rewards/rejected": 1.744831879933675, + "step": 422 + }, + { + "epoch": 0.712421052631579, + "grad_norm": 52.317703020939184, + "kl": 0.0, + "learning_rate": 1.1743704114745029e-07, + "logits/chosen": -1021569408.0, + "logits/rejected": -1079399082.6666667, + "logps/chosen": -271.7890625, + "logps/rejected": -211.10260009765625, + "loss": 0.3508, + "rewards/chosen": 1.666670322418213, + "rewards/margins": 3.195549964904785, + "rewards/rejected": -1.5288796424865723, + "step": 423 + }, + { + "epoch": 0.7141052631578947, + "grad_norm": 44.73164989384191, + "kl": 0.0, + "learning_rate": 1.1619235341197517e-07, + "logits/chosen": -833912896.0, + "logits/rejected": -611463552.0, + "logps/chosen": -261.9710693359375, + "logps/rejected": -261.09979248046875, + "loss": 0.4409, + "rewards/chosen": -0.13229060173034668, + "rewards/margins": 1.3757647673288982, + "rewards/rejected": -1.5080553690592449, + "step": 424 + }, + { + "epoch": 0.7157894736842105, + "grad_norm": 43.20707344363169, + "kl": 0.0, + "learning_rate": 1.1495229691248543e-07, + "logits/chosen": -771930453.3333334, + "logits/rejected": -296132800.0, + "logps/chosen": -194.66646321614584, + "logps/rejected": -196.60357666015625, + "loss": 0.3797, + "rewards/chosen": 0.4760231177012126, + "rewards/margins": -0.01816942294438678, + "rewards/rejected": 0.49419254064559937, + "step": 425 + }, + { + "epoch": 0.7174736842105263, + "grad_norm": 37.98854463213142, + "kl": 0.0, + "learning_rate": 1.1371691456875735e-07, + "logits/chosen": -1023635285.3333334, + "logits/rejected": -639436902.4, + "logps/chosen": -228.6927693684896, + "logps/rejected": -203.55322265625, + "loss": 0.4071, + "rewards/chosen": 3.219998041788737, + "rewards/margins": 3.7488079706827797, + "rewards/rejected": -0.528809928894043, + "step": 426 + }, + { + "epoch": 0.7191578947368421, + "grad_norm": 38.6260673498228, + "kl": 0.0, + "learning_rate": 1.1248624913878965e-07, + "logits/chosen": -868456618.6666666, + "logits/rejected": -591238860.8, + "logps/chosen": -246.4551798502604, + "logps/rejected": -202.9473876953125, + "loss": 0.4088, + "rewards/chosen": 2.625763416290283, + "rewards/margins": 2.438036632537842, + "rewards/rejected": 0.1877267837524414, + "step": 427 + }, + { + "epoch": 0.7208421052631578, + "grad_norm": 42.32410569595673, + "kl": 0.0, + "learning_rate": 1.1126034321732325e-07, + "logits/chosen": -789433344.0, + "logits/rejected": -534869043.2, + "logps/chosen": -241.09903971354166, + "logps/rejected": -267.5244384765625, + "loss": 0.3987, + "rewards/chosen": -0.5567977428436279, + "rewards/margins": -1.7331472873687743, + "rewards/rejected": 1.1763495445251464, + "step": 428 + }, + { + "epoch": 0.7225263157894737, + "grad_norm": 33.14828016562309, + "kl": 0.0, + "learning_rate": 1.100392392343667e-07, + "logits/chosen": -811491712.0, + "logits/rejected": -881575552.0, + "logps/chosen": -205.48550415039062, + "logps/rejected": -208.66879272460938, + "loss": 0.3453, + "rewards/chosen": 1.3217136859893799, + "rewards/margins": 3.224560260772705, + "rewards/rejected": -1.9028465747833252, + "step": 429 + }, + { + "epoch": 0.7242105263157895, + "grad_norm": 45.216833187005214, + "kl": 0.0, + "learning_rate": 1.0882297945372828e-07, + "logits/chosen": -879807385.6, + "logits/rejected": -1048175104.0, + "logps/chosen": -217.96474609375, + "logps/rejected": -228.4263916015625, + "loss": 0.2549, + "rewards/chosen": 2.451664924621582, + "rewards/margins": 4.334318161010742, + "rewards/rejected": -1.8826532363891602, + "step": 430 + }, + { + "epoch": 0.7258947368421053, + "grad_norm": 34.528325072499406, + "kl": 0.0, + "learning_rate": 1.0761160597155286e-07, + "logits/chosen": -767435264.0, + "logits/rejected": -604252245.3333334, + "logps/chosen": -238.663525390625, + "logps/rejected": -211.9125773111979, + "loss": 0.4655, + "rewards/chosen": 0.8303826332092286, + "rewards/margins": 3.884237257639567, + "rewards/rejected": -3.0538546244303384, + "step": 431 + }, + { + "epoch": 0.7275789473684211, + "grad_norm": 71.46182969123905, + "kl": 0.0, + "learning_rate": 1.0640516071486466e-07, + "logits/chosen": -645191936.0, + "logits/rejected": -718282432.0, + "logps/chosen": -243.2042236328125, + "logps/rejected": -196.67498779296875, + "loss": 0.484, + "rewards/chosen": 0.9913086295127869, + "rewards/margins": 0.5928112268447876, + "rewards/rejected": 0.39849740266799927, + "step": 432 + }, + { + "epoch": 0.7292631578947368, + "grad_norm": 37.01038450064629, + "kl": 0.0, + "learning_rate": 1.052036854401166e-07, + "logits/chosen": -916742485.3333334, + "logits/rejected": -1014475161.6, + "logps/chosen": -251.3818359375, + "logps/rejected": -294.1171630859375, + "loss": 0.3733, + "rewards/chosen": 2.161475658416748, + "rewards/margins": 5.233707332611084, + "rewards/rejected": -3.072231674194336, + "step": 433 + }, + { + "epoch": 0.7309473684210527, + "grad_norm": 47.994445942830765, + "kl": 0.0, + "learning_rate": 1.0400722173174489e-07, + "logits/chosen": -653794218.6666666, + "logits/rejected": -922273920.0, + "logps/chosen": -217.94051106770834, + "logps/rejected": -335.4775390625, + "loss": 0.3971, + "rewards/chosen": 1.1465115547180176, + "rewards/margins": 9.679410457611084, + "rewards/rejected": -8.532898902893066, + "step": 434 + }, + { + "epoch": 0.7326315789473684, + "grad_norm": 31.242483556300705, + "kl": 0.0, + "learning_rate": 1.0281581100072939e-07, + "logits/chosen": -1153430144.0, + "logits/rejected": -658751168.0, + "logps/chosen": -223.9492950439453, + "logps/rejected": -250.08029174804688, + "loss": 0.3316, + "rewards/chosen": -0.05509984493255615, + "rewards/margins": 2.957166314125061, + "rewards/rejected": -3.012266159057617, + "step": 435 + }, + { + "epoch": 0.7343157894736843, + "grad_norm": 44.882205337938544, + "kl": 0.0, + "learning_rate": 1.0162949448316088e-07, + "logits/chosen": -1179825561.6, + "logits/rejected": -795787349.3333334, + "logps/chosen": -184.4357421875, + "logps/rejected": -319.9234619140625, + "loss": 0.4252, + "rewards/chosen": 1.6069551467895509, + "rewards/margins": 3.3702004114786783, + "rewards/rejected": -1.7632452646891277, + "step": 436 + }, + { + "epoch": 0.736, + "grad_norm": 53.02185490371656, + "kl": 0.0, + "learning_rate": 1.0044831323881357e-07, + "logits/chosen": -786681753.6, + "logits/rejected": -730507520.0, + "logps/chosen": -197.95892333984375, + "logps/rejected": -195.9046630859375, + "loss": 0.3626, + "rewards/chosen": 0.8055712699890136, + "rewards/margins": 2.56037015914917, + "rewards/rejected": -1.7547988891601562, + "step": 437 + }, + { + "epoch": 0.7376842105263158, + "grad_norm": 41.26140691549541, + "kl": 0.0, + "learning_rate": 9.927230814972382e-08, + "logits/chosen": -1242336402.2857144, + "logits/rejected": -715042368.0, + "logps/chosen": -253.29446847098214, + "logps/rejected": -256.8112487792969, + "loss": 0.4456, + "rewards/chosen": 1.5902636391775948, + "rewards/margins": 5.37732127734593, + "rewards/rejected": -3.787057638168335, + "step": 438 + }, + { + "epoch": 0.7393684210526316, + "grad_norm": 39.11479149303325, + "kl": 0.0, + "learning_rate": 9.81015199187753e-08, + "logits/chosen": -1125344256.0, + "logits/rejected": -827617536.0, + "logps/chosen": -162.90849812825522, + "logps/rejected": -385.02578125, + "loss": 0.2965, + "rewards/chosen": 2.1140575408935547, + "rewards/margins": 4.867137718200683, + "rewards/rejected": -2.7530801773071287, + "step": 439 + }, + { + "epoch": 0.7410526315789474, + "grad_norm": 42.334848723588486, + "kl": 0.0, + "learning_rate": 9.693598906829046e-08, + "logits/chosen": -1135330304.0, + "logits/rejected": -764287616.0, + "logps/chosen": -208.2585652669271, + "logps/rejected": -316.421875, + "loss": 0.4066, + "rewards/chosen": 1.152496576309204, + "rewards/margins": 4.668377876281738, + "rewards/rejected": -3.515881299972534, + "step": 440 + }, + { + "epoch": 0.7427368421052631, + "grad_norm": 41.258031701194476, + "kl": 0.0, + "learning_rate": 9.577575593862775e-08, + "logits/chosen": -715377024.0, + "logits/rejected": -1129872384.0, + "logps/chosen": -234.7706298828125, + "logps/rejected": -306.43499755859375, + "loss": 0.3972, + "rewards/chosen": 1.0637226104736328, + "rewards/margins": 2.9347736835479736, + "rewards/rejected": -1.8710510730743408, + "step": 441 + }, + { + "epoch": 0.744421052631579, + "grad_norm": 54.07697190099627, + "kl": 0.0, + "learning_rate": 9.462086068678519e-08, + "logits/chosen": -766640000.0, + "logits/rejected": -959083136.0, + "logps/chosen": -211.8131103515625, + "logps/rejected": -283.1666259765625, + "loss": 0.401, + "rewards/chosen": -0.5408859252929688, + "rewards/margins": 1.790869951248169, + "rewards/rejected": -2.3317558765411377, + "step": 442 + }, + { + "epoch": 0.7461052631578947, + "grad_norm": 66.5169402578717, + "kl": 0.0, + "learning_rate": 9.347134328501097e-08, + "logits/chosen": -653176448.0, + "logits/rejected": -1222430976.0, + "logps/chosen": -176.54693603515625, + "logps/rejected": -252.2942657470703, + "loss": 0.3954, + "rewards/chosen": -0.43219953775405884, + "rewards/margins": 0.7258930802345276, + "rewards/rejected": -1.1580926179885864, + "step": 443 + }, + { + "epoch": 0.7477894736842106, + "grad_norm": 59.0005118657837, + "kl": 0.0, + "learning_rate": 9.232724351941978e-08, + "logits/chosen": -756462080.0, + "logits/rejected": -722251946.6666666, + "logps/chosen": -305.7142333984375, + "logps/rejected": -234.7782999674479, + "loss": 0.3794, + "rewards/chosen": 0.13024139404296875, + "rewards/margins": 0.5876843929290771, + "rewards/rejected": -0.4574429988861084, + "step": 444 + }, + { + "epoch": 0.7494736842105263, + "grad_norm": 55.62210680654247, + "kl": 0.0, + "learning_rate": 9.118860098861537e-08, + "logits/chosen": -813152153.6, + "logits/rejected": -290959360.0, + "logps/chosen": -197.15120849609374, + "logps/rejected": -282.9163004557292, + "loss": 0.4162, + "rewards/chosen": 0.6745959281921386, + "rewards/margins": 1.0792610804239908, + "rewards/rejected": -0.40466515223185223, + "step": 445 + }, + { + "epoch": 0.7511578947368421, + "grad_norm": 358.03078972226217, + "kl": 0.0, + "learning_rate": 9.005545510232068e-08, + "logits/chosen": -854770614.8571428, + "logits/rejected": -522669440.0, + "logps/chosen": -227.82121930803572, + "logps/rejected": -329.3843994140625, + "loss": 0.2992, + "rewards/chosen": 1.0909557342529297, + "rewards/margins": 7.157499313354492, + "rewards/rejected": -6.0665435791015625, + "step": 446 + }, + { + "epoch": 0.7528421052631579, + "grad_norm": 40.23280955297014, + "kl": 0.0, + "learning_rate": 8.892784508001341e-08, + "logits/chosen": -665233920.0, + "logits/rejected": -786437529.6, + "logps/chosen": -205.84651692708334, + "logps/rejected": -208.0626220703125, + "loss": 0.3587, + "rewards/chosen": 3.913280804951986, + "rewards/margins": 4.716941197713217, + "rewards/rejected": -0.8036603927612305, + "step": 447 + }, + { + "epoch": 0.7545263157894737, + "grad_norm": 331.3992420977293, + "kl": 0.0, + "learning_rate": 8.780580994956849e-08, + "logits/chosen": -991909824.0, + "logits/rejected": -1036429738.6666666, + "logps/chosen": -241.2684326171875, + "logps/rejected": -222.25858561197916, + "loss": 0.3923, + "rewards/chosen": 0.7948417663574219, + "rewards/margins": 0.6548532644907633, + "rewards/rejected": 0.13998850186665854, + "step": 448 + }, + { + "epoch": 0.7562105263157894, + "grad_norm": 33.19053977720104, + "kl": 0.0, + "learning_rate": 8.668938854590763e-08, + "logits/chosen": -644909875.2, + "logits/rejected": -695736832.0, + "logps/chosen": -230.962060546875, + "logps/rejected": -194.96405029296875, + "loss": 0.4064, + "rewards/chosen": 2.0331790924072264, + "rewards/margins": -0.12323621114095085, + "rewards/rejected": 2.1564153035481772, + "step": 449 + }, + { + "epoch": 0.7578947368421053, + "grad_norm": 45.92452873664749, + "kl": 0.0, + "learning_rate": 8.557861950965509e-08, + "logits/chosen": -1397461930.6666667, + "logits/rejected": -905682124.8, + "logps/chosen": -271.2666422526042, + "logps/rejected": -288.8114013671875, + "loss": 0.3724, + "rewards/chosen": -0.3505045572916667, + "rewards/margins": 2.9490411122639975, + "rewards/rejected": -3.299545669555664, + "step": 450 + }, + { + "epoch": 0.759578947368421, + "grad_norm": 40.248447099852605, + "kl": 0.0, + "learning_rate": 8.44735412857999e-08, + "logits/chosen": -1039405504.0, + "logits/rejected": -620940032.0, + "logps/chosen": -259.6157531738281, + "logps/rejected": -244.8777618408203, + "loss": 0.3763, + "rewards/chosen": 0.6175476312637329, + "rewards/margins": 3.04443895816803, + "rewards/rejected": -2.426891326904297, + "step": 451 + }, + { + "epoch": 0.7612631578947369, + "grad_norm": 37.694918257120385, + "kl": 0.0, + "learning_rate": 8.337419212236584e-08, + "logits/chosen": -523443744.0, + "logits/rejected": -701745856.0, + "logps/chosen": -225.3424072265625, + "logps/rejected": -293.00054931640625, + "loss": 0.3628, + "rewards/chosen": 2.430769443511963, + "rewards/margins": 3.699450373649597, + "rewards/rejected": -1.2686809301376343, + "step": 452 + }, + { + "epoch": 0.7629473684210526, + "grad_norm": 35.15022250675745, + "kl": 0.0, + "learning_rate": 8.228061006908738e-08, + "logits/chosen": -789152448.0, + "logits/rejected": -639658240.0, + "logps/chosen": -204.5469207763672, + "logps/rejected": -296.8457336425781, + "loss": 0.3558, + "rewards/chosen": 0.2700985074043274, + "rewards/margins": 10.158620655536652, + "rewards/rejected": -9.888522148132324, + "step": 453 + }, + { + "epoch": 0.7646315789473684, + "grad_norm": 47.5088310425883, + "kl": 0.0, + "learning_rate": 8.119283297609236e-08, + "logits/chosen": -1028607744.0, + "logits/rejected": -645622656.0, + "logps/chosen": -289.964111328125, + "logps/rejected": -224.54290771484375, + "loss": 0.3183, + "rewards/chosen": 0.8806011080741882, + "rewards/margins": 1.526337444782257, + "rewards/rejected": -0.6457363367080688, + "step": 454 + }, + { + "epoch": 0.7663157894736842, + "grad_norm": 34.441433884082336, + "kl": 0.0, + "learning_rate": 8.011089849259262e-08, + "logits/chosen": -749322752.0, + "logps/chosen": -202.4013671875, + "loss": 0.3446, + "rewards/chosen": 0.888677179813385, + "step": 455 + }, + { + "epoch": 0.768, + "grad_norm": 43.86036582680801, + "kl": 0.0, + "learning_rate": 7.903484406558053e-08, + "logits/chosen": -536868650.6666667, + "logits/rejected": -616575808.0, + "logps/chosen": -276.9130045572917, + "logps/rejected": -232.73355102539062, + "loss": 0.43, + "rewards/chosen": -0.4769521156946818, + "rewards/margins": 0.35738088687260944, + "rewards/rejected": -0.8343330025672913, + "step": 456 + }, + { + "epoch": 0.7696842105263157, + "grad_norm": 42.12234531239963, + "kl": 0.0, + "learning_rate": 7.79647069385328e-08, + "logits/chosen": -752578133.3333334, + "logits/rejected": -934050816.0, + "logps/chosen": -224.02742513020834, + "logps/rejected": -277.469970703125, + "loss": 0.4204, + "rewards/chosen": -0.5293665726979574, + "rewards/margins": -1.098210604985555, + "rewards/rejected": 0.5688440322875976, + "step": 457 + }, + { + "epoch": 0.7713684210526316, + "grad_norm": 42.59646325267393, + "kl": 0.0, + "learning_rate": 7.690052415012174e-08, + "logits/chosen": -938434252.8, + "logits/rejected": -1136305237.3333333, + "logps/chosen": -216.8803466796875, + "logps/rejected": -190.39237467447916, + "loss": 0.2499, + "rewards/chosen": 1.1629043579101563, + "rewards/margins": 4.905188814798991, + "rewards/rejected": -3.7422844568888345, + "step": 458 + }, + { + "epoch": 0.7730526315789473, + "grad_norm": 52.399177497226496, + "kl": 0.0, + "learning_rate": 7.584233253293326e-08, + "logits/chosen": -716863027.2, + "logits/rejected": -816465237.3333334, + "logps/chosen": -243.8283203125, + "logps/rejected": -196.03934733072916, + "loss": 0.2746, + "rewards/chosen": 1.0614694595336913, + "rewards/margins": 2.684300168355306, + "rewards/rejected": -1.6228307088216145, + "step": 459 + }, + { + "epoch": 0.7747368421052632, + "grad_norm": 52.891481922508966, + "kl": 0.0, + "learning_rate": 7.479016871219173e-08, + "logits/chosen": -917309440.0, + "logits/rejected": -898528870.4, + "logps/chosen": -282.1151936848958, + "logps/rejected": -213.7822998046875, + "loss": 0.4074, + "rewards/chosen": -0.2861933708190918, + "rewards/margins": 1.0193795204162597, + "rewards/rejected": -1.3055728912353515, + "step": 460 + }, + { + "epoch": 0.7764210526315789, + "grad_norm": 35.541679693506616, + "kl": 0.0, + "learning_rate": 7.374406910449276e-08, + "logits/chosen": -625601749.3333334, + "logits/rejected": -777565235.2, + "logps/chosen": -214.69976806640625, + "logps/rejected": -227.8533447265625, + "loss": 0.3537, + "rewards/chosen": 0.7599695523579916, + "rewards/margins": 2.2433162053426106, + "rewards/rejected": -1.483346652984619, + "step": 461 + }, + { + "epoch": 0.7781052631578947, + "grad_norm": 63.418725969793606, + "kl": 0.0, + "learning_rate": 7.270406991654273e-08, + "logits/chosen": -963097258.6666666, + "logits/rejected": -1547470848.0, + "logps/chosen": -263.2251383463542, + "logps/rejected": -274.23115234375, + "loss": 0.425, + "rewards/chosen": 0.2878738244374593, + "rewards/margins": 1.6172294457753498, + "rewards/rejected": -1.3293556213378905, + "step": 462 + }, + { + "epoch": 0.7797894736842105, + "grad_norm": 38.377982473660865, + "kl": 0.0, + "learning_rate": 7.167020714390501e-08, + "logits/chosen": -783421030.4, + "logits/rejected": -488972885.3333333, + "logps/chosen": -215.7775390625, + "logps/rejected": -179.737548828125, + "loss": 0.436, + "rewards/chosen": 0.9828885078430176, + "rewards/margins": 0.3364712874094645, + "rewards/rejected": 0.6464172204335531, + "step": 463 + }, + { + "epoch": 0.7814736842105263, + "grad_norm": 45.40428262520681, + "kl": 0.0, + "learning_rate": 7.064251656975503e-08, + "logits/chosen": -917309888.0, + "logits/rejected": -730922069.3333334, + "logps/chosen": -207.1556396484375, + "logps/rejected": -247.15775553385416, + "loss": 0.361, + "rewards/chosen": 1.675153374671936, + "rewards/margins": 3.9481043418248496, + "rewards/rejected": -2.2729509671529136, + "step": 464 + }, + { + "epoch": 0.783157894736842, + "grad_norm": 55.99199517489128, + "kl": 0.0, + "learning_rate": 6.96210337636414e-08, + "logits/chosen": -1108944512.0, + "logits/rejected": -576058944.0, + "logps/chosen": -289.2898864746094, + "logps/rejected": -202.31178283691406, + "loss": 0.3961, + "rewards/chosen": -0.21090269088745117, + "rewards/margins": -0.2718646228313446, + "rewards/rejected": 0.06096193194389343, + "step": 465 + }, + { + "epoch": 0.7848421052631579, + "grad_norm": 49.25923339661157, + "kl": 0.0, + "learning_rate": 6.860579408025435e-08, + "logits/chosen": -814812672.0, + "logits/rejected": -933722880.0, + "logps/chosen": -191.00594075520834, + "logps/rejected": -235.2879638671875, + "loss": 0.3267, + "rewards/chosen": 1.0561386744181316, + "rewards/margins": 2.257707277933757, + "rewards/rejected": -1.201568603515625, + "step": 466 + }, + { + "epoch": 0.7865263157894736, + "grad_norm": 52.73809432153172, + "kl": 0.0, + "learning_rate": 6.759683265820293e-08, + "logits/chosen": -749964800.0, + "logits/rejected": -1379509504.0, + "logps/chosen": -182.05026245117188, + "logps/rejected": -213.2086639404297, + "loss": 0.4184, + "rewards/chosen": 1.8726093769073486, + "rewards/margins": 3.3285834789276123, + "rewards/rejected": -1.4559741020202637, + "step": 467 + }, + { + "epoch": 0.7882105263157895, + "grad_norm": 55.06684904272287, + "kl": 0.0, + "learning_rate": 6.659418441879816e-08, + "logits/chosen": -886685760.0, + "logits/rejected": -867823936.0, + "logps/chosen": -242.12347412109375, + "logps/rejected": -234.9542236328125, + "loss": 0.4054, + "rewards/chosen": 0.7248154282569885, + "rewards/margins": 0.6286347508430481, + "rewards/rejected": 0.09618067741394043, + "step": 468 + }, + { + "epoch": 0.7898947368421053, + "grad_norm": 41.061950624386746, + "kl": 0.0, + "learning_rate": 6.559788406484445e-08, + "logits/chosen": -1067656874.6666666, + "logits/rejected": -635095756.8, + "logps/chosen": -228.18094889322916, + "logps/rejected": -251.1126708984375, + "loss": 0.3713, + "rewards/chosen": 2.0004892349243164, + "rewards/margins": 4.07933406829834, + "rewards/rejected": -2.0788448333740233, + "step": 469 + }, + { + "epoch": 0.791578947368421, + "grad_norm": 55.92018991052814, + "kl": 0.0, + "learning_rate": 6.460796607943888e-08, + "logits/chosen": -1008979114.6666666, + "logits/rejected": -580155904.0, + "logps/chosen": -220.10860188802084, + "logps/rejected": -200.3806640625, + "loss": 0.3572, + "rewards/chosen": 0.6659164428710938, + "rewards/margins": 2.2499670028686523, + "rewards/rejected": -1.5840505599975585, + "step": 470 + }, + { + "epoch": 0.7932631578947369, + "grad_norm": 44.05178707657523, + "kl": 0.0, + "learning_rate": 6.362446472477739e-08, + "logits/chosen": -886915754.6666666, + "logits/rejected": -697056153.6, + "logps/chosen": -161.08222452799478, + "logps/rejected": -195.1838623046875, + "loss": 0.3431, + "rewards/chosen": 2.043999671936035, + "rewards/margins": 1.8765502452850342, + "rewards/rejected": 0.167449426651001, + "step": 471 + }, + { + "epoch": 0.7949473684210526, + "grad_norm": 76.34832400410619, + "kl": 0.0, + "learning_rate": 6.264741404096873e-08, + "logits/chosen": -970343594.6666666, + "logits/rejected": -1028333772.8, + "logps/chosen": -240.02298990885416, + "logps/rejected": -181.0073486328125, + "loss": 0.4382, + "rewards/chosen": 1.713447093963623, + "rewards/margins": 0.9383099555969239, + "rewards/rejected": 0.7751371383666992, + "step": 472 + }, + { + "epoch": 0.7966315789473685, + "grad_norm": 53.36902208721296, + "kl": 0.0, + "learning_rate": 6.16768478448568e-08, + "logits/chosen": -770189056.0, + "logits/rejected": -784716697.6, + "logps/chosen": -198.6039021809896, + "logps/rejected": -230.421240234375, + "loss": 0.4316, + "rewards/chosen": 1.7407466570536296, + "rewards/margins": 0.8216514269510906, + "rewards/rejected": 0.919095230102539, + "step": 473 + }, + { + "epoch": 0.7983157894736842, + "grad_norm": 76.44940846297158, + "kl": 0.0, + "learning_rate": 6.071279972884996e-08, + "logits/chosen": -650762581.3333334, + "logits/rejected": -824322304.0, + "logps/chosen": -271.9395345052083, + "logps/rejected": -240.26007080078125, + "loss": 0.374, + "rewards/chosen": 0.2647084792455037, + "rewards/margins": 1.6960653861363728, + "rewards/rejected": -1.4313569068908691, + "step": 474 + }, + { + "epoch": 0.8, + "grad_norm": 38.84706884248548, + "kl": 0.0, + "learning_rate": 5.975530305975806e-08, + "logits/chosen": -953603481.6, + "logits/rejected": -874571178.6666666, + "logps/chosen": -224.2393310546875, + "logps/rejected": -248.4403076171875, + "loss": 0.3032, + "rewards/chosen": 0.2576324462890625, + "rewards/margins": 2.0944193522135417, + "rewards/rejected": -1.8367869059244792, + "step": 475 + }, + { + "epoch": 0.8016842105263158, + "grad_norm": 80.57439772482964, + "kl": 0.0, + "learning_rate": 5.88043909776382e-08, + "logits/chosen": -720325705.1428572, + "logits/rejected": -1631839360.0, + "logps/chosen": -212.89100864955358, + "logps/rejected": -245.09353637695312, + "loss": 0.3109, + "rewards/chosen": 0.8418189457484654, + "rewards/margins": 1.0114402196237018, + "rewards/rejected": -0.1696212738752365, + "step": 476 + }, + { + "epoch": 0.8033684210526316, + "grad_norm": 60.404391161081, + "kl": 0.0, + "learning_rate": 5.786009639464728e-08, + "logits/chosen": -754591168.0, + "logits/rejected": -984781056.0, + "logps/chosen": -213.29608154296875, + "logps/rejected": -191.29742431640625, + "loss": 0.3819, + "rewards/chosen": 1.0502116680145264, + "rewards/margins": 8.452488660812378, + "rewards/rejected": -7.402276992797852, + "step": 477 + }, + { + "epoch": 0.8050526315789474, + "grad_norm": 39.88427529417202, + "kl": 0.0, + "learning_rate": 5.69224519939028e-08, + "logits/chosen": -1256239872.0, + "logits/rejected": -680008704.0, + "logps/chosen": -207.2843017578125, + "logps/rejected": -281.371337890625, + "loss": 0.4186, + "rewards/chosen": 2.776592969894409, + "rewards/margins": 2.579076017652239, + "rewards/rejected": 0.19751695224217006, + "step": 478 + }, + { + "epoch": 0.8067368421052632, + "grad_norm": 46.39968561566463, + "kl": 0.0, + "learning_rate": 5.599149022835201e-08, + "logits/chosen": -757022515.2, + "logits/rejected": -568918784.0, + "logps/chosen": -190.7361572265625, + "logps/rejected": -234.748291015625, + "loss": 0.3371, + "rewards/chosen": 1.5194448471069335, + "rewards/margins": 3.035427951812744, + "rewards/rejected": -1.5159831047058105, + "step": 479 + }, + { + "epoch": 0.8084210526315789, + "grad_norm": 40.26692401415863, + "kl": 0.0, + "learning_rate": 5.5067243319648517e-08, + "logits/chosen": -628281651.2, + "logits/rejected": -631906560.0, + "logps/chosen": -231.4928466796875, + "logps/rejected": -164.00850423177084, + "loss": 0.3951, + "rewards/chosen": 0.1779022455215454, + "rewards/margins": 1.319037381807963, + "rewards/rejected": -1.1411351362864177, + "step": 480 + }, + { + "epoch": 0.8101052631578948, + "grad_norm": 58.215282154133604, + "kl": 0.0, + "learning_rate": 5.414974325703686e-08, + "logits/chosen": -691516928.0, + "logits/rejected": -898130624.0, + "logps/chosen": -199.2517852783203, + "logps/rejected": -193.02496337890625, + "loss": 0.4195, + "rewards/chosen": 0.9336433410644531, + "rewards/margins": 0.8423346877098083, + "rewards/rejected": 0.09130865335464478, + "step": 481 + }, + { + "epoch": 0.8117894736842105, + "grad_norm": 50.746031932726225, + "kl": 0.0, + "learning_rate": 5.3239021796245707e-08, + "logits/chosen": -643468480.0, + "logits/rejected": -744267861.3333334, + "logps/chosen": -202.1339111328125, + "logps/rejected": -236.04170735677084, + "loss": 0.3827, + "rewards/chosen": 0.5528717041015625, + "rewards/margins": 0.6056533952554067, + "rewards/rejected": -0.0527816911538442, + "step": 482 + }, + { + "epoch": 0.8134736842105263, + "grad_norm": 90.38953108333145, + "kl": 0.0, + "learning_rate": 5.2335110458388457e-08, + "logits/chosen": -861357653.3333334, + "logits/rejected": -1220613529.6, + "logps/chosen": -228.4935302734375, + "logps/rejected": -244.1078125, + "loss": 0.3977, + "rewards/chosen": 0.24794367949167886, + "rewards/margins": 1.8121562878290813, + "rewards/rejected": -1.5642126083374024, + "step": 483 + }, + { + "epoch": 0.8151578947368421, + "grad_norm": 40.4236540400168, + "kl": 0.0, + "learning_rate": 5.1438040528872265e-08, + "logits/chosen": -782614656.0, + "logits/rejected": -757872469.3333334, + "logps/chosen": -136.77349853515625, + "logps/rejected": -224.17281087239584, + "loss": 0.5002, + "rewards/chosen": 1.6036109924316406, + "rewards/margins": 0.9859415292739868, + "rewards/rejected": 0.6176694631576538, + "step": 484 + }, + { + "epoch": 0.8168421052631579, + "grad_norm": 121.37099361156412, + "kl": 0.0, + "learning_rate": 5.054784305631546e-08, + "logits/chosen": -910532864.0, + "logits/rejected": -786700117.3333334, + "logps/chosen": -177.5673065185547, + "logps/rejected": -266.01979573567706, + "loss": 0.2893, + "rewards/chosen": -0.057147204875946045, + "rewards/margins": 1.8595226407051086, + "rewards/rejected": -1.9166698455810547, + "step": 485 + }, + { + "epoch": 0.8185263157894737, + "grad_norm": 50.530115528782915, + "kl": 0.0, + "learning_rate": 4.96645488514727e-08, + "logits/chosen": -788890709.3333334, + "logits/rejected": -455711808.0, + "logps/chosen": -256.8352457682292, + "logps/rejected": -267.2051086425781, + "loss": 0.4482, + "rewards/chosen": -0.5332825183868408, + "rewards/margins": 0.43467551469802856, + "rewards/rejected": -0.9679580330848694, + "step": 486 + }, + { + "epoch": 0.8202105263157895, + "grad_norm": 70.07611011272452, + "kl": 0.0, + "learning_rate": 4.8788188486168616e-08, + "logits/chosen": -872919142.4, + "logits/rejected": -701736832.0, + "logps/chosen": -193.6914794921875, + "logps/rejected": -235.78462727864584, + "loss": 0.3179, + "rewards/chosen": 2.0400518417358398, + "rewards/margins": 2.919521633783976, + "rewards/rejected": -0.8794697920481364, + "step": 487 + }, + { + "epoch": 0.8218947368421052, + "grad_norm": 52.808019352043246, + "kl": 0.0, + "learning_rate": 4.7918792292239635e-08, + "logits/chosen": -670976153.6, + "logits/rejected": -896994133.3333334, + "logps/chosen": -245.185595703125, + "logps/rejected": -399.8438313802083, + "loss": 0.4311, + "rewards/chosen": 0.799212646484375, + "rewards/margins": 14.883302561442056, + "rewards/rejected": -14.084089914957682, + "step": 488 + }, + { + "epoch": 0.8235789473684211, + "grad_norm": 54.91548661991697, + "kl": 0.0, + "learning_rate": 4.705639036048439e-08, + "logits/chosen": -952622208.0, + "logits/rejected": -660675968.0, + "logps/chosen": -200.71133422851562, + "logps/rejected": -260.607666015625, + "loss": 0.3739, + "rewards/chosen": 1.0811817646026611, + "rewards/margins": 0.40599095821380615, + "rewards/rejected": 0.675190806388855, + "step": 489 + }, + { + "epoch": 0.8252631578947368, + "grad_norm": 62.12949703126072, + "kl": 0.0, + "learning_rate": 4.6201012539622054e-08, + "logits/chosen": -576909888.0, + "logits/rejected": -580735232.0, + "logps/chosen": -185.22006225585938, + "logps/rejected": -247.2104695638021, + "loss": 0.3748, + "rewards/chosen": 2.8498971462249756, + "rewards/margins": 3.258001764615377, + "rewards/rejected": -0.4081046183904012, + "step": 490 + }, + { + "epoch": 0.8269473684210527, + "grad_norm": 44.89977606216685, + "kl": 0.0, + "learning_rate": 4.535268843525908e-08, + "logits/chosen": -561914880.0, + "logits/rejected": -1542454400.0, + "logps/chosen": -279.1746520996094, + "logps/rejected": -252.60659790039062, + "loss": 0.4187, + "rewards/chosen": -0.5238960385322571, + "rewards/margins": -1.0888721942901611, + "rewards/rejected": 0.564976155757904, + "step": 491 + }, + { + "epoch": 0.8286315789473684, + "grad_norm": 33.62401765550933, + "kl": 0.0, + "learning_rate": 4.4511447408864975e-08, + "logits/chosen": -705677184.0, + "logits/rejected": -642696576.0, + "logps/chosen": -219.29757690429688, + "logps/rejected": -180.13229370117188, + "loss": 0.3157, + "rewards/chosen": 0.919715940952301, + "rewards/margins": 3.393587648868561, + "rewards/rejected": -2.4738717079162598, + "step": 492 + }, + { + "epoch": 0.8303157894736842, + "grad_norm": 44.80536332639876, + "kl": 0.0, + "learning_rate": 4.367731857675569e-08, + "logits/chosen": -1012552362.6666666, + "logits/rejected": -1070745088.0, + "logps/chosen": -210.34134928385416, + "logps/rejected": -197.56212158203124, + "loss": 0.4666, + "rewards/chosen": 2.4052974383036294, + "rewards/margins": 1.760869280497233, + "rewards/rejected": 0.6444281578063965, + "step": 493 + }, + { + "epoch": 0.832, + "grad_norm": 69.77716049036187, + "kl": 0.0, + "learning_rate": 4.2850330809085874e-08, + "logits/chosen": -630044373.3333334, + "logits/rejected": -1326352512.0, + "logps/chosen": -251.13602701822916, + "logps/rejected": -142.5716094970703, + "loss": 0.3156, + "rewards/chosen": 0.20519916216532388, + "rewards/margins": -1.417292316754659, + "rewards/rejected": 1.622491478919983, + "step": 494 + }, + { + "epoch": 0.8336842105263158, + "grad_norm": 36.11198104501435, + "kl": 0.0, + "learning_rate": 4.2030512728849945e-08, + "logits/chosen": -921905493.3333334, + "logits/rejected": -791465369.6, + "logps/chosen": -237.568115234375, + "logps/rejected": -210.0384521484375, + "loss": 0.4533, + "rewards/chosen": 0.15185137589772543, + "rewards/margins": 1.321685512860616, + "rewards/rejected": -1.1698341369628906, + "step": 495 + }, + { + "epoch": 0.8353684210526315, + "grad_norm": 47.31830867232736, + "kl": 0.0, + "learning_rate": 4.1217892710891126e-08, + "logits/chosen": -688228992.0, + "logits/rejected": -928642474.6666666, + "logps/chosen": -180.07191467285156, + "logps/rejected": -247.9576416015625, + "loss": 0.3971, + "rewards/chosen": 2.725332736968994, + "rewards/margins": 4.285257339477539, + "rewards/rejected": -1.559924602508545, + "step": 496 + }, + { + "epoch": 0.8370526315789474, + "grad_norm": 58.64030492711, + "kl": 0.0, + "learning_rate": 4.041249888091941e-08, + "logits/chosen": -930103466.6666666, + "logits/rejected": -520454144.0, + "logps/chosen": -223.98262532552084, + "logps/rejected": -291.0794982910156, + "loss": 0.3902, + "rewards/chosen": 0.41166090965270996, + "rewards/margins": 2.5037872791290283, + "rewards/rejected": -2.0921263694763184, + "step": 497 + }, + { + "epoch": 0.8387368421052631, + "grad_norm": 42.51804510909749, + "kl": 0.0, + "learning_rate": 3.9614359114538196e-08, + "logits/chosen": -594655701.3333334, + "logits/rejected": -1102049280.0, + "logps/chosen": -239.44209798177084, + "logps/rejected": -235.8392578125, + "loss": 0.4429, + "rewards/chosen": -0.807283083597819, + "rewards/margins": -1.4231882254282633, + "rewards/rejected": 0.6159051418304443, + "step": 498 + }, + { + "epoch": 0.840421052631579, + "grad_norm": 60.0326086702577, + "kl": 0.0, + "learning_rate": 3.8823501036279514e-08, + "logits/chosen": -403930848.0, + "logits/rejected": -417444640.0, + "logps/chosen": -302.0504150390625, + "logps/rejected": -168.2909698486328, + "loss": 0.3504, + "rewards/chosen": 0.14431113004684448, + "rewards/margins": 1.6669214367866516, + "rewards/rejected": -1.5226103067398071, + "step": 499 + }, + { + "epoch": 0.8421052631578947, + "grad_norm": 44.106324622609606, + "kl": 0.0, + "learning_rate": 3.803995201864762e-08, + "logits/chosen": -775272089.6, + "logits/rejected": -703777536.0, + "logps/chosen": -191.228076171875, + "logps/rejected": -175.0321248372396, + "loss": 0.3079, + "rewards/chosen": 0.6324722290039062, + "rewards/margins": 2.6547363917032873, + "rewards/rejected": -2.0222641626993814, + "step": 500 + }, + { + "epoch": 0.8421052631578947, + "eval_logits/chosen": -922359676.4770643, + "eval_logits/rejected": -826945158.35461, + "eval_logps/chosen": -217.7674706135321, + "eval_logps/rejected": -228.38810394503545, + "eval_loss": 0.378118097782135, + "eval_rewards/chosen": 0.8047234771448538, + "eval_rewards/margins": 1.8294198813516913, + "eval_rewards/rejected": -1.0246964042068374, + "eval_runtime": 1555.2949, + "eval_samples_per_second": 0.643, + "eval_steps_per_second": 0.161, + "kl": 0.0, + "step": 500 + }, + { + "epoch": 0.8437894736842105, + "grad_norm": 55.94728204942989, + "kl": 0.0, + "learning_rate": 3.726373918117196e-08, + "logits/chosen": -971137472.0, + "logits/rejected": -1374881280.0, + "logps/chosen": -255.43304443359375, + "logps/rejected": -218.9608612060547, + "loss": 0.4355, + "rewards/chosen": -0.6300251483917236, + "rewards/margins": -1.8687539100646973, + "rewards/rejected": 1.2387287616729736, + "step": 501 + }, + { + "epoch": 0.8454736842105263, + "grad_norm": 47.235552657321186, + "kl": 0.0, + "learning_rate": 3.6494889389468434e-08, + "logits/chosen": -602287744.0, + "logits/rejected": -920995072.0, + "logps/chosen": -210.18441772460938, + "logps/rejected": -227.16696166992188, + "loss": 0.3343, + "rewards/chosen": 0.2986224889755249, + "rewards/margins": 1.3155208826065063, + "rewards/rejected": -1.0168983936309814, + "step": 502 + }, + { + "epoch": 0.8471578947368421, + "grad_norm": 49.23690109646782, + "kl": 0.0, + "learning_rate": 3.573342925430925e-08, + "logits/chosen": -778606080.0, + "logits/rejected": -654501376.0, + "logps/chosen": -210.22725423177084, + "logps/rejected": -293.99200439453125, + "loss": 0.373, + "rewards/chosen": 0.8925824165344238, + "rewards/margins": 2.6825954914093018, + "rewards/rejected": -1.790013074874878, + "step": 503 + }, + { + "epoch": 0.8488421052631578, + "grad_norm": 47.196350435325876, + "kl": 0.0, + "learning_rate": 3.497938513070234e-08, + "logits/chosen": -747158118.4, + "logits/rejected": -715651498.6666666, + "logps/chosen": -196.84251708984374, + "logps/rejected": -247.5506388346354, + "loss": 0.3714, + "rewards/chosen": 1.6987966537475585, + "rewards/margins": 2.460467561086019, + "rewards/rejected": -0.7616709073384603, + "step": 504 + }, + { + "epoch": 0.8505263157894737, + "grad_norm": 67.84038276109705, + "kl": 0.0, + "learning_rate": 3.423278311697897e-08, + "logits/chosen": -1138272768.0, + "logits/rejected": -622846156.8, + "logps/chosen": -215.9085896809896, + "logps/rejected": -196.0285888671875, + "loss": 0.3555, + "rewards/chosen": 2.7768580118815103, + "rewards/margins": 3.196271006266276, + "rewards/rejected": -0.41941299438476565, + "step": 505 + }, + { + "epoch": 0.8522105263157894, + "grad_norm": 42.41057229807221, + "kl": 0.0, + "learning_rate": 3.349364905389032e-08, + "logits/chosen": -1031722581.3333334, + "logits/rejected": -827191808.0, + "logps/chosen": -204.5508015950521, + "logps/rejected": -485.5096130371094, + "loss": 0.3319, + "rewards/chosen": 1.5830011367797852, + "rewards/margins": 23.542262077331543, + "rewards/rejected": -21.959260940551758, + "step": 506 + }, + { + "epoch": 0.8538947368421053, + "grad_norm": 40.22957411853309, + "kl": 0.0, + "learning_rate": 3.2762008523713386e-08, + "logits/chosen": -694314709.3333334, + "logits/rejected": -912536064.0, + "logps/chosen": -231.19136555989584, + "logps/rejected": -210.5927978515625, + "loss": 0.4044, + "rewards/chosen": -1.8982666333516438, + "rewards/margins": -1.2363632520039876, + "rewards/rejected": -0.6619033813476562, + "step": 507 + }, + { + "epoch": 0.8555789473684211, + "grad_norm": 56.68241962573519, + "kl": 0.0, + "learning_rate": 3.203788684936534e-08, + "logits/chosen": -1072353133.7142857, + "logits/rejected": -857103872.0, + "logps/chosen": -251.75599888392858, + "logps/rejected": -181.1580810546875, + "loss": 0.4473, + "rewards/chosen": -0.4974602290562221, + "rewards/margins": -0.4106224605015346, + "rewards/rejected": -0.0868377685546875, + "step": 508 + }, + { + "epoch": 0.8572631578947368, + "grad_norm": 47.126301959598706, + "kl": 0.0, + "learning_rate": 3.132130909352709e-08, + "logits/chosen": -802446774.8571428, + "logits/rejected": -754396800.0, + "logps/chosen": -243.97293526785714, + "logps/rejected": -171.71823120117188, + "loss": 0.4578, + "rewards/chosen": 0.9422939164297921, + "rewards/margins": 0.3230052930968148, + "rewards/rejected": 0.6192886233329773, + "step": 509 + }, + { + "epoch": 0.8589473684210527, + "grad_norm": 67.1017756333287, + "kl": 0.0, + "learning_rate": 3.061230005777593e-08, + "logits/chosen": -716743552.0, + "logits/rejected": -928516315.4285715, + "logps/chosen": -392.4697265625, + "logps/rejected": -267.4026576450893, + "loss": 0.3928, + "rewards/chosen": -4.59370756149292, + "rewards/margins": -1.8247188159397671, + "rewards/rejected": -2.7689887455531528, + "step": 510 + }, + { + "epoch": 0.8606315789473684, + "grad_norm": 37.48246855118462, + "kl": 0.0, + "learning_rate": 2.991088428172722e-08, + "logits/chosen": -882642329.6, + "logits/rejected": -579270997.3333334, + "logps/chosen": -216.027587890625, + "logps/rejected": -224.06245930989584, + "loss": 0.3588, + "rewards/chosen": 1.8644773483276367, + "rewards/margins": 4.355569076538086, + "rewards/rejected": -2.491091728210449, + "step": 511 + }, + { + "epoch": 0.8623157894736843, + "grad_norm": 46.701572144028056, + "kl": 0.0, + "learning_rate": 2.9217086042184534e-08, + "logits/chosen": -951701888.0, + "logits/rejected": -1123305728.0, + "logps/chosen": -232.40447998046875, + "logps/rejected": -214.9777374267578, + "loss": 0.3516, + "rewards/chosen": 0.27192726731300354, + "rewards/margins": 5.410427957773209, + "rewards/rejected": -5.138500690460205, + "step": 512 + }, + { + "epoch": 0.864, + "grad_norm": 47.82460894266106, + "kl": 0.0, + "learning_rate": 2.8530929352300087e-08, + "logits/chosen": -747589632.0, + "logits/rejected": -987404083.2, + "logps/chosen": -221.045166015625, + "logps/rejected": -198.009423828125, + "loss": 0.3888, + "rewards/chosen": 0.2495401899019877, + "rewards/margins": 2.997531179587046, + "rewards/rejected": -2.7479909896850585, + "step": 513 + }, + { + "epoch": 0.8656842105263158, + "grad_norm": 50.95999181346899, + "kl": 0.0, + "learning_rate": 2.7852437960743326e-08, + "logits/chosen": -816897996.8, + "logits/rejected": -827434154.6666666, + "logps/chosen": -209.32421875, + "logps/rejected": -241.70332845052084, + "loss": 0.3541, + "rewards/chosen": 0.15461671352386475, + "rewards/margins": 2.3556827306747437, + "rewards/rejected": -2.201066017150879, + "step": 514 + }, + { + "epoch": 0.8673684210526316, + "grad_norm": 70.2346543078714, + "kl": 0.0, + "learning_rate": 2.718163535087864e-08, + "logits/chosen": -805316096.0, + "logits/rejected": -1582949376.0, + "logps/chosen": -179.23245239257812, + "logps/rejected": -174.32058715820312, + "loss": 0.3397, + "rewards/chosen": -0.28184014558792114, + "rewards/margins": 1.7464155554771423, + "rewards/rejected": -2.0282557010650635, + "step": 515 + }, + { + "epoch": 0.8690526315789474, + "grad_norm": 44.88916114173372, + "kl": 0.0, + "learning_rate": 2.6518544739953187e-08, + "logits/chosen": -884145472.0, + "logits/rejected": -1249547136.0, + "logps/chosen": -265.9125671386719, + "logps/rejected": -233.0666961669922, + "loss": 0.3362, + "rewards/chosen": 0.07709883153438568, + "rewards/margins": 0.21887779235839844, + "rewards/rejected": -0.14177896082401276, + "step": 516 + }, + { + "epoch": 0.8707368421052631, + "grad_norm": 46.28385480900765, + "kl": 0.0, + "learning_rate": 2.586318907829291e-08, + "logits/chosen": -802108842.6666666, + "logits/rejected": -925224960.0, + "logps/chosen": -203.9495646158854, + "logps/rejected": -222.9334716796875, + "loss": 0.3661, + "rewards/chosen": 1.3522602717081706, + "rewards/margins": 0.6993664304415386, + "rewards/rejected": 0.6528938412666321, + "step": 517 + }, + { + "epoch": 0.872421052631579, + "grad_norm": 46.93723588820055, + "kl": 0.0, + "learning_rate": 2.5215591048508152e-08, + "logits/chosen": -1172611498.6666667, + "logits/rejected": -601648947.2, + "logps/chosen": -245.01177978515625, + "logps/rejected": -234.228076171875, + "loss": 0.4201, + "rewards/chosen": -0.7683883508046468, + "rewards/margins": 0.7730680624643961, + "rewards/rejected": -1.541456413269043, + "step": 518 + }, + { + "epoch": 0.8741052631578947, + "grad_norm": 37.74401010007106, + "kl": 0.0, + "learning_rate": 2.4575773064708898e-08, + "logits/chosen": -844627520.0, + "logits/rejected": -680875776.0, + "logps/chosen": -191.596923828125, + "logps/rejected": -210.30389404296875, + "loss": 0.3464, + "rewards/chosen": 1.64203679561615, + "rewards/margins": 2.2509559392929077, + "rewards/rejected": -0.6089191436767578, + "step": 519 + }, + { + "epoch": 0.8757894736842106, + "grad_norm": 38.54143890619264, + "kl": 0.0, + "learning_rate": 2.3943757271728816e-08, + "logits/chosen": -821064064.0, + "logits/rejected": -720391232.0, + "logps/chosen": -230.83758544921875, + "logps/rejected": -239.45230102539062, + "loss": 0.2565, + "rewards/chosen": 2.128859043121338, + "rewards/margins": 4.223225116729736, + "rewards/rejected": -2.0943660736083984, + "step": 520 + }, + { + "epoch": 0.8774736842105263, + "grad_norm": 46.42071867786955, + "kl": 0.0, + "learning_rate": 2.3319565544358628e-08, + "logits/chosen": -550017280.0, + "logits/rejected": -889491840.0, + "logps/chosen": -228.99757385253906, + "logps/rejected": -294.79571533203125, + "loss": 0.4034, + "rewards/chosen": 1.803693413734436, + "rewards/margins": 10.621366143226624, + "rewards/rejected": -8.817672729492188, + "step": 521 + }, + { + "epoch": 0.8791578947368421, + "grad_norm": 165.5741913948296, + "kl": 0.0, + "learning_rate": 2.270321948658943e-08, + "logits/chosen": -1067505152.0, + "logits/rejected": -1063348160.0, + "logps/chosen": -216.38136291503906, + "logps/rejected": -210.46290588378906, + "loss": 0.4473, + "rewards/chosen": 0.3500576317310333, + "rewards/margins": 0.7664283812046051, + "rewards/rejected": -0.4163707494735718, + "step": 522 + }, + { + "epoch": 0.8808421052631579, + "grad_norm": 53.27002172942906, + "kl": 0.0, + "learning_rate": 2.2094740430864567e-08, + "logits/chosen": -694395392.0, + "logits/rejected": -859106099.2, + "logps/chosen": -148.91910807291666, + "logps/rejected": -242.549169921875, + "loss": 0.4046, + "rewards/chosen": 1.4957799911499023, + "rewards/margins": 2.2706830501556396, + "rewards/rejected": -0.7749030590057373, + "step": 523 + }, + { + "epoch": 0.8825263157894737, + "grad_norm": 62.65204336260102, + "kl": 0.0, + "learning_rate": 2.1494149437341373e-08, + "logits/chosen": -1015188992.0, + "logits/rejected": -330501728.0, + "logps/chosen": -239.76932779947916, + "logps/rejected": -281.90582275390625, + "loss": 0.3585, + "rewards/chosen": 1.6041777928670247, + "rewards/margins": 3.236354072888692, + "rewards/rejected": -1.6321762800216675, + "step": 524 + }, + { + "epoch": 0.8842105263157894, + "grad_norm": 59.41452814443212, + "kl": 0.0, + "learning_rate": 2.0901467293162444e-08, + "logits/chosen": -898306944.0, + "logits/rejected": -934284970.6666666, + "logps/chosen": -239.46893310546875, + "logps/rejected": -347.1363525390625, + "loss": 0.4087, + "rewards/chosen": -0.3174598515033722, + "rewards/margins": 1.7363191147645316, + "rewards/rejected": -2.053778966267904, + "step": 525 + }, + { + "epoch": 0.8858947368421053, + "grad_norm": 28.835874353397827, + "kl": 0.0, + "learning_rate": 2.0316714511735998e-08, + "logits/chosen": -985848448.0, + "logits/rejected": -997492416.0, + "logps/chosen": -201.36636352539062, + "logps/rejected": -174.35333251953125, + "loss": 0.259, + "rewards/chosen": 2.0680689811706543, + "rewards/margins": 4.799753904342651, + "rewards/rejected": -2.731684923171997, + "step": 526 + }, + { + "epoch": 0.887578947368421, + "grad_norm": 54.565049728962386, + "kl": 0.0, + "learning_rate": 1.9739911332025793e-08, + "logits/chosen": -839355136.0, + "logits/rejected": -553939626.6666666, + "logps/chosen": -199.08831787109375, + "logps/rejected": -176.72452799479166, + "loss": 0.34, + "rewards/chosen": 2.425278663635254, + "rewards/margins": 2.334609886010488, + "rewards/rejected": 0.09066877762476604, + "step": 527 + }, + { + "epoch": 0.8892631578947369, + "grad_norm": 46.20449911567436, + "kl": 0.0, + "learning_rate": 1.9171077717850955e-08, + "logits/chosen": -786516309.3333334, + "logits/rejected": -727864640.0, + "logps/chosen": -230.94038899739584, + "logps/rejected": -252.71160888671875, + "loss": 0.3229, + "rewards/chosen": 0.6411038637161255, + "rewards/margins": 6.392639756202698, + "rewards/rejected": -5.751535892486572, + "step": 528 + }, + { + "epoch": 0.8909473684210526, + "grad_norm": 57.38593942004128, + "kl": 0.0, + "learning_rate": 1.8610233357194747e-08, + "logits/chosen": -670264618.6666666, + "logits/rejected": -815440179.2, + "logps/chosen": -183.37162272135416, + "logps/rejected": -351.1501708984375, + "loss": 0.3711, + "rewards/chosen": 1.5523789723714192, + "rewards/margins": 9.300235112508139, + "rewards/rejected": -7.747856140136719, + "step": 529 + }, + { + "epoch": 0.8926315789473684, + "grad_norm": 68.33693509306372, + "kl": 0.0, + "learning_rate": 1.805739766152309e-08, + "logits/chosen": -777834240.0, + "logits/rejected": -670208640.0, + "logps/chosen": -191.6290283203125, + "logps/rejected": -211.78524780273438, + "loss": 0.3876, + "rewards/chosen": 1.8237395286560059, + "rewards/margins": 1.157799780368805, + "rewards/rejected": 0.6659397482872009, + "step": 530 + }, + { + "epoch": 0.8943157894736842, + "grad_norm": 34.749494357287055, + "kl": 0.0, + "learning_rate": 1.7512589765112994e-08, + "logits/chosen": -896000804.5714285, + "logits/rejected": -150322144.0, + "logps/chosen": -230.301513671875, + "logps/rejected": -289.5316467285156, + "loss": 0.3033, + "rewards/chosen": 1.0773948260716029, + "rewards/margins": 1.1100638934544154, + "rewards/rejected": -0.0326690673828125, + "step": 531 + }, + { + "epoch": 0.896, + "grad_norm": 52.937522598027066, + "kl": 0.0, + "learning_rate": 1.697582852439011e-08, + "logits/chosen": -1075272448.0, + "logits/rejected": -1150560512.0, + "logps/chosen": -186.83663940429688, + "logps/rejected": -228.51455688476562, + "loss": 0.3423, + "rewards/chosen": 1.7373024225234985, + "rewards/margins": 1.5837997794151306, + "rewards/rejected": 0.15350264310836792, + "step": 532 + }, + { + "epoch": 0.8976842105263158, + "grad_norm": 30.475440103504955, + "kl": 0.0, + "learning_rate": 1.6447132517276004e-08, + "logits/chosen": -637437952.0, + "logits/rejected": -962437324.8, + "logps/chosen": -198.0897216796875, + "logps/rejected": -254.03193359375, + "loss": 0.3967, + "rewards/chosen": 1.2216943899790447, + "rewards/margins": 0.7942300478617351, + "rewards/rejected": 0.42746434211730955, + "step": 533 + }, + { + "epoch": 0.8993684210526316, + "grad_norm": 43.2574141571032, + "kl": 0.0, + "learning_rate": 1.5926520042545383e-08, + "logits/chosen": -1247182848.0, + "logits/rejected": -590226432.0, + "logps/chosen": -222.28861490885416, + "logps/rejected": -237.892724609375, + "loss": 0.3006, + "rewards/chosen": 0.06360169251759847, + "rewards/margins": 1.3943130572636921, + "rewards/rejected": -1.3307113647460938, + "step": 534 + }, + { + "epoch": 0.9010526315789473, + "grad_norm": 53.71591400639584, + "kl": 0.0, + "learning_rate": 1.5414009119192635e-08, + "logits/chosen": -915837120.0, + "logits/rejected": -808716458.6666666, + "logps/chosen": -179.90530395507812, + "logps/rejected": -195.09493001302084, + "loss": 0.4779, + "rewards/chosen": 0.5700050592422485, + "rewards/margins": -0.024598677953084347, + "rewards/rejected": 0.5946037371953329, + "step": 535 + }, + { + "epoch": 0.9027368421052632, + "grad_norm": 61.45180482744315, + "kl": 0.0, + "learning_rate": 1.4909617485808073e-08, + "logits/chosen": -1060230348.8, + "logits/rejected": -902803541.3333334, + "logps/chosen": -229.759912109375, + "logps/rejected": -223.30476888020834, + "loss": 0.3913, + "rewards/chosen": 1.0647006988525392, + "rewards/margins": 0.6491708119710287, + "rewards/rejected": 0.41552988688151044, + "step": 536 + }, + { + "epoch": 0.9044210526315789, + "grad_norm": 101.47573520441375, + "kl": 0.0, + "learning_rate": 1.4413362599964118e-08, + "logits/chosen": -850445994.6666666, + "logits/rejected": -538516864.0, + "logps/chosen": -285.9638264973958, + "logps/rejected": -175.34397888183594, + "loss": 0.4247, + "rewards/chosen": -0.0778544048468272, + "rewards/margins": -3.1665999988714852, + "rewards/rejected": 3.088745594024658, + "step": 537 + }, + { + "epoch": 0.9061052631578947, + "grad_norm": 45.25790194746145, + "kl": 0.0, + "learning_rate": 1.3925261637611068e-08, + "logits/chosen": -965904588.8, + "logits/rejected": -1320554325.3333333, + "logps/chosen": -204.27237548828126, + "logps/rejected": -217.86629231770834, + "loss": 0.4245, + "rewards/chosen": 1.1585922241210938, + "rewards/margins": 4.713977813720703, + "rewards/rejected": -3.5553855895996094, + "step": 538 + }, + { + "epoch": 0.9077894736842105, + "grad_norm": 31.80801282899897, + "kl": 0.0, + "learning_rate": 1.3445331492482614e-08, + "logits/chosen": -1130592682.6666667, + "logits/rejected": -715334297.6, + "logps/chosen": -185.36031087239584, + "logps/rejected": -224.770458984375, + "loss": 0.3553, + "rewards/chosen": 0.9161597887674967, + "rewards/margins": 2.255999151865641, + "rewards/rejected": -1.3398393630981444, + "step": 539 + }, + { + "epoch": 0.9094736842105263, + "grad_norm": 39.75979427316365, + "kl": 0.0, + "learning_rate": 1.2973588775511024e-08, + "logits/chosen": -829291840.0, + "logits/rejected": -1113173418.6666667, + "logps/chosen": -194.16685485839844, + "logps/rejected": -230.64664713541666, + "loss": 0.325, + "rewards/chosen": 2.4532127380371094, + "rewards/margins": 2.8305301666259766, + "rewards/rejected": -0.3773174285888672, + "step": 540 + }, + { + "epoch": 0.9111578947368421, + "grad_norm": 49.13868786217956, + "kl": 0.0, + "learning_rate": 1.2510049814252299e-08, + "logits/chosen": -1220529152.0, + "logits/rejected": -959601868.8, + "logps/chosen": -205.97456868489584, + "logps/rejected": -292.898876953125, + "loss": 0.4062, + "rewards/chosen": 0.8909465471903483, + "rewards/margins": 2.5595290819803873, + "rewards/rejected": -1.6685825347900392, + "step": 541 + }, + { + "epoch": 0.9128421052631579, + "grad_norm": 46.27018727053655, + "kl": 0.0, + "learning_rate": 1.2054730652321127e-08, + "logits/chosen": -801094336.0, + "logits/rejected": -564379648.0, + "logps/chosen": -316.7994384765625, + "logps/rejected": -242.9945831298828, + "loss": 0.4345, + "rewards/chosen": -1.3522491455078125, + "rewards/margins": -0.6712818145751953, + "rewards/rejected": -0.6809673309326172, + "step": 542 + }, + { + "epoch": 0.9145263157894736, + "grad_norm": 40.13535818390141, + "kl": 0.0, + "learning_rate": 1.1607647048835462e-08, + "logits/chosen": -819266389.3333334, + "logits/rejected": -731518822.4, + "logps/chosen": -188.42167154947916, + "logps/rejected": -174.18707275390625, + "loss": 0.3224, + "rewards/chosen": 1.533565839131673, + "rewards/margins": 3.497674496968587, + "rewards/rejected": -1.964108657836914, + "step": 543 + }, + { + "epoch": 0.9162105263157895, + "grad_norm": 58.704500927311344, + "kl": 0.0, + "learning_rate": 1.116881447787113e-08, + "logits/chosen": -644635840.0, + "logits/rejected": -1043029376.0, + "logps/chosen": -210.66639709472656, + "logps/rejected": -211.0460968017578, + "loss": 0.357, + "rewards/chosen": 0.7050827145576477, + "rewards/margins": 0.4989025592803955, + "rewards/rejected": 0.2061801552772522, + "step": 544 + }, + { + "epoch": 0.9178947368421052, + "grad_norm": 38.106880910390885, + "kl": 0.0, + "learning_rate": 1.0738248127926342e-08, + "logits/chosen": -678777770.6666666, + "logits/rejected": -1563541504.0, + "logps/chosen": -219.55989583333334, + "logps/rejected": -179.98016357421875, + "loss": 0.3392, + "rewards/chosen": 0.5457992553710938, + "rewards/margins": -1.0953590869903564, + "rewards/rejected": 1.6411583423614502, + "step": 545 + }, + { + "epoch": 0.919578947368421, + "grad_norm": 58.09818425552494, + "kl": 0.0, + "learning_rate": 1.0315962901395802e-08, + "logits/chosen": -517518624.0, + "logits/rejected": -936108714.6666666, + "logps/chosen": -197.85256958007812, + "logps/rejected": -324.84869384765625, + "loss": 0.3334, + "rewards/chosen": 4.236547470092773, + "rewards/margins": 13.935715993245443, + "rewards/rejected": -9.69916852315267, + "step": 546 + }, + { + "epoch": 0.9212631578947369, + "grad_norm": 87.5472135904328, + "kl": 0.0, + "learning_rate": 9.901973414055186e-09, + "logits/chosen": -972204544.0, + "logits/rejected": -999611456.0, + "logps/chosen": -240.75531005859375, + "logps/rejected": -262.8578186035156, + "loss": 0.4516, + "rewards/chosen": 0.7045586109161377, + "rewards/margins": 3.072298526763916, + "rewards/rejected": -2.3677399158477783, + "step": 547 + }, + { + "epoch": 0.9229473684210526, + "grad_norm": 61.8636347167815, + "kl": 0.0, + "learning_rate": 9.496293994555066e-09, + "logits/chosen": -967555788.8, + "logits/rejected": -894452138.6666666, + "logps/chosen": -237.103955078125, + "logps/rejected": -238.02288818359375, + "loss": 0.4624, + "rewards/chosen": -0.08766875267028809, + "rewards/margins": 3.0026972929636635, + "rewards/rejected": -3.0903660456339517, + "step": 548 + }, + { + "epoch": 0.9246315789473685, + "grad_norm": 76.28246005925376, + "kl": 0.0, + "learning_rate": 9.098938683924972e-09, + "logits/chosen": -610943360.0, + "logits/rejected": -872966400.0, + "logps/chosen": -243.48297119140625, + "logps/rejected": -183.1815185546875, + "loss": 0.4655, + "rewards/chosen": 0.6167190869649252, + "rewards/margins": 0.13095130523045861, + "rewards/rejected": 0.48576778173446655, + "step": 549 + }, + { + "epoch": 0.9263157894736842, + "grad_norm": 39.99028773675479, + "kl": 0.0, + "learning_rate": 8.709921235087597e-09, + "logits/chosen": -773891264.0, + "logits/rejected": -839812096.0, + "logps/chosen": -239.84304809570312, + "logps/rejected": -216.54505920410156, + "loss": 0.3937, + "rewards/chosen": -0.7027072906494141, + "rewards/margins": 0.8752220869064331, + "rewards/rejected": -1.5779293775558472, + "step": 550 + }, + { + "epoch": 0.928, + "grad_norm": 176.8083268495628, + "kl": 0.0, + "learning_rate": 8.329255112382665e-09, + "logits/chosen": -674264934.4, + "logits/rejected": -985622101.3333334, + "logps/chosen": -153.71854248046876, + "logps/rejected": -222.71295166015625, + "loss": 0.3363, + "rewards/chosen": 1.8852203369140625, + "rewards/margins": 4.823611068725586, + "rewards/rejected": -2.9383907318115234, + "step": 551 + }, + { + "epoch": 0.9296842105263158, + "grad_norm": 47.516751657127465, + "kl": 0.0, + "learning_rate": 7.956953491100871e-09, + "logits/chosen": -841082624.0, + "logits/rejected": -614038528.0, + "logps/chosen": -277.8362731933594, + "logps/rejected": -308.65185546875, + "loss": 0.3715, + "rewards/chosen": 0.449728399515152, + "rewards/margins": 3.049138320343835, + "rewards/rejected": -2.599409920828683, + "step": 552 + }, + { + "epoch": 0.9313684210526316, + "grad_norm": 92.74872502810405, + "kl": 0.0, + "learning_rate": 7.593029257027956e-09, + "logits/chosen": -639304106.6666666, + "logits/rejected": -1038373478.4, + "logps/chosen": -212.13045247395834, + "logps/rejected": -236.4179931640625, + "loss": 0.3602, + "rewards/chosen": -0.6566390991210938, + "rewards/margins": 1.910294532775879, + "rewards/rejected": -2.5669336318969727, + "step": 553 + }, + { + "epoch": 0.9330526315789474, + "grad_norm": 49.323568144147856, + "kl": 0.0, + "learning_rate": 7.23749500599874e-09, + "logits/chosen": -1499314858.6666667, + "logits/rejected": -812018636.8, + "logps/chosen": -252.00286865234375, + "logps/rejected": -275.40224609375, + "loss": 0.3429, + "rewards/chosen": -0.8385945955912272, + "rewards/margins": 0.727512296040853, + "rewards/rejected": -1.5661068916320802, + "step": 554 + }, + { + "epoch": 0.9347368421052632, + "grad_norm": 48.76961282183268, + "kl": 0.0, + "learning_rate": 6.89036304346105e-09, + "logits/chosen": -709829034.6666666, + "logits/rejected": -862226176.0, + "logps/chosen": -221.70589192708334, + "logps/rejected": -222.41966247558594, + "loss": 0.3214, + "rewards/chosen": 0.810302734375, + "rewards/margins": 1.5894142389297485, + "rewards/rejected": -0.7791115045547485, + "step": 555 + }, + { + "epoch": 0.9364210526315789, + "grad_norm": 51.49949200706721, + "kl": 0.0, + "learning_rate": 6.551645384049897e-09, + "logits/chosen": -880212906.6666666, + "logits/rejected": -670162816.0, + "logps/chosen": -202.01947021484375, + "logps/rejected": -240.55711364746094, + "loss": 0.3524, + "rewards/chosen": 0.031015316645304363, + "rewards/margins": 1.1750317017237346, + "rewards/rejected": -1.1440163850784302, + "step": 556 + }, + { + "epoch": 0.9381052631578948, + "grad_norm": 48.535906716358035, + "kl": 0.0, + "learning_rate": 6.221353751171665e-09, + "logits/chosen": -726684928.0, + "logits/rejected": -747469531.4285715, + "logps/chosen": -186.82913208007812, + "logps/rejected": -262.0355922154018, + "loss": 0.3274, + "rewards/chosen": 4.104483127593994, + "rewards/margins": 5.761045115334647, + "rewards/rejected": -1.656561987740653, + "step": 557 + }, + { + "epoch": 0.9397894736842105, + "grad_norm": 44.18140203861248, + "kl": 0.0, + "learning_rate": 5.899499576598216e-09, + "logits/chosen": -919181721.6, + "logits/rejected": -773407402.6666666, + "logps/chosen": -205.78271484375, + "logps/rejected": -287.0757242838542, + "loss": 0.36, + "rewards/chosen": 1.863102340698242, + "rewards/margins": 3.7114522298177084, + "rewards/rejected": -1.848349889119466, + "step": 558 + }, + { + "epoch": 0.9414736842105264, + "grad_norm": 34.08124127057782, + "kl": 0.0, + "learning_rate": 5.586094000071401e-09, + "logits/chosen": -597630336.0, + "logits/rejected": -568648192.0, + "logps/chosen": -231.41092936197916, + "logps/rejected": -189.46743774414062, + "loss": 0.4097, + "rewards/chosen": 1.2908663749694824, + "rewards/margins": 0.8174479305744171, + "rewards/rejected": 0.4734184443950653, + "step": 559 + }, + { + "epoch": 0.9431578947368421, + "grad_norm": 36.1106808708771, + "kl": 0.0, + "learning_rate": 5.2811478689173686e-09, + "logits/chosen": -405102976.0, + "logits/rejected": -868101461.3333334, + "logps/chosen": -254.44976806640625, + "logps/rejected": -242.90071614583334, + "loss": 0.3742, + "rewards/chosen": 1.0309677124023438, + "rewards/margins": 2.66564400990804, + "rewards/rejected": -1.6346762975056965, + "step": 560 + }, + { + "epoch": 0.9448421052631579, + "grad_norm": 45.82329638700707, + "kl": 0.0, + "learning_rate": 4.984671737671142e-09, + "logits/chosen": -742133077.3333334, + "logits/rejected": -740478361.6, + "logps/chosen": -258.3249918619792, + "logps/rejected": -183.15859375, + "loss": 0.4038, + "rewards/chosen": -0.7480514844258627, + "rewards/margins": -0.37228356202443447, + "rewards/rejected": -0.3757679224014282, + "step": 561 + }, + { + "epoch": 0.9465263157894737, + "grad_norm": 39.6743299184901, + "kl": 0.0, + "learning_rate": 4.696675867711386e-09, + "logits/chosen": -535260979.2, + "logits/rejected": -851761066.6666666, + "logps/chosen": -218.016943359375, + "logps/rejected": -338.2209065755208, + "loss": 0.4127, + "rewards/chosen": 1.0443305015563964, + "rewards/margins": 3.631992403666178, + "rewards/rejected": -2.5876619021097818, + "step": 562 + }, + { + "epoch": 0.9482105263157895, + "grad_norm": 25.945047187533934, + "kl": 0.0, + "learning_rate": 4.417170226905187e-09, + "logits/chosen": -638596096.0, + "logits/rejected": -722204416.0, + "logps/chosen": -176.8044230143229, + "logps/rejected": -205.5111572265625, + "loss": 0.3022, + "rewards/chosen": 1.1705973148345947, + "rewards/margins": 1.9772413730621339, + "rewards/rejected": -0.806644058227539, + "step": 563 + }, + { + "epoch": 0.9498947368421052, + "grad_norm": 56.04530688467595, + "kl": 0.0, + "learning_rate": 4.146164489263054e-09, + "logits/chosen": -803748693.3333334, + "logits/rejected": -645856576.0, + "logps/chosen": -237.97408040364584, + "logps/rejected": -261.6069030761719, + "loss": 0.4415, + "rewards/chosen": 0.1846969723701477, + "rewards/margins": 2.00448077917099, + "rewards/rejected": -1.8197838068008423, + "step": 564 + }, + { + "epoch": 0.9515789473684211, + "grad_norm": 45.941589484462845, + "kl": 0.0, + "learning_rate": 3.88366803460416e-09, + "logits/chosen": -1046677888.0, + "logits/rejected": -937844800.0, + "logps/chosen": -216.30455017089844, + "logps/rejected": -152.97848510742188, + "loss": 0.3607, + "rewards/chosen": 1.318871259689331, + "rewards/margins": 1.2597434520721436, + "rewards/rejected": 0.0591278076171875, + "step": 565 + }, + { + "epoch": 0.9532631578947368, + "grad_norm": 61.47288256142775, + "kl": 0.0, + "learning_rate": 3.6296899482316236e-09, + "logits/chosen": -661587904.0, + "logits/rejected": -835168896.0, + "logps/chosen": -245.52484130859375, + "logps/rejected": -229.88687133789062, + "loss": 0.4462, + "rewards/chosen": 1.543544054031372, + "rewards/margins": 1.7612977027893066, + "rewards/rejected": -0.21775364875793457, + "step": 566 + }, + { + "epoch": 0.9549473684210527, + "grad_norm": 47.39961532223011, + "kl": 0.0, + "learning_rate": 3.384239020618018e-09, + "logits/chosen": -1413642368.0, + "logits/rejected": -993228288.0, + "logps/chosen": -251.99951171875, + "logps/rejected": -258.76263427734375, + "loss": 0.3247, + "rewards/chosen": 0.9964095950126648, + "rewards/margins": 2.568497121334076, + "rewards/rejected": -1.5720875263214111, + "step": 567 + }, + { + "epoch": 0.9566315789473684, + "grad_norm": 42.291585331791644, + "kl": 0.0, + "learning_rate": 3.1473237471012214e-09, + "logits/chosen": -733431910.4, + "logits/rejected": -799960064.0, + "logps/chosen": -177.69178466796876, + "logps/rejected": -213.2462158203125, + "loss": 0.3644, + "rewards/chosen": 0.27896909713745116, + "rewards/margins": 2.7935458501180013, + "rewards/rejected": -2.5145767529805503, + "step": 568 + }, + { + "epoch": 0.9583157894736842, + "grad_norm": 40.929057929848376, + "kl": 0.0, + "learning_rate": 2.9189523275903736e-09, + "logits/chosen": -978359091.2, + "logits/rejected": -652150016.0, + "logps/chosen": -271.872216796875, + "logps/rejected": -174.5239461263021, + "loss": 0.3284, + "rewards/chosen": -0.7042415618896485, + "rewards/margins": 0.11138847668965657, + "rewards/rejected": -0.815630038579305, + "step": 569 + }, + { + "epoch": 0.96, + "grad_norm": 40.061462493884235, + "kl": 0.0, + "learning_rate": 2.6991326662819667e-09, + "logits/chosen": -729751552.0, + "logits/rejected": -752365772.8, + "logps/chosen": -195.5203857421875, + "logps/rejected": -209.675634765625, + "loss": 0.3478, + "rewards/chosen": 1.747754414876302, + "rewards/margins": 3.8098199208577475, + "rewards/rejected": -2.0620655059814452, + "step": 570 + }, + { + "epoch": 0.9616842105263158, + "grad_norm": 37.651701881002374, + "kl": 0.0, + "learning_rate": 2.4878723713864234e-09, + "logits/chosen": -627430656.0, + "logits/rejected": -945733734.4, + "logps/chosen": -186.00541178385416, + "logps/rejected": -253.2623046875, + "loss": 0.3783, + "rewards/chosen": 1.6760249137878418, + "rewards/margins": 4.277830410003662, + "rewards/rejected": -2.60180549621582, + "step": 571 + }, + { + "epoch": 0.9633684210526315, + "grad_norm": 70.57315668742685, + "kl": 0.0, + "learning_rate": 2.285178754864614e-09, + "logits/chosen": -987434325.3333334, + "logits/rejected": -616783462.4, + "logps/chosen": -218.68375651041666, + "logps/rejected": -293.998388671875, + "loss": 0.4376, + "rewards/chosen": 1.6107417742411296, + "rewards/margins": 7.283069960276286, + "rewards/rejected": -5.672328186035156, + "step": 572 + }, + { + "epoch": 0.9650526315789474, + "grad_norm": 69.90429499041211, + "kl": 0.0, + "learning_rate": 2.091058832174891e-09, + "logits/chosen": -730186432.0, + "logits/rejected": -509659392.0, + "logps/chosen": -196.79669189453125, + "logps/rejected": -303.237060546875, + "loss": 0.317, + "rewards/chosen": 0.6219147443771362, + "rewards/margins": 11.697727084159851, + "rewards/rejected": -11.075812339782715, + "step": 573 + }, + { + "epoch": 0.9667368421052631, + "grad_norm": 38.30542476552404, + "kl": 0.0, + "learning_rate": 1.905519322030258e-09, + "logits/chosen": -938985881.6, + "logits/rejected": -598871637.3333334, + "logps/chosen": -183.621826171875, + "logps/rejected": -228.38956705729166, + "loss": 0.428, + "rewards/chosen": 0.9899630546569824, + "rewards/margins": 1.8534589608510337, + "rewards/rejected": -0.8634959061940511, + "step": 574 + }, + { + "epoch": 0.968421052631579, + "grad_norm": 80.5219219878979, + "kl": 0.0, + "learning_rate": 1.7285666461657467e-09, + "logits/chosen": -844385426.2857143, + "logits/rejected": -1569694336.0, + "logps/chosen": -221.71473911830358, + "logps/rejected": -172.7010498046875, + "loss": 0.4116, + "rewards/chosen": -0.09411777768816267, + "rewards/margins": -0.8716171554156712, + "rewards/rejected": 0.7774993777275085, + "step": 575 + }, + { + "epoch": 0.9701052631578947, + "grad_norm": 89.03407725109749, + "kl": 0.0, + "learning_rate": 1.5602069291162368e-09, + "logits/chosen": -925635904.0, + "logits/rejected": -844432000.0, + "logps/chosen": -256.534912109375, + "logps/rejected": -220.78697204589844, + "loss": 0.2972, + "rewards/chosen": -0.9312546849250793, + "rewards/margins": 2.055956780910492, + "rewards/rejected": -2.9872114658355713, + "step": 576 + }, + { + "epoch": 0.9717894736842105, + "grad_norm": 33.63530023405151, + "kl": 0.0, + "learning_rate": 1.4004459980045124e-09, + "logits/chosen": -958183424.0, + "logits/rejected": -700009045.3333334, + "logps/chosen": -239.659912109375, + "logps/rejected": -270.008544921875, + "loss": 0.4455, + "rewards/chosen": -0.18169254064559937, + "rewards/margins": 2.001987040042877, + "rewards/rejected": -2.1836795806884766, + "step": 577 + }, + { + "epoch": 0.9734736842105263, + "grad_norm": 34.10535983128858, + "kl": 0.0, + "learning_rate": 1.2492893823394246e-09, + "logits/chosen": -1030501120.0, + "logps/chosen": -254.66343688964844, + "loss": 0.3929, + "rewards/chosen": -0.19029521942138672, + "step": 578 + }, + { + "epoch": 0.9751578947368421, + "grad_norm": 130.35662841898622, + "kl": 0.0, + "learning_rate": 1.10674231382471e-09, + "logits/chosen": -418937888.0, + "logits/rejected": -518623584.0, + "logps/chosen": -232.24008178710938, + "logps/rejected": -243.5550994873047, + "loss": 0.3157, + "rewards/chosen": 0.7040752172470093, + "rewards/margins": 1.6154407858848572, + "rewards/rejected": -0.9113655686378479, + "step": 579 + }, + { + "epoch": 0.9768421052631578, + "grad_norm": 42.15293348378943, + "kl": 0.0, + "learning_rate": 9.7280972617772e-10, + "logits/chosen": -1371472384.0, + "logits/rejected": -1195231402.6666667, + "logps/chosen": -254.03028869628906, + "logps/rejected": -272.2608642578125, + "loss": 0.3864, + "rewards/chosen": -2.0985825061798096, + "rewards/margins": -0.8987653255462646, + "rewards/rejected": -1.199817180633545, + "step": 580 + }, + { + "epoch": 0.9785263157894737, + "grad_norm": 82.82639202731109, + "kl": 0.0, + "learning_rate": 8.474962549588349e-10, + "logits/chosen": -553952512.0, + "logits/rejected": -1753602389.3333333, + "logps/chosen": -212.57861328125, + "logps/rejected": -235.5814208984375, + "loss": 0.3613, + "rewards/chosen": 1.6823585510253907, + "rewards/margins": 2.548850742975871, + "rewards/rejected": -0.8664921919504801, + "step": 581 + }, + { + "epoch": 0.9802105263157894, + "grad_norm": 39.562779110049625, + "kl": 0.0, + "learning_rate": 7.308062374108692e-10, + "logits/chosen": -836093888.0, + "logits/rejected": -541282218.6666666, + "logps/chosen": -245.46795654296875, + "logps/rejected": -265.79986572265625, + "loss": 0.3856, + "rewards/chosen": 1.5786278247833252, + "rewards/margins": 3.83101216952006, + "rewards/rejected": -2.252384344736735, + "step": 582 + }, + { + "epoch": 0.9818947368421053, + "grad_norm": 49.619503065339295, + "kl": 0.0, + "learning_rate": 6.227437123090539e-10, + "logits/chosen": -1412194432.0, + "logits/rejected": -683454902.8571428, + "logps/chosen": -247.62109375, + "logps/rejected": -278.0474853515625, + "loss": 0.2775, + "rewards/chosen": -2.1934449672698975, + "rewards/margins": 0.6917969499315535, + "rewards/rejected": -2.885241917201451, + "step": 583 + }, + { + "epoch": 0.983578947368421, + "grad_norm": 41.65613682455663, + "kl": 0.0, + "learning_rate": 5.233124198212035e-10, + "logits/chosen": -1068381312.0, + "logits/rejected": -620764608.0, + "logps/chosen": -203.81106567382812, + "logps/rejected": -199.25198364257812, + "loss": 0.3426, + "rewards/chosen": 0.44310152530670166, + "rewards/margins": 1.8038746118545532, + "rewards/rejected": -1.3607730865478516, + "step": 584 + }, + { + "epoch": 0.9852631578947368, + "grad_norm": 74.23272136164195, + "kl": 0.0, + "learning_rate": 4.3251580137831924e-10, + "logits/chosen": -815114496.0, + "logits/rejected": -602160384.0, + "logps/chosen": -206.22401428222656, + "logps/rejected": -258.4097900390625, + "loss": 0.2408, + "rewards/chosen": 1.1723206043243408, + "rewards/margins": 2.8920695781707764, + "rewards/rejected": -1.7197489738464355, + "step": 585 + }, + { + "epoch": 0.9869473684210527, + "grad_norm": 37.528975070907734, + "kl": 0.0, + "learning_rate": 3.5035699955540675e-10, + "logits/chosen": -1021857088.0, + "logits/rejected": -1152371072.0, + "logps/chosen": -198.44970703125, + "logps/rejected": -276.451904296875, + "loss": 0.2932, + "rewards/chosen": 1.2920650243759155, + "rewards/margins": 4.100528836250305, + "rewards/rejected": -2.8084638118743896, + "step": 586 + }, + { + "epoch": 0.9886315789473684, + "grad_norm": 66.83382682650414, + "kl": 0.0, + "learning_rate": 2.768388579627301e-10, + "logits/chosen": -696945877.3333334, + "logits/rejected": -635326361.6, + "logps/chosen": -188.5633748372396, + "logps/rejected": -227.542333984375, + "loss": 0.3505, + "rewards/chosen": -1.1209592819213867, + "rewards/margins": 2.0206165313720703, + "rewards/rejected": -3.141575813293457, + "step": 587 + }, + { + "epoch": 0.9903157894736843, + "grad_norm": 38.439783990376164, + "kl": 0.0, + "learning_rate": 2.1196392114744556e-10, + "logits/chosen": -732253184.0, + "logits/rejected": -485836117.3333333, + "logps/chosen": -229.332763671875, + "logps/rejected": -178.27132161458334, + "loss": 0.3191, + "rewards/chosen": 2.1135427474975588, + "rewards/margins": 2.325268598397573, + "rewards/rejected": -0.21172585090001425, + "step": 588 + }, + { + "epoch": 0.992, + "grad_norm": 84.00543667151456, + "kl": 0.0, + "learning_rate": 1.5573443450545009e-10, + "logits/chosen": -681184563.2, + "logits/rejected": -883190613.3333334, + "logps/chosen": -231.067626953125, + "logps/rejected": -222.4321492513021, + "loss": 0.3878, + "rewards/chosen": 0.3701077699661255, + "rewards/margins": 1.0806505282719931, + "rewards/rejected": -0.7105427583058676, + "step": 589 + }, + { + "epoch": 0.9936842105263158, + "grad_norm": 65.06311520099162, + "kl": 0.0, + "learning_rate": 1.0815234420369357e-10, + "logits/chosen": -837959372.8, + "logits/rejected": -1049096704.0, + "logps/chosen": -237.297705078125, + "logps/rejected": -172.59944661458334, + "loss": 0.3653, + "rewards/chosen": 1.0379376411437988, + "rewards/margins": -0.5275972684224446, + "rewards/rejected": 1.5655349095662434, + "step": 590 + }, + { + "epoch": 0.9953684210526316, + "grad_norm": 131.03458960479955, + "kl": 0.0, + "learning_rate": 6.921929711287134e-11, + "logits/chosen": -793252928.0, + "logits/rejected": -1009639296.0, + "logps/chosen": -249.7359619140625, + "logps/rejected": -276.18560791015625, + "loss": 0.4221, + "rewards/chosen": 1.2013773918151855, + "rewards/margins": 2.3829771280288696, + "rewards/rejected": -1.181599736213684, + "step": 591 + }, + { + "epoch": 0.9970526315789474, + "grad_norm": 59.274731916237144, + "kl": 0.0, + "learning_rate": 3.893664075035885e-11, + "logits/chosen": -958084010.6666666, + "logits/rejected": -776463616.0, + "logps/chosen": -219.0360107421875, + "logps/rejected": -177.7460174560547, + "loss": 0.4209, + "rewards/chosen": 0.09142877658208211, + "rewards/margins": -0.03964764873186748, + "rewards/rejected": 0.13107642531394958, + "step": 592 + }, + { + "epoch": 0.9987368421052631, + "grad_norm": 35.198610073696365, + "kl": 0.0, + "learning_rate": 1.730542323355455e-11, + "logits/chosen": -1364308699.4285715, + "logits/rejected": -682940672.0, + "logps/chosen": -254.698486328125, + "logps/rejected": -193.01634216308594, + "loss": 0.3481, + "rewards/chosen": -0.044670513698032925, + "rewards/margins": -1.6197544847215926, + "rewards/rejected": 1.5750839710235596, + "step": 593 + }, + { + "epoch": 1.0, + "grad_norm": 35.198610073696365, + "kl": 0.0, + "learning_rate": 4.326393243742066e-12, + "logits/chosen": -436645888.0, + "logits/rejected": -1217262336.0, + "logps/chosen": -184.77606201171875, + "logps/rejected": -178.923583984375, + "loss": 0.342, + "rewards/chosen": -0.0059494078159332275, + "rewards/margins": 1.3890884816646576, + "rewards/rejected": -1.3950378894805908, + "step": 594 + }, + { + "epoch": 1.0, + "step": 594, + "total_flos": 2.3597531637009613e+17, + "train_loss": 0.4100885201333348, + "train_runtime": 53730.3673, + "train_samples_per_second": 0.354, + "train_steps_per_second": 0.011 + } + ], + "logging_steps": 1, + "max_steps": 594, + "num_input_tokens_seen": 0, + "num_train_epochs": 1, + "save_steps": 1000, + "stateful_callbacks": { + "TrainerControl": { + "args": { + "should_epoch_stop": false, + "should_evaluate": false, + "should_log": false, + "should_save": true, + "should_training_stop": true + }, + "attributes": {} + } + }, + "total_flos": 2.3597531637009613e+17, + "train_batch_size": 1, + "trial_name": null, + "trial_params": null +} diff --git a/training_args.bin b/training_args.bin new file mode 100644 index 0000000..234d9ee --- /dev/null +++ b/training_args.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4b582adbea2721af7921d1dd62f54fd19ab1dfcbd86fb112da2fbe8e9e865f65 +size 7544 diff --git a/training_eval_loss.png b/training_eval_loss.png new file mode 100644 index 0000000000000000000000000000000000000000..c42df557fc7de1f701371b86a4f1f925a9e20add GIT binary patch literal 24009 zcmeIac|6zczBl?cON1mcN2MY|rKHTFL1{!XtBA~G$ds`Xnlu`cWS)vL&qGn9LX$ZW znTnF}d0+0e_SyS+*4k(7eg1gP>pbUvt#z;EzV-WkhwJ)$-qYv0cwoO48w(!`MNw?K zcWE4=C|W;?qUmI2#6OX1`uqj|v(06vzRO|zQ!Z|1PA92-W-bnA>|M^BHWxX2(#iR> zz1=3MZBiR0MXX$09Gn%TrEUN17o_Z+ETvcOr~7~pS>&+mm@`GOo00$0Bq=AJrYNH= zyERmgxJM1Oc^Dq)?N%Obd|8$y{dlqH+N91sk5BU*dXV(_rC{W}I=S>^2WpDydcS78 z?>>}qGC%Na+k=;@v>o-F%9%G+uV&r1e5=X@xp9@5kLs4ULRs zPrbZtkj~G`d!ekXOkqezee0>0ym zly{&u-eF~0BEV74H=TWyYxnNmqd$K5$HlEYv2n$ZpFdsOR!`kCryeqPz0bC|EoZZQ z{{7VkN&PHpur;%`aHJ%3a?-u%&)0j6hL#^5pUle2QaOG4+v5JN`Q6&uRcXhb#28rC z-B@S!>8Wk;gjsK0q<*Q7cX6%>e`}^mE&lbDn>QZ~W>86g#vBJ*Hn8$s)Xz3& z=t4~L2X&t{TAUjdlHuJNqDR~c&O*s(LFu8Jv=<91Jd)0)_5+Z z6Z!TTk4MIJ;0UD>x_pDk^%wY$LvI~7<)^(yT|_GdauiYvCn6fxqiCa%0VA@4c9_Q-nfGq$$m$)2ZU>aGdnHyBj% zU^D$Ax99jXyXu)gKbz8yZQ|wQdtliZljrj7P>^k=em3of<4*&xUgeybnQ@pL?cy`i z%X93}bU$~l^7(USU0q#XB`^65NZY$I?YvUBtEGsXkIeGFV9@iEti$-=^7^XWJAN=izTQ-hhYA3}!m{Xt{jbw`|!m*{3{zv2<=~ub~n)R2rTL zRaH`VVq^Liya9&Aix<0$v-6b$ry66z zbkhyWU%%#PX=&+iFW!z9Aok0CyuXQ}OiWCkpV&yY97+XV+50TBfIJU2)w(7cMXt{iu_;5FTEGwdDkvup(2!?@7uR8?ohI+we=-(Lp7IItd*3k(h~L_e>wLqP=vX^qg46V zNJm7rMQufW{o+Emk(zhOhsRn?e9c~5TTS_qH!eIs$)s@h`-N@JUs0PWyy=Rjc&+NE zEosLBl9JZ4uGpwCQ?^e>r*^ETR@Uju^YzmX^K;(xjEvoH687Txks^>~`)Lb#3xEFn zxxM-H(jAJ5oWjBp{ry&(ZCY3lKhmbT?>%#L4N6>ldwbTiXRYgFWz1-?-DNFmSREN1 z@Kv`J&u#Jd_a7ZD91#^0qf6CIuXu4?+;P1BpyOFjR0%ArkjzZ!?(Xg@_wI%2DE*<= ziV))VZMQF--I8Ha8m@Tmmxz*5NnOdrK=Wcx&xpCISX9T_aKXyau1Zom$a}-5s+pRK z_{`6`@$&O4yPCx2pps~4YF4~^w=6q5n-U3Ex3iNn_W5&?l*NMw50d2=%*Ow^)Ylh9 zOOTFf$ui$9_%01y-Q^D+Jm~E2zkoLL{nN8dP49Jm-@aYMQ#8eE;8ph7VeHY%O_kM4 zmrAzeJNGS_8;kHYMUmoFa9K+hMptEs#oI^wZtw8e*zVCMcv)cEQoJ|LVAd7>sd||e zFJEfgY_AMv3rJ0sz+I32Y*g-ftD{^Uz1?lJS>NDE@Sth+}4~rBSXVLERWEU*7W1z z6bD`zeVC@$O)V|0#VkQ;&l?+g>apEdVt1;0mHN*6(D-$J`?fBr**bgXx{eYj#cbIS z#YwNq@yG@)5F_o{ABFgccgowHFBQVc*o(}o7KQ+WCPGrU1&5%pU&3s+cOV+%uPM$i%YvH zWrWY)V)uFN&Bz-!n6dk~wfF4{s;E%Q{#;$XlX=mi#*$MSl7hEZJfkL+(9rKP%sTx+yBC}uR3RNX(Q z!Dv`99$t|VKk=oxb^ElnbLM9njScR-T{vd1o*RSN;bOyQBL3nm_?k;=|D2BVVwKWbNyqY(MlG*v? z%c>Yg`kdN9^yNH{pKHj@BOAD{G1m7~thFc#4@(d&wzEMmiqUxy=E<>IVVaqaiT&Hp ze)rRiPtMt}ckf<9v^G4cG)G|p0Rb8FD&}l$6BA+lskQ6ZSD+)?yfe=4@R?>icI;SW z!^Q~yt^1=K8T^XR{eJN9A^ml6ot6U5)3_0tty_6?!eSI$S%8{uqt3Ciu~mDnT)A>| zbTnyvnf1wEL!S~|2RL8P<=?t_v%4kzxPzNpNJ7HNTwlEu30$g=kB>oZG|);-gm8p} zUPkBd4qu)@IoprR$O;}@!OtK1#BibISzd_Yq7#v0*_I&gKAGMO17Vm}O`0?ZE zt*CKTi)&~^Hg8_`d!(Zl7tP+;v}tH57niI{|ABPBbHBbHV-=9+SiXEY9#Keo`i8kb zjlQ%hxwUBpb2C461woumE+l6qKwRlRo6`eV!-=fvAC@U-H zSU0n9iSK`>uyZCy2|uJ%2zKiHihD#*&$)BwCMPG4=>PhXZBP;h=w=lu5xFPl%!fmU z#zz$?@*2*XxZxLtg@w}(zaZNvQOq{2=!WUF)DN`q?4 z*EM-aejIGoNtN8Vap}*####W|EZ0HZnen$e@1Iz*1Muhi3`Y*J`*{}bPI~I)?cJ1f z`taBUHlLnNaf**+Y@b{HW7*X2lg!M_Wxe&cvd<0~pw$SOc#SbAxD8*HE}48_;4PkF zQ?w|MFEB;PJq8zBavo3W>_|Hs9UWb^S6hLr>xWzeXGcdG%CNv?QD(_hivHYampbY9 z0QX)4Q|NOX^fXxODgb=3`Nd|H!CmM@s%B;@f#7KM^z=Z=b{hJ0meG7Ja9zA;(IQ+9 zM?KKFO@U1_`l~rZJxirUyM;aXFk?lt)FE|V{T!QjdnNY zVYOz>8fG@OoxpB;_Uw_ddM$!hCM+u}YmlfN`&}Jycr#yEi=^pE@1%^G883nDXIY^b zG^d!9gk8SON>-_)^tyGbr%pY}^NnadN1E=(=g*(N9uTCl&IQ{owh`EN{PZjbcz(An77fFLrm3o9+F z-)j5uNc!QI0gP<^YH?*)*b*-_nEm}w^G`sp}5@7lAc0&6Gd=B-Aj9qqci zcQ3+DIM(Bx8tyYYg-*&^e@nV`y$Jw3`nP-7NAL=XwBvIe9Y3`EcjRY+)7pM|8YIH3 zGkY@#e`2M$xYl6Ca0t7wIhenUoZQX`Aw}yVlU{blk+wp&+yN#A2G!&v-GEcgeAA$& zO99-?^?#0yosW(d)Xg+bpICcC*ms7ruOa#^i|f(Zl2vQg=z3Zk#2q*yAB={hYj!Gg zX84VI$dZdEdm}_Vu3fyiJE1faFtzd`m-v-y*B&(2ps_B&(i$^p#}#!=O$-`D*kT_E zDWMOt*eWIC5iql`EH&w+$s#WzWrlM{FFu-)>K-#}9vk~2dj^#brN>(F*Q1o|HJ(CX z^Ni)6KJil=XaE3@>S#{raO`qG^IQjV(=NhEDPVu;p%U@oz@2)9O-i1q-{6YJ@yu6~LAN#x+%q$Jp zGx6pIn9g(RWl?Y)2#ocaT7))R_T|O(>MU8y1_1#D7iL_rgQKIWv9S=Td01f=w=NwY zZsT9FWJ!NZI{){fA;;m*+W?>h@SEHjvRMH!v^<*^$|_DF9@AW_C8UC^&D+#!Z`=Kvqztnbxga_dd^2Ofy{IB5)Q> z($plxt_`$wbX`M3Z0Ph+fGD4y*=M$BKYsRXEAAu|9lf%ti4&#rb;0SNBwZJWKG}l_ zL=lnSwrvqh{6bT6 zbJ5$kZ!-%C2~|{GuLN9AAzq~8qmhnD_Kzn_?xIH4cu5X@guK{>J&3e+W(wBE@p08MSB@fo>= zSa;&3^3>gcxueuZ*s zQlPF>b6}u36-c|)(VqN*0?XpXUEjYOre+4r6q-_AOL`N%U%h&@g4h#$E7qBmtbWDS zf$!H?T0`wxxDHmog&!+z@GHB1(5sfA7#`Hsg(`kP@W9$-uiw7CJ}&3eSV4aMn(m=P z{@=f!0qfFfHHuR>apuhBM<`f#M~r-%i085E@n$|Y6sNcYYayC2CmeclEM9Xz2mOs3 zH#)(=pTB&m0d;28bXH%Y8F?Nr)~{Q4UNinYp77g2e#U!|ZELwnZMwHj8!Xr}M%V!Ujt_M&Y72+rvxcy?@H4Z+#jvIbAj zTm+jEhE6iEP2h<7!h4DE(0|nQBK6T6l+7N1;&OBjhpF+?5Lw00y3ntd4{FCM2`=W5 zssb~64F!|kd?*J;r+X9)3ai$CQ)#vP89X>z(rhtQ{mJN?M|Hv zhBDKGudgKnshmxVDRdLLcC8fU9SqtU8sqbdl5b51lWi0}{E~@1_4@D;d*erWdAvlw z!}@G`W^Y23-IcS+BHZIo@2$~^2?h$P4pXt`M9Smnll3=Ou35tfS&g0|&Hm!Wi;bUs zN_}Q!png*kr>1X89`gq|ipS%i{79WQH$tB-!pkEHExCC;oB5-uCIW7bzGHqzEzS=;Z}Z*6#$8HE_tsv2(b?JQ z2cbUuO#RIbli)9oe}0@?#?9RYpoKk@Ip%X{c41vk&5tj0V*Ppg(G%NNIWDg23iezo zL52P&H{RjpJACL6*ljS%7I_8Wx#Sv`Z!kC_?H%{AOO9-(h^E8OU4Nu3LjPVHxkmtv zOj2UQhH6MFP3V)>xhQiq;ICsfoYYzgiAr!Sn|HUwQIuH$>Mw+a1w}-#6D5SGDfu5C z7SygFD^|0ns)~N3WLoOc{%96-md*C<0#L43g0qC+#dkyRP1oIy)qWu~G@!6>JK9s$ znGYM`8!>IXlh)#G)11N$oI)rc1Ud8845P_kpX_bggqqnGc7&!?S4GfktIh5|Uu|{# zJ#2)fVhX5b_2%L3+)uZ+*z!3!ImuYRTMJS8vaqsHd3ibNL+-HN{M;<3pkNpza0q~b zfq|W~zdMMxX5F@?S@Dhtvwz0+CVlSZ%PRn=0Ktp$A}=IyqK?xL6!T!`;)OlFT_Lf_ z3L@e~AXhU85e35y&K@5dZ^m|@ieo3I;aPAg&yI5un<3XA3_MZ{-3bKxooj1S%-9+Z zRM(5w>U#;~B^)&U_|~Xx&Wz`N4+o*gW)x~pT;4?9&C=fQ-_u;hWp^`K{zK39Q8|4X zw}ft4b2E2va4>05Ktd^L_pV%_pd#RYL1TCy2pEJ!qm^6 zeL6V>Lt@a|mzdBl$*S~az<3r=si3E8!9~jUoUn!zZUON#vW1CfQ)^e1U;-BMEw<^oZ zR9G+ef}+v`W)U3#7(NfuL*%DJsm(Ap-g)A29c+fY9za9K_|BscPA?J8qMu1{E3o@* zwQturc5DT>LvG#B(9qTE*AMr4Vp)Lz1|3R(-Ss z348egv9H{|{rDM+FVC+M8C+qZdv_UB>o-$Ufd`OJjfP1TP5Mp#SvI}D`@c&lbV|Tu-*>~ zV_U|;;NWV}1^_QHvy zTGCbk@S+a~J$$&z(F1M~MM)QrFU3>CQ_1}Zr!j))LcrY?-BlOYZrH$mE4kU4)OZIc zC;z{=jU&XF^a)t=5no^3tHfeM%cw$s9I#rIu)B{2Pq_lr7|+}Mj%K{7lheimNJ&Fy zA+@*@eu}1a--~^m8niHogp9e!)Eh5cbDi_e8t3mw&P^4Z}L3 zp@`s-tF_W1l}l_933%Sn zpsA+jpEnGMPWsUIj#6R%EoaK`)!9C?o{k>y9uFNlM9MX4f(6uYg9!WWJ~Q$);ezBF zu==`S5mgs?{Ive9z6QR0(hE*K9f)22AY4698kGWKV})UJT*4FV1~P{wSDSP&p6KWe zqc9@MYipUHnjJ`OhRj|CE}3oJyq^loR^qh+a-$qB zlL;$_lq&@VzE^kUDj}YyHjltS(bw0%fB(MUlk*xHVXp(2*a_FglPH6^i^u<9=yIFF zxB3xhf3vi-_Q+$2IMmF2}nd)`5zY@8-syt}YtbDnw792w=k= z@fxe*AJSb+ye8fUr>`N!~v1~*NA$}rk4 zJ3|xEkQ0q|S{8q}5s3<`wu0Oi1wdq&_2%=8O9YZ~i?AOI67MIU;?tn$h4#%WDK*dJ ziD%M54Qbz$=oA0o0S7nt(OxIW=+=$nfbW}mze$WvN0~<378an)k-&vTZFm=AG!>6e z){}0C-9|@^PfX+v)Sq~~r+zcvtDkN~d@)al9y|~Q2zmJEk#&#;g~IQ>a`yTIMlRCN zUG}ZHesp?sTE>eL8_7*TIj+QOfE?jZ3`AX94^K~XNO)_dq?TLKKrv*&3r90TWfTDI zA{iCpY-YHD6Hos6L*wppip|}{Gk=Y-=eN~2A|qEqb;~UJu_tm?g8f;q&)lCrKHHF- zZ!-({BjpY**#c~o7_@*|SFT*~Ma_n&TZYX@>N#988Vb4ts&pXA>X)xyA7En{d;+VK1DYaWY}Kk&BEUi7`=eA0 z4Wry&nH)K??AoU6-av6BHh#P~zwr2~mEVXz>@XoWJxK z+vZ!I_>tB+-$yz=Y=Vk!&xi<0%(N#%~ptSsY3a_@Vmg)5#fy>aHp#Cb64* z8_Nh$4%A8vSi!ezJtPF&(%+aW=WbRyk^e0}Y6mRxBT_NsRM?bVBO?)5Fh`DU1t&&; z31Utl`fpKD5r?pF=23C5C)D8c$RUA~Hzn!_8y2}AOl?3Rmw}6o23`qM5`;#Yw-oMueFcg6Y1tv>r8K0e#IZ~vK3PuUP&t3q{>wR(M@=W!#EEDA)q z=v804ibE6a8XOD-A)uq6J|N4*xnjjd;A_3qFu*=B3} z{Uf-|%O&;w;CU$UOoDVRt=7Y7a;pq!n58VoDq zo{M+Q%-B(z{1798QoR#_3;i{>>XFb3gxF!3Ad@iC2Mr8Ta*d#&fyosad|Hh)Zk^^LMB%54j~{P*|LK!Wn2!O zScoeMlz8O|hvf06tl-wPqrR$AVAfC%Rbi(PwVT-6A3l7b{NP}Mdt%M%o-0J5M)fg} z#uq%O(d?tidP)oc6c}h4=Fsl|tmGX!DGEzi7{*whLl-Rw3Bnea2`jhnD7jK^Q(Eun z=!kqJhe<<2gKN%K@(QHGV=}gzT3FBpsU7SZ{seG|bmx)$7(=NL@K2;->K6l9sT&#! z00YucP)v61-K(yuN(0XfP;D_1Vj!6m1tm&Dgnds|3=O8Lo zSa^da6}e}15nwLF-J7R3^Q{sWk4ML8wnOysLhGDj1IQ&JUwQ6hqL9IZwzj*|mBsrd z!3oH10|_~C^5oHcBY26o@7=R021~lY#~4E=W|M9IU zZRaI!iOzum7Ik&?$?54H=+Q@8_U+o`hj$GQgpew&w6rvMp-uBTc%*>i;6aS=m~w1C zEk}5(C4~zH#~tlQ6P|t1ZUZOh7nT)^qLCZYhp?Q{F`HBMDnYLobd<@G~K^9NOw;j~Y8#hGYPN}M@n%^;$(n5s> zH;KtF5nfRPxxDad~pV}Ofy%cx12t)#w5eYnChp6M~mM&eozU>qW5rU?(QRkp( zYCJp+H6iyTw9lNJ%@6@sBAg3xhM;~t6F)5)lBca90T+B4VISz3fpDATKgr`^Wfvow zb{`0!|HBh;DU%Y`ty`b5e7bz`;&}k}4j!moA3o&!xZS{-ND9aE`}qF7O56b`lX|H; z&|smW9L*5FRo^u{yo4fb9_lRz4^IfhkR!i2Jn4n6A-Mf0Md>S__PhbGyg&^9BF zs!5e4?7g*~%?}P1H0Y#e9|)t6m=sDG|2aoyBO1(o7&lOuuhi$d>BJpSiEldcLDRs1 zk0N&>_nvN{H0|u$d@G9$Fah4tJTk>5lo5_tB7eL_Nz-AY)iCtYL9fU!NHq6FTzfA23*r zFnZ-u{;~yKS7Y3KL4*F^hvfcS2=d)AN>V~X9q$>!>S4WoWU(AnOmd|#TClNX=Eq(t zqQV4J=ykrcrTq5oJx~W-o0aiUL{ltsm-rE>2qcrD;BRW2I<*Gqh=hKtYHC`JF_8k4 zWY?@fwHJACVO3LphzpTOIYft=(zyv~h_H_u6i64>a7}HYd5x%7>HOSwY;ttFzy+NS zNRSB#!cvfEiVq4B^geM{R#uY4^k&52koiW{q5MFt@K59a-Diqt&I27xWeE@b|%w&MqBYmg5AE~>OZBY3DUdmuPA`65q ztmfraq#F%tiaWiJ)ELUW+TXO-bOl6s%FtiTY;Djd(NA2Jgqb5k~O zndvD935B}O_H~Ain<5g4c(TO!rfFA`Z1`gAex!`3AqR8 z&MA3H)bm~^dIpA-t5^5l8C>k|_vH(&D~yyr?m_5j{xIV0XVS^;ITw0?5PB z{ZssH?aea^ms-Z@%PtZ>Btk0MF%4BZ^J6hm9jR>%kcVK9GQz+~cf#MIlzmGU;5mdf z`dkCHAhq*kkfTwQ-Ver?+05h^5l6w3MQ+t+!~2$9_4UgamEpLS|vC@8Nz0vpdslB(6Q7ZfT7BU zASTxhzJ>T4zLezM!QLd!#6Mo40?}%>iMk42>HAT>vc5b<4hkLf^7H4r$a`D6mKjDX zp@7!!f)6DJLJI_Qs@@o;8Gu*m0?*>(5R7u!sK}c)AE13X<^LL9w~PhEadZfb39 z{b+{{54k1fbH@4eH0V9K#^e!(?Mv680T$;63<+R49JH-Xcp!Kmhtcq|Aivr3i2>3rKFZX^vXsfM6N!-AYIekTol7$L@TBI^7FTm z_+|3p#0u;f^03`UMM|ayBqRaHC{r`D^DySr5)PGPmtyP%=`0OAu~b(knT3sibdbV+ zCe6^}? zY<ku2=ER)Di>$1!W*}+J?IT5EEyw-=HF*3~og@+7t$jDerveB}+_H3o+&Ak> zLI2382zLFu-Aw!&jTc_s!fwo<$pPIlFfa)E0nmiOB}JWBvW$|1nx~eHVzxyF&}atQ zlRI`E&5i=b(ExTOaIBOkOBR|=Tgt>bQ&kS~oFxr%n4$DDQ2=V@K%Rd^%u?TmZ_$WS zTZ1~}HTlO#tC6PY+;1sAEq2SBONFq4G{V@GOJjSMjrKfTK105l?+G76*)3zw13X+8k#POWTexFic0!LL8fUh7H49d|C=3KRP3eDM zB-a-Ci@e*Z?mPEK94ZMQa(KAl_LiO3*^K{9hbgOV$77XSIy2)FeoqAefCWWntT;4u zF&@mo!@~<#Gz+3|~p+X^@*%B|O@+j3UE7*0}{z z(^}-uyQa0!j3UWZfk~@CK+YM|C?XRfqX>5)!gqGi#Px&L!U|D$J;Nwy!VTvBKWxg-Fx38ON^(^inaD6q|m z9wRXTqLsLJckzXKL8mvkqRivx*K$xwxF!oSG7@xsEWZD;xTghmd}<0gxhe(V8Sb0X zhIv^n(_~}4<9(+6jtQ5;fGQ7#i7O$Hs^o*m@p?gTDv<1OL`tXXdaPEQbQ5ZTRsRw(x$j2!)P&LGi1CXR}S7{x+e+c2rMFauO^{| zMg~eGK8jcn5bmmLOM$*kp`*)L+ki0eA|VN6gY3ldG~;4EIPDm=u}zqOoFnVdsRHOw zfiylQ_tY>6mg}Dw5D?H}ga7M3(X7|>_~hPnB6sBs+_`=Gox`ey9dxTtfuRVHizJwl z3NrUVn@6tZoAooKJNyyDLt~mXN&NDUfyMnd3P5k~F~JKf?zq_pP+S1J4N5Ivks~>3 zw>MokJ}c`0@mbOF-c7B!l0th6kP|M^zlt6n=w%v5wzOrL(c#L8E1CWSLwYX5pI5+C zmOuNQ7j^zhbaWVOe!Y+T|Dv@oxrUy>T5ME%djr+o?Ujke}pawu30{nhx_<+d8d~Lf^kE}=JH4K=61SY8a zO;#j;fY)vPEWXKV{j5+o$$&$H?0=X1cGD*)kpa*tqoa*-pY8qYO({^AsRgYe?Fhri zOD|oz^cri9#KZD7K+ORvs)T2XXpw$_%Q|BH3JEQtNFEUC1lT)MXH%~GKLQHrX94W8 zF+GWGaRk!rYH-fCY&4!RlUr<}E1Fy{+~&SocsngnjNy7KoJG{{PctLbf$j#=pw?1n+3;ejALBP+B@jLZXOwk=dt&gw@2V zM=a3u6KN9lU_eXNj}c!{jc+RJ=qP<-iv<4P zBR^R;E5P4Vndb<5X20@AUNTg0A-XZEL%iz)KKyLv=Ys|y}UZVer1G{P*tS? z5u8jm7z`oqXbw?BKga4|kS!(z9{_Wc_Kb0wus3fGp^T8>=jH;s+jvMx+14C1|1k?e zhtj`l_3DdTgW<@iO3jzPsQYI!o^@^^qAyU9agf82f;NIjMMY_$jRYD>)nXTn!FF!x z2wzY>DYouyeqr3PVx*9RsT9%}(k)0%(#!uhS+4&rq3$2ki~rI`{THlWlQ^LNJ~H%< zz)l%hL`g}>^SZjFx0o0i)z(};U5tVCXwMIS?G!bgRzb?|Z^a7(DI^#P;zGhtu-8CC zjwb!h<#l&=hq}>C=1sAz$ng;Z0<2V?*Po3fMGZb|jMI*~R+AxJ~riyJydpyLT@ku$ZkabxREO2a-~w zp$P{)$^e-pqM@OogczPt#pV=BP$f?e- zt?gtaV!@v!XD1*&RZaaTrp8~2kx5xQKAy8#I69X0X?FH`wKx?BsJb3FN&$XRilGj& zoo#J`h{lrO2gxA8BFNsSsHh0POxH#bhI)FTU+kBW5q63k?UGP$VQvmfU8Dr`%^>~n zbx-I&GkOy(;Q}Mw3&0q6v=6u&+J5rmFlZ5!3f_l=5%3}`dgpW=T3SlkB90({GmFaNYpc9wCtCYkGlh|905}rM$9|4$=>f1WD4j>?}L4U+gJ|(NrX-f&hm*5x`SLBAaH%-POcDv~?rZc$?;5N_x9569H@r!t4o|jyQYvENK*O9p2&@w@_>W5z>b|-QDj)#W-rG z_aBII|BeL@3F*RxvtxjKw2;5}3++KEfvrmBUvd1B=fnU51=T_Q;6Y9dVqH-3p6Vp2 zLD*J^2Z;#maOa>9yQE7vv+e|b zK}FlM{}~BJRJ8qH-tK$2z&bkY0NBk2wRr51RcI(=X71x`2?SK3o$Gue8a61m^^-!Mf2H11f{;OLb!O6N8EA zL#JSbV6I8?uWZp77bQAo>R(g5ZN@N!`o91`^Lv1sWau4k|0qr^5k=G-u3#1Nx)g=Z-FT?ABw)AZsq-TGNT-D#Rdv>KtpnX zTo?>V$`2uF!QX*ci7BZw&V@71hL?D$*h`+~nTOoDvjP+H8HMrAfH$z-Y7jFqTQDPcjtzBnYM{NyHInIU*f!lzpQJtY_;M`6ej@G> z4*Lurs*VFVl$A@>_;&WO6;irB6Ljz}Sgf8cl^I)q39ae_{1zme6l z_56L{LS#N3mN${_QEm=Gjzla`kS|O#t=2Ik1}C48;T12zZB817%Q1BYB2Nez@tueU zry9U`vA({pj$`Rkf6#Ff^7-3L0Xc-r==+|4UpbB?$hne<`T3bX<)aymJ^7v~e0^v! zJ;?M=LgXI9c!L=-(j@4MQx~gUJ$CGkQWbV481q;i#%- zj~_3EBQM76FETSXJ%VGGkoxWZfmz9eIQ_2zLjx0PQyuz8zF{%OO7+hbj{(RgZsz(iVhjdGREgAs0muI(%F;bMpr}L;fvoytltV8_K26A)T8+yrUQiwyw<>VaUpciU9go!%OM_3&sV1sl4 z_swEtPv1X2xfnymY)}~C@%EscKn9?v5CbJHKZLz*1^n)n`d(n=$0Dtx!T?&SVD1 z-d?(85ynH{ZZyOwPMffUr$ zj9*pDzI^ebX(~(7#JJ?LkjF=PJ*J4hf#etR9Y}(Jn>8zhHS`?&h)zpmokt=XNVA(Y z#wY^cq~$bBkCa9r?}<1L4>{lv2g1o9?;(Iwaxiu8Uan{t6(iSxKqo;!?cTqQyr#Y- z;>qI_2YQ?t!`s0M`fGOzaBp%{xj6P5*h`!*| zNi%ZlB8>Q8lqzJJb3T%ZU$R)3m_m38l-Zh})Eonx!4@G**RH*25ndRX7Q_*hSO)Y| zxbuc%1_lqnB#E>P3=@R?`K7PV<`pfR;Y$|%7)?dO%qsVw&n=nSeCvn^XV+2pjOel~AGEGStV#CNN9WdDYB>o{RI2I(1l?8KhRBc26`xh*FJ z0UDuWG_V~r#>lP_y}axBRWUV;90x)W>bGL&3lSfAii0ZCaT1Ft4h7nciA)|K5BkVG z3AsyZ|9p6)eMb~rO$Afu%V4Umho`d$Yfa7AI0i$hqnOQP#y00cc0nA6WbEn3$#Ac8 zZS)*x$w3V`e1^yK_wWWh%|)Q&p7Fk-)8hn;;4uU6T?)IB?fw?=6!+ zoc40jcGTVHX=0eOs_J=UWu!5dqly=P4Q|bb5+Nn8wal~;mdO^>Lq3m)hM24+wdoG=ilr z!l>O+a@df9K%eDeHa2d&FH-8tU%c3j6N05;tL2Au30sSZea=^JDQvUg2By8?~VVw>_X2nL*g8dv*>37nFVcn zg-AlP9|G(u8b>-%_hfMvoEHwx;FWg}8E!8Qg`{KwNfe5h8VU#q4HJd4ThuUl?jI1) zgoQ$i6(mq{+zaWZpl^8Wl!}wnMmT9?l#v`xMUuTZT@v|2*Ze{ajm9c)Q{Qt~Kg8xj z>WUu00ZIj304k5BCT8&(y_rJ282$CD2GeWnC7yk_& z60EX7A7BlmQ_7;&qw>&@eb1pflL!?Vqy%h-pl=G_tpC$9X%Hj_yk&#{?_&xW=WS^r zH%tH>vb4m7z@C+YF-WGU(pQwjvwVPTEb%N2(#hbdj7^JV+t6BMP5!2J!V1Uv*|IoY zlMv>_!3+-0X5Plb#0~tn$)V9@Bnfb}%7O9!`9b4z!aI%j*L)E=CyHa$sogvGYhdx=Q-zH>%3pycb%_iE%$b|%*=hoKYsCdhihx8ULs*6fgtFTx|-4h2*N=_5Mr8$ z0Q||VzVSKmn}n;fp6f#gOV^iX&d(rCGgn7j2UpwY=IrjzoL!zf*xwYC5WK<9Ztd#o z=prd3WcR<`An4$1CB&hHI|9Ch*ilX21%fEd;C~Q#a=FhTXsuRVN#T)4+G_gCWTt`I z^Qrq&lhs4U2QioIhua$Q0vqv!i3K)Exf2W@Q<-egLFQy_ zIU|+k*str8f7c#Y_kYQlV(X8}xpSp(Zeq_@uE#w;hPeK{LYn{ISO3Rn5GJdA|Gxi=q8~g(7W9IKVZo&IJg(BOmO3j5 zaR@0`TSlxhq6-T69hpN9CcTR5Lq)heHya5Xozk9;nbv>C*wnNsASZXeJ!i8iWspE;<%kU2j;sJHmr_3nP`QO7;s z0B(&$=FmR%qfV3afXDed%HvjTQ4Ezwe^cM|*Z*E>38at>+glmp{dDHRU_I5Xw-I~m z#o+XNul2^$qn)kg{NmX-RaONb9~oBfO@-Cr(kq_0++lcT%Nqzvr5ht9X1F>9dhiYj zz*2}E{4UPVgpso+EESgBDR{A#*1cn6k>a4D+CeY;W+*7X!m5%0YL-Np6wHqV9h+#j*cGm9&|v$NP(xsYzlW(vQ2Dc>i3q0*Dj2fjR>7AK$b-no=0WD*k)fb;Rw zr#~G| zM6!en&-up1`8|*N>S`viBB^WFu9;RC+}G0!EBhr$I`-WmRx)Ix?u2~dy_evPn>XX$ zzmEZ5K{MrhQn+~zhYHmd6$o0dJFy2lkBErKTk}1f2~8Gx9$&I?YsfHgch{9!)*Gpj z$giNHLQqt*eFswD@K{KHnN?H6tdoBki;2Z#obU1qw{h7Wd&^3nZZ;!jUWpXwTn8)J zoVqxha;%CE{XKUp^4LylkKSh9~FKSD$iaqw1~Ln%TBY?M7c zCDtpkwxYm_&ixhHPs~EcTjiyCD$grV>xHH!8 z)Ph;u5xO{CklUJjbB|L_jtvSTrXtevh2sLGQON13w;9giPR2zPsbD3fYwXD_%}QbH zcjjj3+`)Y~9Z2D-Pi$Yir!zMR&+dIg&-u5*<G>p+HoNZ=A(29L$a;|GGtNIY z&km+6e&=Xb&PH5^po=||i(vTuorwa?AJyOV4i1JC*Pk=Kefw5?KH;GVro^g;m0zy} zdiU;KB?w4U(bTQkNFno?rr<;HyRg+S=F>xroQk`#Q(Am>`+xiwzDf6uyiDto+m z<#bV1F5Y(M$Xl<*1EnRkCUUxT&+oR4%{8P&<#YgPiT#A6;mZlPWLY0c*`qm{ai4<; zaraf^d+%-anZuRhde-9FeMZ5$L-TP^Ex^gp=oA@-|NMEU=48DZLg&7JMFL{vdjId= zTPr3PjF9%DM_lDId#|=YRi6Pxlj>XT-ry5Q$@I#loYdDzNuk}TV%T?9?-h)USSDYs z6y_~l%lYsD>XEyUwy#>^F?8)yKHQ4+BZI`3KE8pV44-|5Uor=VQH-}pC}a=WUwL_L zb=~YSJwH7fuJfth^ck*rLH|qk#Fmzpb{dqYpq_fbE||WL zJCj1D^FQAn_Qm=Jd9D8-^aHKHai?1>J3AXUR`!_B_k1S<3$8qAx390yAGC|WW4sL2cI}7D=Xutq^52yIZzDR8lHv^iN~s{t4G@7uIPhp?wp)~ zj|tXn-WK)To(r?HdiJdJiTw}mW#k;cQB{IIKb`yGrkU9j&?r0G+fC&T8!3Y9RAX76 zLBkfYGCBDZS5#w!A=`;6x2A7k@hJ$?`s0COu9G^@_{pG?i49-g5#Oy=`kOJ>;Y`I) zq(#))aA_dIbjZZ7_Gx|fdQ~LCG{f)wM~U6`Tt`Yq$7sci%LloqlNM2>Z?Eg#h{5W) zxmA>C`d)0PG6CHX~FcZR>Y zE-AfLoU1Ho~j?iqzoO4qP)kX={ZBB6!x8qS;84l zUbwfWu<(-DxFhb}oD!;wpVy0(w6tQ1wmOa$vtxz5_HEPLN1r!+1MO$r0~-xmbJJw4 z7XlBvzP!9#@~o4t``g#DveM_jdHxIyt*lRN99VRx+|sLZy$X6Mx2@3CtBsd+4gKNj z<6bQjbCa~{;Op0~TnlY3zct0wjs14}RD~I%E7!h_agA-%>}UzMZEj)MNKX(A+w+~5 zPquuPXR>2u<11MaOSKD*;=QAz5uQ7L{ZVi))zYngWo>QU5W^zn*nMk=T}(`O*tcJu zcc!517xk~cjRWkPd$NuFpEOY3hmL0T;WWJFTeGdiOtER8^FDMo0!R**-8vmamA36d zJ)1Po8{+!IR-vFFx=4KZMeq1{l-<}j;<5PyLbjprzGtsoU0u0OYlzH0$m6_5_D3;G zJa9V!O%VO?p3G&?xd2EAxL%l_uc2E#8%~X8@7le~*B3LdJeQ2J;5vDCm&a(!(dLn^ zuEpu$R>T?&uU5jY=SJ-Tr;H3snN?5QaH)k;r@;5=REN4a<{fbS<_GH&d_qE))seC- z(0Q)$SV;&72!OhmVgbNdUSVPM%DC+5red1U{s?s`8D4SHxRJ4uQFMR*gYbrFOhbLo z#gwTqpKKtAw5L{9E7ZrQ!*d+?w{G1E;M2HwPgOwyp(S_b3ibM2Ecvr#58iUHvjhE8 zUPFTv)I^l~s$Q$F4JKmi8B#MRm_GpEp%uV3RqxX+4BxeDO%%SHxYEaq+2A|=+_Dcq@&o!4B4Yl6v`5#( zCm8Os;OOAex!I@6HUrY26+}#?8eVUcwa6FyAs26x7)?=dSS{ygCtJrW#n|%STFp`$ zHG#jiq%KiNZ=y?jJ|!7PfHE)*_kicmud;frqVB$z)-QR_au0OU!D5qQgQ;$tiZ-LB zP*r0*2|7>^BYaNw)!Y6aoHz~YTg^06$hFiSOh$s5L6PBm{Q0#SB@tfM{@TF_mi1xQ z=fXl|b@lMMsgO%dt}OX`GHP=NiYw=pprAQTHy{cOe^7%h)Ss<6pqyq~f(%TFv+6kz zYH7VKHx=r4en|lcbO_J}Tax zNl0F&q_xUd_0J!{6SI0UrF%f~<+}OFGz{3&u*!=Lc zJ|~CA)!qG0QPQAH-rtl-(D=xi#nBixKW1 z@u%wV(a{mOc|{OJ0G`njQ}BbH4ihK})8bm@>Yu-SVHEC+ z?oN}S@$~fcIGRtI**OYYy8!THHb&pz>r-M;hsh|wgGk1h$q9{LEpb13^oYnQ-pm-Q8@Vxi_8W^6Ou*{`=Qb`e57zdDD`t?&ACed&^Z5 zoFvRr#{sxHoT2aw(-5kqef;>bUWLtNl~`7mq|39q^~aabk9z!O$6b0_VRSk^ZV2y9 z{M9bsq~`)lUWDxk-g~%qBrrTvj5HmaF(^<&@wpqCV-Uly2b@Kn1Xk_}T97QqLVrc)@V* zf*Da}H^WWK>7#`V%!coQ6wFP6gM*W$UJ4EtKFQ6~5HoASi(-{w1TdE#G@y{KuDf6- zQeT?R7vwG^-h0dv^BdP7hm2mbz&H-j6H@>$;)C7+dU-cRgcxuIg0^U;P;g3j-`&6m zNURZ{nGlyA8G4t6U#)IKM!X6B9Y24@{~<9fHOJQ}HjV~8eEM&yLrBj8&9XwqdEOPgQ*^bqFPaeVA-3RbZ%JID8GKSkqXtIqoT+FV>bJhCXrIwhSyiT`{W#@z|70)duXPkqtix45cts$ z59)U2_5(*d0=m_^0h?zuJayEE^0XCGCSTo((U_0_v`74{rtg>N*Qa)MN6Ng%E zi9{hUP9cG<-QV}hKq(T=ANXn6^S-<8=r5C~MH`v>FNp=ES{|EElH_c%T`(_5Oi%B& z6=h15@e(}&osa7T4)Rq(xkCgDE_L$XMzhMKOaNL@ke?rkqT{b{0wAjF`R|-ku!+RP z#I_vG)VA~OqAf)wuePyd>LYRhsrAs~D z@P|vzvuu@UbGpc2%?#OepY0Cbh#SIDsBdP?1n|LFmg>=Ro5`N6-W2sa_fCwDZvxW6 z2#Cp^{q>1_NdBPx~M9bfO@P;qwBZY&cEAWf)+Ot0VVTSYKV79gG0#nB4I+ zD_*z-9b#b%DJ9b2^;zNgLnI_5n;__rLiMN4eiXS=o&d#dfeP{=Gt;c0)pEjnr+c`{ z?K-H;<5P+f9Qu`x90pIGl!Aia?L)AAra+0obw<#%u(!-RH`-L(;VFV!r?rDatk=~_ zn&^lK5)OBV4?vf5P4hcHI+Kaw=$5aOBbxP)r;`sr1{Ijpv%Va6A{6{FM-2$F*~$Le zA)tv7FuntNLAksP>SLb;f-5YOTpuDfr+ff}!tHHmX>cOo8LIa7f<;x!*8uJDW8YE* zGObcDKl;CVE!h`bwSU?zss$;6vk7{qB48*W5J4xPx#d+sA9Ug(+AVT*~N#jH5tF9XOL=rT8Ve!vE;;-K+*(oxy7)0kpb){KfX5ka>_@q&ilt^O1(eG4s{4#WDC0mU#P<+4@7cC z21RyX7=Sq)``}-+^z`|x4<9_hDH|cn?&Gsf6?fufWMZ1x@|Xo%1!7r&&I`6*A^#-% ztP6(CxLCjiZd!IV09F`m2hRG%P*E#Lr^m7HvTHTka@VWH*qX;s32l(8S> z+lSnIW=e3wy_!W|Uw?6VIg5)jfIUnh0)tghRK!iwvht${R%8GC`Sb0?B<7TQ!f^c8 z@J8bm*eM$Tr~|Qb-@t$*vh}u3wgR~Y8EY&dU@CC$2O@>k(?}WB06ca$I-hQNNH^Wl z248e0!_27wWXA?uQNT+X!6!sh3xQBS&8~hIY?;in{kH3@LdNr_i_8B`K!FUU_Mf|@ zoPjmt-KF0al9oWSf?@cH53`baCQ=^GA|&THXrGlpA+nJCYjSZ+-(~3%ucEX|&jrC* zHj5XEA^>;}Fxro~xfoYldwctM73E#tIUF9M+w^ua_N#eoK|O3?bvG3#)iG=C*l%O@ zie_dUpbWrL7ie$zPzfM|K&J(49iXHxG+;bHPAGJ&1I%H--d0|(l#=h0F%x_H=d2_OT$eNCdW9~`=V3Pirb2+zUjiC42Bb#`fRE+h zzmuMvF6-7ev7K)cKGM;d{rE=fLHQO&?wksS&YQ6N07||8ggpkt27Sc_jQDR_{b~Wp z6bgR8AU5Gjv%fJVRFeVL3Cs<%jmg@7s%VmlFT*|Aqo-X9YFrecisMnyq-)*tqwr^s z59wiRNPV(${AokOiaf9*uSXE;D832GfCUhN!~vp2K%isXkUiUdc)C~Y*IJ-o20S{M zT}rqrcDuj#{L3&7o?p3`6$wG0QSk$*F+1-}2vjJsDes%;4;32#ez#P)tqA*`dO!^h zzB^22Q=nbI_w8}EH{=8~{6$GQ&ZGYrY+(v*;Dmv0hyu;iEOo@T4G`)3`ubOi8s6Qs zvia6GFc1uC0#N&(3|j&t2sB(goH&7a^B0^LCq6+z6&IJXk~O%Z15z3dWby3WTte9l zhjzgqqS}C~z+uU8R|QuvU7}#>SWD#j^XJ44G-%H2$yjj(5&Q8-U7XfP`YYG3 zlL8e)lGX2Y>tj@K7)1o)U*DkO$<8!6Z*2mw)C`D$0NfId0lM(@c?~###Pd3!fnV5s8v;?KR}X{I?vyM(JcUS4X~*Z6GQIEOn)#8e`iVxW6k;T zO7g82bZ}9EL^(7NbAVH0C z0(Gv@T4c73$)RPoH4^SBtV6J%Ewx^W3*dbqAgTZO?fL$9*m=X+zfqHBwust0U?;f# z|J0*to!Ov;3()}xx>IJSCtg>Qp{ht3-Z<|JvtZadvK4&-V-aFJoMcecVY~}_AZIdZ zvQ$fDpN>-h)8tW4KNl6X0Y2aecxp39Za^|xK>-6jPj~mAH;hYBogHpPC?yn<45l~0 zZ3{lyS+D@=<{_Xf!hqnPvogj#ZO8d{|B4BqL96qiP_FF2Bcnw1C9T3m$lMetME^R0 zd6>fgzgJ-&oq(8S3OY5h!%L7jpq6NPR4d~D=gbv$-vOu(N)L;a`|g1@5zGJXC5I`r zk2%ywe}0(D$<2)qII~{4H5D^?=(MxXT%0OU(u06WBfZZCHc$>Y76lSnpW>5$#`hbfF)W`ul25JCzO%{VK zah$oWe&TohJ45@ygMgYE>2c>yfz4O7OOK?o2~>)-BF6p%!{0h3LZ z_PoJQ7b!dmwD8Q2AMsWO3!7n-0~)m1@$OP3K=}{^`!7H)g;lHU@^V_3c>$`!ygFJz z3V$6SDUNV10uKRZoiNEGWc6S#Wow`ev^br(-4Zonud1U%S5;N@`rW&csb-F_kK2B! zHW2ho;;Oihp@Lj&2s?FP=X?{0U=+BKVOvpg8QHs47jJwSIwwQvt_)=L2$uPrxOXL^ zZ~Q1*cAoSD*{KWGGzjH!R>i?maanK?F zI}1VFiFbM0}5>(v{u z%+cxrs2zY@L{KFFa%RHFP54ZJwE^hQZB5NfcQiDDv4AB7jg(nofyMJrhyXgaH!wk) zPT-Zmrvp-9;04!(ZENqTrl9G+X7748RrC_faC5ZMBVhXyZZVRQl0gT&_ulh|$w9CL z`RDV);31&+fON37wHQTe;#m%iw3!Ug_6h=xa%cjYk9@R7+}tGW0L{80|G^M z%uB%9;kU)dQ)g}fr?L?U+~bu_YWMEZpZfq;LEQVWM7Rz_w%TZ#)Jyw6zQlIz_%ME< zZ5MB0$cA9D8 zRQWoE0l)?=K*L@*83ArvEL$wlClV49|78gfQUsX~K&LSU_XV-HoF1K(wrT@&u45A@ zz4rEPz|9rGU?QKqcc?!T&G99=3PU3p4#Kp0v;9h}DTsDhmjr5(P1X=+p z%d5TsHR-ahk=x~U;9^9v$+2oA3ATdDA`H9-r~#fEfRU3Qa0F3yqapzM5Y!)aJIqSz zauPj593Xyja)wYR;NMLPQ~F8EO(2$l&&=?jP7a_jir_8ez5ZhXjJzO#3JBNHoxj~F zCq)JoB+MA~^A10Rp-cCz2p>>f(w=4zD3Bp>5Fy!xb7E#-{;UN}dSEZ{ zo76E2)okJbN*f3}t&#LXFck%p3VE;YT3Yh_Lw!2=CBERx)DoLZ4GP2Aq!q}t_eF=d ze2nPts*orB;@{YqfnyvKu3&-io(<=9Y`!EmJ^G@!6-Wlaq!(fbr5zN{h`%UiOlD=J z-XX^7MmYGYcpqnH=Xg&>Jk$s4MP~%$?Ms5snz#Gvbdzv#hl7Z-l#B$ixaCn|Hzo>n zXSHEVVPuJQAE)hj<^7#XU_ZlgK0iNS37%03JnU9kP{&k3?Sdmb)xPTL>})^iIqua) z+p+8fHxqD0OO~8YHrHQbHaF5S$6kDR3TdX`5ySsstR>UHSj{Xgy}y^dg~eds>KZq~ z92k~mlmV})@!ZkY)#Y7v0Vgr-;CI3_SY#9l2QX}sJA(VLp(`~6G!|$lL4Nb|b2qLK zae9il?&CnOWH440A(>=nNXe}aJt8*(c?g7cK-amh&+7<=S73kw4?MsC1qvWl-?^JE zd6_rel^STN(O_f<3kt-ea!)X`V|KJX-}It-LW}b?@=q@|4!OLkksy(-1$fi$j&+#Q ztg9q~FDdg%0NB+3o9eFEz%nmz9@;=>5;Xt{7huN zX)eMTm!2LCWD2M;pxuTzuw@z$#oXWF;7@>abbXMX2Y!Tgsh%)E$puE$bRfL(Av_!e z{2H{m-vc328<--a>XN4&)ZIP+jKV_)@$vCWWU1^#S@&L(OUQ19;2Gml^TP8?Rnba{ zaS9v|!W3+<61W^Fm%n%6u^S+&(*hyCsCM4~YIv+`6#Vt;4cK3SU=T9%=i$9pKyi9U zM#5nMU|?XN62Mag9*q)=vNKNScmZsOK|jm{OhMBF-QKA_ofpmk?-_A&V4m>t^9K#B ztbzC?2F&x#ybwvC3S{g)^33}A^F}a55etO8yBV@9ps>(UE85uDm%u&=Eu_XZ087UNFO7TwPTH?h`yY0N?D~U+xex+AErrW>G3v!U@dV zE)P~ye$FCMS~`mP-W)94m-@K<@F05$UeFxmk25F~hIPNYcH0FqLX=D$#pQjrY2 zLIrW)!k_8=_fhck7RtbXT>O8%QlcitL)wg`J9?g-$0zI)$ztr{@*eB=j`v)xTkI%V(|$kjKx%jgXz@SeffV%)4Rvkn z|1Et;%*{If2lvbo@B?qt35S2C38^c@i?4J7?Ii5qJ30paX4M}R53klsq8l#` z9$seD+%>%PRhyf&D(63;q}0^sTqZhZHqL7`J{EF{$#H|ojTOnu8$NZ#4CLS0OjaG@Z14G!nR?*F%o(t# zq0qQO#9YT7U|&Ed3L^qXK>c$+g&YrAa*}Mc^*%92^1ZpLgYFqCuA0qqOC|+@d=)9k zLOL!eJcv4WMP$T4)pjrPPtr$9SGgpA348qp;?)Hus_RiG+s`(bnMD7JZzyg8D!e$g zyJs>GrzdE8M8Y8=D_cj&xQGz@B%#09c$w00<(SF3(=%KD6)TGJ0EB4T#|J}wk%5{D zTVhr0wLNQl+&J2{#5~!;3Y6hNc+FBw>K4;$#j=&D z{%xE1*25VaE@j3Bj=|6!W4X(zbaYlOa&=WUN=5 zFft*ynlhm9v9N8}Yx~r9u$qi{lTEK%7!5mC-kQ06``YSm#e{=_iXQRDjN_qcygv6T zwcUyZ>PKA;L_Tm1C2h2cVWH$qG0!-407_71(E%eHAiA`8+m7fK&Y;1^!IFMJ&l>M|6 zZtI=jHP?7dD9UOdh{+Ynl^8dR4AGePsTB#v18Rb5am@TcOTaDaDEy?rle!|($Awz1 zd#|vp*)}Ix^;e!ittZ?LRcr2^rQg01-?AGWTJX^SO+C^X&Z&JYg=^T=z(A}($;%kn zhJOnCu^~+M4ZzbRg}_K9{pL(4+boPwihcL44_JuDC^|WnE_+3xr1$>Yme#y~n6EHE~ zJov@kk@@voa}})={O3AX44>(%p_uNcqz7o_5|YQVJsY`t9}{PeBJmq~xu+5xd%SFV zoi0X5{&~VT+^88n)k;y*EMC*_#QqW2uXxkCgio}`Sfz(}s+n}>E+y5l{ld|@b3nD^ z=wT}2++O}J)^7nfOv!IQ+@K4q{Ywy`U>o-2}{vnE6hG%-U3$M zJF=o2+8o#{Aambhb1>9q^HXwk`<3!2Nd410s@Tg<==by>{|^JgRZxOf zRk&z_+E=vOwT7qN6*Y&EgBx4dqOZ_h2EmL68KSHzo#kYP*6{jxaXWC-$OH|I<<4uf zS(}|72s_;M^`i)7=e)s(XZqsB^;2ItF#F3j78wyy;xHvUU;zVLpd#YK=n%*Nn?MO6 z2H3y(y^NPIu**e6MM>=1$L6NqlmQv9K(CY&VMY6nHHjjfL(JOcyGT*dm_YZ}!o(l? z-74fdZ>SJ$rneYMtj(!LnOPrINScZ=YX4pj8e32;9*QNrDAN^RA8{i@eHI~=lXAlo ze2=>8W;jbNMBXQ!vMM;-rBb2jI`o^gg(^3fDg1@*21h^3i?Q^Q6f(Z9^5@J{QaW#y z7&HC)+0Rt`tk98hmL)W}>EIkU@sVFf(~`w<>Vaez$3TlC%nBv8Tddypi>v#gXMWd4AA+Do zLRT=2D^|n$W8vu)>3rQ^)qQJkv`tg|NcK$2yxUP^EHtQNgE3n!Bu(xL*P2Dd2>;M9 zhm@PYRoB2Dhul{5_>q+*M4DySly3@2$EMj9*x2+(ZN0G&O?3huFSa*R5n^W^`tv6U=nCqozjN>D+BMr$G=UK% zCxTKGr_iM<+x^;UZ*jg4#XeM>Zv^qlMnsoaxRJo~`R?_ern!FpnTl>(bSKjz;UCZG z9MfF4OWs`)x4J_Nk}}92``QNe;afI6oA(qNu)u;*ZG5j{`ZATjoI(>3g##?&^O4hRWgh&G9RUnPb) z983kNrvfE6Rn)G3!t(1o6erPYa0N329(ukr2@9(kp#8zJA6#jHf_PhJ^Zzq3 zxfui`6!+8hm-WqHUILcrPWQjpD+890C}=cllKVL_7)#o_w422#r@41F%g^M?8G&We zP4EXaAo)RJaJja-a4a|t_2#czOOrfgJ({g)aX*G#%USB6=9F5^!x^ke!wSWRA#73$ zo4a`R=9>kv>BO5Vq+@?}?@FW4{#=%bm9t>M%QoRm8tf6hGVA&cUv$*yO5M{3hqY)w z$KO5hDu2OrG~E&dww>AifeM{u+o8 zM_gOf=V9tp=~99B+=O4;xsiTN8oQSNZ6drX#<=TCyMG7LD?zn@Q`vhwT;628evnnn zW;9*=Grz{(Xa;*y7>HA}0yh-if^u+W4Oz&^6258s?k2;hGGCnt2K}p+w9gFsp1!oO z(WmP#P?+NmA{y-lcL_6)aTJh&T(M*=) zzy5Qw)Fmr2xsIsemREpW7#VZuQl{RMEmhB14l$-gPF(s zM|$tbmIt$D`BO`NIq18S@Mj*oAqejBb+kNoQJr>R%D+Fh-g4lU4xu+zs(P`q0U(=` zG&IxrsZG(pvCZFAM-h-*x)Ey`_v(tG0u>isrF_B-;jDUYI(c=DrNcNAg2$Pejxv}G zr{Y_dqA&`+%jQypdUoWy4{ z2{RJyuLUma#9xk5wOx1D(BYKmpOp{&O&$_E_dY)UKpN+$9AGx`l*kqsUarNzhLjoQbVN@Tu( z(Bi0ELL@)GX=Zz$Acwp&ps|6Y+44b6aVz)Dkj4fCM$(HLKYO0@w)E1^1(PJPb4hRJ z@`)dPc{VOt9qWVTWNi^O8`gF#vF5$UWm#{2`>{9lK7HAkv^eYK!%j71)M?L@FE8F$ zA*w)mM)U(sjBp~GUJMWsd0E})B%h-EDs?W$fU8&{Ca|M{Sf|<~T?#QZ5kjroG+}xTeB2Z`Q-{vE``;qW4YjDeSx>9yvcjsoJaAZ?ti;cZ`f#O5eMEKWL+U|KG3+}$ACx%2-_vqvw z1OjSD2?S3ZmHT9-5v^g|4>cSo|4@g^BLcPpt+e=QzlI>DF}Y-l?hy+;0UTRHRQfvB|u0+Mj%fy}?G5T*HDi)Sy~2t`?*W2$^cJE__t zh8bmED-8j3p4kdj84+<_17Zg_!w(&%=)Q*MJSMcyMjbo4oXy_GEzOTqE~F77-)0Do z^iW?5<`6VEd&Z~nn6&NVY_6`d6WnI$4D?3Tb7o!EfP*L?89%P;F2fL;WjGEQRm@n<*~hI1J8o z`P=g5h(G%B!`9QLWkD9jb99Qv&@aY!q^T0)mUf}8!Bi-O{$IPY2(Q;KSwvl{w8z$! zlG<@-Uo+%>q_MJ%we}?Ah&?o%^T~S_$EY;aY6=}{2slWVYV>md!sGj%w*QJC1Yeke z>($lUD6ZM#U%6gPTN&w>tiM%fZQP{{;`w)1Eb@i%!Zt|dSruQ}4V+))?>5JFc+3p) zgq_?{anGUVT(8v4^B9Z%3N*u;8vBV?Ih9jH5}@Se zjAQb*%u+3K$&_m`)3LSLU#)NyC`%Re8-mcox1PtC5u-}=5{Bac3WUr0IqH{1oQkX# ztTc`KbPrGciLQFhy$MQDo^5${=I`K>o{Wzrz8$111fA7+S1!#a@#-wf3JhSI&on+2 z>4}7$Ea1N%zVXv3josA34e?T3dicQr0=l0xgp0NtU_qLsQMt)+M!7OH9Nx&Pm=z*? z@Swv~wa%FeUJo6(tW){u`I&3Ks1r%A7&C8ko2$$Pzfo9LvY)QKz3Gq4&L5 ze76*Y9Rb`R?M2u-A_SkbJKjcnN5pBDnCP;`yhMh7INjKB>F*>ik_LoYz}H zOY))m`5qsAadgb5GWSXS4fj*ds7O=X1U-nO3~EDG(U~qMiYu?aSGhPXo$Y)+E#COx z8vVD~#6QQkxNcPF24a_Nm=|}RWrgOt)txY|}Z9IIQL76fr z-v59mVo|Is%j8dKCHfQ2>)+lKL?p>$Jz&S_if^%7 z7JNDci$h8uCT7%a;%QKo>{BX20$l5=cIK zBABHj-NX0HtY&wB%A5VV6!qs+bU{?;WCePFDDh(-;Lm+R;Rv!#6ne%-;y$(7$J@O& zW#JzE8i^2dp819nyAC}Ex33yiWRl+d`*_+TrkLf4=r5nj!ZL|LcD}6?KuhBO?a9!I z*vUzpzDSqEQP$TnI~i_r<#>2lXy;~4_V(4`@$22p%S%J`BBaSnbZ7&HQk(kZd72-8 zaF9A5yvaU1;tIt#<eY1GJ=lJmxA;7v@9q)6n zzs=I}28NmS$i_Et=Hg0jG90{T1vu2WxkOy)A1{Fwtw>_Qk9w6haod_ zGy2K;RJ0)uc7-%N))~@Q@ys%Lu3jtTE2Fj`OVptbYjV4O=ae$Kn~f2DSS8GJ2Pb^m z`O!<6&%Ij93gFzB8$E`~XfT9sQk6fms(I z5+4;t+`O0p=CMLie%{NuQSJUC)-6HW99R`tXr}P?b-ND9d8!&b=IZ~LDcVSoESvf1 z9e%+B!q*BJq(NEuOic?dT<&=eEOLd$?j&B6@9NYpl{6VDd*NFmE1Lni+h6d6m zHvSHt*^NP{@{HgzuJH4eMJxE>ZM2zWWcO;gnSkABnT>e&&9viZ2pF@brfj}v&Uh?b zd5@V+z7VIcf5@yw&{QCf?iWsCJ^lHU$EM+;Zjv%2FZzio={yWK4{@wlowGlUT?&=5 z-Fr~fJ$_I}?Uo+zSxXq1%(y5x-RR6hj029H3-SyIWZXJGoGXhfdnf>*+thG`%k|#} zaTtwW|8`zs4!ymtFoE9{`?jOtbq#AGsnS+H-!a9LFOr+{{yG3iJ({}5wLI8|l^bKR zU2gNwX9z(N;DOJ z!`WV=&YvLj=0659lUa=`6sAC%{HEj9@aAJv$DK}lZ(fF#v!VqTjgPNwm)x-@Rf66w zc>=@kO$J(m+irD)jJS?!cyJNO>QPIa`|}lQ`wR(ikecQQ@Ig$~uioYd1ZhVi5_p}I zQj`XGPN3g|)bHPLS1#u+w@&OUw-HLH=D#})FEKnjqPh~vokQf8}zJOj^k z4=R(JP8zs{<>n?4$*N8BZxg*3@^tP<7!3MxN^Z#phSSgk3nK#+PWxf-3eO}`e)2)$ zTk0r;?W*$&o@bj`^4DnRPFe>2(51<|s_GopnTnt11Af27%j&`5tNR+quj3B|PRtK@ zn~aCP|I|UCF)Po{^@;56RH~X}pji}pxB8Y|8-LsScK-w07PLQbD6Cw6ps}`Nv|k95 z-%*!CI-FyDe=a!4MY)#}NJ3vyDPN{0^;sWc*e%3o79h~xKuMc0 z+cpwPG2wl;=g$kDK2O6OU+emqcd?^Cxs_=*riF#85GSb&FCyw%Km?$R*1^7ti!@S}c1xC`P%HA$-= zt1PRw4dQ8lb>t&$0|#m~{*4um;}HUhNxBB>+2jS2vDg`~`9&v;@gKc7yc_7QyHN)GS3%?F3sUk50y{yAb)ov&Tj%nVrTkhfQ7vnKnO+#B6DR-c)&7H*^M z?0CMG8$3q;TFVffgRdQBO!`4byAoJ3H5;y=0<#&X|9X%Ij$hIO>Ra`YD z7oMJ*kw`f?KhM77DlYxN>BIKvsKG?$s~VmD;4hSs+Xhh>Hf?pu=|5vjF3yU(1b8bJ zK|b>tNv}Ap&#-s!X1CU52k~<;^joi#kGnJ7$Lmz2>SX#^-Rkw0G?=y(X&!f6={sPF z{q1r(NX=@CLZ4mA-MPBDXKlW!8 zNeYO3UBjVTXuZV?oZ+qoLMQFS*Box#`Na3$a#2QOvomj5*4i;|cOs69`({Z4)B37!&a$`=D4kbQ$kuGaY3?^atg|3(=wJU?K8s%v!4) zJezu>!169rVsNgU98r?kp=qbqE$r#@Pfg%&pDYJ|82FC)YOWO9$@|Vx!qH=?^9qE9 zz4iEa^E2*QmjiU%5H3_xL!3ONd1X9i1ugwShK}xgJfeyy4;C6)W?uJBicNJ9eXKI}Pd|52Q%=-L?90EcWw?qq*UINpg`rwQmj- z+&)o)YM~>vr`;}j+?BBg-tW6@DfvSTIz_j7_5tzA>(|p)6j+bLvR_7TKj6N*6Wo?K zJejo0q=BQ1SBi0(sedD3w)pPe-jhKk@PD3qUR1o5?6VZB@ahsWa_ZH!4mEmiDegdj z>-0NwwF!%qgte8m=A_e5O}kPP7+;Q~^thh>_kp9NC9Q7~^Zoz9*jqYeb+kAbKZ51FRpHF3ytjZmCMT1-ELnP*daeYQGShds$fe(SrZQmc@v;)%1`9k9(d_@ zw=i!%U=sTHNzCXT-a)4`cKr-q9! z3{{Jp%4C}CJxzZv-DXiS(UFH}x-+eHa3OB3fi-P`LeVy;S8;BpRxwkx)smz}3(LnT z`V^v_kfRPU>q4~3?36SK?aD+A<8Iuwc&ZLMMeBbb6PggG$Ou)UZ<*>PY-s8SNOSXQ z1<}tR$Z^~)zjc3>D@5Wxb7qWz2D5l2R1Y0X`@N$bGbQ{bQx6XFf@Tv53pCIId$!RaO`%^+Fk>;zSQvK&)27$Dr|FV6V_Rb ze$?4woe-^2UJWo?o2g@;y&o8%6%nk*{i)0%X`uQTF}@51E06!1eHfSn_dxfKF7sNy z=0?5U9J=Z5^4U5QxFu914gR~yoo(um)5yvb>;Y!;bZi6ci@HR0a=P1vv<&o~WJep+ zt}TwL3A$N%OKWF&^L2TQ>iEJIGm&%3&W||c3*{B|3AykxUL8qmxoS-tRs^pN9}fjY zRW%{i@p1>&Rw#zMrX0(M8D!o0%Li%`x#n@e{oL4Oeo}W1%V__9O7_B=?!dA9<*=%) z#=Hk<_G$tcr~f11BT~kUR@Y1-a#q{K616&Nn6(i0F;&D2yT_fJF~hd(#8ptPGcX?Q zX>r16AL_IckF!3^R(G3rz_NF*H|OJZ*e^0|Cfvv$k*_ev~x(OFz;t{c|?b3^H_Ii9VXl!op5bl3E{0K@TgZ zB)H?PD_s*&?ldrUQRLBQZ&{ov>DtzNzV8s@f46=9%z=gsIn8b>wkeyhE5Ua&D(H6- zs+%2;9m+bsnn(zl%NHmJ@Nj^vdN`4(FBF8Me1YqgKM{ZzzMPM;A#h!QRiorZ1t?fR zOxrM1P57gPA1)Y*s=h_S*U5|w%pDrZpUW2IB2-ow@@9)P63s}uOY_g&7ArQu7$3-` zs*eo_=5+<3tm!$d8iN(SvpBWtzuyLZl8V=hZNP~aNrs`J3Eoas7XCnvm*Zc!4z2Mht;qRzw~=S z^9L+t*@m-{$diO*w3h>AGkFJ&51Sx^JVm1sF2J3`V=&*Jjwhy}FpaugeE5#sf^p&| zt*U-GG664u>)^;i0#9v(k;j4XaY}rEGP3Ao#2#4~Ek;3OBVkY%6)<6>9D~0Wnq5C1 z@R$YP3q0J+cUcZ4P<%Mf29{C;kthyW@HbC4`mO;w%ry(BXTY~W7bjOZ=7ER=)LP1Z z{oIzdAvooE!e{@{lMVS$F6d@HV1J3ypnQ*+{PN z-^WIwT59(vdZfzx$CPVzNjH);S$*r}l*|-X5AEEPvsdnGiu6=k`aHX|vN^yH}L7x(?WXN#rG2ayVZ`Q%_gH@5g?-eTo zi_x2X-LF^qCZ8XopxPB~{83fUa9@N;K?aq4e4YjUssFqIav5>}LAM(Oyw5GCSHOnv z4)`nvQnZEB|>NA+LQVH zi(cE4kJ8rMU(+@n53j~h%`KNyT2b8~rfz4MKw%CC(<*KdYp-ewcy0wDu>y`}nrC0} zc;*ZzrZvy{tRbdWwC?{`V=mht@qlaYva&L_hrd5De<0<4ZU@2q+pAv)IsxHOJOEPx z^MH`&UQ!vP!LYK%urYY@iPuF{J9np0drTzQz$B`%#Io+r%-P2qKs+pZbW?1k!3Prm ziE}$L<;fkCv48L0&Hb ze1b~7pTCSLq{Lb4--|_hO9q^k6I*T*9e`k-_x~(w`!T(&^GVmUXTt54<<5pgCd9wfRQLY?VzWp z7unP#2%vBD$HtEs1nYUqUg#Z97V4rgdXQO24ibgCU>R(~bJ!~Av_IsEJkwE7!W}|U zW&Z|G-uga!I1tfSiOb~o6n@=sts~BQt_6G}G`l?Ap(nHh;W+Z_*>j@|fwCq?EU8p@ zOKszIB$du!%i-=UwRWhG^0Hp+QK_^9~i~stj+gD8Q^{hb{3CF^cKh)D-D_u78MK7n+~B&L;5T8kDD2x zMk8$bAi84ee=L9qY!-cV0U3_}@_&6fC@XgZZg40BWKfHg-_!oh0A_7K8bF{Uz$FwF z6-7k3&OoX>4-hX98D)@3hysB|a{xTLe^VCz*a0fmb)bBk?485Iy*D<8g=3!c5qc z$VU9@1ftpiBD$4t=Tsg;TFO^xyDmHTXpsMS^Q3)BZjg9zhzU5&AzzN_#=cKYg#!OR zf)4n9a%%JuGcz+FO+^iiIM=|ecm^^Gxgf<4((yvKNA+i?eb}iVeIR+Z_OY`_kHX(> z7o?aw_XNg!vy+=aVn-ms@Lax&!$zSW5y}PzA4+WuYCeDnx(NCjr9I;K@&Q{ps67NC zox1WiPx&Lxzh}H$gm>E676N@6fATAb!eZL5Eo<>8kW1z|9Yko~RZYl{Qg7ADUDNL< z--r>rV{TH>beMM8QAN;IK2_l%IAkWaDm!V_e=FwdF19=sw#jZUs^tFieAeYcAM)|Y zWg}pSAR@!;oqm`s*AfLKzvrc;rMqpv{+j;!N1|X@7zSE|1iaJ~`|qo6{k6Wwo5CW= z6Zj87o?p;^{`8{d{ePJra1%`$9n!YF0EiJ_`chHi;}irj>LC#i+4>)#$V72Fw=0op z#9%Qt%NEvkQ+sE-NbiV;j5FFqU~6cx4hJIOab22J@79xte`=0T*HcB~znB}cc+dMl zA-f6Rh$wgW1@Xr%JQA}V3a$OiJdrjhpY|{5x#`bJe&wOn*L5*z_|dPve}PBm#xF_By3*KXulJK0 zx-MAv&R8t=`bU27KRr8HFQd{;g)Xwd;#ib2$R~_6#^u7+rK%isxHfv-$)5`NU~-NG z|Eh?bpWrT{rT{xi!f<9YSnFyt;|zLr4;y@Aib)+PEy^~32-k8Z<@_7s6ChPm&|Hoh}KdorX}1(w5M zqS9%Ji?;{n<(r~~s5QTs$R-k>i%w93WhWlF`(CyGDTaildd@B$U#xLA*z~#Y_V-3P zqY&e>zsADy`7COB{vh8LYRMtK@;E<4y67wWqsB8?5#xPjXgB4!WCbOU7o2-j3Ot9BA=6j z>ABni(uvV&K)4H@RKw=RIl~Zbyh)`{?R5Uh)S+Yg_0QW6XLEDUg=FS*8tGF*v@3+N zllO3K?;B|&vwvMR9XwUcBj@o2Ge@k)Gug?Ol7hAL$g=MR3k(|$%~54w5`*yR!QHJ0 zR66&?3XkpL$dwN8WY6s*3+8^TrNmczvCXS!ggFQUoKiV4Of_Zjy!*FmfDK8x@wn6U z^}wX2qC$rJa;ok#o1+>OoaFTsDXaCxayNPan}SbXpK4-ZMSAr}d!NSf$&i^7e#V+; zGZZoAwaSihyoSnqKRd)f*BI*EWZ%iX$){0f=YczP#G>ZVqp3Ty^C-}F%Drlo%e@Gp zyS<566423y&}CJw`YWzg(*1h&d{p`MWx}w4$mxH*H%rJ9w5P(QyE$XJ^2aQ})K1WC zsv7*4yx@-o7;u^#{P@tx2POu0{78V@^PwEbcmfgE*xl-kGO%?241J_MP#lh;7fcTSzzQpRho7EWLay;WY4^vsWK9|5;@>_Q2BJVm=AX{C~ zjP0t>Gi1xht@z6HBVC5gWSj@iqZDq3|L%*w;wBXT0pDkBDk$@24db22CcD-fEb{`> znX2SzeCazYk=|#cl0##Jd;RP532D6(1@5dx98a zhaoja4|PMqadS|XDPund9m`hlJQ)4ogjx_u9>YPM=C}rxZX8eCIChM1qfF9$yky2v z`{Z9{91e7bN8Re!`!qzS!1(Ze>pa&qz2Ciu&&(>mqygifh;7`)>=PaRj$&LeE|w>$ z6ilP3wfBi$cB->~`O>%4;MTGca`0(^Sl@ERyv!=msT9A9Edi>H-+T8hj_2*$W8eI_ zHeZEv;%SDSHVq5zK2J4iE!lqQ9b+)N$y_YL zwD--Vs|)p(^hjvwpgW|YH9~dqAMSMR@)Sq<4 zoc(S9)(x4aTm)B4XUtT_b6=LsF0Leyar1%D3t~kM*T<5lGoV3=E~p%;beMJW*;6x* zMr~ZsD$>s?{PmMmV36=^pw2;5s zzdOU%j1V~5{Hi1%?}JX8>D&9e*|q%aTu;8nL7MPG>&o8c4Osax2HrD|Hf%(o=DXd# zw<0V?I93zQc%Mkrqyz=(eMb`V(WVS?zp;yZGOO=@P#<r9V+8?;>KW zwbhKBbc9}9BOxLw!0b5sjE*{yDR!}y_)K)CZIWGgsvpCh#*PrvJW4fQ2bZ8qF#f8c zseJPLY%Exeg=t=J%c~o3j^#2HjQkW2-k&2ehE#HuDV*;C4DYleVZ>G` z3^m(8%INtMC#lgE8=3fn?W=0YSG>EJ_;EF9iz1awi8tj8R^4C|-WR@!1lqLzE2_RAT+h-;pZ;FSg>TB9t}1XcaHuh{dlAzmWQRM-L_NF=JyY< zSxrKC-Cgujf!U(DeCkU4SNRut3G!2|>@dzbn-j}Mxw7L*!Z0Qx`{)Rxdw@%KDfZ%K5yA&yr9Y!yoFj%9T{2_VsN z{bSIUi73(o_OFRMc2R|vyvOdpQoH;;ddHM6YZb^}cV+p||Dlk5IauB9W?7`0b_MoEq&d!{2BB`yn%s6@;17{;5nCy;Y|6 zbXh?MGBImI0<1QD8mX@~EWl?^v8foMz4-4?B(b;f`ohcqW z)Ki&?66x$esTOD{Z-ZaH;-GRknXB;1cr-KM;+vlbJb}dBkJe&G(b}w8dabJoQn6?a z$_J!<_Sgfbr;g_cQ%Opz*U>tL_V3SN%Up)~Zz(0&x|qVb$rfv^A>_V5GD1T=;9MUb zD&Zg;$K*DHZ^Yhnq+Gc?55zh#6`|BkBlR*)Og~aY%lD9RP$qJT41e&FiKUoH2qVf35LloXSRjIr;q8ge42lW z&SWT91U1=u@+P7_XjY+;TWRIqUw*mS5=Q2DCK4>VIs1;%3=;XFjbRPiamn8y7EguD z`f-QnyG`pVnM8s{iTMj>?$jA*a$JZ9T@xB0^wXn3W-&NG)@=Itc5M5E|-UdHalbiwr3s0+}wd@&`#$m{XF z4D3VnovJ2DW=Nz*I!zQb>OhHF!?w>CPd;>LKx{Hwun8h;52KOenzogajL@m2TB{O* zO*(BSvDDL^OKFeS9Jx(lbp0E?<6rjwhGM;-e0^hHUK10Q8SGNkGuWLH6Jm}3!!T6<(S|YvB7k;H%d$p8D>4zp<)$hjR5|YARpjF?w$3!YGE3v&Zb~SF$kz$^@F2 zIm@WzW@^!QJUPXQou=50RKm$A5(SHkBOS$QxrBZu-L)(8lkD+l0X}_)m63@nhO@cv zg`_X25D)B-4?ExTEE!TM^OuWk3EvH^*O^!pd9|Fz<<%841onx~$3+HEx9e&g}Ix_VggJ)!EJyVUo4jaY$|nZKNC) zbpf;9PIw)p?f|7=9*H@4pKG1~wzA2hrbOQKm{L>B&*G4-QLmSb2W%nX8?tak z?T?z6vr^xcZ-RL`C_%ib-n7=0cCY-A7Ct;%fzw>^&I0HQE@oXix|<*7>^q(W+a}9a za$-`y@kZwWcnLJr%~#`jif@6!ZxKTaX`I7Ow-O*zPtIUv?Zu!K;_fAHCUAxuXwizPHFxUV?hqS*FzOTR zf8xgv@e?VyXq-OoqoevIZTbO~NSSMv@?*sJC_UqnHh7v*JQ}A12DR7Ac0chrC_<%{ zulweCWS2ewRiEDgzI7A#hR6CUa*!#fRp2nY*_6Z&>*XcNwK>zjFoYPju7I)oq$dg4 zm*JP>2Nvdw_ZR|zD}?p9KfMjBoPrsl8G3aJ+*peLS*_wpF>&f-?DL-c;#Ys>xIE` zts}BI+9V%J5w8X>vuy7moik9A)n|7_s@N<`n80g#FLh->Uz=-tgj>x@!=#Bp+hJ$; zh4^#_8YrGH^Bw;uD@EjjVNtDGsMwr-dT8n(;{kpRnBNS5<08XtEXN`m;j=2(`Okz? z4@f|8^e|%mq)0Y?ib|qu{JfqVac*K~%H&VB#8}&PfJvl=XnkqgmYVE#?d2i-9+xR}q^ayGmWxwZ^GV&L)x$3^f|^}{D%BXBJ$S{%l>ip$1ua}A?- z@@dr2)gzg~RAXR&PE20q%5o-oPyV>!q5_j*5~&35DUK`qk248!sXW*%kr~Y|lomJ` zW8mQuNmG4YaT%!X^^tp5AeZ&!KTinq;)iC^fbkWue*|a0$k&XKm;&Vk+$2x38UmR) zDDIX^{DtYFqxhHwWA%_U>X$KsBfY@_)~Nnt$dcE5sRk&1^_t^fyo26Sb2{fXs`~S% zx7>7A;Rk~caPiOATwN#pO1jn!g^!y;9U&!^&|u3=ny*^wFK^ z-JVcY^@uFvN!`7KvyjV>AcCJI|TF)~9B6j%ms#@JCV>7YOAWgF1 zS)elSw<~i;uLJ1>u!L(cdV4z$1W;_ikLPh+6#-myd7%{QhRKgP7Tj*c$r%v@8X7i@ z(kgO0bmP?Z5vPF%pMnKUy6ezp@E&1BoLY}FdF_Z5B}J^ijz!G*o#8V zS<1h?cy@xjBn|SMUI|;xrQD1es^b6 zp8VBfA>XuNoME1#$PnsK(K0Oif-d&}MfaGg`(1hN9*BlN7_Icq{-mdwJagDdq7|sN zFP!zI)9oQfylGb%A~&r2sNkHzm~>eo7#Lh7^l?00Ih&9P0B=|f&>R&MG0^&40Td2V zG=qRyZU6iU+7(U|Z0Gwil+9AUZTCKXq;?~9TW1(}-gQa6YO;y+w4*PY(WudEl<1RY zMC>ap{7vYr($X-GtlNeP^S7k-Z)(kt2a5g{y@4*AoXQ0Hs&R~=L5`}h(f|HloI%*4 z>Z(`YNSG#lsU}G~BhN00lS3=Q;fXXN?>$r~^nR$Xm+X67Zbo27MM#;L^~Izcv3e^> z^121YY&zz+KxaBv9eyqypEHgJR zN;@-eZKp=Oz*l|5tM*FH^riPpofqGT%}RvOYb#s1=H-vuEivvg8VY%`%|nhlV|7bs z)qaS6>#BaHEFJJ=Z9+Si%l{rqEc_G=x!4f>`_5(G$$4JkkN8iGON+x4s%rfVBs;@z zvjUucl)V5R6e;QAmvu4f6<6W?f$TR%p9O@%NBLA%DS5aR%OJ`Cz7Mdc5!h4-fclMK zpg$=u!cB^psOS(S@ zc8o%_b<8ARhI@quEc;uuln8zd3~d)0X=JB{E+v?P%^t4%r;Q2%K2<6`_^dzaTjzFo z#jm5cKYn7TEfOmTe%>po?8f8TR19#`iQE?TCm?dmP!en;|2xXEbSq0qh8_G_?xuaF z9^Uxl9TW7hgXm%a@+aO-S1^$ae&sVm*Js=NcUnp3>VD#zpUEM5_2TqQaqY-HVN1mT zE$3Y3%R&k4PF=QLa<;I2e{pTaGR$jG%+L#Jff5lFFz8RoyFsZ81mgeaND}CzylsrC z=AKNJC6wjMw!FmJ&9MWf3%msW;Yb#-MF!9q8xtIjfBiY^;yAdT|ep z_>f?JM`bou{DW{2X_#m~VcYx2`>&^sa=C$&PCeX(-Cj;nRZ| zrFZVmf8P~nYxH3A>o*hWCj;AS?lY zx%w*q1T1oMK-BOA0Vlh6t#z9_#Bw}Y$seW z%p4JA$yrqLp*Im$Quwcv75*_K7PP|a>+2Cvch;tA5Ji8#f)~!{;G&r-X(HmDLMURX zr8h;7b9_Hj2iK`&KM%AXaEL?WtXfcbgFzl7x&e?b*(N@Z2Hp!Ud++Llp|)-`h`<&; zR|TVNjff|+&Z;A2tmjCrH<6_e+w=a}?go%2wFZ7W{EH?)Es6&T;m4^$W-r{842qI` ztr9JvC^LLoJ9~uV5nX{jTj~ZOnBLh&==MF=jPL zyb2>|L3+mSsDKefORk7BBI%5j8q1_=v*t!2@#)@VujhH+iqcRpXhs}*hRn-Pn=u7|&O z5RWT;^u9ziJW)rKHT0!1W@UW}lLT{xQ(l+o1Kie9i8l$eUSVFt5*k;#9Otnw{vbf) z@WPL5l&WG9JLW6C1Rl2mcz~?}&N<-YLEwlgJVb#5_~yD@>}k-@gi|LlIn2>jMyna? zs%2A2Y)su#^LFQbSrzuZZ;#U{ zSYcYGW)e3K?mkfY1-)05){VD{KTs={i{zyBxDvb+@Spc_njddjws0vtfLr--H>ho} zoGMU+zi`Mf2M}^u<5d_^gq6U4l)1@-HA|1vaj%e|V{g=;@&HNH zc2sesq9gi<`31O9h*Y^`-zpzLd$6o{sE!?QbiZ+vJxRgPaeS!ZWaLPAJCK25nN%UBK96hGhy)bqca83ULpS+XZ1D^r`EBqW8*N$x zDcZEO$*>$3_epJj|P=sT*AdDf}F)hsA#gf?_9jstPW*OE{vW7(5o2fnJiNa`+Z^I|9wC?l* z7~6;{2|#K`uz{Uc+er}336S;f5M*LOCw=fNdZaVckgZ6ErmmdlfISK>T-vo*8tyPv z<`hAaI?N4=m3r!L-01w%%;w=avTP95Uwg1isfl^?S@y>gIwtiH6x_(34l})u$KD&_ zYx7pCCGFpZnK*Zct+&6^99+3!A*r|fMUIid;3r9#LVdP)MfzfxLa=8BGGkdM8V+hD zt;<2U9B@`n^EY4_dEL^`5FM18o4uU38hO#F4YMXn>;bwJlTk`P-i!Clm^s(#4+eR& zOI59$ve%LApA@8{FQk($R?p~4Vcp;(g#mvFg65bV{P5;oS{f80_HbDa07^>Bn`kH8RyMqZf0p~nm>^&>;jkX?gg9HTzVhO+DGGHTl zwDT;la+P}M@KxxTL42fmMgL%*^Yv}>Ebfo!Ml%RE8P=X#xSrEOwWYy~!fybhoXPgO_~wU|TWzEfI_PSF|SzbX)FR_K*eUSZPOb));qhbX^H}ZkeY(`0LgZ81LEmvIyA>ft+;hp*~U5aLmWC z=m1y8E+SVne%u#|jSv7IC8r8F4Y9lh#K-sJE*Mhy0?G7TZ}Xa5D>%&?mx2wQ4_S9| z&xO+ZNd_}f@f3%53k=-)LN)f>Z@Y_GYSvFSya-eOqFKBfBxT=qa)C%ji%F`a=SGDXAk=U*X?vMl2YsmekUr%Xt1R?W zVFS{U4I=%1om!F=Do`MxDv1Q)v5AwjdPj#rd$u=%1nW07 zOOc?5bq~vrQ2jY@51pDv1J#^wpA8-$CFp*1`!j4zlQ40NxgLw$WA3_&WXJDz`uDxM zql!qCXF0}-3)P3V6&fgU8v`e07Nz)9C8Nm0J`~dZ&f>k0jjw8n*(Xl(Fv(57xjMNU zWGt>=FBAykVhb%)coO~+vz1qA)50lxFUaS&EY@T1{OgjI>1&>Fx{Sg5izx1}k|bXl zY&R6lCPVQv!`CxnT-BD9raNL_KF`x3VsuV9ggSPzU?TR;rZ|cW=Ne#-&aC9_G)>WQ z)T=166f7%0Gt2)vF*Gv(5wEgCDyW=Y8XKX1tIYCKn*KEKelj z;E@$sw26g%{K)|?N`h$9iTB00Gy{Om5>Fv}xte2|9BD?DO9qi{a_o(S6*f1AXV$kr z0MT2*e<+*AhzrBY>==pmUl~FuyiWX zkc7D7bJ!qGz6Z!wKnJoQ$#o4u z4FNrAc+D%`M*p>{X6G~J-xzjJf$yqKh{6FxPFVM$M(@z^+pq{QD!-j4pGgmmrNN+| z@}&;k+l{HvzWbvf8^%O{Zc+_YCiFy$Da9hLD144>z#iPgGx$YLtCzO=7g$0cajw_C z?Xk{Wxism$1$RSjbuAfCbufQ4&3h`14rD!NC4$gzVn4Elz9tXX9OJ z8*-!Y^|ygZVD76-@3YFZO#I42U%A0dAvErV+Evo9bMHLd{Uw%(GRR4>$4oKg+k5^fl*uD1chJN|AjG@n4h$2i|&Z1^pq9ADYVv>@h zRMMhldR1U5kVRa%{E??#H9FAn`!AkLQ4V=*AW7~}beX_-#8w%?8h2ESb(YZxT`QdZ@=_&%aNO2;UM8sFD)Jgbn zHCCPWi6OOfBz0p~^>|3CY6MgxsWWTubMYTXYTtj$Hxs{VY+dnz+oQPc*xY~WpZtOR zub&vDsgsL4pW^hHzAN|gc_tdI72*N6Sb}l65fn%E#}Kk{gEFF#|4F4oz{>#LzbhA1&;c%;sYvlldfYdE zlRJAcGiy-QqYW8dX;USxzn(|SI0omBI}Ed|#xh{g3ugQ|ZRcNZaKn-#M5(k)uG7=v zklXlu;Nd5*;$FK6NCLO|-zofm_fCHKUEzLxnG!AJ<4}9>5dUW(cvyHj$YQar-4j!( zKJ3i0kwSIB#1y!MdnE7gCbHD&??hi|Qvylxsnz2G(BSMQb>G1G${I7r&|*_&{H?<# zYd5i>e`vax4GHcUcA|p2$L0#(295O1(8JOAZNiDh>zNw z(DqpWBYMosw4R#C>MP5G2J+wxlTs=dEpWPyVyr;w6lqP=bwrm0Al6x(EVqDOK=23A z_X~={5in*YpBJD91%SW%0Q4^087&MN3*+boJz)Zg{(HHR2t+o>L#Upu{ulC&*`5!y zL%iM{k6vJ$6j$c_Dy5jN5>*2i2440miQie?ZlxS3%=DVX9<5GWItm(1Wr@yD= zHkaNIUhllOf})7mUwrp?(@4>wVF@YRO7MJ&dkA4&(h;Qhb|$h`sD9P3_W15q*-z3# z7_Y;iG@4|V{MPS4i;1KF!o)`=3OHs=UCIG`MV};p5COZV7>rAqqADqzbVOLqPNB8L6kB8 zbG_|D!f+ukfsNv3+>2C-d`8H?S<2I9{yEivVj0iFeG(-q<8BFGoisb=7Fk{KTD!Ob zXSPx^mbO1VaYx%2scv`*0aXjW5sYFH|0I*y&j=Sq*#kfIJG_w24#(^UNHqC?U_{yO z8dwr!^4icLo<2TXuf%Uj5Jg8|rY$cn!2~Mk5sLxd9OY#2DYmXv1rMCn8PXk|f^~%7 z(!Q-o{dvAl2dJQtBw?mp^%D^q_{64^VdfBjQAzUB5lkI#ZpNdzr|!Gasj@E#BFGYs+$ z0*&ZGB-}*iGwMBLmEY^R@W6vC?M8h6^SezWpwG z;R`bHuQiR7I3oIV{%Eaf1$11*;3ulFLu3PGNNTPF|Gpx5L@%Goy=WE+(LN&b|LMmY znbylUMZ9&09+i%&9P`UJ*$sLP==l3fTLZXLfTD{D5n5UbJ5O?BCBWdOzFxfR5~efvdEa@T{E38h$|C80peoWcuKC;s~pG8Y_@YjK>KXfOX1% z(w+Z}G}mR`!Fz;K<9%!h$Z*QF|H3iwYd{~$NYmQg?lVN|2%^sOT?V2$GauAg0+&4@ z%4blZglO>qWyYlm{~V zm!^Di>eo=dbaLo$Z1H!;2mh7%!`d^qk9plKd9E5%$?Wjffgltki~jHEvU9DS4^az> zH=1tImj6Eq@StHAv}7iefBy982_ywX9fDp2UA3)QfK&)EsQ(GN{jrwU(4eisl_Yew ze)$qiM0p(8LKIM_hh#m=2c-(|*mhb<*PXxN@r=6j2)`|0``n)(f|<_4_?{miM4}<@ z0PexcZr_`ZYlW4iwyv?2f`n92P?i@nhxD!%=(jzjrIG2B_E^+mGfRG3p&HDi+JCCO zpAf<%I0MX#g)M2c<2_g1=c_125{K=aXbgex3 z$Z>-rjpcG}HYZ0oU$tai)f(Z3f=l=^Dy)tx@OGa?AphrruR4C|T{^^SSQ$gi@TkM| zolWEwhT78}@jn9ITff$H=~f&<`|z{fx~dQHmiIkkc%ApsqS=JoApl2$iTWuYSV?!) zd1ca>Eq<=PW;EY_y=D$zpxRLtDJfPqiIlq*M{( z|0^)dvXQmYyC#iJmAJ6g8N)CHTam+Z)YG*Whi4|vOt83-p%!48`^92hTUV80L5Lq~ z{N}4hGRb(J+o8vcykYq5;4(%{wm)a zN4d_tM^Bxcs`uj-t`_nMdn5k>_vTK{w4ePj!vC8S=O@ z*2En`Rb3o8$7}M2jsX)TjooNSYx<;L+2Do8S+kolv{Q5mE#n!y)tia&or1|m&+DX4 z-PVQ7b%$Mw#Z883kS?*pmEN=_DIQ96DwG$kVi@*imn#*}nMNfKt5bjd@=qhyxKAU1 zFjhZpWJRTRh0eG^<%ujT^?Z)DfW85!!Z<6!c*xs~L`w^(E+BgV#3kk55zIfROO><~z(OI8ix_s~7+ag-lRw{a z0#i4OLR?6u4zBVZAV*9dKYd(};9-={n2Pv&)Ye!=f3Q)1@^|_091TTd@D<53&&yG4 z^YRBsh%T5;ha3n-a$?q91lp5ZRrD`W;fe#37LB4t(G&0*1+ru1@7SIja~r-V zArFxDbE>n12qf;8t38u@#{-`_)PUjP{cw1GIyYx{jfC|AATB8l_0qfQEa$f>#mU2t zv@xr)^=RyG$1&d4eCQXz&j0NALBI~x!RL!tFigj}N2qkaug+q_sxHTscABnaz&$UH zhalpl!zDwn2L52~1m(;!W$mY+Jbd)H_K1Kl#^&k(ASI!XFh>*)>0^_P!kmK|HltV`oc_AA4Y@8f9b{DzoVURc2A!z=dQK1*B42c=z;EC_b$2C{7 z$kQEUF?3>8lg{vb-@BXSOZ04U>3C3@>S!Rcjr4TlYh%j26-nEw=)mn6hCwGp+N+H} zM5Z}AuEvU;q}Lj%ObJkhNCz6MVP^ip&)jGr@;% z4R7_N)JcQ1Cx7Qlwxe5ZcnJC*^szlY+uB84uT6q9kUklHiOp0L2Wu&LQ*Htp0}=!4 zup^CT^olXAI7IXRBt7C9X~MXlYl@U@%YGdTwpUGAmwx^4n+{IQsRcuv1r_;p2b?b% zT-1Up42?D|yhrdHMLHp!BHu#+X&b@oNxACxVhqVyAjo*LH2?ic@lbIMxdRR1^wbsO zJ8Qd0gPL`LL$`=vq38?@ZAZ1q@bP#Gdn$oRw0r`U=lT87;So7RPh`d^~4kwIsmik zq~dV^63_zzYiF}Vp;i34^x zjQV29w4?!p9vVeD=YL0-L_vK?Vo*Z@Rtmn>#Lc|afg2z_g3}DQ-4l*K*#7qMRv8vL1V-99}q-ZHbe&}gSPe{ zr}7htwm!PjkogQvzVop%Mu-WoWSgxT9a7xJPIZELDm%~i%EzA=7~%ky-`}=Ifyt!K zZSO2c&D?*%c{NNdT|JZ-9O$FW+|t)4dH=WH4Eyhu>;}@DKKAG^0f2fv@3=7>DY#NzE_ zX7+ym$#i4m*Cjd%QO+{ZDEGEEbtf)2gj9!RPEK^v^);lOpahr(!XB~oYoAFsx6{nC z?0m(=K(^ixM-6}Wj!cKT@|hhE2{(W+6fO#m6KCh|Ra2W7az9U*mEnY-VwyLp;H0>i zUDr=)p~+!_aIl`wy83Hu)4NS>(}#mcA`dkRQ+3~1Ip9lFt-f!;KWJSsy|cNevH1Aq zD2|^}{mPy>A+33guGvnv`Ni2(hCID{jR=DQON!-BBiZt=+{_#no8K`+k6aHThKSTv z_9k1+NRa)kdw(m$YD45ykZ`f!OSggk1r+{=$B#4qgfQA%BLR0SHt5HJdL4dRdd)OT z+SIO~H=Wc5&%b)JI-@yzd}a-Oo%N^sfWO=$Em|h25Ko@I3iaB+hL2OYkUV38b%^kF zM3}VOl+&I^&rj@b5OJK(VBQ>$93+Ji|{+liddKHJU^5SMNt z_%s*Ajt6B;}14 zi1mC5VBo`jmHnCd4E$V&1j&N+endAn&u{!M#?JaHs`hK+Gea|Um$YG|w3P^)AJ~YxD@8SCwyvrY0E*3Cn&Yb(+_qDJ6xl*0u3ILQaz_;gGo=?^) zlFkAGHUMW&c(pO#ACbtR2EeUWv$4iZ|2j*>P&bQ@Uu&YBt-1y&!)dhZ|7pzukmMPV zy)N;jhBFch3zfu8XEMit6MvW#oU^+WiETFG& z{@;Rt{6*4AVSIg1G6E2W6mV zWeN(YxNt@uRCMSc5p)}^e@MtR8v=+WfS3j=eskO|c?Zm!^!-jS=Rt-nK+4VYaY{c4#h=!Jm2nJn04xH{e$2XhFmi2PP zOHzV%@Ug#$)z19_4mO1YTWYvL+eK+|XQJ@C6Ymy6lbi?fdVl2WXTDQrec7v^YG1KI z!CdUoM)g5@-syn?httq*odAvZ9}UobC2u_=quMKmM3dpzeBKs&F|8R}gE__8%vc8W znI)~%EN%s$BYm9oJBoNxsma!ok%sO}CEOIO$u6=R`6uP^Tl;x{?R;!~HjbI#XVQ<_ z*e@GDsMyr3te{xhi?4d5&H6{-LAN&6dWLC0LOi4IrAv;dI$&A40uJ+!wME}gK#NyFqdmAw}q zakH2eeJ4zn=ddi8i=ya@7tE!E*!QlZ{s@u~;NF3j|RR^$o&I5y9 zIM~F_I+Dzi^=Iyo|MjQ|i9pNHNMrInEE2d$X4^|Z^EmY1t49!r3UAPAAh&XRmj@`D}9Hz>q7l{n$=fqai;tu%AV?-sNmB+ zKqd-wTbKG%XG@}tG>`D~pUXG9wV)DJdw z4*A{wg9h~3b(}3*;KRo-g#N$r-e_DN(37)IwZNP))8m#dg_>Q+@E(Kgi66SH=uz-{ zw+z$Fsm4q;4r-BA5>m5e8Rk#7b2bR)z~c(I#s@8%us2@qHp05+La8a=g1_=kyJP>> z4e0tiX*IQ(I1^@MYjR%vW+osVZ+@Fnc$+VEeDxW*ox7EN>gpVK5`HO1f&u)X!}Xl$#3%kAux?cv@-KVzg}`w=wc?lI({vl zd7Z}me;Czf4Qs^=ao&Tu2+qt&8-*7YRCwQJn7rP6#+uRcWj~g7ZnpjbMadKX*K<9q zWNWe+Q^l;t-!zOIP8=>tR(sw+saZCfOmiJGssJ|v7o8n#e^K>Nf(#hVLoehqd9WvC zV1TxN9}!TdHt*ZN=X8YT@fi*QPYZy{+o`H*lK82x4#~(g51;8pdJY6gpmskwhC?5> z9^0!ks;lkK>_wtn^n{5-w;5RoXsVR@isPM`2oEHPzc`k2029{d6Y)T zt{({BY`e)54eY>0nInB?r<9F&<|5+z(``!vocjqaptD;SaAzj39ZIdua0Q`MFJITc z`})o0bJj-OF4g^d>vpNlo=PcoU~kNOB5$X%!Xy)Z7|g=ZC$zb(%PZ4q7>zVKcls<= zH&qvB3@jkP(F#wf>n4wyiK-k#4VS^ysBwa5j+oEepcw$Z>lq3Qa_P zY$0nzVa_pz5BY*^S8!e2C|hfEEq;jE{ZYG+O)ZZ3!$@`e=Aij5AGPW$g`m;_WLT&$MOhngA{;5I23K%eMKw1$3eC-?i^^zgG}`(A(sV&k0z_?S3puM5sH+ zl%6Q@ThyT^?G3M|g`<^*Ca=BDX!}rf=uLvOYW6!OQI*n>uK-i2uiR!cc;|RihkdLb z>8~;XK-;mc(7*_CrMMC`c{kK_DI5GHfA&D!NS2i*)mo;@>o1@ zOGuXT!w8URVi`yj#Zd*|F9;`xU9#z&<-7eCT!b1<$?aDC>ZowY7^@M+SmV80ixEh3 z$?O8d0q`R*U#82^qzka6EWNfC|DA>d=?fA#Aixv~Kx7pS&Jb|O@aj55y#JMPGqUK- zF6<;8?_4~bXO}UdV^ps3IlG^It+*rA{(O@NQN#lP>WH@K(hbHAZIaeL?Ge^=%>JC%qZ|9!XJc|BpxgQ#5&5M0a9 zt3DZIe;fexh^^fVhqosRAc9LUj{+tpAZ<$MVJxt-jk|NWYN?44Zxq;sHuGPPhZ>I1 zwoK_{SOAn;G3Nvjr_%TOjS~VLhQ}5GkavB(pE5WZ@PW#5EGgJVg+W5vm_g_TWV~Sr zGaUTM{15la1-#?Kr{Bzc>Y+^F*u4)>I-CIP!wo`tWS`lHz2|Q%ArZ_Aa z+B6Qy8^rW@RCsaF?<2SQle&RzV&rt*Krs0oq>n(E9q_BZxp4pxnty8r7(ls{z`1zs z%LcLbh6V<0w$Fm`Fe-05;avPbv58#SR=}XOs9?3|0f{Dnjba}wa?83yp*~(;p0e&M zxi3oI;YSRlARwk5Ty@vA^j#=Y->9(0%;k4w`+n?D5~^NedQ`t+g{2+`yKiOsHXr@@ zAs&33hLIJOF^8jd1_}u<zgr670Se)-pcBoSseOOg|M^c0e(xjpiu!Km%M! zO()7g#6{>>Mh^|ZNs*?p2Il88QO_`NPx`Z+smcBDtgL$%pB;{G-(iY~s8QROCb-k-q$(@)D0ZOhmqhJJwy zKB&Yg{p`N2D{!uSPW<=~6*?@ZUh#1 zaSp6vdH>B!-dwJ6#B**FXCopOy7iR`z#yJl1p+K*F~ zo2I*bcajPT=L>b`LkGbg2`!*s1c@PL03p>%Dvs_T1Ur-yH0OD>T@g~nM#yjGBiv2k zX&oi6Hc6nm(1PtOILBmk0>Cl1NkZLV&+^MAQCN-De_kBhp#fQSxT}GD`G~;eJ3uCp zE}svbfjeL$>w3 z@UtX4ARLo%9e<>b`MV}{Wh-cOym6lZ+qo+lSwB@?q*46)C?Ge64aI+v#@TxkE@2WM zL}GaeXTg6<76l%?M`AzN#GwxVuY1MY2s_BslyPA1htjQ9(RIh;OU^6bU`SK3;C}FA zV`gHtIqEM>y&fMaC!&krDbkJib+$Q2z<~Bpvu5;-my|z0(En_GX;jM~9Z!4X)}m(s zdrp6#apZNP+3XBY<=qtNyw%*@PKR&T+x+75i#1nx=t1cA-H|q=j|^gdE+pfw0uhTG z6`s;q%*SzGW2v!5F^&E3krebcw#-Ov=r+q#fSZ_6TkQpyE$^}^t9ULi(q#dfJ>E~< zMc`2V^t@{4e-|L!nM4)}j9qeoY2HG7!Lg7cb`h_^j*&tOU=+u^c@#O7X_t~EsY0!- zUp}STBl`|+HiR%~k(J>6M^@1_hy~;b1ldBm%9}id1u55<)cU6Gm3*Kf`vmXUiV}0P zG*`)-SRIz;r>r>lsA36Dp{hw8u@SzEZ1zXdu9i!P{MzmY|hRf3VYJ@XdSL z*;8m-t7%i3!Tupde9C60JpkN!_7}5qd6n7{VFJ#8TJs<6a>+pfho~4Irqtxa^LUcN zjd4v0T&?~9q_w!*s21(NVnA_ufSp)48@ULsG^POoJeTV424HgCec_eDJb--RjMlPM zAz476k^1fXmC41MMwgKKC9(naJmfy`07@l^`U(K69LMg#3CK+-&<2&P^==kzq2PrOFMyMJekd7O?LUv0KGq(i-{O7 znIhlHl%W6A`kbd*%)NA$QP&6X;*R;0SM6gQ%+0ZwP5&+rOt7P{T`sXrLk|qr?)c9Y zN@W6N?(BX!OzbP_W)`N;_@|0VT8Wnv%JVmd4A+7G#{BOMo z-%~_s?aDDz45o&D_WU}VrO9Ahor*2QPz8V%-@ZI*rcQU05N|(;-4+A@Tf{-1O%{<_ zrqE5u^IX6-B^hs_t6^2>6}oHS&63#>xrIACR(v_U-W@2g+a&afW&j623tu0ev}{eR zd(aoYBr$lgX}4WI&if><8FP@ukwLc~RIv$yl*k{0@C`C||?E{`6-0;IvL1ar&k2S3NKj;F+UWKD82h`WFl zYB2tZ#8QLqtqupJyP?0U2n=R`+rV|4D5~)e#QbRaj23F5aAs>~r{=`-`Yr?s#0G0r7VC zW!MRr7frETHubtc1NypXytj0x>@S>qI<@Dvm13Ge*g9bO(IwFjkjYk*Br&0bW{~HJ zfWO7UD>BTJ%`xMt`(n!{<-eNan$0jPJ*=MELt+=&ThDuSr^m0BhGt>y!vF+l^K_E- z6Hi|j^`WnnF%e)9TR4G^(Ef;JqUF|&;p0^j{;!HPoeCp@@)y-0sNE`=6XxvZM&OUf zjdx!Fqwy>6lBul;o=`7MYtR4yce3yfP(THl`m2Q(sn#vq_5#(d+S-4;*@-@w$*>@~ zG`pSEG&jDKFhKyCwh@`c18flG!I-M?;9b7Wt50yiPgm7S#1OoY_&_&-?9AF`jyo6% z&iAb(tFfX4%#Ep34IsQCgD~!l`4Q~L*xvXJ+k_~KIGA7As-cSC2>*p2>O;iAqc-iU z+)}u?V}?nN1O1=EGPo)6GwH2?tX%p3yo}qWU7rCTTI>EU2Ov?K`TMmh(M=54e1kD- zd5Y-Nlr|ay^bQ%^B85QR^V}X?P&q4SX<$G^_nS0gNnJo4bJ`6bgv6EvUF!YBfnV$1 zNkSVR5L4xRe?Y4ay1-lJs*pDqPD;n%_t2bnrTWLu`ZLNl8=zbxWcgAP4hjjFKf$4p zl-Ax!{{&;g2G?sfaeus>TLs=FF-Je|Arx+j2Ufsz#j@FT?Fk&PDSw&|mgi0-AzG4W zG-5O+NxzcZ?_#bS(VHkw?LhO&xieDO^7$m|LrsB5IY%OUp^9sV&0rRv&0nvE4&J6Rz3sX1N{U2*Q29D#ot5 zPw=FdwPBGzw+E>?2R9J6&*tU;u=Idj;SwR z%2t&B@1L*p^{rc6e#g;5l|jn~qm38G#eaR9-V#beP*V#!brZGbOnW5kw) z0476(4um9bXIOXtz-`5Kx^QSdm-;{v6)b~s)D(U*lj4smSt?IH|6b&ZO!P;ccatXG z7PeMx`wO^$MZ{>=gWluf%G{9EJjPI8+PH;AL*pMr*_ww^p8T9W>vBb1^x;9JcMtEp zYH0TC)p;!>9vyA#bfc@Lai&9Uz7SOb>uIOha<1>GUdwqo+gHhlT-Y;kX_Q5>jjne6 zVfJz{fB;n?Lz7fYk}eNZ$V%E+156;COLsxy^9S%@P&Wcse*I4^x?meZiXUbrOz=8|Qwd+$^@)V?y9{!ATbJl~x-^)i%*v}h$ z#;mI!htx9c&!!B63+y2ihHRyzKU1Y!&pUa3?p1M%&!s)o)_f+gY0}!7aEp<&-PN+4 z19*lYYrJ6)RsmIV4_|}xYseP1?FX$oth*|Y;vs!d)%u9!My>`lZ=>)hnvqZiyv9`| zS^K4ija|8P*ob+X5cxtC5xN?VgMqg+jDl)ss+gsk1(%HulihIkFo*x;v33f+&!_tW z>-Rs9A}ozr1+^Yc0@QOPHvvHRGf{i)gnF$z7mhFe6vi5YpBd zJWb(G1}osC6ycCJhZo%l!T!9L8s=mHjU1U6d);V8K&I6R+jNw(NhEcns!#Ld&SzgS zFyO$B^Wy$pT}UmSQ>qVKkLTF(MI0N}zN9-FXb-izZqPh~R9GxVkQbRq_?Im;*Z`;O z6$0Xw0u~7)!mA;FUhZ&6J5ktBN!anCu{#5c@pMFS=|X1C_j+Wd*GO-@UTPy%5LWP9 zWq0Bz>%6;n&{_c~*C5mY$D9;Ph5*U9Z`Y#|PPNwmAC4D<#WWyC!UyF-?_jE0*9Pn! z4H<`!PCBCvT?tF#Hdtn~Rggvh(k~Y!o;`9r^XYs%fND@J=gGKf8*LBZMrYwN8cpvF z4_y*dAc~cZhOBnj=tns=LA|+{wo-fE^{V)KZBBbfri`09kmO`}!wqyfsYMnF`W{pF zCxs43&RkF@QPWp%wIxfi=ObFd>$P%(7g#>Y(?&PZ&kwaa9w-_v&!CW-2MD<`sSD`^ z@k}gLRQ#YLr`-RKH!LN_wYUGL{W-7M{jE9H4!<2aNSfkeK!$WG4n~sQ1N16l40?!J zK=wvmFwEw5+HymJj7JN>Z;{S8n1KM|%{MwYsp_@4Lvm&3AGJ@$%tS39Qewh^GjEiV z8R033>4OkR0)}l*C-!sqSgJ0rIEKr4u#925i~1h~c2n1GnhwGtkX_3vNfDCguzh4+ zl@0!V&kgK+g|Q}%-}&|IHiRrJE&6=aUhgg@CS2|CHgEJuy&rA2fB1c13k~sFA+~~b zGTR-6DS{)4X-eHWmZ%-Ri^nvA8pgCPsaWD-Sl`x6#`}$3S9b2=zyZI8YNntG zJmvoPrQ9+L;XDLauOgoXI$1DD(}VOp-iJT?d+vso#FY>R#w)a`?Cz#`Yda_DzpZRH zhII$yU`*-0X2E&7%U_jO3Z3UAJdso)o=kDtET*(!evYxS_%KTt$h|vQIO#u9<)-6j z9o{1p?kp8I&&6Z)jMPiT3ZbcCaN((Pr@mjn>-FAV)pC5irlzOKJN_$uf_sXXR+cayLiM#Q`ibu5R z$d2^$6*ggc8L?q+JpAU;#CcR4>2aJ=yTm4z#eub|+F3tOQao%+)*;KMQzXL)3Ke_o z{cv?eLxJz&Bn4QxZibw8@#=k9lOybTG3Vl$mnkb+CIxVVJUR3WN8%}wA)QW`>EC&f z_I_5@b>+qwEv6^`j;tqM9k(kp4$+N+Ubaaq*2L1FmFZxxlW35D>7wzwTJMk6<-!j( zTOW1euCZaO*cuMk_MI7k3Ex*88~KUbpzKYo->H!^9YIy*&)NQ@=svv9#p>1hpld6z|MgFPr9;&76-h~{x12cg_3|o2k-m)zO*lRMlZoKOldY2cm}MPl`!w$iQ!1nEIAkEw|ESFCG)dF$`Df$ZFsZ^4nHs z1szjDIj`HkMTah-+JRebqZV*;%HUMv$T+cfiSWTGzWkZ+9VmxJ}S2+<48$L+S; zg8<$pup~>^d$9ldhKV_P(cbHd6~Z(OOM!LC=cP{gVy4z$wGjoUdY$Pom|Rob-WW@Q zMP7qnrNS2Z3H%LNEO`pkUMac--PHdT)GHt2QDjqbm`Y@26CW3r>c(>3ae_;_$ek$N zP&pDx-S7u&x^fpEbrJxR#SVO+37U_EVtBpY^`)tZ@UG=jBfybUOpkzQZlXE!*IYGy z%D-AQ4cDy+1j4iLm1IZ9>(WYgJ}Hh_4|A|4&uwz=%+uI z9X*?(EvI0NriZbGxd|2L;UBS9&6O%EWblcLuGyu&_v~G`y1AwWCPM>4%L8^yNXJw* z;G<+Q%`v&f98O%5oW-D7O(aE+;=1$9&nLvH+RpUn#I)0L>Q~P1v0BQ6PtP*4%P0)U zlhMu8?7-g%wCbn8o4ux76#ECfx71!3=E%TxyWrJ2%K&8sfB7p@CE-7vG~`9t)#)JnwOh>Ub74a`arIVgkk zhvMb>(eD=!TazoT`r*)Uvo5}idter0l#7i0dnqlFphaeeOPb3H=8lxbVz32%tqe#a zg1@?xIcY7dGS0mQ2w;nVHs0j5Rdp<4TOqMbC zwQ-iGPW@qd7CvQzpSJrJJNrAgqg$*2lh;dW;5b~JE8KjoFOfRoj%9P{*y3bEV;3)~ zS#2@1F?!@Z-L3O$Wb*h%Wv6k|B&!xAtq83eRzfanG-Y%Q*4paR)dEci%ci?>Y`e2K zC^RE@yzcRjw-d1^34x~h)bB5x6NXRC`U<6od`ODbTOLVa4Tn7?7V;OthhB>#dM~-} zImzXd)NA>MFB}S+zu$CEw+GGNX+cz#oo1$LfWIz}QIggTdwid^mp@^cM#!z3D6=u4 z=QvLX^*&$q5{cGJZe!Y1GZ&vni4QN-S?lc*Bw82-9olnh&G8VJFPzilrs*oAz) zPnL6VLhDmkE95{CeXe5D%y*c+6avKLx>FNfZD?alN!Q`Kd9YA3x@vKhlPM}2t^yQ> z((g;>ylDt7H&=%#=~a=8lSy^3Ka%A&b4wpc=IV5_cI*wO$IPs7fxVp}s;dm;feQ&? zj1)bhk_0<>t-Kx{Skx2px;%sPaLue|9j*RGH>He3X*RGQrTcR-%Dtres7+Kl6x%bz zH-Xab*1|`gP5%X%6XL?|FqwMDRF^40Jick?*;by*UNWs8O4=l0t}dlC<@ zEOBJiCYCwue71akM*lF88B7)Nl22uB!t_R4r%IwRh{%vcJc8P5ba~ko(=kAZ3gc7D zYJ>JLa4#pW39F#r5$mDdCRX4-qArkLZwa)D-Xc_FF^S=i<#hG9ds?=-NOaGVLZd49JOZ@*&b_LDo&q@esWkou%_ z`w^oc0X-~FV+*2OipoXx$TJGh_375S4vq~&7@r0Z7F2K5(^c$@LkRp zkR8$fSJQLbBUDPtqSk8#{gyJd>(a7o^lKrQzn)3&_Q9vA$s^cFIG14u6!^+RZZvT}?~w@9+0MTp!J>t;L?KwiK4o@Fa8;6hOSzJn*f5 zWh2!rMsK{n+yZ$vaCUZf(`##0jRx)R^luEbzg<&%LdlT{Qe#^9wZ6LAkaDI?@z--( zCGJx65lss&rwRl)freVtOD09M(p+A(2#^_-TIHq6zg!6=re<9%{zIhWRO?+wd zJU){$k&MD{_ngD>W!l= z*FUq9NV#bRMi*jXw-?<+{?dsV^fq=}-F2g1sappDodJ19TW_Eks(7Jb!P8c%l4sz5 z6+WMVDWR>p5z+zsnywPeAdy^j50A*_sP~oy<~js9>+0b6Bg7TE;GM_!izbw|4%`eC zp--EmY0$ATLQZkc{X;>0aZ zMjdsvM%a0FO+C zHU)GA6&28LJdRNvJ=F612M2vOC##=smTl~tuWxP&fawxW>&f0X+Lz>4v||6K(2T@F zc(K);3qM1kJXNR9HM7=pu>&h=vbIb2k&E|^Lo${=9p|xbfOQSd^(&&mQkCux2#wur z&YhOnRKU$_Ox<$gF)Hb2|00iL&YFDAKJuu{vn7^HXUT3H@Kf|&cZEQ#!H12%S-L+u zhl8zvZYd5XhDJgXKo?G5Ogud8kINkb>$36iXYTV{%D_pG%UJw_!B(}2_ zL@MNsVPRoWR#i3cZ){)?jV^N6>)r!!fC}pBFb4((T-prp##91>FQ-IJmb={#)D2z7 z-e+)FAp@hMUz|Tbt^$K0guqBSY&zo&x|6pz!N9Y@m%>i})5{-kxy= zchwjJw)I38ggtuJ+(=Yz{MpJF=H@zA?B5N{IWbI@gAvs+(_b5I9-q$*QpAE0JBud9 zr_Ou%6xj%LET;OPSruc%UMYHd`o2}?p4h%V@vEyV)A@2OTSrH&KLc->PQN{WW%FkU zda;x!=R~cq8v~Q{d?h^|M+;lu{Z7Z`&g8*wMO1RgNWE|j0oJqCzpE|VuRrF8mhz+7 z1z%J2{e5@&tmo4=cS1tO_nfrC3up9uO)~T&nR7{as*2N)l|RyPuUXSrN>N;>Iue+) zQOFeO>RKM_gLd=-b_sY0q5}gQh32PG;yu=5h_XqN&yK3BI}WgYx~A_zT-te&8(Sz$JRp++oCX+QliS5NrTWoAx~~pI+s^mt}9qQ z_%&5MDKfFq0Ac!+f_lvM+0y2dLw(rP|g0L*1yLgbY!wN%j#2gZPN{n z*ZfaHJc*JK5*eIanszz9KDP;US!2!>-XIn4gh@fAYqu~sijb3&)BK3^b$|%J`n#~-&E|BzWLXVwAi<-* zNJ?6o-3w+CN&%gzzN386=0~*r7miy$!&a9-Dm)L|u$F~^$!myI)y-Wi zkK5yl1#=zEZW~sy*V4(at5d5x z17uY+V8hN3@L&l4`t|(ja%$&%glr@nl^6sH2?=={opU)TPA{mgMicZr?saTg3kR~P zl+xHJ21TsV8;4avxq*Ryye*RU;qvt9K30anqnEkUeBl#kJ&pOWn9&}j;Q)q3`H+eD zE19gamI3x1!-c83PvT$ps}7=rcbtiKQ*f2i$*|`2@mfz6vjIW6n5T7_C#X_Kv{{Lj z&?WguD$W#Yd~q?QNI=aMjaw9rqc;2Xkx2ttij0^vj9UI<6}4*FZY;>K+n*negfD2m zLZ`5%2K{nxY6PGPqz)zkJU4O8X9*%AqJC7~KS99VB*#EB{ckeUdpnZ0*8koW6g2U- zABgQcTe67X8cn3KwY7c2HonDC)!~uTdt*IUbGw3BZPD^6c2!)Z{@Z7MZzm2MW(v^N zhuW#g7a`Hd)V9?2VE2pZ#i<{&9wsLwB(hV~|Xe z8tlNG7BF$dwyv|P9br&ph*9~-HV*OheK^!z%k(H-Dt!2|1k~M$i^X%uLOdKLh!RNx z7j`l_)rOG(tcS?y=;UO+)13AFlmrPHw{5wmSACLBx>L5Bd{ep#>JW4tjL_;cEQzh zZO(M^{^17Ww-p2ZmDq}EjWW#QdJn%R*zLbcOLau6@{k$i-c}>Kgs-)Gmpgp^4zM~N z%G%A`I#Lp#;2mxV!dF>~8`?2Rc_q0oZZ}tU$7BS<-7na8rvE@egI0<6>Mp;06}4d_ zf`e+Qr59M==>)ZrUr#23x8r|RPjeUuOh&5CE+$6faN=?OOI>XC)4DG8b8fuKdAVD; zMT@sFM{S%1cB3~{n6r?`I6VSbZDMZOPbU0NjE<-imXOps3XFn-)c&q7cv>Dnnh|ML zThwmzdPIgZfzaM5`HdMf#Tw?mbnAt@f+grw>lnZKgLMXkO$j*BsBd~3QW8mmNeI}c z5?Z3Su+-$%pY^xBEH($}7@|(B6ry)jDr;eXLT(OUk4$W^4I+{79OvrH{*xFCe65S* zeuz;y3fv)6V2rZtBVFxSrqz$xRiiexvn26M*iy8WgOycd4oH$%~U3fl5Y3R@1k4G&> zO<(^W5+VUM0#-`1y0vxo9D~BxKxzUjDT5RH&(y!hdqEG}kvUK+LIW1ldHWWH-ZAzn za#-~p2#-W}H_)W8t!CnImP-m|4+_NqtsodN81;JAqwT02yIF|4wf*qJo=!Vdg-p)x zS)RE};hI~3np9qEEtw4=-)$-7IGn5rC82qT0rlFN;x=kr23FJ$dan3C>!3^9tmLVp z)Rn&`Mxzb3d?Ui7p#bUB(}CX zYF7u3@O+M#W7MCnC_&;Bwre+awU$0gyLmn>bQvNMC#_bgJB0WPoDSa*>lpp0=X1{4 zCk=b?RD|17i#^6CAsM}EyH{RnT5RpzjSqodzhku&3z~cOg-x|W1y*W8#0Nj-LJ1$k zKt{d5732LpRQNU0Y?B=o4Bj-V@ucAS0QD}@wjq1RP zIBWtaeDy5>W`w24W8R-Eckb95)Q}|{gi8OiD-M^5q$$kdDaBQnVQwf@%EnD`j&Je| z{IKd|An59rDK`Fn?N#AL$T*WFPr{O1oFBmssH+K$NlUl>kOdqn=w?O}luswrDF#hd zc-3ZJHXXUMG9YP-!#J4Sbo@?WgjO-_m#b^J?{m5nG$+TNvRCik$+4vVVLj-uDTuvL zOGmTD0xrkXe?qw`gbvxIBU^ATTfhB@6hEqyg@l+;vTQVZK$4alrg=7}iU-10)eO7( z2tQ+}7P`~!#~?s!H&Nb@T-OZxZ+PRc)K{QSkfhCVt>t71CQ%SKv|wN-4w;8h>4I;d z1t;c{2v$S+9t28IVuoIj+C-#JjGGl2GVny=+=Q^d{7`XHHD4uZ4N`z1U9G4ay!q?j z1O0Eo!dezR+r+;o5KKB}R0S>=mlPxO4OOQzT*xYck%ukmGh`eNv-l)A9(gnj8@FDCXlZ2MHoW;0ejzvT7o9nK4;b#JtEcGwtn;0~u zweb2JX-jg;c$}DJCJi_54tih6BFBUini7 z;XZ^W6eMSaVrrm-L^J_2{^&4xIXdOB43~Z6F~q^%ZE2YLqk5Tf|0Y@oQNZ(c^r16H zgAtSAf~bWZQ%)bhwy$^louG1OLGqna;iYGX6qgQO-;w8ww{p4@2mb5F_EOiUe;x&5 zZ#R^G`Q;_9J%~!4+*%$!6R-5ZJ7GmwMm<1(Qd44Nwk^eRj!^mVL+1OyedZt;-T7;U z1$C>v&Yz=2a#1>EfA$;9b0_k&aA+fWsDYLX+|qQ{4G|7+m>u9%cI zi(UC2b02}&nqT4KkN)GpxkXL)Z0V!Mt~ks3+*rdu2OUjK!jEfp+se!-6~V^YoLwvR zvcQ&x^dg%@#Rd;NX0AyA5wv7Acn4;dhM;z%8;jcC^4)1Ia!s8WuzR9duv!$omg70L z%s~QR+i8(~ucH-emKF1ng2TnFpBZChdWSDZ6HmI+BvBs=Jp))XjqqDkCjQ=Jqr_38 zyxr<5%UK8?$0#dRu`YM0f##@6l*Li4BgLZegW}{T52j3`==?1; zO&og*l%MtKRimLtqQA}TIyet+Iy1TH50b4+>s7Y=8L=do0|2>`RWuX&$TN|{lx_G$LD^m^ zAlsEQsOz5Prrl+I($0N~r}{m?D(x#q11hiu0PAn8jbS62>25%Qebawb9|5}MvECPl zTLkfj!grVlSDlkgF^7KQCWr!rW|@e}6C1X}iNOS7JtMf0pM>mQb+zZx;kJ2QmuThR znq_al?w?QBrUvboq6_f%=%V-V)7uO*hVU}i+5v69&A##oYE$CzahlQZ3?@`pY3;Ki zQ7eCTqfIrN+8c54-Y>Eey`%}lKvT$2cRW31xfx1Fc3=JP?Kfk94h#@kh|%GnL)sFO z#D0Gv&w$>0_ZL&_j@Za54_=kZH0FbHXiRK}tpAY2**1+@R&JGISU14v}g# z22!&2#a7?wQ-=WMni>!f{@aQCOy}r#I8-V5zXcEgu1=;31l>POW1FUq*>(NHFVX(r z>)O4O_pkL5n>TSEu@t^KUnt>jxH^cpcV;%aKH7z^W`@~ucRupd#|ce&F2j}vmvX0< zmbDFeH zWtk*%oc5(|R;R!#UMSQuNi4>5e#(O8wIn&xOml}?Fq0aA`G4#FYPWZrWuvHqo)^N0WOY<3JHkI2!e+D*ZLj75US$>7+%a1kp50dmCQ{a}8 zcvV#O*a)8~;423!aIqGgNGa-5{44)Q)qvN#gw5wA!wod)P+AnkUeDEx{6a*2bMu)a zzI`l_WEJshGl)Gv*GHr9oz;!d-FixNe|Y;A0Sa8n#&K`0Xa+%1scZw7pMfCi14vv$ zhd-G%o_#XyiXEksGOf9Kd~>Mcad&V+_aI9LS16>_R-EeagGJRJnarW(%4pq)X^0$R zWWq_QH1ekdQtCEc1h(95^eUOGQ9Y?cWV?-0q6Q|@iGX8E@TzW;luGYWzciGHTucn; z+?1g`keoQ<2YfzYIsJ~9p1pu54+)o%v1nF%wZa2Lc)4GR$05DNZufTD5c#c7NfZd& zKj@VgA#oeIBANYqv~^=>brQ0~7W-v>M;6~ms08qKo_C9XL)OclB9nMP|RU!+_V34Xq zCornxPMEO^d~(*hXBH1sAoB2%EgpEoYNK!03uem~>SYg}I{KYPu4Zg=msqG0I*Jbq z0zQSuFHgFYt|p#SlFY~E$ztlFgq|1O*!9LVYH`Ddcgl}ZdlL8wfl$-(*TB{+IqSEY zRgurUK4`7&)^sdXjYA6}hCx~IpulOb;TjXV4WHADpAA(kwK7dIGLt!iC=a3LKYvM# zW-%q?Y~dawc`F4#y9)!92LxgjOq;vy$`oy!Nt)PG*JbhaiY)}qexl32KbOlKdUs37 zm0v}RYd5_#7~UuIu;@FU={Id7Mq zBxOxQ(Vfzxyj3bfiOHes8dQrWMI?JJ_)Vg!}P*17V5vnCS#YO@xpNTb+%=hD+zN36n+Jfx)~oXjxE zyX;9y5czQL_hGI^iiD`~VR@Vi>xjJoxe@tIOUmN5Kl|^Sv-x_b&l@l7<@phHRYy~E zv%2H7AE1QQITlDIpU>ST zni#`ijn)RS{p`*9cFLa7sA$HR#BebT_tXo6z6pe14*^&{5C~oP3Ih;m`GtU7npPl$ z#(V_`GHA=X*<_ND-B?uowg+u7T{Wjt(O zOt7jv@?%QG04=$GIJZO0E817wR09ddh7kE-n2d{+aA$(zbA}JlItcFOqdJpYZKuSk7Nc z=k4iXUYoaTqlpSuIxp=){|)rpY}kyAGRe5#zg)fRMf>!p8$!8Q*v*vS-e<=-9wo%* zY|Y&*RQl_@TP8jEfwf_Ghrr4+!?@q>tu&DuL;Io@mXA2h> z3?K+zx5*9{;9Oe#8SvC=ae8+Obe`tWl7ORL5`FVGySVrm`kDz>L(t0mnPsgG-+8=V zO~TU;>KGyk69gk52A!FmE%e7j`wDyw`SnYA z^*MCFY$#07Y8+>7e*WXb@xOl~|NZ=?^mujePshwW|3nqKOY=^H0WiBgWGIe0;F*5c z(m}2<80@5RQ_G6h)w7mT6T3YY0FF>wWm9#v0t&osKyPXH%Xl8^52-Mz(YXtInZ4`^ z1yp5*_}0juX_hHwSaQ~5#KiaIS`{wW^E!oHUGLt|tw2|lor|l;63)b9UJJmlX)^f( zUR_B)Kf$+#$<~&c(CgEs_tW`Eo!J09D3LY$n{ziIKo%To{5Z=7ynO#X!3}XqNf?&) z{Q`#^8VYJ@#9-5X;OI6!N)6aq4Sn@3l&c{Hv}@;|3e7C?Qz8Nd8mf?4okfG-e&&ac zBqO8`;z-)-;|-h1IQ}48Yq00mdzQJ(*^xO%k_8qnfabXaL>}mN`EoxzJg~h*A^d4o zliyxyvd5RwB0ry=#(0DN9xpc_d=Ue2+WZ>CRuTFF{1l_36M*_gK}FsAw(&$0N5UW6 z@Jj$n%o#d!-tNiGNQHsjYPZk8lw(8;4S&T)~-C9s&{QK zGKHctRWg(@GGvI%JJXJgKO{0l?L<+^JliEh=FCH8Qe-M*7DC8SLYe0>&upA$`F+>- zo$FlZI^RFX<&V9uz4qE`t@nAK=YH<@Uh6LTqe)Bh({-V;bYsv}S4mczC%A0Tn4y=f;w?(;=dqmynnGSJ%!85K1GF z!yG$kA*T#5_7s|=<_#w55p_ZK$i%Rqp&JuPNJJy}D)t+SXBKiYsOmYMB`I2 zD<@HKrV!RF_FyIUFCz;@?HWR6=rVxw+O4df#$u7&LotGqY$X{U#@}Um=NKBClp^eZ z#oZB=Be;IyM-dA}fn};}OgCrv#W@XeV}nag`LPct9|hC6_$a{TDsIjK4&C0 z@#T?kH+Yy78P(pcxcA#CFZpPM`~mZTr&Z&6bDK-0x4q;F;j)*r17`R}P3$v(+d5ib zXK^7md9gMm`SDRsH+Xw-twLl%RAW4rC|cp=HfpVqd`d>{I{ryI!<@Q~vefOAFs!;d3uHpkv2LA|)7VG?YQ*VdQciE@;UntP6 z7`l(A%?bK!mv<4z7;es~Q1J5f5QoM@ShcGA%M}VJ*ehvo(})M(a4TK8fhdPAi!p^- zl7esxwhPupE_0penkMMpE6Cp`;d+VFKwdZFs9wFG}hBPq62TvtB`{-VRy{9E2jzRRRn2|6z0Hv{~tf zXG)iU5XbnHcZ42(Rl7o2?sb?Q|A{T8B1!e`Erk11x6pq2E4L#g=C`h!ulOH+*b0%3 zWi4AY({N6!#vgJ_}pRxx;McUylGKdU(C==y~^BdU)>F=JD3Z{E{4!Ss^h%e!uj#Thj zP&gIM)MMO?2EN1#*Ar#tBpS4R(bIuE5-CRRupAt06kT1K^I1!xz*MiTYbiD~_3UBJ z$w+sy<5yC33U-bute?D=5&eoV89Xr(Nk!Ci&bO~?9p{=QZm-_zaJ&ncp7!{aJeiR0 zGec1!0>edF1{VDM6B$k~&Dp2_gc&f@kh8kcUbt{!ac375`b1qYQXh~dEY@$1GFCrd zUVa3R#7)c1<;&a^>oq>4Kr6Z*C2%utzMBmnT5*=~jfluSHKE)pl0e+I^fw*;p(NoQ zqcqP4jU?>?>6x9+A;~VYyl-xlP+EU~Qwzn<-+5&+1t9Rw#=<3=p@D%wzPwUm!OGR+ z=%e~GGr9N;3rPUx=Vbos%1|qCTrim!%J*2yRZlp|jvp5-)8EmgBjvv2#3kZIZ*rEJl3tTQdOrQ(NL`KED>O3r)YP*m1b#N>xfEzR{Ot6D8qj13 z+fS+!RUwxVpLm#o&8V#(-A?(FTw z<%WP7=FeRs=sYz3{zq>Ft5pCL7c)L|dT>Q?Drt&Ahwe_nP=I$IApsJQTCHH7+BAPFM>PxI| znf=tQ*`A!~xw(eXhCqw$GLSM9fZ}Vy-P2P}2?BRq7^K0CKVtA+(ZoWhKi7pY{P+Th z^xeNGB>;jM&hO-dnLz3O-p*2%-0orfsrC|^f&Tu0*RRjJt#+zBBmO$IQaiOGDMtj^ zSA^r+a^Z=szMo8#maY^ej}e@2g&mP=r3t@rsa8Y{JNM(q4i0hi8@{XrWaF13tN4ST z&)3)2OPl(Na5P8HT*d>Ii42OT#Vpf5aXuEpC&)t25pv8q@OA=-e28oQEkwgZSRhDV z`RsX7aq(Lyl=WKh+}}Zwk;hi6DG2$L{OPI5d@0C$nV6YB$!z@eRTHeFd-CMTj{*yF z6bu!YltAE0Sx;dp*vq`s=)Q{Ok{#zku-jVt#3@IF#{U0Rc>Ird=7*G(l^09$-LG|b zKsRXzr&mAm$Q&F2oc4`}&tQ ziCp062Cic4Z&288tKjV5(=Jgtt|L5^j|(=nlx=znjMXHo-)KI1sB#K`K#2F@9h9)O)d z!q2GWv8NQIX2+?isSzYDF0N1-0028);BkuYr^c9Zm7nV^O@I%1~b9Okeen5(o6bwSSn zU>sJ|{*;c5==$zyWyM=f%UrYdh-|9$WP%{ht}3B+>yh&L$y}`qo)@3;xba^DfX?=aW@luDo!0V~k#oSfRX z9|i+DJ3G~rt}3I`0&FyM=_2O-{Q1PXw@7|lYN{n}XQ7(ahiw=L7k$~W1f{Z_`H$=r zK9fUVVntrgCFEY0-CflsjLGetJI}{=jDkc6*UntA)OfnZ!DA|m$bXz#Q+6wTe9a|8 zg~eHMyVZ2>>`m}C(cz&BhfX(oF6DN@RI;Z@Irwd)!)e?gW|LX@A`&+TUOd%s=v*>8 ze~P-G(>>?DmTUSlQ$61#sfyJpn<@Ri@j*(IP;|6*I%T4u*R~#Kb ztbz;BRWLzGT7{ln&2(=$I%9J3j+LFAmX-alC%xEf*RJV6)md#kZL5k)c=&Afsi2!e zpX}w!=M!CKb9A7-tbRAL#dgT;SNx@?$N9^7DzQ%@qj-5>Xiwh-y8ZO(-8tJXIE4&U zkr>`I7B=X*AcGB;FwnQ@D^$8?Wfcb3lJD2!?rSYm2}L|mcf;;A z{mKSi-WO!p>CBRil&JiFy4J7ZA2{gMM4zPh)wpoXOH1F@vt z4^Bv)p(F~SFkan3FJz4V3{{&FvsDt7Tf;e&CY#2VRMG8nino57?w=9F$1jbRE~Ki zh+IHV7{~xz=?i=MR(OoOK;o0{hM5CfiODR`JRF1k`|inaeV_?>ll@blL!jIA7xM<5 zYnAyQVyVse<;#~TX=zttb7AM~z%^D@-n8oe_Kj4Vt^m+WP>5tVGY~+CmbNw#@}v9- zpGJ~&PHa)E2rjj(%uoaeek}_3@4xNJ1egQj&`k)v@Y+sXR6&8T^wxL;)2UM!9FBXc z#pkqb#WC<@^S<){88lT!=NrsNc~BSTRV-iq(_cb`Otm~80%M)Kwzf^^W8T4Qi~ssN zghr0=t}tfE7DD) zq4t9|Ki(06*-$-jpuGOh?V7jEkb8 zVpNR^vfR<$>OHZ%s_lCf5G#Ew`Wl6<8xMX$U^~HpgV9ToOhN*#ukd!uZ-6qXklrG= zH5)w%P4HNG3d@I==n z#9?rik_9Mp+v=|}e`!{Idp!dKzulb;>-QnS!K2{Mal%Cw6``R@cPEep3R!h=Osv2R z2nq@s2HjlT`nIG(5^{C z=OyX$??|zB{$TzMRIEHZJ1c7cZD=S2oW}U}d^U`!FxloneALVt_iyqj^7t`HGorm2Een`f`8 zsaaGdaGxEq1L<=;j533uNs!}^J0mzxzF*DIb%7_{!aVwFdM3x22Os^lq|t{ zGFJF@E@}~FzC7NHY$hg)7bZT%q@wd7ykjcDW4$ODyHLGfwc2Lup-cG`6Q}+T1C>?Y z<@ehU=^qAN)F3@C)*R|GvaRIY-CgjE=HS&i&PGRxt=?+0O}{_z9t-?+iaFMcl!`$$ z{*rB!iR+x0Re!P7iT50+MqLo*G`6y`GOp`HQ}7wxeq*?ILC4;{XcY)=Ll=m_^!9d* zoT4J#K7r_@>xGv72iY6=uE^K9x!jgB{vh+e;sVe)BT>rnD!{df`O>kYSXz>q5KlrH z2R4CED!a&w)89UG612i$F0QTz8}6@}?sEc$#Omqkb-{80CwA0$1|l|a+~H%!0@z!0 z`LTNN>_Vdz8W0eW&Yk3hv8rDn^jsl|u^?|Z#{$>#> zDi*o5chi0C8zj&cVw6J+1Ja%|xr6(HNyYX4D)*0>s3;!`uEkI3sL3**cw523KvzwT zJVQIzE>jylG6U*my?$mWY2_o&PNzegmy?~XS`LWb$6s%96wK@Sb4iCQ3|p#G<1j)!@FUNL$>&1sDmjGq1d8~s>J zT>L7$?d>TZGj3S^IJB#Ad96nmJ=C@YN!{%Y&wa->v|zI+z&+Pe?^RH2MjL(vl@P&; z{{mX%y_;7ydCK^4r&1pa8MH`r{|u9KXbIcB)r|>~_OuivT84)HP!4`V%%Na>5tvUh zA2xYflGXKfV#Hx#@CDdN{f2vcva_>mo-&#|2Y{hCqxqhB(7r<^J198ZTN*SKDBv>qCGD!{l}OM=uu#tDGrgKGSt1xJS0T$aEO`Bio{MG>x}5i)cj%sqOv9a9WD%79Uwue0UB ze8%Gd3tg{p5(Y)FZ>zvZYN$;@vik)4?)8O=m7}uT)8`^1BiWd!0Scy~XQz=+6MRNW zbeDmio_9e?N=jHl;`y(Sj#+DN(03c6pjO=p79&jDmXrt(9GCWB7K}E@?Fpj>f8gWW z_ih(b3xk01b5p`7qwoMs&u$rGV0)wvC_ES~G}%Z1ltW!m&{OY_UZI96ht|mZno@J3 z(hz?DKozX4a;%ckgA8Vs3zla?5GR;mQ2IK1dLqBuLpx*n_lk92g|i4+LPlO(8m!5? z`EhZzTN5SKW5lc(Zrr$mF8bLFRh=QHVfh4z>fyvogH&jc(d9G~_cgsgKg&))-r4;A zq3S7!uX%QZpgr!Ls`mDmHe@I^7#L0w9r+f?iz(T5> zq#)0o51#eV_&_?6Nf9N<4`mX=nl>EykLHkrZk>3ut@81(g;18>$p_qp;^Z5sf CE7dOm literal 0 HcmV?d00001 diff --git a/training_rewards_chosen.png b/training_rewards_chosen.png new file mode 100644 index 0000000000000000000000000000000000000000..7d49d18beda97d1e2afc4a8f09569de336e81c5f GIT binary patch literal 53405 zcmeFYWmFek*e^VUG?F46(hAZgDIuYhAl-;`cXu};pdej>l(cjU2q=nlcX!v>qxbWy z_uKpBd^~IE68R4^d-lHK7Z>453ewmZq!cWas#Vjm`RhFL-S0 zV9G|LfHV$nf^H|H=?FnMhVWm6LeTO8`EBRE_Ep6+gYZ~G4^{sxTciCw-g=^1G z!C}H&4dwDo<+p2We>yu!Eu-`ake*2yTUA)%bBvJkDCeRxA-+V;?Lcrv9rc`^e3CIs zRmmmjZF+iwF|~T`##CFGasJ)3Z*cY6T|ER>Tr5k8i4Y@LoP;~&!V`kO0^dk%4#x2N z@9RLUhtSKvmtk>vZ)lAfezRzepOA)$*B?(Qc6-1kp>G< z zZ#I5^s=Gd!^jtoBoQ(NAiKQ(Bo6IBc){Mk#w^t@3cBEK+QN88j-kYKH=-OEyGO$j% zqT*us>02{R4W}q_B)A$NeLI>Yy>kbD6v9}iw&&%Hokg{mkd`^}wSQ=5+mwNUr zBwy`wem+Wr%XYhQtni;lJ+AdS8eg55AuOhCU)vT&TAB~5#?xP&+26l7>({?QOdUXh z#wv_alOF3vfMY1a6B@`O6_blK)FiU070mYDauNKjW~tg!Id0Y7U(^!5GIjZ(w6s)j z)3Yn$UW%}%ON*jU0Vy^%HiY}rd9u!)_Q{hc>OvoD|A)~I)&hvs5^uV~*4ddsIGWSqsDtRH@7T$$^H%Rufg| zr_-)foLpRZR8+B#O6$Z^SlpLH{K$B%pbmDo+w0T)P69=0QBkV9p-8JAT`6NIg^T!4 z9+fo)7vu zde7W^^N~>R&P`aBNm*Dt92B{ybz2QGNJjAUE2 zCmKKb-Q0qL#jVPr?hibcblVz9U48E!-6I$jz1he4p2C4-Z*MOnFW)s)@A#^^X4dNv z`_1+#F%`x=7}mCp`_Jg;=;lCVY1G-#;1dwoPdkqy%ptd3?zT|fxpN1c zCozu&;mui(XaMem^EEO!g6wQ2DOvQs;WV^mo2w}J&Sk~g!k?l-riiFZ4{RU zx%yw#9)Kk;ra2D!|5`_GMWgwvc_59Shj0_yG(5WdM9iQ{%g#9K^O4c?u z@VlG6&K+K4h)_UwzvTI3Yg80O3(-@u=lUL%b;i?5@9p~Dk`0O83UP9BLSk;{NK=@n z*Qc{RA6;j{qoT0p^A16pEqv*J+~GW`i&{1s={oCuAKn;|%PlU6N52|&zG8m*^yzGR z*C+Jgz@c%k9TTn~`!k-q!PBRo3ep35U$-OS2>I32@LtX1__l)EgqCR6Vp!{WZu3mJ zuYOG7GQGRgbgGw_oNTMOYV(~wAa0F2!|>b=dKrUdZ+}WtbuGq<#sp*DHv%)B=sP_OVa za`K)h8}%m8r|?)3QF?B1j9KwoO@F<&`KRPK_`CngfY0e-e0>l+2$9OG0o_TD2krU26`A5kI}4Zq4E5i;MaMJiB2 zP{q+6H*9f!)^DMN-+)0eJ?d4l+B`^B$ycxLrTgBv8{THz92K~M0%$k#N`3z3a<}JC ziFSq6I48M~8!naa6$dm28g8qu>vV8X5Hj3$AU{wAvv4woLaQxD7kU{p2y47SYS`85 z)NR*H#-*pnSWni>=Oycjo0!mxfSO*u`jCN(i=eFOgw&>Cn^(ua>t5YK-y;NwfKdS% z`mEbPbYI_#1GLZ(`sYcR4Gm;$O-J}}?*xa1v76uIdAiV!$tDQ`3nVUL%WUlI^^R+T zV>mMHZZ8O*1<Shmh%Ik1Iqt@o* z5-}?)D_o9B>sLsxK;`KM#VB^trU9Eb6`OC?t1`Fb>inz?$H(vb@^rDZVMkn98E<2A zvjQ~PG=6(TDJiM5%h}r)a4ErLjde%H?UgCi8iq%^H)b3=4~Jw)?M#G#;|c_U5^!&X zCmYKSEjYWKYe+}}iu>|~v&$6<>wWzIuIyf`4)t^yO?qr3qCP?br&EsflbD#8z1VzH z)Lvlewdjz;&G{-pl1cFP>5L~EN9sUsb@i(;UfUTVPVjgB@ZxwA9pZfRop@p2Eohsq z`RoCL*vZcHb2OUH#r-(Y9XY)(oE8qxgIxNP*%%=@Z;LGF9XJ$v2L|Y)qoWhHMK6BF z$~ZWDvC~a=Um-E#O_c?iLK74xdcCfgTV5UqdP<;PuV}?frdeeczjSdkdwWfVKwW?TEra}nNFr7Y zPgbAH$=QVVcJX;fb8}{n_b-+r9+gu2i-`@Zd3^`p?yB3|+zg+#VLYAnZ7JVQ_uQd@ zzI8{F1zS^4P~_ItrGRdL-cxsV(BE5Uzv$FsadboMom8dK=*CeoZnXs3O?%D8hDAb3 zip_UIXCNUSq%5ZGI!te0A`=t3iC*UAM-H z#E>{dRaF&ja;e2+zdk&pBqRg{(eG-9RLh~F)2hDYuBt-rat!AS42xo3rkp^Z^{lSJT^AjYO>i|5J7AP#8LUGqCj;P zgMpmt_f$R{#b}!MW3~Ccl3luAK7>{WN0N7a+I*bTSAZAKz}Xq&B;;ksRzXZnw6kX887G zZE*}&!es}+j^;~RTr>j!x8e8!RfxfAXkehLdQ>kGG#`}h?d=bjn=nG66%f#LV0D8_ zSnn#RxUE*ai=e6}S+xKP@f58M6fv4ABx-xvxyk(=?)Hd#kLdEH#V#6MoSm~OD)1`b z3?R}V14PdGyA035!lK;s)aEbjfBu}HMlp>a@4k+mzg>*K7*woQ(!S6Y$>Fr7lr^2g zVO(y|7LXDxJqS85WBp^$29R6LR{@y1H@Q5N+-1}@nQcc1HYkAM=kCf1H~?pdM*uuBvYp|5LNX|H_yR_gd3oq7si5UcTu0~+Ldb33 zzddJ}YHq|8@H(@T6^!gM`i2@Q?Xv@VrJ1#LSE_Au`1kKGdVTq(od}@8aZ+fIe-W#* zs|&rIk_{UCD0m;VS-;6`qGkfclYxPOvt>5laL_LRN`L+44aTBUL(Kul!Kw&=vb`tZ zM!|cRploJUSKp(Rj_C|SC)k?{#4#F57f#>>XbE)m0!c{7WsCR)!`qE^v6w48gsE%% z+0}>$j3`nbxt}(mJS8}KtYwA9#l`uNaJ?}A=%8*fMzGxc7ekge-Kn&t*R%8vkcf3U zj|Vs^St8#uDq8H6^}rYvldVzdiycCpsrt@W5byqtjY8pmZL$8h>9?b=IhlQM1Ox;C zDdVVGTj*hM-Q~Am2%=7;r=z0=&FpYx04ejP@foJ*vl7c05WS_PkGZ+ILHE+$zp~tZ z#DI&ISyQ8SkTGQ~j(P5wn3jeGF)=YIO+WT+J~%kQrJ`aR=XF?;f#S}v`u@Vff(##0 zeXspHUn(m(Tz7O}1SV!}U6N(5f`a-RH4gvY!ab28Pz3GAO_B|Frs~}Q7K%?z#Vj6) z031_$KCUcy+)8a);q~BV#S9<*((2$B7HIDFy5%+$P}`@@R5SX$7S%=o@}6C6mf0Vz zDgXZS#|Y%)x52>>KrrC8Z@>5~ic(l>;knZQ&!_=-M>o*eIouCk0~GIPqN(HbQ>uOc zxSJT=adof@K>WS+f(#>&IcJ9%w^3kADa%@Jp7>n+%0NVu@=gc*MVW#&>C^%o62I3O zJB&j=>(t{@c^~1y-C5{jEeDWNbIH>JzV1;KoyYfit;YOD^*rVGKTg}kKKY0rt0PE; zZen7>@3a>8+6trw$Pt7Osj2%vBybFm)_#V?#!3SqV*_|b?_nQ%)CLbWGNN)U^Pe;! z_m>r!S}i_JS(zLqt4v8l$$t**hlEe(hM%)_F=luLz&NP3<(f5#RXfCa^N%6Gpm`0| zF>1fVAyHAQOVIk~z`k@(R9nW!$NT@xkp_Geri^5KHl5%A=?A9(0yzVO17`m4CbXZ} zF=%mI)h8z<$$ftdGOb9fq^gR0%ITLp%ol!)kB8SguEHQS0Ah;0K=w;Wz%OTs1l+Nc z&Q}BQue($tpT~UGe$ey^Ac%Q+3Hh9LP%VUtUePq_v6VI+$O!?i>vlFDO4}^@GXWIH zx;p*=j?@T3X-0M%NZe&c%-Ebl$pJ@zC%(gEqa?hXN*`{v2 z^BqVha;0T|F2x48n_PGG$ORmbcN-7ymbTn@7&Zt+kk4@!TfA|uKT+O1yCSYGb$|8@ zWnpmD_j_Z8?!b_U*X0omvCOn-kr>WlDV zKwD{qg(+;Cj)|^(TFS3sWEOk%`uiI$SjCyki4|#Y_P!*X?f*6Q{XG{dWh^RN;~wH) z+8r!tI!JzXz3VIO>|EU`8Z%1ttr`h@8ZgfRQ1W@XR*BnsUNYTv;W3RO>swH2<~u?$ zKRrDQs&uHF1-06c^F0jzn5z(^g~s!hRKSe!Ukn`VPgwCr zQ3w(R;huRlhwUm>r3}2$ahnf5JNl(azfup##MP$$ZTq-sMl>L$VVAlUMtAb%Q{V7w zIRk=&hK%TTzMM=ynE{YHI~FY(hSyQe6^R?rRUk13-^InnSF?8>seM*T$sb8q7e~|c z+L!9Op{J*Z^Jj_D3y2>01R@Zq)(A??u?~)oaB4RK4w$fevOO6JWGl>7Gv3q+gEmAj zz_8$g_hVI>5lDwY(Oa)x-^S-++(78sySz?N<}I1su*Bs?f$z zX#eudB}P_e=BGwR*>qj8q)pxmiT+3<&`Z^M{l~FF2WUP=IaF;H)e9gOU!CQI0hovf zFrENhUg}OJES2^3W7YNC?{tLXsGr3hqXJwGvpJZ8I1UNlAJuWh2B}v8$Z}$0;*r_% z5!|RLV6ahyDpOeZA!>zFjGaH+3(2XiwwzatV_%N-90AmMgAGwGZaqWypak)3B zpaCS(ED!@dJ-UdHNCu8X2v*z1NU^e-+8Z~o_wV1k&H1ASaT8+Rf81HB+pq`n18x10 zTXzgaB!D!=Kmn9_{=6fGLU3j0qcLDM!pV0Yt_ zlkeN#iG;!C8PGVQBiWjX|1!qm;XS|@Ge3XE0wpKj-~#65@o(P-j#rw}3JQ{;;!t+> zv$aG?MG~QtmdorYD!N_m_xvRz-Y4U*3TarO0m~4ri~DmpzGo7u-lZB9;(M>S+W`fO zfHfp|%fW_mm=Du^)@vfo@VQ_H5kmULhoP_%_V8uJoWY#*7Ln~21QT-pMBBk35Uy;MkC-6!~za+ytjUuPekGdt=7~f5rEhp?Kt!xt`C!Qoo(rPD3+!NFadAhi&k^mk zaZ+L;H9h?uxOD+g1*?VcxlAoCyj2toFP?0rIBe_zx|fbc!WjfGyKDp^v=k$_qcpwB z=6jAw!eNA5F}eL3^rEHh+F7EPZ^M4dAys-`y2uKOu!izq2dj~$;E0@wePPS6kVqf& zQ*1J-o$QSw;ra&B`rGeMD!o$obqP;CtbO#<)upHh$VKMqQ-lhw5;{i4P^Npg_uccsv2S1D+BDYrDB6CGx|A-~MSGtAih3w*}mxqoMH!^(Y?1X{gmq zqh@uBo%s`xF49mOD51>FD$nQJe*OIU6qcP5l9Q!Bt387E+y z0I=4mT1yt}ojd4r;gj1*RmCMG^nh+Y2oDbz@5>z$xr_pwZewGEut(Hx4idP%I+O)k zX79Ie)S{Q;L*D?`Ec3v=1^hJ*Xr*|m1J8h5@&fe#JQ*YjG@4XCTLzT^1$y38`TeOf z-G-#^>B6Z1)g~k+ND|Ia!}*!c3S7ka0M!s6kFK?C2?)IZo+j`XwkHx? zw1$2Fcqj)B<~a)yraZbMX$S5wj@|%a9~5As34rwitPyJ4HjT`7knF(?CRxWp_BSbO zg#FE4sOVh%YPy8H{D9x#z*kT(0C2l&HCa>A66&DE39J^t*NVQf?qMyh26FBJn%I}w zzU|UsQ~O*m$rljyA5&98nGxS{Q~{Xy3oJfZmagw+VPQl>L?F*L`FAh%zO@J4@#K^p zg$L>UJQK^S5G9VY5D4B#@UiK5ACJf=D)xYKyr0~1b;r0nstr^=>BwvqrbGmPJZ$RV zVS>Lu0h**ZI6)vtiDuQ;lbD;E*8%BtAC!0H`B>PT3q&bG$od;LS`|JSE`N-Ydv ztE##>EeA(@$h->d$=OGhWLbp0HA)s!d2aT2wAz?zEM{66tWJm6ae_g2ihGMsP!{lnXEx|*pn*;q%;iwB3{(3i&$L08xoyi;d9&}0U>$iP)`(PQC8AUGE zXkiEii_alAR576CW!BegnjHchN2(}%On8@4C@h?S@!^Hr#d^U$py&iZ%jN~*chJd+ z3kD4^K2~jc-=^i-Wy)pJ8bFEiJ#QnU2f%TRfn6d1K;R4y1~f}#!4VEA zxvAcBUal?d#p7I4ble5Fk?s7^napO>dZ_g?7g>bh-LkCp?x1baA; z9{@m@mVZ9C0V(?aU%TP2atT!NA6u&cGqiW$h(`Srt^r3e1_qyDY>VBzINW(5e(PDl za$=7+M~#0MDnov=S1WwtlfW{B&)E$)Zu>yF?)z~L2Z>ZT8-@wO^6wDfk)!v-Py_;& zxiwxHYz>5MAj!7^yX6jo9eD2#-c{ArS*4|S!FO?M-VeNi4Lz3P+XOsI0+NuY3JQ0D z{u*q(+jL3^*M@l@^sH@d1zW2gqvOKr4wf&G4>K@BSNs^5HwF<0DaC9NqGA<{^Lhhu0Sc-dP1gSUc=z0A{(cH{bDBS4m1rqI`evSroB9P}1i> zmi*=91)|sJK)AR6eGMN%7r*{iaaVJZ%T|fS=5Cgf5yXRl1gFr9+amUe*`Ky=DvH8) zl1%pBW?Ufyc@@~+q@d{q17)wn8_u7?l9I0DQGJ}s@hA{}_b`cA{KlK%uEu=M%=U(o1Tt ztHU9j$v=zxnN3Z)2UJoYLA`4{{-e$Bu!IIg^GMhO0EHB$AeWwI|B9FgN;Uc!ytCkt z>Dw?ofX$8t(CGHE@<&Pl2U+x+$#I1CAT%0cpy0#QnZvmI==2BJ_~2{_{>r(z8nSc| zEG>jRP$U1I8IR>3V%|mJtG$l8-R8@np?|$(t%N3hn2?ZA=0W(szJW!KQUbvb&T4L^ zG)=P)DQCq+o7*gF=>}u~E4C*GO#_&x;E%VkDqJlkWhFvUdlep8B|Y8Ue@$F8Peu+u z6;!5_h3_#i;#SuPIip_onKVW;r}`(UM6XC zTU+J`R8#>F&|`W#?W9Jq)ayRPR4%YTO-TEMC} zmGJtVRGZ1j!9|>N%xr9^9qa(lyMh7-FCp<4;=h@k<*FcyFHr=&dD+RwM&KvaWULJy@UmQM>60OZtwb^sqX7M9M2XTcWP14@K( zJq~&|aA^fL0LRw^@VX8gc#pBM zmf<(>Mx#M~T-Gvl8`)`C3x+p0P+>KEnCjI*!F-LNB-{JJID`+4=g~32d*!i@xY5ig55U2Dhlkr1GyRxfMOQd ze1`>BjdBA7*epy;NBoW_pu#V7u-qTK z3lty)tMM;)Kr}x-*%}WlE^8*euj>^5;}j^0_Mmvp0eaiHGu>c6D6l%W-$NC=~0*KR)Nj z5vqagd5uYYU3R9zfe8zX`T#~J?7|wL)mS+K@Q*`#J4+}4r-3m8>`g&+4jzC<{uL>y zuGgg9fElm{6vTOuli-9h{<{$%wR?bl=WyDiI6#Pn60#p);I-nAeI}A&noFlMP!yv!sH1 z9q=l`f!9j}-tG*Pq2OI#pn(`J_q_w6`m3gA7N<$#Sy)2NLK>l8U2veeiT4th?%YVJ z`3g8{+hs_Qj)(xxzRPmjepv}41ex#i>Trt`DE8d9JX+apLj=+a-HQ+0=AozuO))gZu|N@G)%A14$7`E;&t2 z6hN5+Lp?RG$tz8VxyMbQ30 znt;mis;243dna{pY(II*eRFsAgF}%x!P8S3g<=v^O6?zPJvnRC2!m&7ikbl}4Fp3$ImN}#e<0d`xgdtO=sC}# z#b3U}2OI!43V6*V(6YXvvU-q`)RSsK_e-wLcoKkv&E zSU7>v@E+W4$H&JZKU69&Tv)yUQy9XqI@NM>Tvh?Bp{s-B7G&tPi3vQ41mXnf6%yLo zsaeKAEX5zyb)|q+SMa3Ur<$51AYDUD`jgN_d|^pRtyCu*E*?x^SPpaxmT<9rP&nKC z_z(EfSY(2`ej$MpN=HXeKM(hl_Fzm{Hvu@Vr46w6NFYq{7u^6LTy8P)5NKz{7f0)L zfaTi(RcQV%Tb%;)PH^+G1A3xydk{K2l63<#5+mTfKoFRFg1uuf(8d>+oE!=kFtxWo z>vL(X<9mHv2k0z^#cw8{E)h@G+JcgpCUN%OG(8-~q_BMr!Z8RQpW|yemseedIR`jt z#Aw7Fg!pF&DhfDjO?gTR*k**~_8XJa8lG{OncCbO_j?tJv$3h}q~V)(wWHU2KWWyG z$npDx@;_wj2N3oDQVJ9S$wQ=Nr~jSsL~ef);fK7SD zq5i_1E>WXnR>hsY&zT2e|Enak7Yuc0d*lp1wP7LTW07*SRI-;NSp>;UY2@pn?n9!G3_I zh*0rDdRhnN@8`rJPi<@-1Ep^6@chD)Kam)u)TfnL@pW1qJ<&fDMLl7kC8PRequhKg zoWf(utNkut6DH08knRnDUA{O{Fy}X#ASJ#yv~iX3kP{ox=3=vOIVd-RWl^bQb}kUb zADoFCn`3S%64h3rq(o=|&=KU?US{R$XC`~wiNRaVDJI}uWg^(8J!gtuLvFjVo0ax5 zGdvyj-w`B=1VnJf8)lP=VdRbeH&}{7_ofJ8 zi;GT8w+JF9{70cL(29OUU~M3ci7K7j+KK&Y{ktanx*8L@rs+7Wr?4hvE-Pmrw zG|GQJ;t{_4Q0u}It$^EzO3c?UxQqO~q|9>SS|;MmoL^}fjMzA-@h%~hLe=5oFg9C$ zYwBox5#fJ}IKumGPK3IJQ*O4lcci7qzdvF$p1!S^rIh>c=1j`_Bk?Go1(ZT*oj>C3 zdS>?qt0k+~Mr(XNlBPj;Dg4fUt(w&9@3mOOX2Jd{kzZX(tEPWmd_!*i>cZD7FcEgo z)6#h|oGR3e+mCQD%-3_?C*;V~!{gt68t_juCjH_mmdzD0WHsdWwUEp~AV*WJsVn#J zWqmSLpul}*e#4cEoCT{8Bc?z5CrXkca=VZEFgOBXt_%H3eK*!MrmHT|seu72Dn&jYK||w#{9# zN^^ZlJ{;sWnZc8TH;pZebWenDQjSKK_ZI^x7%R>-j9*+tBFmDd_`7~$Zi4xEqYwA!Djfu0ZL>t;j9=->c`GFOWDh}WJ{=Q)2gK({N;7q-&gZ^9{;eC4R2HFT^# zvlcPWWIVKfBS^vW-_8}T(D15q+apTKrwVR}pwR%POD{5}O!DXZxOTXI*ERgYLXFQR zsSNhpV&0z3rf=L(#35cRl3D$C1BFuzq=E#^Nm@=S&x>z~EYjHsTk{*4eAg2X*1uLy zq9#GQ>0)Kk1+`fs0smD6B$Ov*Smd3j?cC5C3||~?NHN&7SoF@L4J7_EVo1~yE-YBw zh~nGS_4w{+^6X@&68xT41Cax8Uo0D{3^IiWf+~W=(V6+)&93 zp~YVb+jUE=3;)}0tco<8CLf|#CDS2?Rb8aVjsnhAV)2P6o7EiDF2=+%?Lst>K!}3Q&!4GJsh`EK1$w5TnHpYkuH#`@rt_7xv^8bE+9r*Z$4* z_dQ0}Ra-*liDgU~N19I+7i{Hb#sAg_YJq`OpN=DST+By5p%X=XlYMoBBm-?tt|F}l z+O>w;fia<|Eq&_m^`{3ona$JDWwG3u{RR}qpLlCryCwgQ4TAv1YKo6AqE2CRf8dOu z{wHHriJ7q}D4jet zxt-meCf%=DuKdT6ER8R}p(@QHlTR9#2&%@`*iikU_#A89gR*9Kx!7Q^kvV2?3t+qf-c!AgMX-J}IL9eJ;|k618l81Otz^ zm~8zAjM`tR92&d40Esn22zl;VS9$m7WLad2sw@52up^6UWOy)aWcJfmZzA)sDu~>eF`{ z=$E{6T|av0UFVoyp7K??B-=S?)*oNfV?9E8$NX|(70lPI;&$+7$w$v>DGIG5gf zv8c3la^hqPwa~7k{<8WUiyz4?;DDf>&AdHdvK5Y^wC$zf94#b;{%^TUsu^~_yQySx z%kfOo2ObbW#-MpoP_>3}`&9^9eXs7XSd&*oB91m-5s|hG67f^*;5+;8d2y5qeg%)T zRO)q*#w>w!JePoFM5$5;M4CV_*%gPT_ zHiOV%DSZpK$7NPscg!lcVO`-P<}3AGXs=*3C+CHc{@UuWZC zcp(rC=<0R3l!UA8tlpcXj?=h+cN;8xS=DwHkx6LVBfT77ctf=O{xaN8 zgK!rE-aj2!%+HB7+c+}nm=BS0*~M$3(UA*kM!Wco!+?Amu5x;GU`(|+`%1R~l`*xU zqBa6xANav7z`dEW@1Zz5nY4u+)-x{=*qjG40!9m0u=x5b1!<}ryMo^{+o(_?pR?3* z=)LFI1}w!dzb%u>x^kc)uAh(=dQNsY+Fc@kQ2wJe@zrjqd>B>fde?66I!wMbK0iME z$fxIGl%zKx5T6XC8fOl;n)DgTK;nhDDv(c81YK=Jqa(xxyv;UGvhw;Pv!mQ3wiQnW zTKCp_X-%+qh*YRr^{~Fhqmgki-7%GfE?3NQDh5?ShrTj3z4x7AF7#LE!tAMr@x^& z^5M|@o?z?QRE10L=!vsv3!ZygU)X!XD&Nl^pcoY)M9~fDO14As?=L*rU*C9;uG6Nm z=SwB`#Og9?A=vd+XePD3HhR69NiODGENC_R)g9{CdUSW#aHLcs*I!`1S^W&^KDjTq zD0@Nq5JCaJUjn9Hao}NJAU74)?7Io=xruj;PEK|d&}1qxiK!&_q(EbsKCg6U1hO^d zb83|cFrw1&Mc*_vcTyZU>l?2l5$6eNX=v2rWX=$`lfM_8PFEZFVZ7X|<#t6gP!wxo z$oJmZ=t1%>13DtQBil7Tf>O!uT3Pr{ra299+K9{ar`t$OD=^&$vmr2rx2VwsqntqD z4ZZ~q$%S1IARv)H6A2|(RjST314d9cjE2?qjU?Z9C?0r;i1}vLa{CEysx$AAYzj^p zo^x3A%2nXL>};ok&JYi{!XBGaW$q#o+iV$Gmwi~NW&GsM6bSei;@{DGFy{d?!f!)E zqgJPT9XL6_AwAm#vV=Fk9LDnRycWz~e9jM5ijR0V7!1}|ZEPXbqRIIa*+6168A&bT z;!K37k2skV_C_P>$jm6t=mqDS(Txc+nPSA|eMCGCTMztzXxCrC;8f>bXJA!hkUqj$ z3|gJV#Mk)b1#Z=)W>H4mB>FL@&v^E{m3_6t_9db^v1-XU&f7L=Tu&bTmRmtCG)GvG zZ*42WX<9}aQ#nFJJGx8oNncAJ7r(~C1~G}?ak=CRvE1$B1LKHz99B`RSWzf-967|$ z$&siV@!8THnYS-au>-ZBMxd+hmHA#{!ea}-vr*R8#)Ri#Lw>Dqt^QPID!^d~@t?f! z*FlOO(MMYE^gQ~b-fIR2%tx%V<+ibWSzq$Vq}J_Ic!X%T_J1=JG#?JjJ$raYrnklOxI80CKM?N=N5U!!aN(d4&otD`d{vz>znjJ-7OK3yVy6+h%2X2`RtZIwvbe3AIVk2lft1a!4u zxfdlR{SGx(d&3yfowb5QywN)JHW1>&egT<{Tre`R3SbVZLl@E)R`sYFu}1 zW_;{SefL(tT#K~6R+N118Z+qZdEj@I{E2*S_1J{%p$LlLpt9?kK`j3QMLlfZi9ZZ8L6Ku)@v^28rDPd&deWFO*f z#uZYrpIlR^r|<9s;9nihQ%5T_<8sSVdZa-!EGskEJ^YN@*`s3F?d_LcN{r;mz7NVX zOX5^@P ziAnLbH8)NsQ3ZZjCwb}ioCVINxkogm#A-^P<69rKIxO7n-%mp=1s~|u1sx8IVDe9B zkB7Xwv>;rMWZTJLX3LsRy&-Z{$|L2z36> z_>9js{L)`MTsYj4eML~>sQkeX0Hq5CM*;$i`;q-7ov4oz7MXH%N90=9k@}hL2uZ+^n9OhGVqRX|b?s!Lr|CIkqJ|G;Ye`qJ-ci@VnOcI*yf1o&GjXdAa zNBT){8`BQ)t5j0swV1r%$*XEwp23Jn#-)d`qILSk-Y7skW)k*$Q#|@iK5Xtzi(d%T z+Gk3`#Q4E1klh0x`9Ue_H8r}))bH-|yuQ)-m%eyZo$2v{xJ8iaU-!YT2V zlT&A2iqST=7Tiv5gwAfu-JH#UB==e_pGD+|U{T{Fy_dQ}O<=S3$?R{~bsC)tg%!fVf(ecgV21?Tg9 zzj`iMnB$W@@4W4e&oMU?g)aca-mD|Pwo>S~pl@}!#t+Ei4Wb$bii1irTdSO0A4wkL z+X#)dt$W+Zv;)P@!Ya^tUnAaKMu_U#Yd_EUR2`0N{~_8JR%o1+U4tN{5CwycEX<_%( zwa_PMj1guH*%E&Apq<1M*$T1-3G@6RFO*1T^h%uK(@GW!kU-B*apr_u=< z)OCsu2ue5OOz{}c$exoa|61w(-LBf}!ir$%>CV+hmNSIM)S~C1aFVfAl;lP9qSf%K zH7c83w8%V)4pHg|b=vDQ`lo$fI>8d|R$No>aUES@MNhDrR2we|8u*b~mB!UNwoW`80@QP1H=Vf#YHckry zWyY50Uvj8g5_-oHO;h7y_jp&gER%}rCyMCB4-WCPZQ9qBcEsTr1V;!AGhubCtt7lv z4O0?m)+4l2yhBimQ_VYZ{ed=3^UO({V_%={s^#ag$!}!xL1`fWsNSq)Au4foC1lX^ zk5849Lrf2fIwYC-TC*wv6=1S&7r>l22cRx-5ZN5Al2dc2H@_Se$m+b0(Hb#;Fy zTS`JfV3(WRzl~{dd^1t*y>?9YsYXck0ekkes`4);set(F-|7B#?q{m8^cR9U=GEyl zqcX@W)n&_*otRwvm6km3A8>9Ki)yPPZehZmlvvDC9CMY?@$2Go<(NmK?8vF?G-}-K z<0|>*Ty3}XWsP{posTa(H}~KCu5q_3f2LcWymG@VMN?FA{T6tvF z>&L_W)h%Cj=X;0;=@>=x1==lts+4?8X_leM&PRxP<~Bl**sdFyo+3-zH0Xh#3(>Wp zx~DtkFq`J?BFO2(N>$!{v#p~{Ir{6i^=I0(I6b6aflB&uL6B9&+>btF*l&*@U}{;> zHei%Xm=v8|Ignm*Vb`fV9|=IPJf_=Gg8f*-c1FaN^&_T_EAkJhTFv?s_6v32c2rZG z?VgUT9fak$O7+Caj~hnh72--@Q$O0NaHW2@psslRr~MpySQIT;gg|&(gH?5}f6rpO zi}ZN~#PrB0XI>=piqJd`BX1QJBRTy3ES2W1X*9Lm)u*r#3l%)@GAO{*Vkm*0a|cUQ z>n;thc!&q2KU0&(<;#-AkwhF@4lHio?}@`N4+4n@OJim_x9M}=_B39 zQWT{TWGkJVJihTlZ}`0Rr}I5CG>lk|U*u}BNd*&7U}v5^N|qi#jk7HvLdGAjJNgkd z#xQ(||4CqK(G+Ah5u&X}05x<{$7o$A@hzj@>~dbIcnf4LV)mIk*8_v_`zkETJG8u* zk-nk_oFN;|_0{)r+7Mn|KmeGa(0dk7?48e$O5HS);&xP)7F1vR&68uo={BcRG zw_W8+rD2i$8CQ$bg3lG0vl&(@C2DvRu}oLEBVnqWuO3FQn6?(aINP}V^h%uLy<~W- z=e__p7uHlW^iW#|pEaRpXd!U-;hl^=8E?o~R$nr%=#H0U!8-o6wVkUtTJw`(zoMjX zRe1}h{8JTZOAH3fcYUy_#6%fcF^EgP=VOx9~NW^I(SqyBx+rTX*>-W1?P<-{8Wn}pMq(@yp4+4zhkbp5`<5Gc_(GHI*3 z4&xxO!=YkW&V4j(SNM$7Dci*gao89;WyU;(80DkQAhPaEp{^@$lI$;Znzf^|_%$^8 zh*C?}4k~Kmgbn0YVF}QOzsb8`tj?18yk;Ok;%yy4*!h%+Ez?uz=CeGF7wv-bv#8k^ zSv*lt_Kui&`U`U>$FrNsuij%SBM*fs^G1Q+VWMO_qWnt6>jU>h*s4I*_i<`p&kB~d zzU%4_TGf8-$mHVkWMroE!;nu+`@hbo!; z?bGDYbc_xoUJA?hh$fyo3kwm34n1n+j6yKYr%~w6g>d0)Yz@(t`jb&84 zeo~*+a!m9@ST=&yv`bHc|7@>8PUvLZr}682VO+r;K#)*f=M3~?V!NYhcl*zk^$B4m zbFMjdK?P4hdj5;z+Vnb$z0>b9F;A=gSfFy`GJpQ?uR3o()uwPD$|FznfkMVvSn=N3 zV%#F-x>-my76{E_dNbKtztoBcw+(30{HhDSU>{Q~bpG7fJ}+rJd?LDUru(vb+a*#n zn}Ul|L~Omz<~8DG_>Y|YdIC(1`R=RbwkM7C`}{Z6$3a<7ntx`$YFCVj?`iJXmMG|n zeLL_Nu%kWT?G%~~^FBiIV-us}v7DUtal1lDv%33k-g4t$L8;`oj?b6z;1UygVIr0n z*``WcIdW*q(TUihrRW+zYgYGq&?b{#q%t?#xD+#wshb(nX%d&q&1<;ZWDQxkGopt# zukdy81GFblLLHR(sXvmi;2sh7VW~!HP8kB>YR!W}^UY^z0|6nKd&3(C4`|yMs?g6N z@d|79ETX)kP&HgMoY{|E-&@N#>-P%LC-+?{GqEE&C8^|pNGo2=Xk;Hv6X0-7xh2^? zMs1bi={I$?7q!ucJReR1nv4KuI|{R0NuO00%xhIUU+!&fbVM9(oZPXy*4)qX!I~=e zwHEfzH=vmKEGf7lC{$?eX#7GU;m1mKSquX7GL08FyWCE9E%ts6SSg7eTxf!$W+F!6 z_q9^{eeK}Ak-Bp}Bph#e2SF(Z*c2HX;vu5?evYXe90=t>hu4)U@(C5i>d-KyQ$s}Y zH$X;HD1 z3KxRI4OP{90S%W!-W-)C-P4tFJbNo3#evUG6JmDFdbVYM8hJVz)rm}QS{a5&hr2c8 zhr%AAS;NR+Q1MLG8f(5=hdBb{-6$W?ZhgsT5}1ZNWK{hrJF1)L;b4J6>V00ZP8W$= z>o#!X7}U!33Jc^Nr=9aL_WEzpp+acrd79$o^L9*Kir%4QJ)}?QM{BA|kx+}G-ywgt zah})Le)MB(9)+g0dl==ntDDm3PTM|;`q|yr->$bj0#ATttQImz2z#(mmkh_UR>*1<)EgFd!}mc;2mK?0{NTam^u^%SWW_<<@_Bub=WNhW`P zyK;0i@ZJ9*>8yjQ{Jt)JFWueU-Hk|xbV*Bhr=*m0r*wBWN=k=xgOoIg0s7a_GhgvDg&I7rxh9+s`LqP6CO)nm>m`_<;Mw*yWcM{Fg394eI=5F zt+ZgMaq>n@{kJ&4H)T;UE@R(0$HRXSbaB&fznpleovMO!LZJ1fb9U`Onog2xPR8JR z=rrDyw-)XK>xwV9n>nqbQ{|OiOoLsGYA5MV`>W9u@xKOPU4G=n?q0_i0%!`J=4w_s zyA%S5{2$Egv$HWPmg4vkf2p(L5ZPcTNxx-W>G?8+4ih5Hf2zlB&>Db^=YNzqIh1IP z29H(!GbLkJqL~-Zu6Bg6tEQ1jwBR@ZUxtZAEXTq6w|UICI9%cdGJvD-2BHovISJ7b=T{S^(_anr=_R}yhug9$~QWwMNIaG`mLLc^zplqDAOwZ|5kr75;P>i4$X>OZkQ_v3%k;A&liMx3%p<`Y`I= z)F?@lkD~F<%;qoGg};?8D%^|eu4{jBw0iy^oRf2S8^>)5*JtvuOZY>nPE z@Fn6|ol_CO+@?K(&qZD*1uzk!R@y=;>Eea?e-wG#CXD^boqH=qGlcvKfw$e}gF~3V z%fD04kS{|>51vY~ZN);gP$1HT3Fkj~1%_ zmT8;fd-ENR?6u`P!HT*`1q_u%zf{XqN1S?lH;XAy!}vVh9xdbgnD#U6h)5$(c~X3! zJ$zL2)(sB`YV-7%(Frsl14K(mXif}Fn5l_2={)WUQOfQoTLCD?ECX|8Sq={$e8(6D zxQ9*6iS`*p!nPB$zc?>hPMPnc0{WOx+G^%gCV1K;aX$bpSjEVgnuK_}HTEcROy{JH zH`U=jGSC0&#|Q2A9UVcFYUDzN!{wIUh2cbzo2z$!eg6I&&jjxaq2v(S)r zX$QlO?}6>(Fxq?H2Uoo*shH@VzJq@y)n%l^#1|}E@NherQLmH22_Xx(ClS*M?Eg@o z(W!XiqiMbfrVH$@WoZ_D>OEWDh9hwiv~bauhin)u$JLbM6QP`|cik*=;kM*YzFqzv z{_+I5N^xlJErxQ!gaO&_0$e(c`pQ0sh`64kgS0CHCsmI4lLwG=tXv^ zglu)H3ej5;d6cJRHU`{XN9*=(Xm{5m9R%DK)Z+YfN%&=>dx$n@*lD&g zwuH*zPD;EhPXka9^`I{|)jREdEp0Xq?gjto=soT8<2nkY&j#tByT{dO{!wi;9ZW{p@a~t7=2|~Mm*0SoQkI^&hg4GsGRENAI25%_Q^ZNW4;TwZhRFiv zQ4sjCwl!FoyIc8{>;Sj4KwA(R)Lnz}eBUF-8bJqwCn9CL)d{Mm$9OHer#F9s*qm*kHuAmYg8JNc6thO5eQoLY`kQeQE_r`f6 zOy|N3PhMfqi`9RB7y6%98}(^^HnA3`Ioa$AB=B}q;iSQNl(gSgD<9lNFN~RLA|;!W zYhgKgV0zyHdSR#0iCzOE#NGPa7(6pXsK30Ny|v+%f|T8^GCW)7wPw&*OgRS``hTU- z0cR8R@h0Qe|6betd5ti8dJs5qMUjA6TMs(+NhO)DGGPq&@# zjt|>${tz&r_S$*WrBLoz*f@~YR%(-hMUqG{c=MmJl_@uOj43TeQ@S1*Kf5TR#mr=t zfzDqovp%F??@dp_v)<>x7m)rXH4D(2w0DV8JIoGfDpc-^le(A4iUzo;Tb!*^_={ld zV2#zL5f798_2+(a9)5pDEgz@xiICy?$LBy3c7<*ET^ zn!soJ`+~tauJ}=G{cT2<2IqnuF-b0Rw7+Qw{RELRvb%EcrCmp9$t&@Igac^=sMCw`YHBG$>80KlKa1%gq7 z+!AR63Esxc#IcbLtj49!x;SrvOAHFcq73xrIq`v3xlWsS;R=bX7O1+05%K#bV-1wS z!MN_#(!^1wgvmt08~uP#JiZqdTBsxOD%@=m}+7b~baQY7Gm8>DdIbTofl zVCm12`}M#5y!DL(d}Sw{mYzt9nu*x|OkbhbS;Ph~)HRoGTE*v8I(u5YnO#Zk&3FjA zy>Xe4MB*wD!neVkfrI4QddWJ$e7WkA;aq;d^ttQe^n#(g?Gerk9zJ*tAa>wgW6@Xx z+#ROzLo*&|>B!>A$8Gh37p(unnCcXhgF8evCqUcj`v7Wi@mt$J*FcAP?dS1%1JQN3 z#{f)z+XU10#_$p8tu=%}+j={?j3OzWOW~4gdlgaDPJ2E?iVacOaLJf)@hM)V@21Ta z;C0T`M$1_ZTctm4r}|l=JGYOLrboM$UA4CDNs>(vX-15k@+XB%2Y|~h5oM_BpwkC{ z&Uu)E#aqt#%gdRKqkH>{@&8Ivep+!~VviH*AH*B^?576Udf%Jz^Jen!;S;1H^TtX; zS=PsM=I9ORz%>zbbKE)jZ{Q!6Nsf10frJ(6k;NEi58d8gfVVyY>&;S8aTO{!tdt1E zCx(S@kR3m(f9Sp5FS~bApM`|L6h=%A;e-$_rw@>tax;q)Gv?HR zANIZO1A+duT!%P?xO1S?K0npl$nqj;Cc29rvmN!f?XvdSxC`_=sOkJ(W_dAV&x3i| zDKihZ&}qk4blSv)vx~>@N1{u?|Y$Tf<)pZRh`Zri+9>m8AZ96$D4+7p$VOkz+mzKhHLJ0s=H6(<&`;yE-vQ^NCxa4D7 zit0CM=OC|Y89b0xV~g0Ecp{v`z}S#ov@L zNXfwavmRyQ?2sTwQR9=l$o#>;aovy{J;R9Db)JS1!bknrc zJA|glK$DX9GDya;8Jv#Msa=~X+w5>3$BLU&jS^D903Oi{C@qA5fZ+NDi>U%~`H9+y zqE}0#@Ukb#!GFjA?a7K|`8v<>Yy?cacZTaDb*FM2XW>q4^v90CygvL`y|IaJ-{DiW z##c#oZp#!nF{bLwcBGs?Z#AK6&3mwRh$N8X6kB+<-=7{5{= zVdD0dVR%88-zTU4x=*c>qLS(s^%oPu9Sk<3vW2j|>x8G2uxZSO6X$4&mNbrm1y~O_ z&JN%tiUYS^hG##hr5ENy3u&z=(oM_ik#7*>L zRuICicg7z;`F_^pn?}1j|CO^wS>L+-)jo7}r=i|5hU!5WKanhR1M_PpT5r!u2KJjc ztzExOBRDRRmQ>T+&3(r<_+8F+>#3=KWt%h=P*C!#A#n)^BE#FcNyRYd42P{BrXl%W zV`#OpYEt2szQRW(HjctA(0V{ee66`HS!K4`NxnOr1=elh}KXH^Z`!ZJJzP zZ`8;oI;1(uKlKBnA~j*&@3dY07f{|pT|d5J%r>fYblZ_9Kg!rVyDq71QbLm25w11o zz<8tPAo%v`r5^>oXasN-V}dXc83%`&q+4MS7y;s8q@FPncubQVCLwn03fhzsu#!|_ z_#T{p%ecAGKvisZIV0anJ!S`mRb8mjF_KC!7)JFV zLN6q+A6OSI;c2p!;@}|4iJxFNP_S22Jg)*s<@R zJ$KbX7Cif^?G$n@R6|FOY4Vyxyg)ZZ;f`6p@m&%Tp1rTQaynM@OpTJoPN5Eyk3R;4BPoh-mQ1H*-}m86CKd~O(gDIqSlWlJ z4FC5(-O=-32zKx9JEi}*-s=smPPW^UdB^c+kWiFVIac1TbfFR9o`Yn%uosU>EFtFz zFuc+`7JzOT1ebgaBqO$>qR!9om8=I!-~4|D%0wx(_6D;z(8XoQ7sS*hRTlY|!;|`# z8V`j61ojUX`PxtHtbg|Fn<>`-uK$FTv2(a-JDz&+?QzACAhkMiFJkat%)DxHAF3HB zOoGSg;$iB+1Gc_5^<8(9fxBZRDopY#ey^bqwnLUzo~>-k0*)hLb|nvvW8*vJ#!O{- z^h2qIYK;DtC)cl~UzigOR&AS5LzcfC43I*=u`grh$Mln0pAIj6N=3@K8TZd0B-s=% zzXx#9ZeKLX$F?cZ*g*@a_!D?3akeA$WrgU*p`&ls1ibvvR)56V8l~jvIAZE-$~Sua zOSi$R*l8&obN_$nnw~$#cYbk$NQ~1SfBd zwMz&GH!iz89`M-|`38W{)P!iX0Z2a2uRaI*UcSu}a(VxSD$H244#Gz5W8)$G>X(Q# z43*L>2TO8=HlqF$2^OdFE78DP{Q-7_T!Nj>OAAXPgtlCxI3R+)jVLz-Rpd7}y-C-# z(xW?2c|ABzb5X0aa}HN^QjNHvN(3tBM$R=NXT}779Sfd=q^G4- z$=$D=dcF7btU8e-j*HU$+1;pNuo7La$|*qRjMSCrBTl7N9-T<1J!r;nVQHlOIP`}q?C+|T55#ISLRi{!-ze2;sWmyQ(F zY8o=SU5~ZfHD#U=XeEn~adSDZoV(pOI|1S`vU{Y=LQ?bxjG zijQ6qAr7Gt_-VVm|LDMlzU0CIhxaj%O3kMLYLvk5wOea;95MCEym&6etw*$|vr9~$ zx!uWXT)UB|YdFlwYU=i1(6zaI3~p=%T7N}XjmFYEdc@A8%iB=g=_I@gOPEu!909RnwA$; zWakEsw(Z93*;DlRAoD{nlz#+0|GQ=x!OB64xs=L^{VHt{bq4SVRW$}NQ1sDAQ` zzOt*H3OKb|y*|LVP!&EIR^CCLT-5k`M1KHrDXgaAa4&UpX~5h3v}0-VF7cu#8eF0M zQ73TStqButHoPAA;*df*UZ}7F#?U%@HNVy8#!O;mR^?$}>!x4tunD`rU+x$TTN}9S zBcyO^{MO=tVWDRYW1aXdY@M+0zrlYgvN)XT;{}p6SsHB0nlfp|xjHT7s{ytKWL{dW z+c_mFKCU0<;U;mWt@=4WsM#aS`ZuCuDe6p7zH>rCgH=#pU3&XhV`&_|chxd>GnIYw z_Q^5_iz&slZ98{PU;e#b($7Zo#~nYs@?kT*az5GV*(;%;Rkne@x(jRZ*sCKg#OmlT z^$=X&%k}EL;f4sT{{RRFAyiKwL(5{d+3Ll%3*hgPA*4W<&38)?$Jpg?km4Zf^HjdV zNh^lH<8d=0Rh6fqn9y^0B*0Ai?A7ey{GELI^w;Cox%zg^{ko(UT&*Ji zSHbhq4*uBZ5!&4ws5#tL#VuJQ91fIAv_Im?Cx`0iCrgIyL|wI^x&Js6Fyy$+-|{I5^^T7d z5I)@md`>FBhFF6uV6srW*zh}V4$*K6vsc?JDj5keURaO#Y7Fes={aH)4WVrtXCW2@ zpG3lS=IBi^pVULk7^RU45g&>AKHTh-8I@ol&HnZcd_t*75c2yw`_Ai07$A5_3E zUeu2h2Lmq|tRM~&{lysik{S2!=A4OJjy<{syZB#dYJ~}QiSEuvtPoNV9q~sN7l!k} z>YHXd0P}a6W-y3Qc+|zy-E)i;iP__1q;7Ql;EM`H8=z>;NAMxBqerIJZk<$+3+uVw zpm!hJ1SnQ&1N*4>vYPvC?Gx3EXr3Hh(APQxh(McuSJDg)ftiH%ktz-FUpRl41mBQ` zA4UDq4O~U0{!Ua2>%f}>Mt7y;a}&(pf`)OJl-=x4A#hWv{l81xJ$$Jvo3#z@WT%F{ z{Sp7*^9ocpT43^_xrpVOeMX;=Fj$fiH@GNO#mBDtUG$X*c9m+CoHBaJ4u836X~V-M zJ5dI%zZj8O9O(5yzC$@-8I6d7FZn6KfgUhs4)+^WMT@?%3`x)VFh%kdeQmnhcOjkP z_Vk@?_y%HAPLA!*5crfxe%@4T&RtD_RD4a6)a$_$GT>xOUB1CsX*&YUpy9$RmgeV+ zm<+pfO6{-1HpS{|DSCe@Hin1U;|`iWBv}rvvNw*8(`Sb!v^@MBy}u{HKZ!$#RWG&x z0v^N_MEVy2bvrLF4@8B?f=n{!j)U|k3WBnND(BOYX8C7S?0VVo`gv@};CT&vNtjDz4$F*pi7 zRmhui9M*euSbl%$a2Jp9f5<-YzeR;h<~%2gsPUtbn}wiAQvD*4sGCqly7{_kPY@{h zkGzQL3tW76+@-zfgI;QAZ167u!S29i^%!>NAn7S4jhA+Gb90j_jEaK;1FVF2F|UJc z++g78e+gbc`Lm;8u1p7mM7fbnV)NE$jN~I>@ZxyzaT>zjV|8G!&uh)SIz4>00MF}l zG`!eiA(uQ}G(`Kg7e}lXU&r~tn&$ftudlgngWfBmi2l6-O1w3GZ{LgSa`&;G z`QlxAYVQ5~#!)Z#9GJRYPkt8YYm*Ft-QtT5l%1C|xYk-ue%FDU z`?miR^>^brm?w_kMoOvC$s3JD{YPxH)O=_9^v6OLwj}err{C1ljHN%1c`8b>fanET zDZETSD)J-SZ=mns{Brq7j|wPq470CUjf6CUf8td zXNL5oe5-|z4h)zJ6}Q&oLZwXkdB3T)?pZ3e{i4x4t*F6+2!CX?gwZ)01Qh2RMll(K zTznw+Bz2`ix78JDU|mUif!fvB)QX*Sr(JU5iHJ}l&9jRy}ucCE>CU4D%0rOY8e{3!%)=}$v`cA|p~#u8bDEIkr154gfIV4Y3$uyD*)Fo_!b^yFx5 zDD2PJzH2i{?&1{{yS>N4yBPn-HM;=8+1VVtk6T8-tr4lUriLgcEDdcBGzqr{1V+;d z3pE_;{xw+&We7O~A0MP0Y*+i~S-+m&b>iYhtIyX7etsu}u&RIo#vrdii^=J)j7x%}?}KsX>SDwHHJeHcu2w5Q_vkcL3!f}&-GAtdg6m__W%S`Qkn(;zS+ zzWtlaiRumlOSl9$_2u#Bs?GE1!{qtM1aq~cM5dYhK^HQ==X85o+$0r_Aol#dHbq_3 zpaWS?307V!9r4{n_W9PIny^5|2{-`kOoe1+Z+RCmLAkK1PpAXCuL2-Ty)Bu_?PfaI z966sDvx4L8XJV|fZQByl;4mPXJH<_@xME=R_^>}I70oPYXk=;T*l+Ef&r62NxWC)1 z&~dwS1%oz|A{SzBSN!SmxD8jikFIp{Qx5$SQ8q>58&2Zbc2nctcSmT0kA!hU#Nkyb zSDYPV^KhFRU+2$E)V{&A+c1D}S!05LYZ`Ba4LucWC2>Xn2cuTa>U}^ygqpzF~b#rSEZLv2d zKlFwJb_DLky(1~}+gCiD2IF|Jd**LeL=4=sVE)3k`{lQ=G)HI zA^8x7-BW@|@~r5`W}2&kn=eTP$wVf>eS0=;G(@tmON7u%+*2Tswi!-nA2PiA!sXA6 zq~Ly3E#wf$w*y@ZA_Q`0N&s8$PMRHKP5DBCG1@Q-V&?O}4^WE)ByguT#A z6O5SH^|2ddqq_cBs4(W}27V#vDtXi@iP%n=*$CXZSo|_NMsUIVcQBu?u?%k|2L@;d`vLPqJ;L)nqwusxkK*akt+AXOD3D+j%f&6O&xQTD zCiD2p|KB22m}!uwhg!N*yQKFxYQ{g-XDs%2WtPe%eI!lvCOuH~kAhEpboIAPP^^s=$lFFo!x1fT(wN2OLcV z;wnGt{8`@xm-fw(Bl>=8Zfc-Q00{Hq`q2G75WIe0Bcl?#iCT#XQgU8Jg%bO02?{9x zqH=rBQ^;OQA^;_Ah2l!V?ex>cU|>Y1o0_*}nzd!(ud${gn?~iM>zQ!y9gItu?NkB|R`IA&!%D4WI-AEUvh=3bj|CPvGR1X^t24wO}m<9_V4%U!wAIKb&<4 zf$Ud6I!Sy+q5kwXR^<&TJ#wb+Q0fV2*zu!fhHqi%PT*#bFZTY5E@kTHCj5tiDYj|^ zj|+QJV>kGatk2}Iuk4F9h#c@5@PcqVdz@RJkk^BRN+9(8+zOW;>?WB(^Ll5|?Dmv- zV!DJ>B@WWUFdqKgC+Qs922`4MzTV1s?bd7vc@G{d;F!?iwaGr@#(^iRnwQ!zo(Q9r zfgm>EHXu(ZHV>Nf*nxeI)H`v4BA3YTMe``$`}hxBqowhfyLrVX?Gr=hck5DQW*#i9 zxU-V>8w)p2M@g%al(xjNAM=EeiXuJ)dXv0?vt#`-M+y*CU`^KzA^AfWB?nf;!eePWvF7NhF8G z;WJhB{SyfZZ`&Yf)$yXUJ|cyP#C<_sNP5mryDO1%>(EMM6qJ0Y%mAl%8l@xmnh75! zj7o4d+Ah56yT)7y@o*JUYHF!|JVKKF@U8bU_$J&oGT)A-9nIl()SNaR1ZY?2`BneR zX~|N_zE^9q_y}Ih<&! zD0TyX(q{_JnX6(gkoPchljh{(vFGN&OZw1h}F1 zoJ7%9y2@Bz#{SlaFD%B5seoc_JFbYCD0RZFNsvra$88%qLtNe2ez&23lS;*;m;#+|M6D)T<^rk|5y@8^pZeHK4Uk%;& z_{mKomCwW6x{HS&b#ny2(O9L0M3^*CVW)tvJ+O_PPmW-FvYr3N7{WI;!j^vYnZVh< zUU<%we~tJGibl(FxiZh_7nDg{LJ={Nz<6`8_yIFagtO=UE`U+DNAzWH=dOzSnbd^LkFP%wO;skHw4(T;L+i^M@-Yi9g*F79$lHHgvf8h20H3@* zvogj-vCqT>0}0Z6EMT*+ARtodCG86Y1(!6Z#1`g&7%-3@w>d*w5@vDw@f8q~bnB{E z3ml=&#TpApXLa_2=@u&0ThO(GUWl_42maUB)$caBxDCnGASv!|(Qy5Qpa0z3hr9mQ zr8K%*|5r}lc>8k6sr|;rP40D|_py*6|7VCVbMA37?N37Q;~~R`vy>*H{4^^6m5qa? zfK}#JpIv|F-9K8L+v|b8l@!m|W%}GPdbemcFo&tVm&03kYn5QKy@7`~kLyU3m&&Bj z$1aE|A`4@x9g7;6?iyk|tHqq3q>L77nfyii43VCq3yYk2U?)pHzT~Dw)DuQUofX80Jx9+$ zj7LNxCHEC05fQ!dHnx7LA9#QAncoqO1w1XDrTJt3Zq3?mWtN3$iS9Q#!XTXX1n=Aw zX?Y;onMu>Tk%Hy4-8}XKpTJY>Ps<8~|8@;l_?r`e?WNuY)2EOWyD$1f{+-{#`0V|*2#L{74nlIHa_|y3QKmr8|5DGc_w# z2$i7E;}lw`hn$sJe>++DodU(s4xZi0t-r@zf2#;2i9vDILV%k|K2{n`kds%D_2?95 z*(nGir6ZAW!vn2Cxd7!7gy(0+y%gsCIcA-q-t=<70oD>wk!TzwrALz$7`ty9{7qI- zOo9u!L@_{#)_;=n-q05zqj9ygDU)l0!JIZE%^pi$2%aiA`_FJJLYbQ^`fed&FDn$lTGEh;d_NR0B1eFL-ED$;V9P5| zhe1$%bRY$_Y#hk00rO3ikl|C`#V}ZY=L$b%1!M>k<94ezr;x1hpw*NF8>>8Fw?{;nuwjjj7A%nj?{8h zMD*~MLOU|C*8CO^FdC0j!hFA3krEu29EeQY{uJuum2bLym050?tv!Mx<||OlBEg zvBmfuj&I|~UnsP;tQ`#~{@|4Po4fDKv+KmRLlAmbHEtNg>95bCfbF@{P6i4NDS%{+ zfPhlUB)Pk{=7Yi5ker+x?SRLZ=+#4OtM|lZK|3m6FUfT52R>#)e%eIWQ)UZ6uRt`9 zkM$Du!&3{akKYnC@GSz!@yn%k^tQ+P15U2E0BO1Dz(ynoNqn+!bO`6vPA3LndQlQN zga0cWI~(PV*v*~GYRp!t=H%qobP569j38)@+tQ-HA#WXoFmXmhdAhsc`$`$QeHThp zUOBx&F}?49i=h%Up5;G$i{XO|1Bf{5L}Kc!#&PuB{fiYcZs7B{==3w$fR{qzn=UQ{ zbj>IXruoUM)RZ)CPqH=oEMni%h;|Qc(p_;bt5AR2s7!?x<20kW9iT#=~ zp;`vXzBC8+_(E%e75j>Cx?2`U@or>25VMmO%uE>?ZT9{x3-JX$0WD#l4=-24v4yN8 zmA{mS?N;8W;em>~0M$Iu)cj^(2#7v4G0A>9F zxnb)*>xxQ0C<-gTdHxv zcs>3&+q6Xni@O#ue&A@vIR)v1-A$~eMkox>X(cxrFgpXkK+Gb)ToNkHT$GtyB0LCh z;+}_8BdZqcFF2a>@$X9PA31OvuMz?}CrEk< z1^)jq>M>xydl_%_fo@r3jvCIKi$_Jts~8s8fQsrF3M}jm%(6>cGw7N%4|0cL;uOx> zUCdn%i0>j$^8VgO2ec~k;DX}07^HaRq`M4ee>kc-o2XELmS*=fgwU@{6X#XEq8|G! z?$_QO*nRPjJG$@o^uqdOtO9IB4Z=GY4A*d7vhwx;hmCw>^#?R9gIQ%jj*S3Zt&oit zgXUf>4m$~~sg^2DXLlghK$iP(`I`$--JFx%kOBISfpEVkci$_Zzq{~obIGt5_evY^bUpNwm~gDl_x!_0NN6beBTu#bIX@S)nTfW z1(BEW@_(Wz3rB`PTNV46T@sRi592t+g9M;VL6Zr13E#UnP2v{CgCWs(%x0tiFYio) zb((Jy$ogNlhD%3cKp`&~-7k9*z$OmV7Qr&{pB;AQu#rbf5`& z38lExqy3W3Z(&GLQ6{Xp!i@d?ooCC6O$l-z+o=X^UP*=P5!w9xWwIOp|}e9zJS zzyI_l`*d==eyxdy%;XWJ(?^W=dip)wt}Nr??EC?c#dr5^N&s>-nHnbS zl6>xLKPv2)h{Z^xMD{`*Ib=WuxqM^vpw!$l9po?YkHw7>7ob3RKulhbB$MWtE zGPf7G&rnFZf;8;Ue=R9dQ`F2D;V$^S;)z11@!fP8`Cj3K0;Q5v8@=uqs>ZOo6!2)S z**mSK_PJoWLsWRR_q9X<7rL6lz&UB2^mbBm&S$6 z!IVOoU3u6WpPdiUloN2I7j^P`!7)wQP95k`T~A; zf$sZbAC_1|hZI)7rqW^DChMu6>Wb4 z>WDQh?_9mcrHawSAdbaHDwC5s?BXh@Dd-a>iIYF>#tHdjeelI|W%~<>K`ua4H*%=mEYKptxP!REuqz39Cmt=6 zVLvo563|(ADIlzE2CA3{pDI$Nm?RVdDDr1JZ`A$K2dl8AnylahJewifF+1#QkOc*+ z2yp~^MnR&rx`|1{w?}Uf;C{1E$@Bz?t?TDegSS)FHdCHV?!?$<5Z<=wShBB~hy*S# z2h2s}f$;Uw-9PVG=xZAo+4fTIw5 z+yEk%GAVm2LkWaBQ=A7E1Tpk|%=YST-eH1p2&S(NA(dYdcd@wr-oROkn>5Tuisxe| zad``JG(8;zK^ZQE!2xNvHuz{t(`#=4eIY5lpGC3w1`!L(V_UL`$&eV_5BNKeqG;@f zf2xc>=z{BL+f+q_Bd62_9;pV3_B3IqezhJm$Fjvx0o3`G0&Or%jFb{#dthHrXSv?Q z5pUr+%?HkspoL+pe^Q5%SqSO^rcG5&a&9DF()bY>J?isV_52!_4u*G31NM{{BHUSc zNGujZ@Bb-;+tzs*=+VqVE-kLc{2?Xu8y;5X1Gf-4A0I0$B=3UfCntDNv;tKu4xHFH z{<@h2kymgc8#EK4$y7=5uv$KP#gFU!RY;$T;5jRxXl+QW(@})77;ljq|7XzzTR6lG zfqMsROOVmjOi?Bs1gnTB?YtxTV1Y};O(51D2N33jK_ivJ}r9|gR6^w0VOHTme3tGUyuRy1MB+i?j<$rTtbtt0+Y)Lj{ChG z^^Ne(6NVF^$khys!pc;70|`IlyUps)E7?R-4AUD7&hK)u69L#Vvd;keY*Ij9y#zom zpTC!kf(a4EmDN1vTG1%2#r9hG?;T(Jl42+v;auy^8EI zMuX*I$uL;MFS*NmZ4_5=rAuk{Gk40+ z=vQ)MavGnl4Xj0G_!$Gxqos>Uo#YWr0A9|UHj>n`DLaPI8d%+{d0JS`089jN+z zKvw(Si)-g)5kY%<`?-j^A9+yl@HFoHe?1@1m48?LJVl%6`qsNvXO+9%g-S@{TxFU?AyPakX3fiv zxO4Z)$B^3!2_)!Yw+4E}BENurAJ)~1Ul9?N^t@VDz1dwz3hY5a7(VyiI*gwj3UL!o z5s2?kdE0%hoB#CYy$`kS4xL7DJjsr7U-PoY4gHgGEj7Nw3aCvIT@nPC@>!#oW4Wy- z_uPA#-%xN&ssHSn&9KI$MdCHM|9e?LgEUkdfc4b2i-|Ra7-V52iB4q#^4uJFImIIM64^;X-10Jnql=)HgQ2bJ~^>YxhZ(Qg50PJ2%WAa-dZt3}a}NilI0 zU@Cm?{mxT)W0SIM-&hwLHMf_G1%I6Y?Nv>6wJXR7eY?|8Q00WWJfq^iZ7W?-$|VR{ z{L)3w?@221T1acg+Zx6E#msv2wszJBTF7Hu1c$$$} z;b@J^2ELlZ%>(Vp>a!>!!8P~`rVyATF~a2NPHIS5n!uVmw=I*qxF@Uxq^mhclm# zhDs2SyBM-ELnF9~z3iekW5|)g8K?TNvlbI>-8D>Cir=iSjDiU8BI_dQ(#K^*zB&OS z{>>cc0n>OJwYuHN2N#MC21PX`8|;IOJy=cfB5|Hf)T~5t@BX5S)82F03@FADIJk!v z{l$wt1BgO%3Yl2}GduX_8_peWY{P#`3ztp*d|I{=lU8z=F#?WfCbR zpy<(G=68Luuqyl8p6P)|bOh=w6*P{lZyoPn8r@*}nB26v|4uY%UXUlS0;vyd{s4=@ z8lNb}3x~l#U~tE;5iV0BHwFbi)xmC7u(EzL9yq@he?_U1rNpV7WyZgZ>J~PanCkun zFW8SXPQq=x@+Q*Ut^KYGEb^RG-9uo+Ma%yqBC)3iPiODa)c`g8>K-{JJ=R2nX{Bb287Txg(od5OO}i$bH;Xr_WxpHv*dRubeHZk-W?_2$wc z0_8HKuu9I52%5zCEMOxemq_-9mVIaV+gZF$(5_NyvT4TaUUQFkVBo4Bjq$!y8kOqK zT*~`3OKQ{Fer3c^r1oA1_vOhLyaD&_uySr8^r~d=6qmqfmek@v31drhAN6?w<7QABxQYn}# zuw`eGqI{X{ieR}>68j<_*pN*kkp};gS$dt6trGXEg)V`t{EFg{1@Z*EGWbu%V8s-L z|GDJLax7d1<=`b2d>F8?rn!gWM`20NCT(}_m$c9FJtA5Q34lZeay;JJQ?}Q8U!nN^ zpSd!*yez2Vd2i8Dqa}+6V-(<-GqsXJ;^0VubAr7a4ID$Gps0|9?1#P{;_P%yRT5gL zd5t8q2{m{Oue|x%N>JUoEce^9tZXbfF(Z<&@rB~tLpKs~aZI1BA3g|OCsc2^E!wkW z7xd-_5hVi|WgD504bL;zO@+Q8iEp28J?&t78XSlJ*$^nUQos2n3{?kNN}2M?eLLml zt|Wc4<+H=gV~P@fQkCg-LQXN+Nv3?+sb-4>uw(7}B&u^;2H5@fs93(?2;7%^T;@Z# zo-qpuxPzh|6Fow7jv1)I=H_`OGx9*e-=B$3LXevu@#&RY18E>KMqTeJ4{3X+c8|)L zXemZXTPH-vM>&mctr*t0LEs1`2N6852;)dZ+DpdXC*Rb#v+2WJ+WxTx@Foz;&{Sem zOaExvL!zlE4dt$!%fMNr)Qb`fj1)SKA5NM^@z#MopQ>=jL*|&77wkeaS`~4}NAs@$^OBlim8->Pqc$_!8aXwm~-_x>=)$3)E`0m7*q+EFM zcD{xxh?`F#4u1Wp{hzvXDg*AaILA>1j&p7iVW$2U7a^bCzE47=(@d@XlU#tdHs#IP zMt9ZdGB`MD)tS1CVat(yl7ZXdhbXAB;+*96QJ5lUcXi*Io3J})YQqJi%5B4Hl@pXl zh`tg1sAva+QF4b|VuYJhB!!E6lsH6-k)5Pw4Cy0c4`Lt&A{h zhfeD2L(gyckC|4zE4&fq=5J(e=WO^rJOhV+!i)Uuw+-ZylMxF>>3`FHTPzT_zm;#! zpX2%?({YLW8|F@HHn){63IMwlH&CVa(u`@`8u1&d;#>3!H6K-O*6hVxG>l;`M4PgR zsr6fQ_h%*rlNmze%=Sq<}$Q+gW;KI_3~YeEL%Z~t)wRvnfJ0qX>_FLgA$dWTxP0p)8=^n&J( zTEB~oze-Nn>D!N|I<$Qnt+q@1?~V)2hD9N|r5H5gfXBouXXfVv5>mn#A6hpR3n775 zfh|~G&w<>~dGuh85col&}OS~l`NoZO`2 z_eCw?oHQwq=Av&iFxC)0bDbv3F|AdpO%3#N7lY@b52XHy8TaHC2eStEn+R$r4E4G> zzTK-Z+#E`Rn$=YTe{85#kA}u>!{F_k){mNGHz$Cx|2<|nwL-}$nxI;W+4jdTKM)=y zueFT%)cAecT(K!|F-b7e|s; z48YpLpji{qoWPV?eaQi$$e_66-HM1MJDlm%XQFxDkih1Y1tihH0IKu4WQ)4H{VsSp|3Ea#eSw*^GPh|+L?5AHHo$DxqGQRKJLa*1*LD^A+7^o(4asu$(4o< z&u1jQzuIA-@wFr72&b=w;;XAU9GVMXaDKz?#ap9UOlRM{sYObm?T@_Ca9%B;DM*ac6=!R4_*~S6Cff%! z_iI#mOb5dzn($`}j?K`au>}sr?D5;6AQAbPdKGE79Et^tS7Ax&3PDu9UK{n~G~-`0 zhF4%oyO0GjZscaxL!1f>q7f zkDWh{pz&&WmHjHHNWXgQ!jSoRb%Ql&;l$*$Ui>$`evJ!G}51db{-c$yz5 zqlz=HraR3x1{!z32#brBjwlwthscX{`0Dywve2NboJ@ zk7pu$uSs$Hdkacnrd+%_$(Lp(8M#5c;B!SPHAXl(Dpda4ONEEb4ipwo*Ma1?7~pDZ zgBs)5!Obo+U%OC2CWsC^1WR*sA^RWptH4K3$k5PmC}Q^a??B+#w!d%Le+mBcmdioH zktm`UE`qYg&D_RD&1zN&h+cKI)ujcR5}g#wHfrNFgI*ytKrv$;9N)qLnZZCXUaIe* zW8b~Q(V3OUwEN!UkU0eJ=V_=%kXRQz4n&1bNUHmY;zEje!pi(n0|S+~n_A$Y71c}v z_t5)&hdBYz?DAhQ1XJ8NL-at207NaTv3jN9&{R<70BsA`_h``br@n*P2WVWdSJz2; z+Y3G?Y__fWs$jRxYaXRTN~gliN<|Q%wjvc#Yg{fdY2PDPsZUxNo zfCf?z7)t_SqH_%$iWkB+J*yvPh{`?KrW+xO@w-Kf$kOvkf0r@TjV~ZHr5n0Jab|1J zsD`f8pl$J$W?Z1}lI2hhAs1n|7BCL=x=htHgveCHz(|YG-=+M_icGDXt(qiT&@FX= zqXm@7h3pzMv=TfNIILVQJH=RjbO;Fe@SvcQaKQJ@+eRh{u6V~Jw{zZ$@h;YkWfh2H z&lV!ci8tj;_w@0(36lAo{$XJfko^INj20pUS+fuRcMNi17<`|n$EDF=*vs3{d z&U9LG3Jag&3@w&y4!lTDU|4^sM_H_e@+<+3qsAAITED~(u>8@5I#u}%jA^d}w^5;J z29Szb0k)DN!oqMPyjNO~cEDi|`p3ri{)w5HS&uzMK~0VFkB*vJKx*pSX-`70YyAb_ zjn}yaOqYSrYPW|aKW7ON^bf7H{LGZ!)Q6PFGs&=>UuMt=+}G3g7thq;k<@n00My2e zxlc^&i>_fqIvT%}5GXT7Nc1MIfg{!II}2H{+(?!z@vY9MpNzTK`akvJ!Q?C`tHE^e z2lbqauhu)KojA)M*MUa?P!$E(-{QQT;s@&=lm4>xAEx2e76u~*5t>>%*!oA8Lf_wI9#At8~gC~PuC&+QYY7HaGk9FcT`bR~m{b&>OPVV2w zbIkZm{DVp_XZt3vn2_`%CfixP3lS;3pvT>U5hdD;12JG)He(e$%ElJrg8BDhjwRqZ z`;9W)1-9+!;RpsIDk=)~l(q%nZ124bdkU&jf$T5v4BoGk8fm-+XLO#>i+)3>p+rE0 zVi-0B=C|mfV?NDOVpN}$QU=_ZFbE08)6Wjr3?fFF{_x#8>t9zZ&z?UV)dIrzxVs&G zSmNh)vlU}>x!&6@RQ0Ng*I!g7j*pM$+Q2y3@wEmxGYyA3#(XKsM~Xiny2e)gJ_?kB ziCM2)?a}z<;zqtj4Loh1cj}XMylI5hLnnJV9VDL%L-rl#dG5i~$M$F@(M%XD=iwJo z=8p=ycLA-j>o8a;vnZkDS~VrzqJkzUVv_cnV~b_yOz}vq3g;z`f44%1y3=FIa7l|a zXst>qSTE`j#@yVxB7WdH$OHu26e3c4!?%BJ8lbI*Zyy;+iT1wL1vHA zeBAj0YGRx1E9Z#tWqpbq^KAJ=GLuT&7kwQgF^XbaSIG27{?oRN1IS2!az!sbzT9>m z9}_>`9y$k-mxtyt=i7kK6j1Z5fp}}l(L|5O0niDi zoCv<)xjD^`={$HeYURrC{_?WCKk;KyBePWyt0^nL7vAZc)g&N&W6t~Dobg^*scO?_ zitX#(H;X>RcHnyw%+3Qi^!sI&aP8Bd0QAsYmyCCBJg%-jO!4cxMr1x~ZD9jyAi>0o zNnzw7pq*r9kuUUw@u$x86=gg*m?aB-&a6NrE$xH#P*&aM@+vDXxv%fIGi9%a&1ZbQ z50pWG?-$rrn+6df?{K>|WQopE+*9{eRkJ4ui=au)N^wlj0kX}Wx~_D~5n4baAeUcRDD|=0hLq{j=L+;}Gpup;2&*uiEjpkDQl?GI+m&@Il?XHGD zBI62=>faOY04D2t0T)vlm5MVFUCsAjrm&1t@96S8?Vs!^mi4&*IHD&YDuy>x2itl` zRK(1j9~`bo|MrV=IsmyyTjrmFSZYP2HPWA??MzFZcGZ*4Z-($-N%-f@b6wZ z;7-?73CX>nr?qCKuWUfX;VVw{dv%F@h-(}K!JP|OT9_*OsLVhlE{C&)wRYYF`QWfE zzW}V?p`OOOySuidyjOZhi!F&PCTRYsZ~Or<8*y)@ET`7zy+RK4OOYLj!}#^vAu3s6 zFZ0x0Y8|axS!pVTk1wCgU?r?s+T~3bs+v1fXlB+cn_!V;g;&Hk#4iKkX*F0R+x-S`*Fka@xM{@T z)e0{ofs0w)%$u3END(cm$g&Zbj&3XmA60l2dzg`d9{|=UDTZaNrgmQC?0lTSRJJ4D zw_c-0mIm_!7wIkm$JWknhjczVK>u!hZ2w^abx%;+cp1p7~0>ROYZqWjHDh z7-KAH6F!r0s!1baQ5GY|*)|59Hfo>vB==3G>%vla*IAx6rQG}{{i_;3c-?KIQ&SWlk zV8z03K#PMK1utYOnrlH<*$u3l5rLE$uFbJCiP)-iu`LTy0dmN-Z82`kre@dpqP2&Q zfir)#q~&d3g)3i(c51C8M5%@M;Oo${#XUagwvTs(*82hAH3ANCds+Wk8#R5Ywo$XdLCG zrR()ZYPoSgCK=Fy|2BixoS$w- zUU){{h4t%(Cx4Dl9IJ}<#TIP*7RM#a-gMl5NNveALsl~?7p8<45G`>}a<{tiHkmc& zO+qn({CRymIG#^*nf;S1DTwMLoq7rp!b13Nw@J0^`^jBS`zfoyd>*aV&&mvOan3je4rFQ$G*Hn1zCoWPyJ`m+5Jc{V zFIAdUaz1_WIEdj3>IqZ==*UpjnaB(F5O^qJ|5~-R#J=U*qg7{uB?jm(gz-Z`VAVK?
$pS}dI`lho^5V-v?TZb9#1XOU^U304SaBiOUTsqDLY!)#ZYin3c z^H1Ndj$=L#8}Rl|#`|bY$2x19j$W{l%E62Y95Sg!0@b{Ve7(qiWYOQI9?N~L%TE)h zd;Z&+NngatKrR3gQW@+4@tZI~km*?QkOEMIeKY~13UwP8MgY)03TD?fJ@YTqhaQ7! zQ&wx(aMoUOa(2M>Ki;n%?Ewzf0Z_B>`8q2XV8aE002Ys+mzNiRM7Vup)`d`TZY&-I ztmhX?NQ!0Q*6ECJbC^Pbda&`;wnF6wDfe1s#v>%|66Wg{Y@5U>Okwc=) zu>*~T#6^YSJl`vbxP4rIT7b(F{P-XgsX=KN!O3!Qp!1UX+D z5V89s>GYd6zc9T*;-!Ta+z(7N8l)s#J2Wd~G5-(r3~stQq+TvjT9j;MC7V@Y&eB07 zP~sRc7umu3-DX&@rzqJ@_3N`k?B2kO$EXI)KtB_Nt+4){w+aWAJ2zJ&zmO_R zbFaQqY?~r}EsS7|_KUsdz_QjSunL!!8pUPAZ_g4ASOrewN|xS_H(Pi5tg(|pi@{DP z0l=g&0ZrpJnWqBS40zq`DFp$iP`F`ZX9m=KfqJ)XkNHbX&XMG{%A^ zqC?`5ttJ8nQxSJB8NLfP^A%7X-{HLB#fyK19#7u)v8Kz&z_w>h8PZRIH*9;Qgq46mam@h(Exnh#ogk2e!_UgH*~hR9BZ1-?h6$Aw?*@J% zQdSheY)j(N>@ej%`E%3j$-Bc8!57ZvMG2s^C-S@(f!U+*HSWC4OruVIMcV(v{or@D zAt&uEte!4B;k7HOp6!MHdi`txY;}HS!POgPe;uJOqP9atr?wf!=OvKD=xBrMo=HX9 zSX1N#obq~*Q~DeMc{|EKlZMhG3F5A)- z@J^g@gpfzw%9&@G!-lt?n@o92+Ay<3hoA@QC>fu(;Ou?Ergu~bMVko0)kXR~X@4cU zEGYjN5`_Gd!>e911fwj&-iXiV9cpHwFiTf^#c^2ZA#QB?cm?e8?Zuv7+u-0J6o$p; zSacyB;o75xw8ztZItGQi9uRhUU=tZI6Yy@42jnsmmJX13ykRG_nzA&R!b^K92l6z4 zDE#wg{~fFIO(cHmb~nSqK3M=UWT0ukjH|-%%pflhJ=upeF*8gVW4V@bWe&!l?w7wj z8%%HIlf*XWuyf+*%2k9{}~Oy1ulZ$1DjYt{2QIddUxMnA&aIyH*01lJvb>2e_IJ(0g*M8 zS}ggS3SC@Ah4xQ%mK~s=#F{UVN%6Ouue<091h$C-_0n<^S|Z=dkrSCN4bX9nNccN} zIuFpH?eG$PIA8u;!p>726r_2r#Bp_s-(J`_mHH8jLa&9DswKUg%pS86xDo^LAG<7B zt5u0HZtT0K=5S#D^2Yh;Is5OfdnZ5dHv~k_ zud(-w&@-0`c5((-Ygj3H%o??K{SykQD3xOTvC^L0)r`I4V`DNz=sD{Ar|Wb}Ps63SYHiT6n4AgYR~9qWZqlN&Q8PI9lJ{jgQ@~ zhwd#eZ}5m5$yD1FL){QchJXB~sfg>+sJDSi)P3si_`?>{ZZ{5T@eC)3+FkzqeCOy`S}@ZsU-yO&#j+RyDq4F6%_bdNHjt_2XalPge6a0uwjisTXru zxqmyV5T47PM@+P`@rrwukfwN+CHqK1(pHl3+e<21HMyM+r+@}fiD7T}&hIM2+xsp4 z3IcAY%GZpyzff@!P}=0W2JGoe(g9l6};GGLqYrLlY2W+A+Gv?7qmKfGNLl}nhc$8Ya5G*UvvN@$*YQ6a&sYsumOn%Dd#_`Hwl<7n5 z^1WYB=3Q?(Pw|uId;PW(6`Yx4nL}2_8HU$4p3k4%{zk4$*^6>SB_`sb{F#q@4Z_xp zoYyI4L6-pjkH$JOcP!ic#S|RrL$e(O1AO{$7WiloP*3&n< zf85+Ee!k*nNMg$$ul2%t90)DUqX|@Z=1GsRb$z_~s?7E`w6yM)P=s%FhG=W24}5CT z4ww5HyeHk5w||ek3f?Q_Q*m%`nBRfc&%V5ogkzC|TnOhY2lP=HF(QHa6dyXY?fO!f z{gn@P&8q9J^KsiB2^5MSdika)%S>wSw)+$B;<+AF~{mAQ$WD-h0nT1)9v#GIP&aYS68|hLm2}wy#W(9?k)c zz(43;+}Q24R5W#y(Xixf9SIIm=(%%keB!OwH58!5Y+oeeNH-df7sSGOAb4(My6)pq zTSsto$%&&1dt%nJeX7%M}^_;MJWjqqXhV zWHeTb)5z5jgF$&2GlyrgWFRT1;j3enUo0b8;Sm*;ek;qnIExd`^TZt-)KZu`Ml+;|U#oVqf zwn^>IE832qoFCe8lkJX09TgE&jIZe>!+WbNXr3K@-;K6B;CGTrQ2%KC*U?nDW}s4 zrcR#<(L<$I7;Jdea9x~0X|m6q>&Zy7DYpMeid05{`A&Xjwgu$Q=~w~V^>$EK&QA>` z$m71{+i>ywow<|0<*iOSMR!0fOMR%rsFE==jX>u1lbl_&)fLYCCv(+ zsegjwlK3yls^sOyCNVTx9)1kRxxgPk)8{K{eeNEoe)k@b2p-%#j(s^RCzxzUp&gp3 zO+cT(PCyBC2C3g z)$%vv!y5l%4g%WSZ3h>L&#!P220Cds;I+@Hj*8K9e8kl@Ep7{c?u0Mx*!&y>6T3L0yxsc(dEzH=O6Y%CFoQPYBKP zAIkbw%I`2J7nAGzUbRRNl^4M*w7A!oH*DfdDl+0u^~X?doZXz)R(Q%p@Lnc%jg!P( zI8wM%b_bU?wTAK)_5Ru?&G^%1rgy8zJF-(37V`Nl*k-F{Y_E4lAViZzEd?>f-8?tC zW-WdOLF4==FG8|z}fU<^_OO)-dB8l{I}s~MYM5TxV?mM z6WXYHqgyyF^d?6J!ER;nGAnv#$=3%K{#m+H6FW2StXFkgmy4NvW8lR_E&j|n>#X<1D?IL>2d}IAn@adG{Y1P(2 z0>`>7u%|d}PYzXYU9^_p6H+FAd~sv@syA>SZhP-EkwB*1NzOYaIM_a0jievV_&2)% z72(4C+FCay+xV=QH^2B*mlueC(o*#I%*S^Rnst^*-^Pix8(REHHx+FkK&r2<(CMeP zT*w#vSRV1GqrAvgaXfX9`h2BIj?Jr5w7O!j4f6xTtE1?OlqbrR-q!H)m1K%#2CtJ4 z%@(&>r=!IK2Pz3e>k4=&b<9c`6^3}Fn%`fxD_cvSEu^eTjV$iFb(P~+Pc%nTpuaz4 z-pZaj!{U+Pt7t*LN3!#`z-ZTFEX1Z!HJvI{CjA@k!!vuZ$LEWlOKLczFfMRi#KXhqY@H=^EAW7i%+~3cF$ANE@%S6mv_< zsm=$+jNINnMB$PXs^dO~7I zsK=e?QABK4nd3@lYb-@0<<2P3XOg}Ulxr985;WxJ9s%==xP%0Yo70VC4x4l^dwJj> zA%R)sAXue|Q=?Qb)83Mbknc^4@b$OH8WPdyCVo>Pg&uQSW$Q?NV7H{QjQjqqLmj%F zs{~}Vzb8HY=xSkVGg0&Hp#PZg3wC_jE;#?jcN&`agz=o-B~&Ox9K|F z!*07zn^%$qIOJbi(vTYYqnVL%zqB%@+~t23d6q$)B)iHDBQ^`kmuetRJ1+VHefB^k z!+f;hRN$ zn~ogi0sC0Nou18?=`IHze?f%Mlavh7{Pc;&cw?4)SBfVu^gU#V$L4{1#sbVH+M? zsvPe{Y`$DZxvy`!g&?G2caO)Tt_0!j1s53v< z!$XWQGh2l^3T5nKGLFR9*)MlguDO(aP9wH5ur9-Fvl7ux2XlFUo186rLh%MhM&g5>Iy z$zuX!BEzEG7pNBnfjG$Oc5bTec`?o9a-2!FGxN!28Z?r2Y#n^K)i1QALG~G>UyGGG z-B|mu5@8N28;0L*6iE0+24Nc^-olzP^Ia}2L!n-K#YFS^6r*#1TpEj348-feLRZPy zIRElsz92O<^>~fruS1NE%gQrEQn&@uUIW%IT9fU+XW%XE41e65{&mfHVf34ui;Yy& z0)k10EiT?g2=c&98T?WjhB8}LG|ih-Xp~COagzjsp#}+#-r`~SdFcH2V_Ag|XAmd1 zD>CZDd0&s#R)hL4AZ7mc;Wo$uwv|{5&h)H1thJBN=RGqL8J_x!-=5JoeBH?&d>W|{ z?J3|&(|KH(i#IDK0pg4Gyo#NeShB(MEw(0qw(`;A#&hAP z=!0BUg2;U23Nww$pkl`_9z#)#z&POtmiZo>1ZpPREu<59C-t~7$-*4ccK7#a7gTFU zwPuqPX`Elgbf0c#Ypv$lL3#8M+M-WGL$`fa;utn*rMNcI%z(@J{C9j%Cc{0yO8!^L zI@UX?JlYEKRUK~KdRtN0-;0aoVDw&ll<%%|rtDL}{XIy0>&zFMT{RmWivCF~0)1$d zvA3u=9JDCsQ#J6J^uau}Cc=a2H-X!y=g*%pvpR|tfhz0s&&;KmPa+l1P{>r7HDA3X zQu}v(6C4ua2)b?`2+)=_@w_t1n`;y;IEaw=AG1KDUd+|G& zLTahLs>xUNJm88k+SPPDr*~fZ#M+(=lO%D3hyV@7?JWP%!}t2OF#QP2GT?-U>E2q^ z{F%!RbG16YxNXhmTSO}5c-(D5$oOH7uK7ajn6w*@%ts(tvNvR--IrDc>$|f3 zxX85dpgmqgik`wZfr*o@XYCls#E}N6&7zdW$iI)Siu6mXUre~aK7%yMa&4`Ayxk{s zD1{k&p^7O;8YhYlYsdH!A}I=sl#5H5vLo?wLhf9OxVgi+y~)Rj7zSQ3G)JagH6jW- zA9Yd6Sz_xb){!InoL85F6b2pwImQK5#IFxGhJSVwxZ3_sB4RTgw$J+*9UH;^bUxDg z>$X>h&u*Fcqq>^Spc{!!w^hqzB-M_KHB&tN62@UaE(+xw#sLw-j_ey{!PlGRZ|GCw z<5@&c*S+|h_PNQ)$u%w~25CI5is*dTnUgCpKBN*n3;Hgu>0F@>HF?83^3lE#X)HWR zkdZWQHDIoY`o!yTZELmEstr11WH9gs#DWqw%`JzFrL<}Q9jG;4o1<6m2c;_|7i+sp zsHhb@r||7^3w$$anK>kZ?h{4ED9+guB9X_-i&Qv7G}FU{hGc&4*4GR=Z_lo2R7>Uc z_4TiH0kZN=4iOR^eDix59>iyf68v`Vqh_U$*L1-RqsJ>lt1 zj&3}Upha8e{kY|g_VjmQ;Ufl;ToyM5BsZW&3If1X72p*z+lq+xb_`ohD+|=urWd=@ zr+4n9MNFdd+BWKV)iD8yDs0i$On4U~h)MsH&l{qh};= zMAo#^t?mdbA~lHqnTr6HCzp(4+X3nNjU0s!XoOFeYd6(c%+jll%ohty0jX%clRP(Jx;3Uzp-k~l5Qov-H=Vn$iYj>c zb35Of!$!RY%T*;-SSE84JUQ6yH(sPLm2ddpr3u0!K_JoPWE_P8{&JsPrx3V0K)LbN zA?ki2mBeFri|tC2P$5O3 zW_pnGfw3Q21;P->IVPJA1Og!x277~XW(*#3{tTHI0`ckn_mBIpUK9;I`{twZ0#^b> z?1nT7{0<0&$N%5gbl$(bYPsRhtgI2a;XnoP(J`RYBlhF`BW$|4ZI5TTo`XpY$$e`M zfjP(PCxYZsK<^V0LGmAa?Z0}Vh?`>z1M&Rl(_3aD^8xSq*Cpfo{1<->f&35e`A;u$ zGw(}%$clXfXj`v1%Kc6cu`?7mMI9m4M-$wx-2Ba`_Z-Kd_o$S3L~b0yM+*& z!O9{o#{&MQ1*ihuHj_+ZV`A1EK7cEiTdp|Y`2lTtQSUQqBWFC`4bCs)^MuHy2`md*H6g>_)pcy0nneX(D^cDsE{Qw2u8 z8ynON3`cftN8V3&wemM`(y%z!j>n|Sz*7NsR&;MZ1fcsFIpoR&Z zjU*Xw%RN0kC6go-3zb(;YP-wxe0|I_**^ZEb%3Z^ZWkFs%S6d4-_2lrcK2qaK# z%d_c>?@*M?=V5GO;+-TYcbz_9KpCdQ)_THZ1Je?(V2fmoGz3_hDD9D?-5-$K!5(DSmV-7?yTd~_LTBf@naw@5+8iU|u zO)2|6%M09e^3%i86B|f+8Nnyau>=emWI&;2W^T^hBd=Ozx$tuwg@9}7V;5OQ zpU`I8<6c`RQx9}HODy_kjpzw?aSMoTa`0u0g6U9VvnuY5z9>?$XpJ_n7N$*)8zW=m zWav{}@r1G>N&A~`HdA^M&m`-YjyM9)2%aBRg9V`^FRwQ@Hy7;=i$+CH&kjJP>{0_j zv0};*3u6XJ5eq;NP$+$3HEK2n6H)+NMi06viI9MmtVy1Tn8Cxzj$hykic5Owv*T5m~q_Pgd$ z5WuHExB#tPa;DpLD2Osss*wX!go;~R(|-i(?@Z)D_d)i6*y`Em&j4n_XElukPQ>Ma zdSwnaNi-D=P20nPjmoD_B{yeV7VSPT$*gA4gx>cj$4=!pL4vT`7)0LpRbYeST=*sA z0{$ogmqTAx;X+Y0k3rlt2>z^k?LLJEUj+on$;r)h(*c%qyQ|h#Y&My1w(!#~dL%)d z5f|K~!c;3xLXt|7leIJln5ruW#|qYSzZ#={j56&eRZzOW|2!SWW-+rZ=Y1OkHvIVK zH=kdZfnyS?aDD;I2uwEy;^wMNBA}>AeEe(RTM1oJWm*l2;2}+fFtip{cx=!>!6oP^ zDYcpm#Ka5WAbJ%5GA}MFlC)_)@t@54IY*wLl}-1lZo4}O^Y)^`yX=#7gDAkfoVjS# z=Gwg54u7tpm3^w0)zHw84Onwi({@z3egZSc^4i*IU^$-yaYDsEfBv9}ZOVdMg{A_4 zQA8;T{nS7S1Db{p67>uAAzkBn4CWARD7_624^Lanmw?YhJuWT|O>{#xZR1r*DK!m^ z<9c6I;ep9$`ZPcnCK`Ecnhw|>MxX8@bUZIW`mN$eR8d+wmBeD=aDRPL_`pEVQo?+& z5O3| zRs#jl_wjk%Y5-95YiqEx{VSX!7$p_sWp~>DZClMbd*4kA;}8TuTJXcSK>$JHp8+)* zp!aKBTwH*A=IioDRr>t-v$6&_=q$jCl^?|w6|-PHUR=)G1YEfSiXybCYiw-H?M*f9 z!Tk%!?wU1$mpt1@sv;6Czyo@tqxQ9W2}`2?c!`q^jwxf(O= z{Iay?0E0MiZwPL*jWCqKCW*q1lMHeQBx)_={x`N1{#9tKFJ8L?ab)?0L0LuE0a zDy&JvSfW|z;KB?6#;P6^uD0I|0tsW%#olZRulr>pSbNFfHD1Q336#rfi1HabqEDf{N8UK< zu=z}O^7n63;FN6W^>nwC0*HxH|*X@3SssEme2Qp4E@w+skw(W4Y6 zt@j%YpmB?jiGm>G>e^stYKlbZ4Gx`?pC4dyhvmO9^hX9M8}drGdaM6 z$^i`pWktno@HSb%fJG&$sNOtYdOTPHEe3!C!_sN|5P#>*a=rhoNm5vZCBv5h!ng#Uvc8vkUNX)4W95 z=3cr9-yBIR0ZWP+7ey*s2Mo}He~PNA6u|GFPG~(RkBg7rg_0aj9bhuW$$TIaiyhw( z9YHI9IE6!wjg2ij7i?|mXmF3yGc)-E0|Up8P~@76i>t;rqNfw0(>pUgm}MQUEJ$t(I7N%IvIN(9BW6MK z?c#bZVz0vc!J_`I4XlZ|T8o6!vomN?*WAnlfSYZQ>CLscJ5SktvA6H^o}Ha7sjXGH zyE>BFzJUmME5IQU1KmR-)4z?5|8&)$Sa1a2Z6Qd!2(5lscGy3GfFT16tD%(lCQBLy z204eHpfm*c%+mpiMG!!e*{zaWFUoA7pb4}s40@XlU$(L@0VMSU^uKgX!Qn_vMYVmI z2<=_Srg7zgflA4VPc|)i;2KrH)Y-WfB(u#F79sKyM2Sogw&XzmH34GW`T3;W-u8G7 zKDSdTG)V?e#Qg5^;1=BUG(cd%VGQ!-kF(m&X;6ax0Z0@;IQi!&OK7_%_gDbw$2Jss zcRy+=*h+Vo9ZX=DwW|Ls_QVe<7e^)bsZo=c&xGdMPd6D)MWFPXo|@Xu3Lu;STP+Wa z!mt4kle#+9va&KakScLddcPsZxo9;~)xu^_AKEIqgd-ylB9)u!X`pgq`8p0m)&EQmnGxGbm*{#~bIGD0_z fJly|xh4n<_Bd53cJ2_<<>|KCY%8HZ=>G}RQ{-YKX literal 0 HcmV?d00001