From 1c090adbdc45a844335579aa1848a725c295f7f4 Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Mon, 13 Jul 2026 21:23:18 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: bsq1989/qwen_4b_sql Source: Original Platform --- .gitattributes | 36 + README.md | 131 +++ all_results.json | 13 + chat_template.jinja | 85 ++ config.json | 71 ++ eval_results.json | 7 + generation_config.json | 10 + model.safetensors | 3 + tokenizer.json | 3 + tokenizer_config.json | 30 + train_results.json | 9 + trainer_log.jsonl | 342 ++++++ trainer_state.json | 2443 ++++++++++++++++++++++++++++++++++++++++ training_args.bin | 3 + training_eval_loss.png | Bin 0 -> 40648 bytes training_loss.png | Bin 0 -> 37015 bytes 16 files changed, 3186 insertions(+) create mode 100644 .gitattributes create mode 100644 README.md create mode 100644 all_results.json create mode 100644 chat_template.jinja create mode 100644 config.json create mode 100644 eval_results.json create mode 100644 generation_config.json create mode 100644 model.safetensors create mode 100644 tokenizer.json create mode 100644 tokenizer_config.json create mode 100644 train_results.json create mode 100644 trainer_log.jsonl create mode 100644 trainer_state.json create mode 100644 training_args.bin create mode 100644 training_eval_loss.png create mode 100644 training_loss.png diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..52373fe --- /dev/null +++ b/.gitattributes @@ -0,0 +1,36 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +tokenizer.json filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..5c4522c --- /dev/null +++ b/README.md @@ -0,0 +1,131 @@ +--- +base_model: Qwen/Qwen3-4B-Base +library_name: transformers +pipeline_tag: text-generation +tags: + - text-to-sql + - sql + - qwen3 + - llamafactory + - spider + - spider-test-suite +--- + +# qwen_4b_sql + +`qwen_4b_sql` is a `Qwen3-4B-Base` model finetuned for text-to-SQL generation with full SFT on a cleaned split of `PipableAI/pip-txt-to-sql-spider-bird-dataset`. + +This repository tracks the stronger 4B checkpoint from our H20 single-GPU training runs. In our internal comparison, this checkpoint outperformed the corresponding `Qwen3-1.7B-Base` baseline on Spider execution accuracy. + +## Base Model + +- Base model: [`Qwen/Qwen3-4B-Base`](https://huggingface.co/Qwen/Qwen3-4B-Base) +- Finetuning framework: `LLaMA-Factory` +- Training mode: `Full SFT` +- Task: `schema + question -> SQL only` + +## Training Data + +- Primary dataset: [`PipableAI/pip-txt-to-sql-spider-bird-dataset`](https://huggingface.co/datasets/PipableAI/pip-txt-to-sql-spider-bird-dataset) +- We used a cleaned local split derived from that dataset for train/validation + +## Training Setup + +- Hardware: single `NVIDIA H20 96GB` +- Precision: `bf16` +- Context length: `2048` +- Per-device train batch size: `1` +- Gradient accumulation steps: `8` +- Effective batch size: `8` +- Learning rate: `5e-6` +- Scheduler: `cosine` +- Warmup steps: `300` +- Epochs: `4.0` +- Template: `qwen3_nothink` +- Best-checkpoint selection: `load_best_model_at_end = true` + +## Spider Benchmark + +The following numbers are from Spider dev using the official evaluation tooling: + +- Official `match` evaluation from `test-suite-sql-eval` +- Official Spider `Test Suite` execution evaluation + +### Main Results + +| Metric | Score | +| --- | ---: | +| Spider official exact match | 35.0% | +| Spider Test Suite execution accuracy | 67.6% | + +### Difficulty Breakdown + +| Difficulty | Exact Match | Test Suite Exec | +| --- | ---: | ---: | +| Easy | 64.9% | 87.5% | +| Medium | 37.4% | 72.9% | +| Hard | 16.1% | 50.0% | +| Extra | 3.6% | 42.2% | + +## Notes + +- This model is stronger under execution-based Spider evaluation than our best `Qwen3-1.7B-Base` run. +- In our experiments, exact-match metrics were often stricter than execution-based metrics because semantically valid SQL rewrites do not always match the Spider gold form exactly. +- A later 4B rerun with altered training settings underperformed this checkpoint on Spider and is not the checkpoint published here. + +## Intended Use + +This model is intended for: + +- text-to-SQL research baselines +- schema-conditioned SQL generation experiments +- single-turn SQL generation from natural language plus schema text + +It is not validated for: + +- production-grade database access control +- unrestricted execution over arbitrary enterprise schemas +- multi-turn agent workflows without extra prompting / tooling + +## Example Usage + +```python +from transformers import AutoTokenizer, AutoModelForCausalLM + +model_id = "bsq1989/qwen_4b_sql" + +tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) +model = AutoModelForCausalLM.from_pretrained( + model_id, + torch_dtype="auto", + device_map="auto", + trust_remote_code=True, +) + +prompt = """Generate SQL from the given schema and question. Output SQL only. + +Schema: +CREATE TABLE twitter (TweetID INTEGER, UserID INTEGER, LocationID INTEGER, Lang TEXT, ...); +CREATE TABLE location (LocationID INTEGER, Country TEXT, City TEXT, ...); + +Question: +How many tweets are in English? +""" + +messages = [{"role": "user", "content": prompt}] +text = tokenizer.apply_chat_template( + messages, + tokenize=False, + add_generation_prompt=True, +) + +inputs = tokenizer(text, return_tensors="pt").to(model.device) +outputs = model.generate(**inputs, max_new_tokens=256) +print(tokenizer.decode(outputs[0], skip_special_tokens=True)) +``` + +## Limitations + +- Performance drops on more open-ended and heterogeneous SQL benchmarks than Spider. +- The model can still produce invalid column references on out-of-distribution schemas. +- Benchmark numbers here reflect our current internal setup and should be reproduced with the same evaluation pipeline for strict comparison. diff --git a/all_results.json b/all_results.json new file mode 100644 index 0000000..0363d63 --- /dev/null +++ b/all_results.json @@ -0,0 +1,13 @@ +{ + "effective_tokens_per_sec": 1153.798380212981, + "epoch": 4.0, + "eval_loss": 0.24901165068149567, + "eval_runtime": 46.8427, + "eval_samples_per_second": 18.786, + "eval_steps_per_second": 18.786, + "total_flos": 3.545203061907456e+17, + "train_loss": 0.08566030774240586, + "train_runtime": 13933.4985, + "train_samples_per_second": 3.772, + "train_steps_per_second": 0.472 +} \ No newline at end of file diff --git a/chat_template.jinja b/chat_template.jinja new file mode 100644 index 0000000..699ff8d --- /dev/null +++ b/chat_template.jinja @@ -0,0 +1,85 @@ +{%- if tools %} + {{- '<|im_start|>system\n' }} + {%- if messages[0].role == 'system' %} + {{- messages[0].content + '\n\n' }} + {%- endif %} + {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }} + {%- for tool in tools %} + {{- "\n" }} + {{- tool | tojson }} + {%- endfor %} + {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{\"name\": , \"arguments\": }\n<|im_end|>\n" }} +{%- else %} + {%- if messages[0].role == 'system' %} + {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }} + {%- endif %} +{%- endif %} +{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %} +{%- for message in messages[::-1] %} + {%- set index = (messages|length - 1) - loop.index0 %} + {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('') and message.content.endswith('')) %} + {%- set ns.multi_step_tool = false %} + {%- set ns.last_query_index = index %} + {%- endif %} +{%- endfor %} +{%- for message in messages %} + {%- if (message.role == "user") or (message.role == "system" and not loop.first) %} + {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }} + {%- elif message.role == "assistant" %} + {%- set content = message.content %} + {%- set reasoning_content = '' %} + {%- if message.reasoning_content is defined and message.reasoning_content is not none %} + {%- set reasoning_content = message.reasoning_content %} + {%- else %} + {%- if '' in message.content %} + {%- set content = message.content.split('')[-1].lstrip('\n') %} + {%- set reasoning_content = message.content.split('')[0].rstrip('\n').split('')[-1].lstrip('\n') %} + {%- endif %} + {%- endif %} + {%- if loop.index0 > ns.last_query_index %} + {%- if loop.last or (not loop.last and reasoning_content) %} + {{- '<|im_start|>' + message.role + '\n\n' + reasoning_content.strip('\n') + '\n\n\n' + content.lstrip('\n') }} + {%- else %} + {{- '<|im_start|>' + message.role + '\n' + content }} + {%- endif %} + {%- else %} + {{- '<|im_start|>' + message.role + '\n' + content }} + {%- endif %} + {%- if message.tool_calls %} + {%- for tool_call in message.tool_calls %} + {%- if (loop.first and content) or (not loop.first) %} + {{- '\n' }} + {%- endif %} + {%- if tool_call.function %} + {%- set tool_call = tool_call.function %} + {%- endif %} + {{- '\n{"name": "' }} + {{- tool_call.name }} + {{- '", "arguments": ' }} + {%- if tool_call.arguments is string %} + {{- tool_call.arguments }} + {%- else %} + {{- tool_call.arguments | tojson }} + {%- endif %} + {{- '}\n' }} + {%- endfor %} + {%- endif %} + {{- '<|im_end|>\n' }} + {%- elif message.role == "tool" %} + {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %} + {{- '<|im_start|>user' }} + {%- endif %} + {{- '\n\n' }} + {{- message.content }} + {{- '\n' }} + {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %} + {{- '<|im_end|>\n' }} + {%- endif %} + {%- endif %} +{%- endfor %} +{%- if add_generation_prompt %} + {{- '<|im_start|>assistant\n' }} + {%- if enable_thinking is defined and enable_thinking is false %} + {{- '\n\n\n\n' }} + {%- endif %} +{%- endif %} \ No newline at end of file diff --git a/config.json b/config.json new file mode 100644 index 0000000..3e55494 --- /dev/null +++ b/config.json @@ -0,0 +1,71 @@ +{ + "architectures": [ + "Qwen3ForCausalLM" + ], + "attention_bias": false, + "attention_dropout": 0.0, + "bos_token_id": null, + "dtype": "float32", + "eos_token_id": 151645, + "head_dim": 128, + "hidden_act": "silu", + "hidden_size": 2560, + "initializer_range": 0.02, + "intermediate_size": 9728, + "layer_types": [ + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention", + "full_attention" + ], + "max_position_embeddings": 32768, + "max_window_layers": 36, + "model_type": "qwen3", + "num_attention_heads": 32, + "num_hidden_layers": 36, + "num_key_value_heads": 8, + "pad_token_id": 151643, + "rms_norm_eps": 1e-06, + "rope_parameters": { + "rope_theta": 1000000, + "rope_type": "default" + }, + "sliding_window": null, + "tie_word_embeddings": true, + "transformers_version": "5.2.0", + "use_cache": false, + "use_sliding_window": false, + "vocab_size": 151936 +} diff --git a/eval_results.json b/eval_results.json new file mode 100644 index 0000000..1443bbe --- /dev/null +++ b/eval_results.json @@ -0,0 +1,7 @@ +{ + "epoch": 4.0, + "eval_loss": 0.24901165068149567, + "eval_runtime": 46.8427, + "eval_samples_per_second": 18.786, + "eval_steps_per_second": 18.786 +} \ No newline at end of file diff --git a/generation_config.json b/generation_config.json new file mode 100644 index 0000000..c35d61c --- /dev/null +++ b/generation_config.json @@ -0,0 +1,10 @@ +{ + "do_sample": false, + "eos_token_id": [ + 151645, + 151643 + ], + "max_new_tokens": 2048, + "pad_token_id": 151643, + "transformers_version": "5.2.0" +} diff --git a/model.safetensors b/model.safetensors new file mode 100644 index 0000000..b8431bf --- /dev/null +++ b/model.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a04a88d8d7c026499bc570cdb42e1cbc0af368cf779b2f1060543a5d47dee1a5 +size 16089918232 diff --git a/tokenizer.json b/tokenizer.json new file mode 100644 index 0000000..c7afbed --- /dev/null +++ b/tokenizer.json @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506 +size 11422650 diff --git a/tokenizer_config.json b/tokenizer_config.json new file mode 100644 index 0000000..271320b --- /dev/null +++ b/tokenizer_config.json @@ -0,0 +1,30 @@ +{ + "add_prefix_space": false, + "backend": "tokenizers", + "bos_token": null, + "clean_up_tokenization_spaces": false, + "eos_token": "<|im_end|>", + "errors": "replace", + "extra_special_tokens": [ + "<|im_start|>", + "<|im_end|>", + "<|object_ref_start|>", + "<|object_ref_end|>", + "<|box_start|>", + "<|box_end|>", + "<|quad_start|>", + "<|quad_end|>", + "<|vision_start|>", + "<|vision_end|>", + "<|vision_pad|>", + "<|image_pad|>", + "<|video_pad|>" + ], + "is_local": true, + "model_max_length": 131072, + "pad_token": "<|endoftext|>", + "padding_side": "right", + "split_special_tokens": false, + "tokenizer_class": "Qwen2Tokenizer", + "unk_token": null +} diff --git a/train_results.json b/train_results.json new file mode 100644 index 0000000..0616efe --- /dev/null +++ b/train_results.json @@ -0,0 +1,9 @@ +{ + "effective_tokens_per_sec": 1153.798380212981, + "epoch": 4.0, + "total_flos": 3.545203061907456e+17, + "train_loss": 0.08566030774240586, + "train_runtime": 13933.4985, + "train_samples_per_second": 3.772, + "train_steps_per_second": 0.472 +} \ No newline at end of file diff --git a/trainer_log.jsonl b/trainer_log.jsonl new file mode 100644 index 0000000..4aa0fe1 --- /dev/null +++ b/trainer_log.jsonl @@ -0,0 +1,342 @@ +{"current_steps": 20, "total_steps": 6572, "loss": 1.119423007965088, "lr": 3.166666666666667e-07, "epoch": 0.0121765601217656, "percentage": 0.3, "elapsed_time": "0:00:54", "remaining_time": "4:54:53"} +{"current_steps": 40, "total_steps": 6572, "loss": 0.8098940849304199, "lr": 6.5e-07, "epoch": 0.0243531202435312, "percentage": 0.61, "elapsed_time": "0:01:37", "remaining_time": "4:25:19"} +{"current_steps": 60, "total_steps": 6572, "loss": 0.6310151576995849, "lr": 9.833333333333334e-07, "epoch": 0.0365296803652968, "percentage": 0.91, "elapsed_time": "0:02:14", "remaining_time": "4:03:37"} +{"current_steps": 80, "total_steps": 6572, "loss": 0.5526751518249512, "lr": 1.3166666666666666e-06, "epoch": 0.0487062404870624, "percentage": 1.22, "elapsed_time": "0:02:51", "remaining_time": "3:52:34"} +{"current_steps": 100, "total_steps": 6572, "loss": 0.4834024906158447, "lr": 1.6500000000000003e-06, "epoch": 0.060882800608828, "percentage": 1.52, "elapsed_time": "0:03:29", "remaining_time": "3:45:38"} +{"current_steps": 120, "total_steps": 6572, "loss": 0.44873433113098143, "lr": 1.9833333333333335e-06, "epoch": 0.0730593607305936, "percentage": 1.83, "elapsed_time": "0:04:06", "remaining_time": "3:40:39"} +{"current_steps": 140, "total_steps": 6572, "loss": 0.38286705017089845, "lr": 2.316666666666667e-06, "epoch": 0.0852359208523592, "percentage": 2.13, "elapsed_time": "0:04:44", "remaining_time": "3:37:42"} +{"current_steps": 160, "total_steps": 6572, "loss": 0.38465352058410646, "lr": 2.6500000000000005e-06, "epoch": 0.0974124809741248, "percentage": 2.43, "elapsed_time": "0:05:21", "remaining_time": "3:34:27"} +{"current_steps": 180, "total_steps": 6572, "loss": 0.3305965900421143, "lr": 2.9833333333333337e-06, "epoch": 0.1095890410958904, "percentage": 2.74, "elapsed_time": "0:05:56", "remaining_time": "3:31:11"} +{"current_steps": 200, "total_steps": 6572, "loss": 0.2875316619873047, "lr": 3.316666666666667e-06, "epoch": 0.121765601217656, "percentage": 3.04, "elapsed_time": "0:06:33", "remaining_time": "3:28:43"} +{"current_steps": 220, "total_steps": 6572, "loss": 0.3140716075897217, "lr": 3.65e-06, "epoch": 0.1339421613394216, "percentage": 3.35, "elapsed_time": "0:07:09", "remaining_time": "3:26:36"} +{"current_steps": 240, "total_steps": 6572, "loss": 0.2985520839691162, "lr": 3.983333333333334e-06, "epoch": 0.1461187214611872, "percentage": 3.65, "elapsed_time": "0:07:45", "remaining_time": "3:24:30"} +{"current_steps": 260, "total_steps": 6572, "loss": 0.2449601411819458, "lr": 4.316666666666667e-06, "epoch": 0.1582952815829528, "percentage": 3.96, "elapsed_time": "0:08:21", "remaining_time": "3:22:57"} +{"current_steps": 280, "total_steps": 6572, "loss": 0.24566495418548584, "lr": 4.65e-06, "epoch": 0.1704718417047184, "percentage": 4.26, "elapsed_time": "0:08:58", "remaining_time": "3:21:42"} +{"current_steps": 300, "total_steps": 6572, "loss": 0.20863604545593262, "lr": 4.983333333333334e-06, "epoch": 0.182648401826484, "percentage": 4.56, "elapsed_time": "0:09:34", "remaining_time": "3:20:20"} +{"current_steps": 320, "total_steps": 6572, "loss": 0.19693111181259154, "lr": 4.9998867856224845e-06, "epoch": 0.1948249619482496, "percentage": 4.87, "elapsed_time": "0:10:11", "remaining_time": "3:19:13"} +{"current_steps": 340, "total_steps": 6572, "loss": 0.21833207607269287, "lr": 4.999523005839606e-06, "epoch": 0.2070015220700152, "percentage": 5.17, "elapsed_time": "0:10:47", "remaining_time": "3:17:45"} +{"current_steps": 360, "total_steps": 6572, "loss": 0.2382877826690674, "lr": 4.998908383543311e-06, "epoch": 0.2191780821917808, "percentage": 5.48, "elapsed_time": "0:11:21", "remaining_time": "3:16:06"} +{"current_steps": 380, "total_steps": 6572, "loss": 0.21118538379669188, "lr": 4.9980429804147276e-06, "epoch": 0.2313546423135464, "percentage": 5.78, "elapsed_time": "0:11:58", "remaining_time": "3:15:01"} +{"current_steps": 400, "total_steps": 6572, "loss": 0.18367968797683715, "lr": 4.996926883302385e-06, "epoch": 0.243531202435312, "percentage": 6.09, "elapsed_time": "0:12:34", "remaining_time": "3:13:55"} +{"current_steps": 420, "total_steps": 6572, "loss": 0.22483460903167723, "lr": 4.995560204213496e-06, "epoch": 0.2557077625570776, "percentage": 6.39, "elapsed_time": "0:13:12", "remaining_time": "3:13:22"} +{"current_steps": 440, "total_steps": 6572, "loss": 0.22673561573028564, "lr": 4.993943080302715e-06, "epoch": 0.2678843226788432, "percentage": 6.7, "elapsed_time": "0:13:50", "remaining_time": "3:12:57"} +{"current_steps": 460, "total_steps": 6572, "loss": 0.185296630859375, "lr": 4.992075673858379e-06, "epoch": 0.2800608828006088, "percentage": 7.0, "elapsed_time": "0:14:26", "remaining_time": "3:11:55"} +{"current_steps": 480, "total_steps": 6572, "loss": 0.18437937498092652, "lr": 4.989958172286214e-06, "epoch": 0.2922374429223744, "percentage": 7.3, "elapsed_time": "0:15:04", "remaining_time": "3:11:23"} +{"current_steps": 500, "total_steps": 6572, "loss": 0.1850834846496582, "lr": 4.987590788090533e-06, "epoch": 0.30441400304414, "percentage": 7.61, "elapsed_time": "0:15:42", "remaining_time": "3:10:48"} +{"current_steps": 500, "total_steps": 6572, "eval_loss": 0.2562323212623596, "epoch": 0.30441400304414, "percentage": 7.61, "elapsed_time": "0:16:30", "remaining_time": "3:20:23"} +{"current_steps": 520, "total_steps": 6572, "loss": 0.16346561908721924, "lr": 4.984973758852904e-06, "epoch": 0.3165905631659056, "percentage": 7.91, "elapsed_time": "0:18:27", "remaining_time": "3:34:45"} +{"current_steps": 540, "total_steps": 6572, "loss": 0.18838067054748536, "lr": 4.982107347208317e-06, "epoch": 0.3287671232876712, "percentage": 8.22, "elapsed_time": "0:19:03", "remaining_time": "3:32:49"} +{"current_steps": 560, "total_steps": 6572, "loss": 0.177593994140625, "lr": 4.978991840818816e-06, "epoch": 0.3409436834094368, "percentage": 8.52, "elapsed_time": "0:19:40", "remaining_time": "3:31:13"} +{"current_steps": 580, "total_steps": 6572, "loss": 0.20775914192199707, "lr": 4.975627552344638e-06, "epoch": 0.3531202435312024, "percentage": 8.83, "elapsed_time": "0:20:18", "remaining_time": "3:29:49"} +{"current_steps": 600, "total_steps": 6572, "loss": 0.16498700380325318, "lr": 4.97201481941283e-06, "epoch": 0.365296803652968, "percentage": 9.13, "elapsed_time": "0:20:55", "remaining_time": "3:28:12"} +{"current_steps": 620, "total_steps": 6572, "loss": 0.17565951347351075, "lr": 4.968154004583374e-06, "epoch": 0.3774733637747336, "percentage": 9.43, "elapsed_time": "0:21:31", "remaining_time": "3:26:35"} +{"current_steps": 640, "total_steps": 6572, "loss": 0.16204673051834106, "lr": 4.964045495312794e-06, "epoch": 0.3896499238964992, "percentage": 9.74, "elapsed_time": "0:22:07", "remaining_time": "3:24:59"} +{"current_steps": 660, "total_steps": 6572, "loss": 0.17068564891815186, "lr": 4.959689703915272e-06, "epoch": 0.4018264840182648, "percentage": 10.04, "elapsed_time": "0:22:42", "remaining_time": "3:23:27"} +{"current_steps": 680, "total_steps": 6572, "loss": 0.1589680790901184, "lr": 4.95508706752128e-06, "epoch": 0.4140030441400304, "percentage": 10.35, "elapsed_time": "0:23:19", "remaining_time": "3:22:07"} +{"current_steps": 700, "total_steps": 6572, "loss": 0.17568455934524535, "lr": 4.9502380480337e-06, "epoch": 0.426179604261796, "percentage": 10.65, "elapsed_time": "0:23:56", "remaining_time": "3:20:51"} +{"current_steps": 720, "total_steps": 6572, "loss": 0.16204804182052612, "lr": 4.9451431320814715e-06, "epoch": 0.4383561643835616, "percentage": 10.96, "elapsed_time": "0:24:33", "remaining_time": "3:19:40"} +{"current_steps": 740, "total_steps": 6572, "loss": 0.16562143564224244, "lr": 4.939802830970762e-06, "epoch": 0.4505327245053272, "percentage": 11.26, "elapsed_time": "0:25:10", "remaining_time": "3:18:26"} +{"current_steps": 760, "total_steps": 6572, "loss": 0.17697544097900392, "lr": 4.934217680633646e-06, "epoch": 0.4627092846270928, "percentage": 11.56, "elapsed_time": "0:25:46", "remaining_time": "3:17:03"} +{"current_steps": 780, "total_steps": 6572, "loss": 0.1649466037750244, "lr": 4.928388241574327e-06, "epoch": 0.4748858447488584, "percentage": 11.87, "elapsed_time": "0:26:23", "remaining_time": "3:15:55"} +{"current_steps": 800, "total_steps": 6572, "loss": 0.1602837324142456, "lr": 4.922315098812883e-06, "epoch": 0.487062404870624, "percentage": 12.17, "elapsed_time": "0:26:59", "remaining_time": "3:14:43"} +{"current_steps": 820, "total_steps": 6572, "loss": 0.142719042301178, "lr": 4.9159988618265585e-06, "epoch": 0.4992389649923896, "percentage": 12.48, "elapsed_time": "0:27:36", "remaining_time": "3:13:37"} +{"current_steps": 840, "total_steps": 6572, "loss": 0.14508233070373536, "lr": 4.9094401644886e-06, "epoch": 0.5114155251141552, "percentage": 12.78, "elapsed_time": "0:28:12", "remaining_time": "3:12:31"} +{"current_steps": 860, "total_steps": 6572, "loss": 0.1821539044380188, "lr": 4.902639665004641e-06, "epoch": 0.5235920852359208, "percentage": 13.09, "elapsed_time": "0:28:49", "remaining_time": "3:11:27"} +{"current_steps": 880, "total_steps": 6572, "loss": 0.16131887435913086, "lr": 4.89559804584665e-06, "epoch": 0.5357686453576864, "percentage": 13.39, "elapsed_time": "0:29:24", "remaining_time": "3:10:16"} +{"current_steps": 900, "total_steps": 6572, "loss": 0.17404688596725465, "lr": 4.888316013684435e-06, "epoch": 0.547945205479452, "percentage": 13.69, "elapsed_time": "0:30:01", "remaining_time": "3:09:11"} +{"current_steps": 920, "total_steps": 6572, "loss": 0.14134640693664552, "lr": 4.880794299314732e-06, "epoch": 0.5601217656012176, "percentage": 14.0, "elapsed_time": "0:30:38", "remaining_time": "3:08:12"} +{"current_steps": 940, "total_steps": 6572, "loss": 0.14891813993453978, "lr": 4.87303365758786e-06, "epoch": 0.5722983257229832, "percentage": 14.3, "elapsed_time": "0:31:15", "remaining_time": "3:07:19"} +{"current_steps": 960, "total_steps": 6572, "loss": 0.1696299910545349, "lr": 4.865034867331967e-06, "epoch": 0.5844748858447488, "percentage": 14.61, "elapsed_time": "0:31:52", "remaining_time": "3:06:21"} +{"current_steps": 980, "total_steps": 6572, "loss": 0.14085158109664916, "lr": 4.856798731274874e-06, "epoch": 0.5966514459665144, "percentage": 14.91, "elapsed_time": "0:32:29", "remaining_time": "3:05:22"} +{"current_steps": 1000, "total_steps": 6572, "loss": 0.133053982257843, "lr": 4.84832607596351e-06, "epoch": 0.60882800608828, "percentage": 15.22, "elapsed_time": "0:33:04", "remaining_time": "3:04:16"} +{"current_steps": 1000, "total_steps": 6572, "eval_loss": 0.24901165068149567, "epoch": 0.60882800608828, "percentage": 15.22, "elapsed_time": "0:33:51", "remaining_time": "3:08:37"} +{"current_steps": 1020, "total_steps": 6572, "loss": 0.12680984735488893, "lr": 4.8396177516809695e-06, "epoch": 0.6210045662100456, "percentage": 15.52, "elapsed_time": "0:36:06", "remaining_time": "3:16:32"} +{"current_steps": 1040, "total_steps": 6572, "loss": 0.14880582094192504, "lr": 4.830674632361178e-06, "epoch": 0.6331811263318112, "percentage": 15.82, "elapsed_time": "0:36:43", "remaining_time": "3:15:23"} +{"current_steps": 1060, "total_steps": 6572, "loss": 0.1447562575340271, "lr": 4.821497615501186e-06, "epoch": 0.6453576864535768, "percentage": 16.13, "elapsed_time": "0:37:21", "remaining_time": "3:14:16"} +{"current_steps": 1080, "total_steps": 6572, "loss": 0.15530819892883302, "lr": 4.812087622071104e-06, "epoch": 0.6575342465753424, "percentage": 16.43, "elapsed_time": "0:37:59", "remaining_time": "3:13:10"} +{"current_steps": 1100, "total_steps": 6572, "loss": 0.14426586627960206, "lr": 4.80244559642167e-06, "epoch": 0.669710806697108, "percentage": 16.74, "elapsed_time": "0:38:36", "remaining_time": "3:12:03"} +{"current_steps": 1120, "total_steps": 6572, "loss": 0.15436025857925414, "lr": 4.792572506189489e-06, "epoch": 0.6818873668188736, "percentage": 17.04, "elapsed_time": "0:39:13", "remaining_time": "3:10:54"} +{"current_steps": 1140, "total_steps": 6572, "loss": 0.14860854148864747, "lr": 4.782469342199915e-06, "epoch": 0.6940639269406392, "percentage": 17.35, "elapsed_time": "0:39:50", "remaining_time": "3:09:48"} +{"current_steps": 1160, "total_steps": 6572, "loss": 0.1313084125518799, "lr": 4.7721371183676205e-06, "epoch": 0.7062404870624048, "percentage": 17.65, "elapsed_time": "0:40:26", "remaining_time": "3:08:40"} +{"current_steps": 1180, "total_steps": 6572, "loss": 0.150812029838562, "lr": 4.761576871594841e-06, "epoch": 0.7184170471841704, "percentage": 17.95, "elapsed_time": "0:41:02", "remaining_time": "3:07:33"} +{"current_steps": 1200, "total_steps": 6572, "loss": 0.13278884887695314, "lr": 4.750789661667318e-06, "epoch": 0.730593607305936, "percentage": 18.26, "elapsed_time": "0:41:38", "remaining_time": "3:06:24"} +{"current_steps": 1220, "total_steps": 6572, "loss": 0.1612934350967407, "lr": 4.739776571147943e-06, "epoch": 0.7427701674277016, "percentage": 18.56, "elapsed_time": "0:42:14", "remaining_time": "3:05:16"} +{"current_steps": 1240, "total_steps": 6572, "loss": 0.16211290359497071, "lr": 4.728538705268116e-06, "epoch": 0.7549467275494672, "percentage": 18.87, "elapsed_time": "0:42:49", "remaining_time": "3:04:10"} +{"current_steps": 1260, "total_steps": 6572, "loss": 0.14386119842529296, "lr": 4.717077191816824e-06, "epoch": 0.7671232876712328, "percentage": 19.17, "elapsed_time": "0:43:26", "remaining_time": "3:03:09"} +{"current_steps": 1280, "total_steps": 6572, "loss": 0.12942540645599365, "lr": 4.705393181027463e-06, "epoch": 0.7792998477929984, "percentage": 19.48, "elapsed_time": "0:44:01", "remaining_time": "3:02:02"} +{"current_steps": 1300, "total_steps": 6572, "loss": 0.14771063327789308, "lr": 4.693487845462413e-06, "epoch": 0.791476407914764, "percentage": 19.78, "elapsed_time": "0:44:38", "remaining_time": "3:01:02"} +{"current_steps": 1320, "total_steps": 6572, "loss": 0.1276724100112915, "lr": 4.681362379895349e-06, "epoch": 0.8036529680365296, "percentage": 20.09, "elapsed_time": "0:45:14", "remaining_time": "3:00:00"} +{"current_steps": 1340, "total_steps": 6572, "loss": 0.1319241166114807, "lr": 4.6690180011913524e-06, "epoch": 0.8158295281582952, "percentage": 20.39, "elapsed_time": "0:45:50", "remaining_time": "2:58:59"} +{"current_steps": 1360, "total_steps": 6572, "loss": 0.1557891011238098, "lr": 4.6564559481847795e-06, "epoch": 0.8280060882800608, "percentage": 20.69, "elapsed_time": "0:46:27", "remaining_time": "2:58:01"} +{"current_steps": 1380, "total_steps": 6572, "loss": 0.11075855493545532, "lr": 4.643677481554947e-06, "epoch": 0.8401826484018264, "percentage": 21.0, "elapsed_time": "0:47:04", "remaining_time": "2:57:07"} +{"current_steps": 1400, "total_steps": 6572, "loss": 0.1580789566040039, "lr": 4.630683883699607e-06, "epoch": 0.852359208523592, "percentage": 21.3, "elapsed_time": "0:47:40", "remaining_time": "2:56:08"} +{"current_steps": 1420, "total_steps": 6572, "loss": 0.16006500720977784, "lr": 4.6174764586062556e-06, "epoch": 0.8645357686453576, "percentage": 21.61, "elapsed_time": "0:48:18", "remaining_time": "2:55:14"} +{"current_steps": 1440, "total_steps": 6572, "loss": 0.1462727189064026, "lr": 4.6040565317212685e-06, "epoch": 0.8767123287671232, "percentage": 21.91, "elapsed_time": "0:48:54", "remaining_time": "2:54:20"} +{"current_steps": 1460, "total_steps": 6572, "loss": 0.14725338220596312, "lr": 4.59042544981688e-06, "epoch": 0.8888888888888888, "percentage": 22.22, "elapsed_time": "0:49:30", "remaining_time": "2:53:22"} +{"current_steps": 1480, "total_steps": 6572, "loss": 0.1304166793823242, "lr": 4.5765845808560334e-06, "epoch": 0.9010654490106544, "percentage": 22.52, "elapsed_time": "0:50:07", "remaining_time": "2:52:27"} +{"current_steps": 1500, "total_steps": 6572, "loss": 0.1293134570121765, "lr": 4.562535313855094e-06, "epoch": 0.91324200913242, "percentage": 22.82, "elapsed_time": "0:50:45", "remaining_time": "2:51:38"} +{"current_steps": 1500, "total_steps": 6572, "eval_loss": 0.25538697838783264, "epoch": 0.91324200913242, "percentage": 22.82, "elapsed_time": "0:51:32", "remaining_time": "2:54:17"} +{"current_steps": 1520, "total_steps": 6572, "loss": 0.11359381675720215, "lr": 4.548279058744451e-06, "epoch": 0.9254185692541856, "percentage": 23.13, "elapsed_time": "0:53:39", "remaining_time": "2:58:20"} +{"current_steps": 1540, "total_steps": 6572, "loss": 0.15145074129104613, "lr": 4.533817246227024e-06, "epoch": 0.9375951293759512, "percentage": 23.43, "elapsed_time": "0:54:15", "remaining_time": "2:57:18"} +{"current_steps": 1560, "total_steps": 6572, "loss": 0.11953675746917725, "lr": 4.519151327634685e-06, "epoch": 0.9497716894977168, "percentage": 23.74, "elapsed_time": "0:54:52", "remaining_time": "2:56:18"} +{"current_steps": 1580, "total_steps": 6572, "loss": 0.13977375030517578, "lr": 4.504282774782605e-06, "epoch": 0.9619482496194824, "percentage": 24.04, "elapsed_time": "0:55:30", "remaining_time": "2:55:21"} +{"current_steps": 1600, "total_steps": 6572, "loss": 0.1338045358657837, "lr": 4.489213079821551e-06, "epoch": 0.974124809741248, "percentage": 24.35, "elapsed_time": "0:56:07", "remaining_time": "2:54:25"} +{"current_steps": 1620, "total_steps": 6572, "loss": 0.11776142120361328, "lr": 4.4739437550881355e-06, "epoch": 0.9863013698630136, "percentage": 24.65, "elapsed_time": "0:56:43", "remaining_time": "2:53:24"} +{"current_steps": 1640, "total_steps": 6572, "loss": 0.12504475116729735, "lr": 4.458476332953051e-06, "epoch": 0.9984779299847792, "percentage": 24.95, "elapsed_time": "0:57:19", "remaining_time": "2:52:23"} +{"current_steps": 1660, "total_steps": 6572, "loss": 0.08379222154617309, "lr": 4.442812365667281e-06, "epoch": 1.0103500761035007, "percentage": 25.26, "elapsed_time": "0:57:55", "remaining_time": "2:51:23"} +{"current_steps": 1680, "total_steps": 6572, "loss": 0.08407147526741028, "lr": 4.426953425206322e-06, "epoch": 1.0225266362252663, "percentage": 25.56, "elapsed_time": "0:58:32", "remaining_time": "2:50:27"} +{"current_steps": 1700, "total_steps": 6572, "loss": 0.08041079640388489, "lr": 4.410901103112434e-06, "epoch": 1.034703196347032, "percentage": 25.87, "elapsed_time": "0:59:08", "remaining_time": "2:49:28"} +{"current_steps": 1720, "total_steps": 6572, "loss": 0.07876392006874085, "lr": 4.394657010334908e-06, "epoch": 1.0468797564687975, "percentage": 26.17, "elapsed_time": "0:59:43", "remaining_time": "2:48:29"} +{"current_steps": 1740, "total_steps": 6572, "loss": 0.10302903652191162, "lr": 4.378222777068406e-06, "epoch": 1.059056316590563, "percentage": 26.48, "elapsed_time": "1:00:19", "remaining_time": "2:47:32"} +{"current_steps": 1760, "total_steps": 6572, "loss": 0.08733606934547425, "lr": 4.361600052589358e-06, "epoch": 1.0712328767123287, "percentage": 26.78, "elapsed_time": "1:00:55", "remaining_time": "2:46:34"} +{"current_steps": 1780, "total_steps": 6572, "loss": 0.08532609939575195, "lr": 4.344790505090447e-06, "epoch": 1.0834094368340943, "percentage": 27.08, "elapsed_time": "1:01:31", "remaining_time": "2:45:38"} +{"current_steps": 1800, "total_steps": 6572, "loss": 0.08734336495399475, "lr": 4.327795821513195e-06, "epoch": 1.09558599695586, "percentage": 27.39, "elapsed_time": "1:02:09", "remaining_time": "2:44:46"} +{"current_steps": 1820, "total_steps": 6572, "loss": 0.0913870632648468, "lr": 4.3106177073786684e-06, "epoch": 1.1077625570776255, "percentage": 27.69, "elapsed_time": "1:02:46", "remaining_time": "2:43:53"} +{"current_steps": 1840, "total_steps": 6572, "loss": 0.08115516304969787, "lr": 4.293257886616318e-06, "epoch": 1.119939117199391, "percentage": 28.0, "elapsed_time": "1:03:22", "remaining_time": "2:42:59"} +{"current_steps": 1860, "total_steps": 6572, "loss": 0.08891176581382751, "lr": 4.275718101390975e-06, "epoch": 1.1321156773211567, "percentage": 28.3, "elapsed_time": "1:03:59", "remaining_time": "2:42:07"} +{"current_steps": 1880, "total_steps": 6572, "loss": 0.07950961589813232, "lr": 4.25800011192801e-06, "epoch": 1.1442922374429223, "percentage": 28.61, "elapsed_time": "1:04:37", "remaining_time": "2:41:16"} +{"current_steps": 1900, "total_steps": 6572, "loss": 0.08310645222663879, "lr": 4.240105696336687e-06, "epoch": 1.156468797564688, "percentage": 28.91, "elapsed_time": "1:05:13", "remaining_time": "2:40:22"} +{"current_steps": 1920, "total_steps": 6572, "loss": 0.07682961225509644, "lr": 4.222036650431715e-06, "epoch": 1.1686453576864535, "percentage": 29.21, "elapsed_time": "1:05:49", "remaining_time": "2:39:29"} +{"current_steps": 1940, "total_steps": 6572, "loss": 0.07520227432250977, "lr": 4.203794787553032e-06, "epoch": 1.180821917808219, "percentage": 29.52, "elapsed_time": "1:06:26", "remaining_time": "2:38:37"} +{"current_steps": 1960, "total_steps": 6572, "loss": 0.0754019558429718, "lr": 4.185381938383821e-06, "epoch": 1.1929984779299847, "percentage": 29.82, "elapsed_time": "1:07:02", "remaining_time": "2:37:44"} +{"current_steps": 1980, "total_steps": 6572, "loss": 0.08085885643959045, "lr": 4.166799950766793e-06, "epoch": 1.2051750380517503, "percentage": 30.13, "elapsed_time": "1:07:38", "remaining_time": "2:36:51"} +{"current_steps": 2000, "total_steps": 6572, "loss": 0.0884653627872467, "lr": 4.14805068951874e-06, "epoch": 1.217351598173516, "percentage": 30.43, "elapsed_time": "1:08:14", "remaining_time": "2:36:00"} +{"current_steps": 2000, "total_steps": 6572, "eval_loss": 0.2661186456680298, "epoch": 1.217351598173516, "percentage": 30.43, "elapsed_time": "1:09:01", "remaining_time": "2:37:47"} +{"current_steps": 2020, "total_steps": 6572, "loss": 0.06684748530387878, "lr": 4.1291360362433965e-06, "epoch": 1.2295281582952815, "percentage": 30.74, "elapsed_time": "1:11:15", "remaining_time": "2:40:34"} +{"current_steps": 2040, "total_steps": 6572, "loss": 0.0720324158668518, "lr": 4.110057889142601e-06, "epoch": 1.241704718417047, "percentage": 31.04, "elapsed_time": "1:11:52", "remaining_time": "2:39:40"} +{"current_steps": 2060, "total_steps": 6572, "loss": 0.08799988031387329, "lr": 4.090818162825804e-06, "epoch": 1.2538812785388127, "percentage": 31.35, "elapsed_time": "1:12:29", "remaining_time": "2:38:47"} +{"current_steps": 2080, "total_steps": 6572, "loss": 0.09275985956192016, "lr": 4.071418788117926e-06, "epoch": 1.2660578386605783, "percentage": 31.65, "elapsed_time": "1:13:07", "remaining_time": "2:37:55"} +{"current_steps": 2100, "total_steps": 6572, "loss": 0.08431113958358764, "lr": 4.0518617118655845e-06, "epoch": 1.278234398782344, "percentage": 31.95, "elapsed_time": "1:13:43", "remaining_time": "2:37:00"} +{"current_steps": 2120, "total_steps": 6572, "loss": 0.09995608925819396, "lr": 4.032148896741717e-06, "epoch": 1.2904109589041095, "percentage": 32.26, "elapsed_time": "1:14:20", "remaining_time": "2:36:07"} +{"current_steps": 2140, "total_steps": 6572, "loss": 0.07387629747390748, "lr": 4.012282321048618e-06, "epoch": 1.302587519025875, "percentage": 32.56, "elapsed_time": "1:14:57", "remaining_time": "2:35:14"} +{"current_steps": 2160, "total_steps": 6572, "loss": 0.07667248249053955, "lr": 3.992263978519398e-06, "epoch": 1.3147640791476407, "percentage": 32.87, "elapsed_time": "1:15:34", "remaining_time": "2:34:22"} +{"current_steps": 2180, "total_steps": 6572, "loss": 0.09203824400901794, "lr": 3.972095878117904e-06, "epoch": 1.3269406392694063, "percentage": 33.17, "elapsed_time": "1:16:11", "remaining_time": "2:33:30"} +{"current_steps": 2200, "total_steps": 6572, "loss": 0.07835246920585633, "lr": 3.951780043837107e-06, "epoch": 1.339117199391172, "percentage": 33.48, "elapsed_time": "1:16:48", "remaining_time": "2:32:37"} +{"current_steps": 2220, "total_steps": 6572, "loss": 0.08577624559402466, "lr": 3.9313185144959835e-06, "epoch": 1.3512937595129375, "percentage": 33.78, "elapsed_time": "1:17:25", "remaining_time": "2:31:46"} +{"current_steps": 2240, "total_steps": 6572, "loss": 0.0789969801902771, "lr": 3.9107133435349025e-06, "epoch": 1.363470319634703, "percentage": 34.08, "elapsed_time": "1:18:01", "remaining_time": "2:30:53"} +{"current_steps": 2260, "total_steps": 6572, "loss": 0.07151145935058593, "lr": 3.889966598809557e-06, "epoch": 1.3756468797564687, "percentage": 34.39, "elapsed_time": "1:18:38", "remaining_time": "2:30:02"} +{"current_steps": 2280, "total_steps": 6572, "loss": 0.09204544425010681, "lr": 3.869080362383437e-06, "epoch": 1.3878234398782343, "percentage": 34.69, "elapsed_time": "1:19:14", "remaining_time": "2:29:10"} +{"current_steps": 2300, "total_steps": 6572, "loss": 0.11079612970352173, "lr": 3.848056730318881e-06, "epoch": 1.4, "percentage": 35.0, "elapsed_time": "1:19:54", "remaining_time": "2:28:25"} +{"current_steps": 2320, "total_steps": 6572, "loss": 0.06770140528678895, "lr": 3.826897812466728e-06, "epoch": 1.4121765601217655, "percentage": 35.3, "elapsed_time": "1:20:30", "remaining_time": "2:27:32"} +{"current_steps": 2340, "total_steps": 6572, "loss": 0.08210510611534119, "lr": 3.8056057322545763e-06, "epoch": 1.4243531202435311, "percentage": 35.61, "elapsed_time": "1:21:07", "remaining_time": "2:26:42"} +{"current_steps": 2360, "total_steps": 6572, "loss": 0.09583572745323181, "lr": 3.7841826264736888e-06, "epoch": 1.4365296803652967, "percentage": 35.91, "elapsed_time": "1:21:43", "remaining_time": "2:25:51"} +{"current_steps": 2380, "total_steps": 6572, "loss": 0.09235450625419617, "lr": 3.762630645064547e-06, "epoch": 1.4487062404870623, "percentage": 36.21, "elapsed_time": "1:22:20", "remaining_time": "2:25:01"} +{"current_steps": 2400, "total_steps": 6572, "loss": 0.08658097982406616, "lr": 3.7409519509010985e-06, "epoch": 1.460882800608828, "percentage": 36.52, "elapsed_time": "1:22:56", "remaining_time": "2:24:11"} +{"current_steps": 2420, "total_steps": 6572, "loss": 0.08892765045166015, "lr": 3.7191487195736915e-06, "epoch": 1.4730593607305935, "percentage": 36.82, "elapsed_time": "1:23:32", "remaining_time": "2:23:20"} +{"current_steps": 2440, "total_steps": 6572, "loss": 0.07849371433258057, "lr": 3.697223139170748e-06, "epoch": 1.4852359208523591, "percentage": 37.13, "elapsed_time": "1:24:09", "remaining_time": "2:22:30"} +{"current_steps": 2460, "total_steps": 6572, "loss": 0.07469035387039184, "lr": 3.6751774100591716e-06, "epoch": 1.4974124809741247, "percentage": 37.43, "elapsed_time": "1:24:45", "remaining_time": "2:21:40"} +{"current_steps": 2480, "total_steps": 6572, "loss": 0.0782626211643219, "lr": 3.6530137446635265e-06, "epoch": 1.5095890410958903, "percentage": 37.74, "elapsed_time": "1:25:20", "remaining_time": "2:20:49"} +{"current_steps": 2500, "total_steps": 6572, "loss": 0.08304058909416198, "lr": 3.630734367244012e-06, "epoch": 1.521765601217656, "percentage": 38.04, "elapsed_time": "1:25:57", "remaining_time": "2:20:00"} +{"current_steps": 2500, "total_steps": 6572, "eval_loss": 0.2771773338317871, "epoch": 1.521765601217656, "percentage": 38.04, "elapsed_time": "1:26:44", "remaining_time": "2:21:17"} +{"current_steps": 2520, "total_steps": 6572, "loss": 0.08037537932395936, "lr": 3.6083415136732374e-06, "epoch": 1.5339421613394215, "percentage": 38.34, "elapsed_time": "1:28:46", "remaining_time": "2:22:44"} +{"current_steps": 2540, "total_steps": 6572, "loss": 0.08990358114242554, "lr": 3.585837431211845e-06, "epoch": 1.5461187214611871, "percentage": 38.65, "elapsed_time": "1:29:23", "remaining_time": "2:21:53"} +{"current_steps": 2560, "total_steps": 6572, "loss": 0.0773526132106781, "lr": 3.563224378282978e-06, "epoch": 1.5582952815829527, "percentage": 38.95, "elapsed_time": "1:29:59", "remaining_time": "2:21:02"} +{"current_steps": 2580, "total_steps": 6572, "loss": 0.0777865469455719, "lr": 3.5405046242456396e-06, "epoch": 1.5704718417047183, "percentage": 39.26, "elapsed_time": "1:30:36", "remaining_time": "2:20:11"} +{"current_steps": 2600, "total_steps": 6572, "loss": 0.08037815093994141, "lr": 3.517680449166943e-06, "epoch": 1.582648401826484, "percentage": 39.56, "elapsed_time": "1:31:11", "remaining_time": "2:19:18"} +{"current_steps": 2620, "total_steps": 6572, "loss": 0.08837634325027466, "lr": 3.4947541435932976e-06, "epoch": 1.5948249619482495, "percentage": 39.87, "elapsed_time": "1:31:49", "remaining_time": "2:18:30"} +{"current_steps": 2640, "total_steps": 6572, "loss": 0.08201563358306885, "lr": 3.471728008320532e-06, "epoch": 1.6070015220700151, "percentage": 40.17, "elapsed_time": "1:32:26", "remaining_time": "2:17:41"} +{"current_steps": 2660, "total_steps": 6572, "loss": 0.07379403114318847, "lr": 3.4486043541630066e-06, "epoch": 1.6191780821917807, "percentage": 40.47, "elapsed_time": "1:33:03", "remaining_time": "2:16:51"} +{"current_steps": 2680, "total_steps": 6572, "loss": 0.08871785402297974, "lr": 3.425385501721696e-06, "epoch": 1.6313546423135463, "percentage": 40.78, "elapsed_time": "1:33:39", "remaining_time": "2:16:00"} +{"current_steps": 2700, "total_steps": 6572, "loss": 0.07757498621940613, "lr": 3.4020737811513107e-06, "epoch": 1.643531202435312, "percentage": 41.08, "elapsed_time": "1:34:16", "remaining_time": "2:15:11"} +{"current_steps": 2720, "total_steps": 6572, "loss": 0.08565697669982911, "lr": 3.3786715319264483e-06, "epoch": 1.6557077625570775, "percentage": 41.39, "elapsed_time": "1:34:52", "remaining_time": "2:14:21"} +{"current_steps": 2740, "total_steps": 6572, "loss": 0.08129348754882812, "lr": 3.355181102606816e-06, "epoch": 1.6678843226788431, "percentage": 41.69, "elapsed_time": "1:35:28", "remaining_time": "2:13:31"} +{"current_steps": 2760, "total_steps": 6572, "loss": 0.07875375747680664, "lr": 3.331604850601533e-06, "epoch": 1.6800608828006087, "percentage": 42.0, "elapsed_time": "1:36:06", "remaining_time": "2:12:43"} +{"current_steps": 2780, "total_steps": 6572, "loss": 0.08989614248275757, "lr": 3.307945141932556e-06, "epoch": 1.6922374429223743, "percentage": 42.3, "elapsed_time": "1:36:41", "remaining_time": "2:11:53"} +{"current_steps": 2800, "total_steps": 6572, "loss": 0.08251298069953919, "lr": 3.2842043509972294e-06, "epoch": 1.70441400304414, "percentage": 42.6, "elapsed_time": "1:37:18", "remaining_time": "2:11:05"} +{"current_steps": 2820, "total_steps": 6572, "loss": 0.07760271430015564, "lr": 3.2603848603300026e-06, "epoch": 1.7165905631659055, "percentage": 42.91, "elapsed_time": "1:37:56", "remaining_time": "2:10:18"} +{"current_steps": 2840, "total_steps": 6572, "loss": 0.07395396232604981, "lr": 3.236489060363329e-06, "epoch": 1.7287671232876711, "percentage": 43.21, "elapsed_time": "1:38:33", "remaining_time": "2:09:31"} +{"current_steps": 2860, "total_steps": 6572, "loss": 0.07028600573539734, "lr": 3.212519349187766e-06, "epoch": 1.7409436834094367, "percentage": 43.52, "elapsed_time": "1:39:10", "remaining_time": "2:08:42"} +{"current_steps": 2880, "total_steps": 6572, "loss": 0.08469281196594239, "lr": 3.188478132311319e-06, "epoch": 1.7531202435312023, "percentage": 43.82, "elapsed_time": "1:39:46", "remaining_time": "2:07:54"} +{"current_steps": 2900, "total_steps": 6572, "loss": 0.09567424058914184, "lr": 3.164367822418029e-06, "epoch": 1.765296803652968, "percentage": 44.13, "elapsed_time": "1:40:22", "remaining_time": "2:07:05"} +{"current_steps": 2920, "total_steps": 6572, "loss": 0.07239987254142762, "lr": 3.1401908391258474e-06, "epoch": 1.7774733637747335, "percentage": 44.43, "elapsed_time": "1:41:00", "remaining_time": "2:06:19"} +{"current_steps": 2940, "total_steps": 6572, "loss": 0.0891954243183136, "lr": 3.1159496087438098e-06, "epoch": 1.7896499238964991, "percentage": 44.74, "elapsed_time": "1:41:36", "remaining_time": "2:05:31"} +{"current_steps": 2960, "total_steps": 6572, "loss": 0.07796849608421326, "lr": 3.0916465640285426e-06, "epoch": 1.8018264840182647, "percentage": 45.04, "elapsed_time": "1:42:12", "remaining_time": "2:04:43"} +{"current_steps": 2980, "total_steps": 6572, "loss": 0.08645985722541809, "lr": 3.0672841439401223e-06, "epoch": 1.8140030441400303, "percentage": 45.34, "elapsed_time": "1:42:49", "remaining_time": "2:03:55"} +{"current_steps": 3000, "total_steps": 6572, "loss": 0.08427774310111999, "lr": 3.0428647933973103e-06, "epoch": 1.826179604261796, "percentage": 45.65, "elapsed_time": "1:43:26", "remaining_time": "2:03:10"} +{"current_steps": 3000, "total_steps": 6572, "eval_loss": 0.2774485647678375, "epoch": 1.826179604261796, "percentage": 45.65, "elapsed_time": "1:44:13", "remaining_time": "2:04:05"} +{"current_steps": 3020, "total_steps": 6572, "loss": 0.07381275296211243, "lr": 3.0183909630321865e-06, "epoch": 1.8383561643835615, "percentage": 45.95, "elapsed_time": "1:46:17", "remaining_time": "2:05:00"} +{"current_steps": 3040, "total_steps": 6572, "loss": 0.07289664745330811, "lr": 2.9938651089442184e-06, "epoch": 1.8505327245053271, "percentage": 46.26, "elapsed_time": "1:46:55", "remaining_time": "2:04:14"} +{"current_steps": 3060, "total_steps": 6572, "loss": 0.07124295830726624, "lr": 2.969289692453773e-06, "epoch": 1.8627092846270927, "percentage": 46.56, "elapsed_time": "1:47:32", "remaining_time": "2:03:25"} +{"current_steps": 3080, "total_steps": 6572, "loss": 0.08125877976417542, "lr": 2.944667179855109e-06, "epoch": 1.8748858447488583, "percentage": 46.87, "elapsed_time": "1:48:09", "remaining_time": "2:02:37"} +{"current_steps": 3100, "total_steps": 6572, "loss": 0.0724608838558197, "lr": 2.920000042168871e-06, "epoch": 1.887062404870624, "percentage": 47.17, "elapsed_time": "1:48:47", "remaining_time": "2:01:51"} +{"current_steps": 3120, "total_steps": 6572, "loss": 0.07104775309562683, "lr": 2.8952907548941057e-06, "epoch": 1.8992389649923895, "percentage": 47.47, "elapsed_time": "1:49:23", "remaining_time": "2:01:01"} +{"current_steps": 3140, "total_steps": 6572, "loss": 0.0677955150604248, "lr": 2.8705417977598277e-06, "epoch": 1.9114155251141551, "percentage": 47.78, "elapsed_time": "1:50:00", "remaining_time": "2:00:14"} +{"current_steps": 3160, "total_steps": 6572, "loss": 0.07164496779441834, "lr": 2.8457556544761687e-06, "epoch": 1.9235920852359207, "percentage": 48.08, "elapsed_time": "1:50:36", "remaining_time": "1:59:25"} +{"current_steps": 3180, "total_steps": 6572, "loss": 0.071807599067688, "lr": 2.8209348124851187e-06, "epoch": 1.9357686453576863, "percentage": 48.39, "elapsed_time": "1:51:13", "remaining_time": "1:58:38"} +{"current_steps": 3200, "total_steps": 6572, "loss": 0.095755535364151, "lr": 2.7960817627108965e-06, "epoch": 1.947945205479452, "percentage": 48.69, "elapsed_time": "1:51:50", "remaining_time": "1:57:50"} +{"current_steps": 3220, "total_steps": 6572, "loss": 0.07055851817131042, "lr": 2.77119899930997e-06, "epoch": 1.9601217656012175, "percentage": 49.0, "elapsed_time": "1:52:26", "remaining_time": "1:57:03"} +{"current_steps": 3240, "total_steps": 6572, "loss": 0.07278798818588257, "lr": 2.7462890194207513e-06, "epoch": 1.9722983257229831, "percentage": 49.3, "elapsed_time": "1:53:03", "remaining_time": "1:56:16"} +{"current_steps": 3260, "total_steps": 6572, "loss": 0.07153088450431824, "lr": 2.7213543229129956e-06, "epoch": 1.9844748858447487, "percentage": 49.6, "elapsed_time": "1:53:40", "remaining_time": "1:55:28"} +{"current_steps": 3280, "total_steps": 6572, "loss": 0.07440360188484192, "lr": 2.6963974121369242e-06, "epoch": 1.9966514459665143, "percentage": 49.91, "elapsed_time": "1:54:16", "remaining_time": "1:54:41"} +{"current_steps": 3300, "total_steps": 6572, "loss": 0.0517767608165741, "lr": 2.671420791672093e-06, "epoch": 2.008523592085236, "percentage": 50.21, "elapsed_time": "1:54:51", "remaining_time": "1:53:53"} +{"current_steps": 3320, "total_steps": 6572, "loss": 0.03812239170074463, "lr": 2.646426968076052e-06, "epoch": 2.0207001522070014, "percentage": 50.52, "elapsed_time": "1:55:27", "remaining_time": "1:53:06"} +{"current_steps": 3340, "total_steps": 6572, "loss": 0.04107579588890076, "lr": 2.6214184496327865e-06, "epoch": 2.032876712328767, "percentage": 50.82, "elapsed_time": "1:56:05", "remaining_time": "1:52:19"} +{"current_steps": 3360, "total_steps": 6572, "loss": 0.04673115909099579, "lr": 2.5963977461010022e-06, "epoch": 2.0450532724505326, "percentage": 51.13, "elapsed_time": "1:56:41", "remaining_time": "1:51:33"} +{"current_steps": 3380, "total_steps": 6572, "loss": 0.03674449622631073, "lr": 2.5713673684622524e-06, "epoch": 2.057229832572298, "percentage": 51.43, "elapsed_time": "1:57:17", "remaining_time": "1:50:46"} +{"current_steps": 3400, "total_steps": 6572, "loss": 0.03422380387783051, "lr": 2.546329828668949e-06, "epoch": 2.069406392694064, "percentage": 51.73, "elapsed_time": "1:57:53", "remaining_time": "1:49:59"} +{"current_steps": 3420, "total_steps": 6572, "loss": 0.035878732800483704, "lr": 2.5212876393922657e-06, "epoch": 2.0815829528158294, "percentage": 52.04, "elapsed_time": "1:58:29", "remaining_time": "1:49:12"} +{"current_steps": 3440, "total_steps": 6572, "loss": 0.03577531576156616, "lr": 2.496243313769986e-06, "epoch": 2.093759512937595, "percentage": 52.34, "elapsed_time": "1:59:06", "remaining_time": "1:48:27"} +{"current_steps": 3460, "total_steps": 6572, "loss": 0.04281675517559051, "lr": 2.471199365154283e-06, "epoch": 2.1059360730593606, "percentage": 52.65, "elapsed_time": "1:59:42", "remaining_time": "1:47:40"} +{"current_steps": 3480, "total_steps": 6572, "loss": 0.042955422401428224, "lr": 2.4461583068595014e-06, "epoch": 2.118112633181126, "percentage": 52.95, "elapsed_time": "2:00:20", "remaining_time": "1:46:55"} +{"current_steps": 3500, "total_steps": 6572, "loss": 0.04432957172393799, "lr": 2.421122651909918e-06, "epoch": 2.130289193302892, "percentage": 53.26, "elapsed_time": "2:00:56", "remaining_time": "1:46:09"} +{"current_steps": 3500, "total_steps": 6572, "eval_loss": 0.3123805522918701, "epoch": 2.130289193302892, "percentage": 53.26, "elapsed_time": "2:01:42", "remaining_time": "1:46:49"} +{"current_steps": 3520, "total_steps": 6572, "loss": 0.03356837034225464, "lr": 2.3960949127875556e-06, "epoch": 2.1424657534246574, "percentage": 53.56, "elapsed_time": "2:03:47", "remaining_time": "1:47:19"} +{"current_steps": 3540, "total_steps": 6572, "loss": 0.036935809254646304, "lr": 2.371077601180031e-06, "epoch": 2.154642313546423, "percentage": 53.86, "elapsed_time": "2:04:23", "remaining_time": "1:46:32"} +{"current_steps": 3560, "total_steps": 6572, "loss": 0.0395690768957138, "lr": 2.3460732277284994e-06, "epoch": 2.1668188736681886, "percentage": 54.17, "elapsed_time": "2:05:01", "remaining_time": "1:45:46"} +{"current_steps": 3580, "total_steps": 6572, "loss": 0.044693085551261905, "lr": 2.321084301775689e-06, "epoch": 2.178995433789954, "percentage": 54.47, "elapsed_time": "2:05:39", "remaining_time": "1:45:01"} +{"current_steps": 3600, "total_steps": 6572, "loss": 0.03243565857410431, "lr": 2.29611333111408e-06, "epoch": 2.19117199391172, "percentage": 54.78, "elapsed_time": "2:06:17", "remaining_time": "1:44:15"} +{"current_steps": 3620, "total_steps": 6572, "loss": 0.04325798749923706, "lr": 2.271162821734225e-06, "epoch": 2.2033485540334854, "percentage": 55.08, "elapsed_time": "2:06:56", "remaining_time": "1:43:30"} +{"current_steps": 3640, "total_steps": 6572, "loss": 0.03856868743896484, "lr": 2.2462352775732653e-06, "epoch": 2.215525114155251, "percentage": 55.39, "elapsed_time": "2:07:33", "remaining_time": "1:42:45"} +{"current_steps": 3660, "total_steps": 6572, "loss": 0.041602414846420285, "lr": 2.221333200263637e-06, "epoch": 2.2277016742770166, "percentage": 55.69, "elapsed_time": "2:08:11", "remaining_time": "1:41:59"} +{"current_steps": 3680, "total_steps": 6572, "loss": 0.04286134541034699, "lr": 2.1964590888820233e-06, "epoch": 2.239878234398782, "percentage": 56.0, "elapsed_time": "2:08:48", "remaining_time": "1:41:13"} +{"current_steps": 3700, "total_steps": 6572, "loss": 0.041756758093833925, "lr": 2.1716154396985526e-06, "epoch": 2.252054794520548, "percentage": 56.3, "elapsed_time": "2:09:25", "remaining_time": "1:40:28"} +{"current_steps": 3720, "total_steps": 6572, "loss": 0.0359495222568512, "lr": 2.1468047459262882e-06, "epoch": 2.2642313546423134, "percentage": 56.6, "elapsed_time": "2:10:03", "remaining_time": "1:39:42"} +{"current_steps": 3740, "total_steps": 6572, "loss": 0.04322676360607147, "lr": 2.12202949747101e-06, "epoch": 2.276407914764079, "percentage": 56.91, "elapsed_time": "2:10:41", "remaining_time": "1:38:57"} +{"current_steps": 3760, "total_steps": 6572, "loss": 0.04191597998142242, "lr": 2.0972921806813468e-06, "epoch": 2.2885844748858446, "percentage": 57.21, "elapsed_time": "2:11:19", "remaining_time": "1:38:13"} +{"current_steps": 3780, "total_steps": 6572, "loss": 0.041278204321861266, "lr": 2.072595278099247e-06, "epoch": 2.30076103500761, "percentage": 57.52, "elapsed_time": "2:11:58", "remaining_time": "1:37:28"} +{"current_steps": 3800, "total_steps": 6572, "loss": 0.04312986135482788, "lr": 2.047941268210849e-06, "epoch": 2.312937595129376, "percentage": 57.82, "elapsed_time": "2:12:35", "remaining_time": "1:36:43"} +{"current_steps": 3820, "total_steps": 6572, "loss": 0.04236046075820923, "lr": 2.0233326251977426e-06, "epoch": 2.3251141552511414, "percentage": 58.13, "elapsed_time": "2:13:14", "remaining_time": "1:35:59"} +{"current_steps": 3840, "total_steps": 6572, "loss": 0.04433901011943817, "lr": 1.9987718186886724e-06, "epoch": 2.337290715372907, "percentage": 58.43, "elapsed_time": "2:13:52", "remaining_time": "1:35:14"} +{"current_steps": 3860, "total_steps": 6572, "loss": 0.04127628207206726, "lr": 1.9742613135116986e-06, "epoch": 2.3494672754946726, "percentage": 58.73, "elapsed_time": "2:14:29", "remaining_time": "1:34:29"} +{"current_steps": 3880, "total_steps": 6572, "loss": 0.04586326479911804, "lr": 1.949803569446828e-06, "epoch": 2.361643835616438, "percentage": 59.04, "elapsed_time": "2:15:07", "remaining_time": "1:33:45"} +{"current_steps": 3900, "total_steps": 6572, "loss": 0.03624185025691986, "lr": 1.925401040979171e-06, "epoch": 2.373820395738204, "percentage": 59.34, "elapsed_time": "2:15:44", "remaining_time": "1:33:00"} +{"current_steps": 3920, "total_steps": 6572, "loss": 0.035064518451690674, "lr": 1.9010561770526076e-06, "epoch": 2.3859969558599694, "percentage": 59.65, "elapsed_time": "2:16:21", "remaining_time": "1:32:15"} +{"current_steps": 3940, "total_steps": 6572, "loss": 0.042050021886825564, "lr": 1.8767714208240312e-06, "epoch": 2.398173515981735, "percentage": 59.95, "elapsed_time": "2:16:59", "remaining_time": "1:31:30"} +{"current_steps": 3960, "total_steps": 6572, "loss": 0.038166466355323794, "lr": 1.852549209418154e-06, "epoch": 2.4103500761035006, "percentage": 60.26, "elapsed_time": "2:17:37", "remaining_time": "1:30:46"} +{"current_steps": 3980, "total_steps": 6572, "loss": 0.040885674953460696, "lr": 1.8283919736829332e-06, "epoch": 2.422526636225266, "percentage": 60.56, "elapsed_time": "2:18:15", "remaining_time": "1:30:02"} +{"current_steps": 4000, "total_steps": 6572, "loss": 0.040162667632102966, "lr": 1.804302137945614e-06, "epoch": 2.434703196347032, "percentage": 60.86, "elapsed_time": "2:18:51", "remaining_time": "1:29:17"} +{"current_steps": 4000, "total_steps": 6572, "eval_loss": 0.31583163142204285, "epoch": 2.434703196347032, "percentage": 60.86, "elapsed_time": "2:19:38", "remaining_time": "1:29:47"} +{"current_steps": 4020, "total_steps": 6572, "loss": 0.04170995056629181, "lr": 1.7802821197694426e-06, "epoch": 2.4468797564687974, "percentage": 61.17, "elapsed_time": "2:21:49", "remaining_time": "1:30:02"} +{"current_steps": 4040, "total_steps": 6572, "loss": 0.03834344446659088, "lr": 1.7563343297110375e-06, "epoch": 2.459056316590563, "percentage": 61.47, "elapsed_time": "2:22:25", "remaining_time": "1:29:15"} +{"current_steps": 4060, "total_steps": 6572, "loss": 0.03928310573101044, "lr": 1.732461171078486e-06, "epoch": 2.4712328767123286, "percentage": 61.78, "elapsed_time": "2:23:01", "remaining_time": "1:28:29"} +{"current_steps": 4080, "total_steps": 6572, "loss": 0.03358933925628662, "lr": 1.7086650396901489e-06, "epoch": 2.483409436834094, "percentage": 62.08, "elapsed_time": "2:23:37", "remaining_time": "1:27:43"} +{"current_steps": 4100, "total_steps": 6572, "loss": 0.03547535240650177, "lr": 1.6849483236342322e-06, "epoch": 2.49558599695586, "percentage": 62.39, "elapsed_time": "2:24:14", "remaining_time": "1:26:58"} +{"current_steps": 4120, "total_steps": 6572, "loss": 0.03600102663040161, "lr": 1.6613134030291217e-06, "epoch": 2.5077625570776254, "percentage": 62.69, "elapsed_time": "2:24:51", "remaining_time": "1:26:12"} +{"current_steps": 4140, "total_steps": 6572, "loss": 0.04347077012062073, "lr": 1.6377626497845278e-06, "epoch": 2.519939117199391, "percentage": 62.99, "elapsed_time": "2:25:27", "remaining_time": "1:25:27"} +{"current_steps": 4160, "total_steps": 6572, "loss": 0.02908192276954651, "lr": 1.6142984273634505e-06, "epoch": 2.5321156773211566, "percentage": 63.3, "elapsed_time": "2:26:04", "remaining_time": "1:24:41"} +{"current_steps": 4180, "total_steps": 6572, "loss": 0.03611198365688324, "lr": 1.5909230905449846e-06, "epoch": 2.544292237442922, "percentage": 63.6, "elapsed_time": "2:26:40", "remaining_time": "1:23:56"} +{"current_steps": 4200, "total_steps": 6572, "loss": 0.03758668601512909, "lr": 1.567638985188012e-06, "epoch": 2.556468797564688, "percentage": 63.91, "elapsed_time": "2:27:17", "remaining_time": "1:23:10"} +{"current_steps": 4220, "total_steps": 6572, "loss": 0.033633843064308167, "lr": 1.544448447995773e-06, "epoch": 2.5686453576864534, "percentage": 64.21, "elapsed_time": "2:27:53", "remaining_time": "1:22:25"} +{"current_steps": 4240, "total_steps": 6572, "loss": 0.036797890067100526, "lr": 1.52135380628137e-06, "epoch": 2.580821917808219, "percentage": 64.52, "elapsed_time": "2:28:29", "remaining_time": "1:21:40"} +{"current_steps": 4260, "total_steps": 6572, "loss": 0.039166563749313356, "lr": 1.498357377734201e-06, "epoch": 2.5929984779299846, "percentage": 64.82, "elapsed_time": "2:29:07", "remaining_time": "1:20:55"} +{"current_steps": 4280, "total_steps": 6572, "loss": 0.03717599511146545, "lr": 1.4754614701873703e-06, "epoch": 2.60517503805175, "percentage": 65.12, "elapsed_time": "2:29:44", "remaining_time": "1:20:11"} +{"current_steps": 4300, "total_steps": 6572, "loss": 0.03962793946266174, "lr": 1.4526683813860792e-06, "epoch": 2.6173515981735163, "percentage": 65.43, "elapsed_time": "2:30:20", "remaining_time": "1:19:26"} +{"current_steps": 4320, "total_steps": 6572, "loss": 0.035475924611091614, "lr": 1.4299803987570396e-06, "epoch": 2.6295281582952814, "percentage": 65.73, "elapsed_time": "2:30:57", "remaining_time": "1:18:41"} +{"current_steps": 4340, "total_steps": 6572, "loss": 0.03256964683532715, "lr": 1.4073997991789078e-06, "epoch": 2.6417047184170475, "percentage": 66.04, "elapsed_time": "2:31:35", "remaining_time": "1:17:57"} +{"current_steps": 4360, "total_steps": 6572, "loss": 0.03712306022644043, "lr": 1.384928848753792e-06, "epoch": 2.6538812785388126, "percentage": 66.34, "elapsed_time": "2:32:10", "remaining_time": "1:17:12"} +{"current_steps": 4380, "total_steps": 6572, "loss": 0.041410398483276364, "lr": 1.3625698025798322e-06, "epoch": 2.6660578386605787, "percentage": 66.65, "elapsed_time": "2:32:47", "remaining_time": "1:16:27"} +{"current_steps": 4400, "total_steps": 6572, "loss": 0.03158504366874695, "lr": 1.3403249045248907e-06, "epoch": 2.678234398782344, "percentage": 66.95, "elapsed_time": "2:33:23", "remaining_time": "1:15:43"} +{"current_steps": 4420, "total_steps": 6572, "loss": 0.03525224924087524, "lr": 1.3181963870013604e-06, "epoch": 2.69041095890411, "percentage": 67.26, "elapsed_time": "2:33:59", "remaining_time": "1:14:58"} +{"current_steps": 4440, "total_steps": 6572, "loss": 0.03239959478378296, "lr": 1.2961864707421345e-06, "epoch": 2.702587519025875, "percentage": 67.56, "elapsed_time": "2:34:36", "remaining_time": "1:14:14"} +{"current_steps": 4460, "total_steps": 6572, "loss": 0.031032082438468934, "lr": 1.2742973645777394e-06, "epoch": 2.714764079147641, "percentage": 67.86, "elapsed_time": "2:35:13", "remaining_time": "1:13:30"} +{"current_steps": 4480, "total_steps": 6572, "loss": 0.030566230416297913, "lr": 1.252531265214662e-06, "epoch": 2.726940639269406, "percentage": 68.17, "elapsed_time": "2:35:48", "remaining_time": "1:12:45"} +{"current_steps": 4500, "total_steps": 6572, "loss": 0.041362547874450685, "lr": 1.2308903570149048e-06, "epoch": 2.7391171993911723, "percentage": 68.47, "elapsed_time": "2:36:23", "remaining_time": "1:12:00"} +{"current_steps": 4500, "total_steps": 6572, "eval_loss": 0.3391737937927246, "epoch": 2.7391171993911723, "percentage": 68.47, "elapsed_time": "2:37:10", "remaining_time": "1:12:22"} +{"current_steps": 4520, "total_steps": 6572, "loss": 0.0388390064239502, "lr": 1.2093768117767613e-06, "epoch": 2.7512937595129374, "percentage": 68.78, "elapsed_time": "2:39:12", "remaining_time": "1:12:16"} +{"current_steps": 4540, "total_steps": 6572, "loss": 0.032555675506591795, "lr": 1.1879927885168733e-06, "epoch": 2.7634703196347035, "percentage": 69.08, "elapsed_time": "2:39:48", "remaining_time": "1:11:31"} +{"current_steps": 4560, "total_steps": 6572, "loss": 0.03606459796428681, "lr": 1.1667404332535504e-06, "epoch": 2.7756468797564686, "percentage": 69.39, "elapsed_time": "2:40:25", "remaining_time": "1:10:47"} +{"current_steps": 4580, "total_steps": 6572, "loss": 0.032086309790611264, "lr": 1.1456218787914128e-06, "epoch": 2.7878234398782347, "percentage": 69.69, "elapsed_time": "2:41:01", "remaining_time": "1:10:02"} +{"current_steps": 4600, "total_steps": 6572, "loss": 0.033362787961959836, "lr": 1.1246392445073438e-06, "epoch": 2.8, "percentage": 69.99, "elapsed_time": "2:41:36", "remaining_time": "1:09:16"} +{"current_steps": 4620, "total_steps": 6572, "loss": 0.03638745844364166, "lr": 1.1037946361378027e-06, "epoch": 2.812176560121766, "percentage": 70.3, "elapsed_time": "2:42:12", "remaining_time": "1:08:32"} +{"current_steps": 4640, "total_steps": 6572, "loss": 0.030933958292007447, "lr": 1.0830901455674977e-06, "epoch": 2.824353120243531, "percentage": 70.6, "elapsed_time": "2:42:49", "remaining_time": "1:07:47"} +{"current_steps": 4660, "total_steps": 6572, "loss": 0.02879139482975006, "lr": 1.0625278506194538e-06, "epoch": 2.836529680365297, "percentage": 70.91, "elapsed_time": "2:43:26", "remaining_time": "1:07:03"} +{"current_steps": 4680, "total_steps": 6572, "loss": 0.03345020413398743, "lr": 1.04210981484649e-06, "epoch": 2.8487062404870622, "percentage": 71.21, "elapsed_time": "2:44:01", "remaining_time": "1:06:18"} +{"current_steps": 4700, "total_steps": 6572, "loss": 0.02593054175376892, "lr": 1.0218380873241314e-06, "epoch": 2.8608828006088283, "percentage": 71.52, "elapsed_time": "2:44:38", "remaining_time": "1:05:34"} +{"current_steps": 4720, "total_steps": 6572, "loss": 0.03906567096710205, "lr": 1.0017147024449674e-06, "epoch": 2.8730593607305934, "percentage": 71.82, "elapsed_time": "2:45:15", "remaining_time": "1:04:50"} +{"current_steps": 4740, "total_steps": 6572, "loss": 0.03371626436710358, "lr": 9.81741679714493e-07, "epoch": 2.8852359208523595, "percentage": 72.12, "elapsed_time": "2:45:52", "remaining_time": "1:04:06"} +{"current_steps": 4760, "total_steps": 6572, "loss": 0.03090968132019043, "lr": 9.619210235484333e-07, "epoch": 2.8974124809741246, "percentage": 72.43, "elapsed_time": "2:46:29", "remaining_time": "1:03:22"} +{"current_steps": 4780, "total_steps": 6572, "loss": 0.0322105199098587, "lr": 9.422547230715931e-07, "epoch": 2.9095890410958907, "percentage": 72.73, "elapsed_time": "2:47:05", "remaining_time": "1:02:38"} +{"current_steps": 4800, "total_steps": 6572, "loss": 0.035210177302360535, "lr": 9.227447519182353e-07, "epoch": 2.921765601217656, "percentage": 73.04, "elapsed_time": "2:47:42", "remaining_time": "1:01:54"} +{"current_steps": 4820, "total_steps": 6572, "loss": 0.026842504739761353, "lr": 9.033930680340097e-07, "epoch": 2.933942161339422, "percentage": 73.34, "elapsed_time": "2:48:18", "remaining_time": "1:01:10"} +{"current_steps": 4840, "total_steps": 6572, "loss": 0.03439584076404571, "lr": 8.842016134794682e-07, "epoch": 2.946118721461187, "percentage": 73.65, "elapsed_time": "2:48:55", "remaining_time": "1:00:27"} +{"current_steps": 4860, "total_steps": 6572, "loss": 0.04011322855949402, "lr": 8.651723142351603e-07, "epoch": 2.958295281582953, "percentage": 73.95, "elapsed_time": "2:49:32", "remaining_time": "0:59:43"} +{"current_steps": 4880, "total_steps": 6572, "loss": 0.03800423145294189, "lr": 8.463070800083562e-07, "epoch": 2.9704718417047182, "percentage": 74.25, "elapsed_time": "2:50:10", "remaining_time": "0:59:00"} +{"current_steps": 4900, "total_steps": 6572, "loss": 0.03839131891727447, "lr": 8.276078040413879e-07, "epoch": 2.9826484018264843, "percentage": 74.56, "elapsed_time": "2:50:47", "remaining_time": "0:58:16"} +{"current_steps": 4920, "total_steps": 6572, "loss": 0.02721840739250183, "lr": 8.090763629216589e-07, "epoch": 2.9948249619482494, "percentage": 74.86, "elapsed_time": "2:51:25", "remaining_time": "0:57:33"} +{"current_steps": 4940, "total_steps": 6572, "loss": 0.023991990089416503, "lr": 7.907146163933102e-07, "epoch": 3.006697108066971, "percentage": 75.17, "elapsed_time": "2:52:02", "remaining_time": "0:56:50"} +{"current_steps": 4960, "total_steps": 6572, "loss": 0.01451514959335327, "lr": 7.725244071705871e-07, "epoch": 3.0188736681887365, "percentage": 75.47, "elapsed_time": "2:52:40", "remaining_time": "0:56:07"} +{"current_steps": 4980, "total_steps": 6572, "loss": 0.014327619969844819, "lr": 7.545075607529104e-07, "epoch": 3.031050228310502, "percentage": 75.78, "elapsed_time": "2:53:20", "remaining_time": "0:55:24"} +{"current_steps": 5000, "total_steps": 6572, "loss": 0.017832010984420776, "lr": 7.366658852416788e-07, "epoch": 3.0432267884322677, "percentage": 76.08, "elapsed_time": "2:53:56", "remaining_time": "0:54:41"} +{"current_steps": 5000, "total_steps": 6572, "eval_loss": 0.39115142822265625, "epoch": 3.0432267884322677, "percentage": 76.08, "elapsed_time": "2:54:44", "remaining_time": "0:54:56"} +{"current_steps": 5020, "total_steps": 6572, "loss": 0.011674411594867706, "lr": 7.190011711588101e-07, "epoch": 3.0554033485540333, "percentage": 76.38, "elapsed_time": "2:56:57", "remaining_time": "0:54:42"} +{"current_steps": 5040, "total_steps": 6572, "loss": 0.013690856099128724, "lr": 7.015151912670562e-07, "epoch": 3.067579908675799, "percentage": 76.69, "elapsed_time": "2:57:34", "remaining_time": "0:53:58"} +{"current_steps": 5060, "total_steps": 6572, "loss": 0.011978642642498016, "lr": 6.842097003920903e-07, "epoch": 3.0797564687975645, "percentage": 76.99, "elapsed_time": "2:58:10", "remaining_time": "0:53:14"} +{"current_steps": 5080, "total_steps": 6572, "loss": 0.013893941044807434, "lr": 6.67086435246406e-07, "epoch": 3.09193302891933, "percentage": 77.3, "elapsed_time": "2:58:46", "remaining_time": "0:52:30"} +{"current_steps": 5100, "total_steps": 6572, "loss": 0.009910025447607041, "lr": 6.501471142550194e-07, "epoch": 3.1041095890410957, "percentage": 77.6, "elapsed_time": "2:59:23", "remaining_time": "0:51:46"} +{"current_steps": 5120, "total_steps": 6572, "loss": 0.008863755315542222, "lr": 6.333934373830222e-07, "epoch": 3.1162861491628613, "percentage": 77.91, "elapsed_time": "2:59:59", "remaining_time": "0:51:02"} +{"current_steps": 5140, "total_steps": 6572, "loss": 0.010502541810274124, "lr": 6.168270859649761e-07, "epoch": 3.128462709284627, "percentage": 78.21, "elapsed_time": "3:00:37", "remaining_time": "0:50:19"} +{"current_steps": 5160, "total_steps": 6572, "loss": 0.012096930295228958, "lr": 6.004497225361786e-07, "epoch": 3.1406392694063925, "percentage": 78.51, "elapsed_time": "3:01:13", "remaining_time": "0:49:35"} +{"current_steps": 5180, "total_steps": 6572, "loss": 0.013278065621852875, "lr": 5.842629906658226e-07, "epoch": 3.1528158295281585, "percentage": 78.82, "elapsed_time": "3:01:52", "remaining_time": "0:48:52"} +{"current_steps": 5200, "total_steps": 6572, "loss": 0.013548998534679413, "lr": 5.682685147920481e-07, "epoch": 3.1649923896499237, "percentage": 79.12, "elapsed_time": "3:02:28", "remaining_time": "0:48:08"} +{"current_steps": 5220, "total_steps": 6572, "loss": 0.013736458122730255, "lr": 5.524679000589256e-07, "epoch": 3.1771689497716897, "percentage": 79.43, "elapsed_time": "3:03:05", "remaining_time": "0:47:25"} +{"current_steps": 5240, "total_steps": 6572, "loss": 0.013177134096622467, "lr": 5.36862732155366e-07, "epoch": 3.189345509893455, "percentage": 79.73, "elapsed_time": "3:03:42", "remaining_time": "0:46:41"} +{"current_steps": 5260, "total_steps": 6572, "loss": 0.011971819400787353, "lr": 5.214545771559879e-07, "epoch": 3.201522070015221, "percentage": 80.04, "elapsed_time": "3:04:17", "remaining_time": "0:45:58"} +{"current_steps": 5280, "total_steps": 6572, "loss": 0.014422819018363953, "lr": 5.062449813639528e-07, "epoch": 3.213698630136986, "percentage": 80.34, "elapsed_time": "3:04:53", "remaining_time": "0:45:14"} +{"current_steps": 5300, "total_steps": 6572, "loss": 0.010663678497076034, "lr": 4.912354711557856e-07, "epoch": 3.225875190258752, "percentage": 80.65, "elapsed_time": "3:05:29", "remaining_time": "0:44:31"} +{"current_steps": 5320, "total_steps": 6572, "loss": 0.011400717496871948, "lr": 4.764275528281892e-07, "epoch": 3.2380517503805173, "percentage": 80.95, "elapsed_time": "3:06:07", "remaining_time": "0:43:48"} +{"current_steps": 5340, "total_steps": 6572, "loss": 0.008456526696681977, "lr": 4.6182271244688355e-07, "epoch": 3.2502283105022833, "percentage": 81.25, "elapsed_time": "3:06:44", "remaining_time": "0:43:04"} +{"current_steps": 5360, "total_steps": 6572, "loss": 0.014539115130901337, "lr": 4.4742241569746407e-07, "epoch": 3.2624048706240485, "percentage": 81.56, "elapsed_time": "3:07:22", "remaining_time": "0:42:22"} +{"current_steps": 5380, "total_steps": 6572, "loss": 0.017625690996646882, "lr": 4.332281077383177e-07, "epoch": 3.2745814307458145, "percentage": 81.86, "elapsed_time": "3:07:58", "remaining_time": "0:41:38"} +{"current_steps": 5400, "total_steps": 6572, "loss": 0.007641100138425827, "lr": 4.1924121305558563e-07, "epoch": 3.2867579908675797, "percentage": 82.17, "elapsed_time": "3:08:37", "remaining_time": "0:40:56"} +{"current_steps": 5420, "total_steps": 6572, "loss": 0.011799700558185577, "lr": 4.054631353202121e-07, "epoch": 3.2989345509893457, "percentage": 82.47, "elapsed_time": "3:09:14", "remaining_time": "0:40:13"} +{"current_steps": 5440, "total_steps": 6572, "loss": 0.011455408483743667, "lr": 3.9189525724707634e-07, "epoch": 3.311111111111111, "percentage": 82.78, "elapsed_time": "3:09:51", "remaining_time": "0:39:30"} +{"current_steps": 5460, "total_steps": 6572, "loss": 0.012499115616083144, "lr": 3.785389404562259e-07, "epoch": 3.323287671232877, "percentage": 83.08, "elapsed_time": "3:10:28", "remaining_time": "0:38:47"} +{"current_steps": 5480, "total_steps": 6572, "loss": 0.01148865669965744, "lr": 3.653955253362351e-07, "epoch": 3.335464231354642, "percentage": 83.38, "elapsed_time": "3:11:03", "remaining_time": "0:38:04"} +{"current_steps": 5500, "total_steps": 6572, "loss": 0.012819178402423859, "lr": 3.5246633090968205e-07, "epoch": 3.347640791476408, "percentage": 83.69, "elapsed_time": "3:11:39", "remaining_time": "0:37:21"} +{"current_steps": 5500, "total_steps": 6572, "eval_loss": 0.43404534459114075, "epoch": 3.347640791476408, "percentage": 83.69, "elapsed_time": "3:12:26", "remaining_time": "0:37:30"} +{"current_steps": 5520, "total_steps": 6572, "loss": 0.013325585424900055, "lr": 3.397526547007832e-07, "epoch": 3.3598173515981733, "percentage": 83.99, "elapsed_time": "3:14:26", "remaining_time": "0:37:03"} +{"current_steps": 5540, "total_steps": 6572, "loss": 0.011712662875652313, "lr": 3.2725577260517396e-07, "epoch": 3.3719939117199393, "percentage": 84.3, "elapsed_time": "3:15:02", "remaining_time": "0:36:19"} +{"current_steps": 5560, "total_steps": 6572, "loss": 0.01580573171377182, "lr": 3.14976938761867e-07, "epoch": 3.3841704718417045, "percentage": 84.6, "elapsed_time": "3:15:38", "remaining_time": "0:35:36"} +{"current_steps": 5580, "total_steps": 6572, "loss": 0.012312603741884231, "lr": 3.029173854273909e-07, "epoch": 3.3963470319634705, "percentage": 84.91, "elapsed_time": "3:16:16", "remaining_time": "0:34:53"} +{"current_steps": 5600, "total_steps": 6572, "loss": 0.011797953397035599, "lr": 2.910783228521269e-07, "epoch": 3.4085235920852357, "percentage": 85.21, "elapsed_time": "3:16:54", "remaining_time": "0:34:10"} +{"current_steps": 5620, "total_steps": 6572, "loss": 0.012182456254959107, "lr": 2.794609391588504e-07, "epoch": 3.4207001522070017, "percentage": 85.51, "elapsed_time": "3:17:32", "remaining_time": "0:33:27"} +{"current_steps": 5640, "total_steps": 6572, "loss": 0.013599888980388641, "lr": 2.6806640022349897e-07, "epoch": 3.432876712328767, "percentage": 85.82, "elapsed_time": "3:18:08", "remaining_time": "0:32:44"} +{"current_steps": 5660, "total_steps": 6572, "loss": 0.009272868931293487, "lr": 2.5689584955816497e-07, "epoch": 3.445053272450533, "percentage": 86.12, "elapsed_time": "3:18:45", "remaining_time": "0:32:01"} +{"current_steps": 5680, "total_steps": 6572, "loss": 0.008165979385375976, "lr": 2.459504081963421e-07, "epoch": 3.457229832572298, "percentage": 86.43, "elapsed_time": "3:19:21", "remaining_time": "0:31:18"} +{"current_steps": 5700, "total_steps": 6572, "loss": 0.009182130545377731, "lr": 2.3523117458041865e-07, "epoch": 3.469406392694064, "percentage": 86.73, "elapsed_time": "3:19:56", "remaining_time": "0:30:35"} +{"current_steps": 5720, "total_steps": 6572, "loss": 0.0107998326420784, "lr": 2.2473922445144485e-07, "epoch": 3.4815829528158293, "percentage": 87.04, "elapsed_time": "3:20:34", "remaining_time": "0:29:52"} +{"current_steps": 5740, "total_steps": 6572, "loss": 0.014469687640666962, "lr": 2.144756107411733e-07, "epoch": 3.4937595129375953, "percentage": 87.34, "elapsed_time": "3:21:11", "remaining_time": "0:29:09"} +{"current_steps": 5760, "total_steps": 6572, "loss": 0.0121701680123806, "lr": 2.0444136346639333e-07, "epoch": 3.5059360730593605, "percentage": 87.64, "elapsed_time": "3:21:48", "remaining_time": "0:28:26"} +{"current_steps": 5780, "total_steps": 6572, "loss": 0.014668506383895875, "lr": 1.9463748962556096e-07, "epoch": 3.5181126331811265, "percentage": 87.95, "elapsed_time": "3:22:25", "remaining_time": "0:27:44"} +{"current_steps": 5800, "total_steps": 6572, "loss": 0.010488402843475342, "lr": 1.8506497309773885e-07, "epoch": 3.5302891933028917, "percentage": 88.25, "elapsed_time": "3:23:02", "remaining_time": "0:27:01"} +{"current_steps": 5820, "total_steps": 6572, "loss": 0.010667071491479874, "lr": 1.7572477454386257e-07, "epoch": 3.5424657534246577, "percentage": 88.56, "elapsed_time": "3:23:37", "remaining_time": "0:26:18"} +{"current_steps": 5840, "total_steps": 6572, "loss": 0.011079683899879456, "lr": 1.6661783131032726e-07, "epoch": 3.554642313546423, "percentage": 88.86, "elapsed_time": "3:24:14", "remaining_time": "0:25:35"} +{"current_steps": 5860, "total_steps": 6572, "loss": 0.009308797866106033, "lr": 1.5774505733492263e-07, "epoch": 3.566818873668189, "percentage": 89.17, "elapsed_time": "3:24:50", "remaining_time": "0:24:53"} +{"current_steps": 5880, "total_steps": 6572, "loss": 0.012319787591695785, "lr": 1.49107343055111e-07, "epoch": 3.578995433789954, "percentage": 89.47, "elapsed_time": "3:25:27", "remaining_time": "0:24:10"} +{"current_steps": 5900, "total_steps": 6572, "loss": 0.00843576118350029, "lr": 1.407055553186701e-07, "epoch": 3.59117199391172, "percentage": 89.77, "elapsed_time": "3:26:04", "remaining_time": "0:23:28"} +{"current_steps": 5920, "total_steps": 6572, "loss": 0.00938587412238121, "lr": 1.3254053729669564e-07, "epoch": 3.6033485540334853, "percentage": 90.08, "elapsed_time": "3:26:41", "remaining_time": "0:22:45"} +{"current_steps": 5940, "total_steps": 6572, "loss": 0.011934128403663636, "lr": 1.2461310839898656e-07, "epoch": 3.6155251141552514, "percentage": 90.38, "elapsed_time": "3:27:17", "remaining_time": "0:22:03"} +{"current_steps": 5960, "total_steps": 6572, "loss": 0.013170333206653595, "lr": 1.169240641918104e-07, "epoch": 3.6277016742770165, "percentage": 90.69, "elapsed_time": "3:27:54", "remaining_time": "0:21:20"} +{"current_steps": 5980, "total_steps": 6572, "loss": 0.014534834027290344, "lr": 1.0947417631806539e-07, "epoch": 3.6398782343987826, "percentage": 90.99, "elapsed_time": "3:28:31", "remaining_time": "0:20:38"} +{"current_steps": 6000, "total_steps": 6572, "loss": 0.011021688580513, "lr": 1.0226419241983865e-07, "epoch": 3.6520547945205477, "percentage": 91.3, "elapsed_time": "3:29:07", "remaining_time": "0:19:56"} +{"current_steps": 6000, "total_steps": 6572, "eval_loss": 0.44063475728034973, "epoch": 3.6520547945205477, "percentage": 91.3, "elapsed_time": "3:29:53", "remaining_time": "0:20:00"} +{"current_steps": 6020, "total_steps": 6572, "loss": 0.010764393210411071, "lr": 9.529483606337902e-08, "epoch": 3.6642313546423138, "percentage": 91.6, "elapsed_time": "3:31:58", "remaining_time": "0:19:26"} +{"current_steps": 6040, "total_steps": 6572, "loss": 0.012128306180238723, "lr": 8.856680666647882e-08, "epoch": 3.676407914764079, "percentage": 91.91, "elapsed_time": "3:32:33", "remaining_time": "0:18:43"} +{"current_steps": 6060, "total_steps": 6572, "loss": 0.011729901283979416, "lr": 8.208077942828713e-08, "epoch": 3.688584474885845, "percentage": 92.21, "elapsed_time": "3:33:10", "remaining_time": "0:18:00"} +{"current_steps": 6080, "total_steps": 6572, "loss": 0.009119105339050294, "lr": 7.58374052615457e-08, "epoch": 3.70076103500761, "percentage": 92.51, "elapsed_time": "3:33:46", "remaining_time": "0:17:17"} +{"current_steps": 6100, "total_steps": 6572, "loss": 0.017101363837718965, "lr": 6.983731072726818e-08, "epoch": 3.712937595129376, "percentage": 92.82, "elapsed_time": "3:34:23", "remaining_time": "0:16:35"} +{"current_steps": 6120, "total_steps": 6572, "loss": 0.012368235737085342, "lr": 6.408109797186118e-08, "epoch": 3.7251141552511413, "percentage": 93.12, "elapsed_time": "3:34:59", "remaining_time": "0:15:52"} +{"current_steps": 6140, "total_steps": 6572, "loss": 0.008782628178596496, "lr": 5.856934466669212e-08, "epoch": 3.7372907153729074, "percentage": 93.43, "elapsed_time": "3:35:36", "remaining_time": "0:15:10"} +{"current_steps": 6160, "total_steps": 6572, "loss": 0.008994438499212266, "lr": 5.3302603950119994e-08, "epoch": 3.7494672754946725, "percentage": 93.73, "elapsed_time": "3:36:13", "remaining_time": "0:14:27"} +{"current_steps": 6180, "total_steps": 6572, "loss": 0.011286454647779465, "lr": 4.8281404371981755e-08, "epoch": 3.7616438356164386, "percentage": 94.04, "elapsed_time": "3:36:50", "remaining_time": "0:13:45"} +{"current_steps": 6200, "total_steps": 6572, "loss": 0.011785905063152313, "lr": 4.350624984055196e-08, "epoch": 3.7738203957382037, "percentage": 94.34, "elapsed_time": "3:37:26", "remaining_time": "0:13:02"} +{"current_steps": 6220, "total_steps": 6572, "loss": 0.013624191284179688, "lr": 3.897761957196877e-08, "epoch": 3.7859969558599698, "percentage": 94.64, "elapsed_time": "3:38:01", "remaining_time": "0:12:20"} +{"current_steps": 6240, "total_steps": 6572, "loss": 0.011700452119112015, "lr": 3.469596804214548e-08, "epoch": 3.798173515981735, "percentage": 94.95, "elapsed_time": "3:38:37", "remaining_time": "0:11:37"} +{"current_steps": 6260, "total_steps": 6572, "loss": 0.011029987037181855, "lr": 3.06617249411581e-08, "epoch": 3.810350076103501, "percentage": 95.25, "elapsed_time": "3:39:12", "remaining_time": "0:10:55"} +{"current_steps": 6280, "total_steps": 6572, "loss": 0.010965974628925323, "lr": 2.687529513012488e-08, "epoch": 3.822526636225266, "percentage": 95.56, "elapsed_time": "3:39:49", "remaining_time": "0:10:13"} +{"current_steps": 6300, "total_steps": 6572, "loss": 0.012378603965044022, "lr": 2.3337058600575722e-08, "epoch": 3.834703196347032, "percentage": 95.86, "elapsed_time": "3:40:25", "remaining_time": "0:09:30"} +{"current_steps": 6320, "total_steps": 6572, "loss": 0.011792077124118805, "lr": 2.0047370436317437e-08, "epoch": 3.8468797564687973, "percentage": 96.17, "elapsed_time": "3:41:00", "remaining_time": "0:08:48"} +{"current_steps": 6340, "total_steps": 6572, "loss": 0.01145942509174347, "lr": 1.7006560777798608e-08, "epoch": 3.8590563165905634, "percentage": 96.47, "elapsed_time": "3:41:35", "remaining_time": "0:08:06"} +{"current_steps": 6360, "total_steps": 6572, "loss": 0.011088228970766067, "lr": 1.421493478897945e-08, "epoch": 3.8712328767123285, "percentage": 96.77, "elapsed_time": "3:42:11", "remaining_time": "0:07:24"} +{"current_steps": 6380, "total_steps": 6572, "loss": 0.010828402638435364, "lr": 1.1672772626704909e-08, "epoch": 3.8834094368340946, "percentage": 97.08, "elapsed_time": "3:42:47", "remaining_time": "0:06:42"} +{"current_steps": 6400, "total_steps": 6572, "loss": 0.01165580153465271, "lr": 9.38032941258965e-09, "epoch": 3.8955859969558597, "percentage": 97.38, "elapsed_time": "3:43:22", "remaining_time": "0:06:00"} +{"current_steps": 6420, "total_steps": 6572, "loss": 0.010783226788043975, "lr": 7.3378352074163215e-09, "epoch": 3.9077625570776258, "percentage": 97.69, "elapsed_time": "3:43:58", "remaining_time": "0:05:18"} +{"current_steps": 6440, "total_steps": 6572, "loss": 0.011295531690120698, "lr": 5.545494988045963e-09, "epoch": 3.919939117199391, "percentage": 97.99, "elapsed_time": "3:44:33", "remaining_time": "0:04:36"} +{"current_steps": 6460, "total_steps": 6572, "loss": 0.013613662123680115, "lr": 4.003488626848073e-09, "epoch": 3.932115677321157, "percentage": 98.3, "elapsed_time": "3:45:10", "remaining_time": "0:03:54"} +{"current_steps": 6480, "total_steps": 6572, "loss": 0.011696261167526246, "lr": 2.7119708736486615e-09, "epoch": 3.944292237442922, "percentage": 98.6, "elapsed_time": "3:45:47", "remaining_time": "0:03:12"} +{"current_steps": 6500, "total_steps": 6572, "loss": 0.010873865336179733, "lr": 1.6710713402015577e-09, "epoch": 3.956468797564688, "percentage": 98.9, "elapsed_time": "3:46:23", "remaining_time": "0:02:30"} +{"current_steps": 6500, "total_steps": 6572, "eval_loss": 0.441041499376297, "epoch": 3.956468797564688, "percentage": 98.9, "elapsed_time": "3:47:09", "remaining_time": "0:02:30"} +{"current_steps": 6520, "total_steps": 6572, "loss": 0.012961818277835846, "lr": 8.80894487179651e-10, "epoch": 3.9686453576864533, "percentage": 99.21, "elapsed_time": "3:49:12", "remaining_time": "0:01:49"} +{"current_steps": 6540, "total_steps": 6572, "loss": 0.01224210560321808, "lr": 3.4151961369188745e-10, "epoch": 3.9808219178082194, "percentage": 99.51, "elapsed_time": "3:49:48", "remaining_time": "0:01:07"} +{"current_steps": 6560, "total_steps": 6572, "loss": 0.010245455056428909, "lr": 5.300084932574612e-11, "epoch": 3.9929984779299845, "percentage": 99.82, "elapsed_time": "3:50:24", "remaining_time": "0:00:25"} +{"current_steps": 6572, "total_steps": 6572, "epoch": 4.0, "percentage": 100.0, "elapsed_time": "3:52:13", "remaining_time": "0:00:00"} diff --git a/trainer_state.json b/trainer_state.json new file mode 100644 index 0000000..d44e87b --- /dev/null +++ b/trainer_state.json @@ -0,0 +1,2443 @@ +{ + "best_global_step": 1000, + "best_metric": 0.24901165068149567, + "best_model_checkpoint": "/root/workspace/finetune/checkpoints/train_run_06_qwen3_4b_formal/checkpoint-1000", + "epoch": 4.0, + "eval_steps": 500, + "global_step": 6572, + "is_hyper_param_search": false, + "is_local_process_zero": true, + "is_world_process_zero": true, + "log_history": [ + { + "epoch": 0.0121765601217656, + "grad_norm": 2.8322572708129883, + "learning_rate": 3.166666666666667e-07, + "loss": 1.119423007965088, + "step": 20 + }, + { + "epoch": 0.0243531202435312, + "grad_norm": 2.00534987449646, + "learning_rate": 6.5e-07, + "loss": 0.8098940849304199, + "step": 40 + }, + { + "epoch": 0.0365296803652968, + "grad_norm": 1.4319002628326416, + "learning_rate": 9.833333333333334e-07, + "loss": 0.6310151576995849, + "step": 60 + }, + { + "epoch": 0.0487062404870624, + "grad_norm": 1.3265702724456787, + "learning_rate": 1.3166666666666666e-06, + "loss": 0.5526751518249512, + "step": 80 + }, + { + "epoch": 0.060882800608828, + "grad_norm": 0.8091968297958374, + "learning_rate": 1.6500000000000003e-06, + "loss": 0.4834024906158447, + "step": 100 + }, + { + "epoch": 0.0730593607305936, + "grad_norm": 0.8584772348403931, + "learning_rate": 1.9833333333333335e-06, + "loss": 0.44873433113098143, + "step": 120 + }, + { + "epoch": 0.0852359208523592, + "grad_norm": 0.9426456093788147, + "learning_rate": 2.316666666666667e-06, + "loss": 0.38286705017089845, + "step": 140 + }, + { + "epoch": 0.0974124809741248, + "grad_norm": 0.939085066318512, + "learning_rate": 2.6500000000000005e-06, + "loss": 0.38465352058410646, + "step": 160 + }, + { + "epoch": 0.1095890410958904, + "grad_norm": 0.8642272353172302, + "learning_rate": 2.9833333333333337e-06, + "loss": 0.3305965900421143, + "step": 180 + }, + { + "epoch": 0.121765601217656, + "grad_norm": 0.7353930473327637, + "learning_rate": 3.316666666666667e-06, + "loss": 0.2875316619873047, + "step": 200 + }, + { + "epoch": 0.1339421613394216, + "grad_norm": 0.7686158418655396, + "learning_rate": 3.65e-06, + "loss": 0.3140716075897217, + "step": 220 + }, + { + "epoch": 0.1461187214611872, + "grad_norm": 1.153424859046936, + "learning_rate": 3.983333333333334e-06, + "loss": 0.2985520839691162, + "step": 240 + }, + { + "epoch": 0.1582952815829528, + "grad_norm": 0.7382195591926575, + "learning_rate": 4.316666666666667e-06, + "loss": 0.2449601411819458, + "step": 260 + }, + { + "epoch": 0.1704718417047184, + "grad_norm": 0.7017386555671692, + "learning_rate": 4.65e-06, + "loss": 0.24566495418548584, + "step": 280 + }, + { + "epoch": 0.182648401826484, + "grad_norm": 1.0982152223587036, + "learning_rate": 4.983333333333334e-06, + "loss": 0.20863604545593262, + "step": 300 + }, + { + "epoch": 0.1948249619482496, + "grad_norm": 0.6582516431808472, + "learning_rate": 4.9998867856224845e-06, + "loss": 0.19693111181259154, + "step": 320 + }, + { + "epoch": 0.2070015220700152, + "grad_norm": 0.6201784610748291, + "learning_rate": 4.999523005839606e-06, + "loss": 0.21833207607269287, + "step": 340 + }, + { + "epoch": 0.2191780821917808, + "grad_norm": 0.7551426887512207, + "learning_rate": 4.998908383543311e-06, + "loss": 0.2382877826690674, + "step": 360 + }, + { + "epoch": 0.2313546423135464, + "grad_norm": 0.6009605526924133, + "learning_rate": 4.9980429804147276e-06, + "loss": 0.21118538379669188, + "step": 380 + }, + { + "epoch": 0.243531202435312, + "grad_norm": 0.7086856365203857, + "learning_rate": 4.996926883302385e-06, + "loss": 0.18367968797683715, + "step": 400 + }, + { + "epoch": 0.2557077625570776, + "grad_norm": 0.964198648929596, + "learning_rate": 4.995560204213496e-06, + "loss": 0.22483460903167723, + "step": 420 + }, + { + "epoch": 0.2678843226788432, + "grad_norm": 0.6596850156784058, + "learning_rate": 4.993943080302715e-06, + "loss": 0.22673561573028564, + "step": 440 + }, + { + "epoch": 0.2800608828006088, + "grad_norm": 0.5922938585281372, + "learning_rate": 4.992075673858379e-06, + "loss": 0.185296630859375, + "step": 460 + }, + { + "epoch": 0.2922374429223744, + "grad_norm": 0.6671269536018372, + "learning_rate": 4.989958172286214e-06, + "loss": 0.18437937498092652, + "step": 480 + }, + { + "epoch": 0.30441400304414, + "grad_norm": 0.7064187526702881, + "learning_rate": 4.987590788090533e-06, + "loss": 0.1850834846496582, + "step": 500 + }, + { + "epoch": 0.30441400304414, + "eval_loss": 0.2562323212623596, + "eval_runtime": 47.3626, + "eval_samples_per_second": 18.58, + "eval_steps_per_second": 18.58, + "step": 500 + }, + { + "epoch": 0.3165905631659056, + "grad_norm": 0.7698928117752075, + "learning_rate": 4.984973758852904e-06, + "loss": 0.16346561908721924, + "step": 520 + }, + { + "epoch": 0.3287671232876712, + "grad_norm": 0.4859939515590668, + "learning_rate": 4.982107347208317e-06, + "loss": 0.18838067054748536, + "step": 540 + }, + { + "epoch": 0.3409436834094368, + "grad_norm": 0.5321416854858398, + "learning_rate": 4.978991840818816e-06, + "loss": 0.177593994140625, + "step": 560 + }, + { + "epoch": 0.3531202435312024, + "grad_norm": 0.593889594078064, + "learning_rate": 4.975627552344638e-06, + "loss": 0.20775914192199707, + "step": 580 + }, + { + "epoch": 0.365296803652968, + "grad_norm": 0.7926055192947388, + "learning_rate": 4.97201481941283e-06, + "loss": 0.16498700380325318, + "step": 600 + }, + { + "epoch": 0.3774733637747336, + "grad_norm": 0.2909524440765381, + "learning_rate": 4.968154004583374e-06, + "loss": 0.17565951347351075, + "step": 620 + }, + { + "epoch": 0.3896499238964992, + "grad_norm": 0.4575704038143158, + "learning_rate": 4.964045495312794e-06, + "loss": 0.16204673051834106, + "step": 640 + }, + { + "epoch": 0.4018264840182648, + "grad_norm": 0.5366008281707764, + "learning_rate": 4.959689703915272e-06, + "loss": 0.17068564891815186, + "step": 660 + }, + { + "epoch": 0.4140030441400304, + "grad_norm": 0.5129569172859192, + "learning_rate": 4.95508706752128e-06, + "loss": 0.1589680790901184, + "step": 680 + }, + { + "epoch": 0.426179604261796, + "grad_norm": 0.4709528684616089, + "learning_rate": 4.9502380480337e-06, + "loss": 0.17568455934524535, + "step": 700 + }, + { + "epoch": 0.4383561643835616, + "grad_norm": 0.6092886328697205, + "learning_rate": 4.9451431320814715e-06, + "loss": 0.16204804182052612, + "step": 720 + }, + { + "epoch": 0.4505327245053272, + "grad_norm": 0.5957323908805847, + "learning_rate": 4.939802830970762e-06, + "loss": 0.16562143564224244, + "step": 740 + }, + { + "epoch": 0.4627092846270928, + "grad_norm": 0.4758240580558777, + "learning_rate": 4.934217680633646e-06, + "loss": 0.17697544097900392, + "step": 760 + }, + { + "epoch": 0.4748858447488584, + "grad_norm": 0.865627646446228, + "learning_rate": 4.928388241574327e-06, + "loss": 0.1649466037750244, + "step": 780 + }, + { + "epoch": 0.487062404870624, + "grad_norm": 0.466294527053833, + "learning_rate": 4.922315098812883e-06, + "loss": 0.1602837324142456, + "step": 800 + }, + { + "epoch": 0.4992389649923896, + "grad_norm": 0.6357060670852661, + "learning_rate": 4.9159988618265585e-06, + "loss": 0.142719042301178, + "step": 820 + }, + { + "epoch": 0.5114155251141552, + "grad_norm": 0.6055647730827332, + "learning_rate": 4.9094401644886e-06, + "loss": 0.14508233070373536, + "step": 840 + }, + { + "epoch": 0.5235920852359208, + "grad_norm": 0.45214834809303284, + "learning_rate": 4.902639665004641e-06, + "loss": 0.1821539044380188, + "step": 860 + }, + { + "epoch": 0.5357686453576864, + "grad_norm": 0.5735688805580139, + "learning_rate": 4.89559804584665e-06, + "loss": 0.16131887435913086, + "step": 880 + }, + { + "epoch": 0.547945205479452, + "grad_norm": 0.6279348731040955, + "learning_rate": 4.888316013684435e-06, + "loss": 0.17404688596725465, + "step": 900 + }, + { + "epoch": 0.5601217656012176, + "grad_norm": 0.6474089026451111, + "learning_rate": 4.880794299314732e-06, + "loss": 0.14134640693664552, + "step": 920 + }, + { + "epoch": 0.5722983257229832, + "grad_norm": 0.5808464884757996, + "learning_rate": 4.87303365758786e-06, + "loss": 0.14891813993453978, + "step": 940 + }, + { + "epoch": 0.5844748858447488, + "grad_norm": 0.5440990328788757, + "learning_rate": 4.865034867331967e-06, + "loss": 0.1696299910545349, + "step": 960 + }, + { + "epoch": 0.5966514459665144, + "grad_norm": 0.6859214901924133, + "learning_rate": 4.856798731274874e-06, + "loss": 0.14085158109664916, + "step": 980 + }, + { + "epoch": 0.60882800608828, + "grad_norm": 0.3178713619709015, + "learning_rate": 4.84832607596351e-06, + "loss": 0.133053982257843, + "step": 1000 + }, + { + "epoch": 0.60882800608828, + "eval_loss": 0.24901165068149567, + "eval_runtime": 46.6976, + "eval_samples_per_second": 18.845, + "eval_steps_per_second": 18.845, + "step": 1000 + }, + { + "epoch": 0.6210045662100456, + "grad_norm": 0.3812738358974457, + "learning_rate": 4.8396177516809695e-06, + "loss": 0.12680984735488893, + "step": 1020 + }, + { + "epoch": 0.6331811263318112, + "grad_norm": 0.5174199342727661, + "learning_rate": 4.830674632361178e-06, + "loss": 0.14880582094192504, + "step": 1040 + }, + { + "epoch": 0.6453576864535768, + "grad_norm": 0.4705193042755127, + "learning_rate": 4.821497615501186e-06, + "loss": 0.1447562575340271, + "step": 1060 + }, + { + "epoch": 0.6575342465753424, + "grad_norm": 0.42298850417137146, + "learning_rate": 4.812087622071104e-06, + "loss": 0.15530819892883302, + "step": 1080 + }, + { + "epoch": 0.669710806697108, + "grad_norm": 0.30658382177352905, + "learning_rate": 4.80244559642167e-06, + "loss": 0.14426586627960206, + "step": 1100 + }, + { + "epoch": 0.6818873668188736, + "grad_norm": 0.4838867783546448, + "learning_rate": 4.792572506189489e-06, + "loss": 0.15436025857925414, + "step": 1120 + }, + { + "epoch": 0.6940639269406392, + "grad_norm": 0.716833770275116, + "learning_rate": 4.782469342199915e-06, + "loss": 0.14860854148864747, + "step": 1140 + }, + { + "epoch": 0.7062404870624048, + "grad_norm": 0.36538004875183105, + "learning_rate": 4.7721371183676205e-06, + "loss": 0.1313084125518799, + "step": 1160 + }, + { + "epoch": 0.7184170471841704, + "grad_norm": 0.5409316420555115, + "learning_rate": 4.761576871594841e-06, + "loss": 0.150812029838562, + "step": 1180 + }, + { + "epoch": 0.730593607305936, + "grad_norm": 0.5275493264198303, + "learning_rate": 4.750789661667318e-06, + "loss": 0.13278884887695314, + "step": 1200 + }, + { + "epoch": 0.7427701674277016, + "grad_norm": 0.5485584735870361, + "learning_rate": 4.739776571147943e-06, + "loss": 0.1612934350967407, + "step": 1220 + }, + { + "epoch": 0.7549467275494672, + "grad_norm": 0.5949460864067078, + "learning_rate": 4.728538705268116e-06, + "loss": 0.16211290359497071, + "step": 1240 + }, + { + "epoch": 0.7671232876712328, + "grad_norm": 0.43323376774787903, + "learning_rate": 4.717077191816824e-06, + "loss": 0.14386119842529296, + "step": 1260 + }, + { + "epoch": 0.7792998477929984, + "grad_norm": 0.6409174799919128, + "learning_rate": 4.705393181027463e-06, + "loss": 0.12942540645599365, + "step": 1280 + }, + { + "epoch": 0.791476407914764, + "grad_norm": 0.4871342182159424, + "learning_rate": 4.693487845462413e-06, + "loss": 0.14771063327789308, + "step": 1300 + }, + { + "epoch": 0.8036529680365296, + "grad_norm": 0.5108008980751038, + "learning_rate": 4.681362379895349e-06, + "loss": 0.1276724100112915, + "step": 1320 + }, + { + "epoch": 0.8158295281582952, + "grad_norm": 0.915285587310791, + "learning_rate": 4.6690180011913524e-06, + "loss": 0.1319241166114807, + "step": 1340 + }, + { + "epoch": 0.8280060882800608, + "grad_norm": 0.5282526612281799, + "learning_rate": 4.6564559481847795e-06, + "loss": 0.1557891011238098, + "step": 1360 + }, + { + "epoch": 0.8401826484018264, + "grad_norm": 0.46745216846466064, + "learning_rate": 4.643677481554947e-06, + "loss": 0.11075855493545532, + "step": 1380 + }, + { + "epoch": 0.852359208523592, + "grad_norm": 0.40246087312698364, + "learning_rate": 4.630683883699607e-06, + "loss": 0.1580789566040039, + "step": 1400 + }, + { + "epoch": 0.8645357686453576, + "grad_norm": 0.3718211352825165, + "learning_rate": 4.6174764586062556e-06, + "loss": 0.16006500720977784, + "step": 1420 + }, + { + "epoch": 0.8767123287671232, + "grad_norm": 0.4359384775161743, + "learning_rate": 4.6040565317212685e-06, + "loss": 0.1462727189064026, + "step": 1440 + }, + { + "epoch": 0.8888888888888888, + "grad_norm": 0.3503302037715912, + "learning_rate": 4.59042544981688e-06, + "loss": 0.14725338220596312, + "step": 1460 + }, + { + "epoch": 0.9010654490106544, + "grad_norm": 0.3662220537662506, + "learning_rate": 4.5765845808560334e-06, + "loss": 0.1304166793823242, + "step": 1480 + }, + { + "epoch": 0.91324200913242, + "grad_norm": 0.45357516407966614, + "learning_rate": 4.562535313855094e-06, + "loss": 0.1293134570121765, + "step": 1500 + }, + { + "epoch": 0.91324200913242, + "eval_loss": 0.25538697838783264, + "eval_runtime": 47.0026, + "eval_samples_per_second": 18.722, + "eval_steps_per_second": 18.722, + "step": 1500 + }, + { + "epoch": 0.9254185692541856, + "grad_norm": 0.31984779238700867, + "learning_rate": 4.548279058744451e-06, + "loss": 0.11359381675720215, + "step": 1520 + }, + { + "epoch": 0.9375951293759512, + "grad_norm": 0.6947388052940369, + "learning_rate": 4.533817246227024e-06, + "loss": 0.15145074129104613, + "step": 1540 + }, + { + "epoch": 0.9497716894977168, + "grad_norm": 0.5685822367668152, + "learning_rate": 4.519151327634685e-06, + "loss": 0.11953675746917725, + "step": 1560 + }, + { + "epoch": 0.9619482496194824, + "grad_norm": 0.2805669903755188, + "learning_rate": 4.504282774782605e-06, + "loss": 0.13977375030517578, + "step": 1580 + }, + { + "epoch": 0.974124809741248, + "grad_norm": 0.333103746175766, + "learning_rate": 4.489213079821551e-06, + "loss": 0.1338045358657837, + "step": 1600 + }, + { + "epoch": 0.9863013698630136, + "grad_norm": 0.5493115782737732, + "learning_rate": 4.4739437550881355e-06, + "loss": 0.11776142120361328, + "step": 1620 + }, + { + "epoch": 0.9984779299847792, + "grad_norm": 0.4903205931186676, + "learning_rate": 4.458476332953051e-06, + "loss": 0.12504475116729735, + "step": 1640 + }, + { + "epoch": 1.0103500761035007, + "grad_norm": 0.2682284712791443, + "learning_rate": 4.442812365667281e-06, + "loss": 0.08379222154617309, + "step": 1660 + }, + { + "epoch": 1.0225266362252663, + "grad_norm": 0.34869125485420227, + "learning_rate": 4.426953425206322e-06, + "loss": 0.08407147526741028, + "step": 1680 + }, + { + "epoch": 1.034703196347032, + "grad_norm": 0.38522422313690186, + "learning_rate": 4.410901103112434e-06, + "loss": 0.08041079640388489, + "step": 1700 + }, + { + "epoch": 1.0468797564687975, + "grad_norm": 0.4700411856174469, + "learning_rate": 4.394657010334908e-06, + "loss": 0.07876392006874085, + "step": 1720 + }, + { + "epoch": 1.059056316590563, + "grad_norm": 0.3719494938850403, + "learning_rate": 4.378222777068406e-06, + "loss": 0.10302903652191162, + "step": 1740 + }, + { + "epoch": 1.0712328767123287, + "grad_norm": 0.406753808259964, + "learning_rate": 4.361600052589358e-06, + "loss": 0.08733606934547425, + "step": 1760 + }, + { + "epoch": 1.0834094368340943, + "grad_norm": 0.19822958111763, + "learning_rate": 4.344790505090447e-06, + "loss": 0.08532609939575195, + "step": 1780 + }, + { + "epoch": 1.09558599695586, + "grad_norm": 0.3569728434085846, + "learning_rate": 4.327795821513195e-06, + "loss": 0.08734336495399475, + "step": 1800 + }, + { + "epoch": 1.1077625570776255, + "grad_norm": 0.4549162685871124, + "learning_rate": 4.3106177073786684e-06, + "loss": 0.0913870632648468, + "step": 1820 + }, + { + "epoch": 1.119939117199391, + "grad_norm": 0.5802178978919983, + "learning_rate": 4.293257886616318e-06, + "loss": 0.08115516304969787, + "step": 1840 + }, + { + "epoch": 1.1321156773211567, + "grad_norm": 0.30524641275405884, + "learning_rate": 4.275718101390975e-06, + "loss": 0.08891176581382751, + "step": 1860 + }, + { + "epoch": 1.1442922374429223, + "grad_norm": 0.33804091811180115, + "learning_rate": 4.25800011192801e-06, + "loss": 0.07950961589813232, + "step": 1880 + }, + { + "epoch": 1.156468797564688, + "grad_norm": 0.33472684025764465, + "learning_rate": 4.240105696336687e-06, + "loss": 0.08310645222663879, + "step": 1900 + }, + { + "epoch": 1.1686453576864535, + "grad_norm": 0.3032575845718384, + "learning_rate": 4.222036650431715e-06, + "loss": 0.07682961225509644, + "step": 1920 + }, + { + "epoch": 1.180821917808219, + "grad_norm": 0.47542238235473633, + "learning_rate": 4.203794787553032e-06, + "loss": 0.07520227432250977, + "step": 1940 + }, + { + "epoch": 1.1929984779299847, + "grad_norm": 0.39914897084236145, + "learning_rate": 4.185381938383821e-06, + "loss": 0.0754019558429718, + "step": 1960 + }, + { + "epoch": 1.2051750380517503, + "grad_norm": 0.4697635769844055, + "learning_rate": 4.166799950766793e-06, + "loss": 0.08085885643959045, + "step": 1980 + }, + { + "epoch": 1.217351598173516, + "grad_norm": 0.28078529238700867, + "learning_rate": 4.14805068951874e-06, + "loss": 0.0884653627872467, + "step": 2000 + }, + { + "epoch": 1.217351598173516, + "eval_loss": 0.2661186456680298, + "eval_runtime": 46.548, + "eval_samples_per_second": 18.905, + "eval_steps_per_second": 18.905, + "step": 2000 + }, + { + "epoch": 1.2295281582952815, + "grad_norm": 0.2885560989379883, + "learning_rate": 4.1291360362433965e-06, + "loss": 0.06684748530387878, + "step": 2020 + }, + { + "epoch": 1.241704718417047, + "grad_norm": 0.24163688719272614, + "learning_rate": 4.110057889142601e-06, + "loss": 0.0720324158668518, + "step": 2040 + }, + { + "epoch": 1.2538812785388127, + "grad_norm": 0.52589350938797, + "learning_rate": 4.090818162825804e-06, + "loss": 0.08799988031387329, + "step": 2060 + }, + { + "epoch": 1.2660578386605783, + "grad_norm": 0.35164448618888855, + "learning_rate": 4.071418788117926e-06, + "loss": 0.09275985956192016, + "step": 2080 + }, + { + "epoch": 1.278234398782344, + "grad_norm": 0.4981421232223511, + "learning_rate": 4.0518617118655845e-06, + "loss": 0.08431113958358764, + "step": 2100 + }, + { + "epoch": 1.2904109589041095, + "grad_norm": 0.2867731750011444, + "learning_rate": 4.032148896741717e-06, + "loss": 0.09995608925819396, + "step": 2120 + }, + { + "epoch": 1.302587519025875, + "grad_norm": 0.5612165331840515, + "learning_rate": 4.012282321048618e-06, + "loss": 0.07387629747390748, + "step": 2140 + }, + { + "epoch": 1.3147640791476407, + "grad_norm": 0.4880141317844391, + "learning_rate": 3.992263978519398e-06, + "loss": 0.07667248249053955, + "step": 2160 + }, + { + "epoch": 1.3269406392694063, + "grad_norm": 0.3143049478530884, + "learning_rate": 3.972095878117904e-06, + "loss": 0.09203824400901794, + "step": 2180 + }, + { + "epoch": 1.339117199391172, + "grad_norm": 0.47859013080596924, + "learning_rate": 3.951780043837107e-06, + "loss": 0.07835246920585633, + "step": 2200 + }, + { + "epoch": 1.3512937595129375, + "grad_norm": 0.28448912501335144, + "learning_rate": 3.9313185144959835e-06, + "loss": 0.08577624559402466, + "step": 2220 + }, + { + "epoch": 1.363470319634703, + "grad_norm": 0.32139304280281067, + "learning_rate": 3.9107133435349025e-06, + "loss": 0.0789969801902771, + "step": 2240 + }, + { + "epoch": 1.3756468797564687, + "grad_norm": 0.4797567129135132, + "learning_rate": 3.889966598809557e-06, + "loss": 0.07151145935058593, + "step": 2260 + }, + { + "epoch": 1.3878234398782343, + "grad_norm": 0.2404891699552536, + "learning_rate": 3.869080362383437e-06, + "loss": 0.09204544425010681, + "step": 2280 + }, + { + "epoch": 1.4, + "grad_norm": 0.328392893075943, + "learning_rate": 3.848056730318881e-06, + "loss": 0.11079612970352173, + "step": 2300 + }, + { + "epoch": 1.4121765601217655, + "grad_norm": 0.2993980646133423, + "learning_rate": 3.826897812466728e-06, + "loss": 0.06770140528678895, + "step": 2320 + }, + { + "epoch": 1.4243531202435311, + "grad_norm": 0.47816380858421326, + "learning_rate": 3.8056057322545763e-06, + "loss": 0.08210510611534119, + "step": 2340 + }, + { + "epoch": 1.4365296803652967, + "grad_norm": 0.38082119822502136, + "learning_rate": 3.7841826264736888e-06, + "loss": 0.09583572745323181, + "step": 2360 + }, + { + "epoch": 1.4487062404870623, + "grad_norm": 0.3811774253845215, + "learning_rate": 3.762630645064547e-06, + "loss": 0.09235450625419617, + "step": 2380 + }, + { + "epoch": 1.460882800608828, + "grad_norm": 0.3869916498661041, + "learning_rate": 3.7409519509010985e-06, + "loss": 0.08658097982406616, + "step": 2400 + }, + { + "epoch": 1.4730593607305935, + "grad_norm": 0.5042543411254883, + "learning_rate": 3.7191487195736915e-06, + "loss": 0.08892765045166015, + "step": 2420 + }, + { + "epoch": 1.4852359208523591, + "grad_norm": 0.4064173996448517, + "learning_rate": 3.697223139170748e-06, + "loss": 0.07849371433258057, + "step": 2440 + }, + { + "epoch": 1.4974124809741247, + "grad_norm": 0.3958194851875305, + "learning_rate": 3.6751774100591716e-06, + "loss": 0.07469035387039184, + "step": 2460 + }, + { + "epoch": 1.5095890410958903, + "grad_norm": 0.39424121379852295, + "learning_rate": 3.6530137446635265e-06, + "loss": 0.0782626211643219, + "step": 2480 + }, + { + "epoch": 1.521765601217656, + "grad_norm": 0.46179333329200745, + "learning_rate": 3.630734367244012e-06, + "loss": 0.08304058909416198, + "step": 2500 + }, + { + "epoch": 1.521765601217656, + "eval_loss": 0.2771773338317871, + "eval_runtime": 46.9989, + "eval_samples_per_second": 18.724, + "eval_steps_per_second": 18.724, + "step": 2500 + }, + { + "epoch": 1.5339421613394215, + "grad_norm": 0.3447970151901245, + "learning_rate": 3.6083415136732374e-06, + "loss": 0.08037537932395936, + "step": 2520 + }, + { + "epoch": 1.5461187214611871, + "grad_norm": 0.544224202632904, + "learning_rate": 3.585837431211845e-06, + "loss": 0.08990358114242554, + "step": 2540 + }, + { + "epoch": 1.5582952815829527, + "grad_norm": 0.2808915376663208, + "learning_rate": 3.563224378282978e-06, + "loss": 0.0773526132106781, + "step": 2560 + }, + { + "epoch": 1.5704718417047183, + "grad_norm": 0.27800413966178894, + "learning_rate": 3.5405046242456396e-06, + "loss": 0.0777865469455719, + "step": 2580 + }, + { + "epoch": 1.582648401826484, + "grad_norm": 0.4247933328151703, + "learning_rate": 3.517680449166943e-06, + "loss": 0.08037815093994141, + "step": 2600 + }, + { + "epoch": 1.5948249619482495, + "grad_norm": 0.748912513256073, + "learning_rate": 3.4947541435932976e-06, + "loss": 0.08837634325027466, + "step": 2620 + }, + { + "epoch": 1.6070015220700151, + "grad_norm": 0.29709526896476746, + "learning_rate": 3.471728008320532e-06, + "loss": 0.08201563358306885, + "step": 2640 + }, + { + "epoch": 1.6191780821917807, + "grad_norm": 0.3287765681743622, + "learning_rate": 3.4486043541630066e-06, + "loss": 0.07379403114318847, + "step": 2660 + }, + { + "epoch": 1.6313546423135463, + "grad_norm": 0.4224306643009186, + "learning_rate": 3.425385501721696e-06, + "loss": 0.08871785402297974, + "step": 2680 + }, + { + "epoch": 1.643531202435312, + "grad_norm": 0.5213542580604553, + "learning_rate": 3.4020737811513107e-06, + "loss": 0.07757498621940613, + "step": 2700 + }, + { + "epoch": 1.6557077625570775, + "grad_norm": 0.21830426156520844, + "learning_rate": 3.3786715319264483e-06, + "loss": 0.08565697669982911, + "step": 2720 + }, + { + "epoch": 1.6678843226788431, + "grad_norm": 0.30557703971862793, + "learning_rate": 3.355181102606816e-06, + "loss": 0.08129348754882812, + "step": 2740 + }, + { + "epoch": 1.6800608828006087, + "grad_norm": 0.453703910112381, + "learning_rate": 3.331604850601533e-06, + "loss": 0.07875375747680664, + "step": 2760 + }, + { + "epoch": 1.6922374429223743, + "grad_norm": 0.33889397978782654, + "learning_rate": 3.307945141932556e-06, + "loss": 0.08989614248275757, + "step": 2780 + }, + { + "epoch": 1.70441400304414, + "grad_norm": 0.21036894619464874, + "learning_rate": 3.2842043509972294e-06, + "loss": 0.08251298069953919, + "step": 2800 + }, + { + "epoch": 1.7165905631659055, + "grad_norm": 0.2764931321144104, + "learning_rate": 3.2603848603300026e-06, + "loss": 0.07760271430015564, + "step": 2820 + }, + { + "epoch": 1.7287671232876711, + "grad_norm": 0.3566988706588745, + "learning_rate": 3.236489060363329e-06, + "loss": 0.07395396232604981, + "step": 2840 + }, + { + "epoch": 1.7409436834094367, + "grad_norm": 0.3616081476211548, + "learning_rate": 3.212519349187766e-06, + "loss": 0.07028600573539734, + "step": 2860 + }, + { + "epoch": 1.7531202435312023, + "grad_norm": 0.41925984621047974, + "learning_rate": 3.188478132311319e-06, + "loss": 0.08469281196594239, + "step": 2880 + }, + { + "epoch": 1.765296803652968, + "grad_norm": 0.4866960346698761, + "learning_rate": 3.164367822418029e-06, + "loss": 0.09567424058914184, + "step": 2900 + }, + { + "epoch": 1.7774733637747335, + "grad_norm": 0.2977049648761749, + "learning_rate": 3.1401908391258474e-06, + "loss": 0.07239987254142762, + "step": 2920 + }, + { + "epoch": 1.7896499238964991, + "grad_norm": 0.3723915219306946, + "learning_rate": 3.1159496087438098e-06, + "loss": 0.0891954243183136, + "step": 2940 + }, + { + "epoch": 1.8018264840182647, + "grad_norm": 0.22685140371322632, + "learning_rate": 3.0916465640285426e-06, + "loss": 0.07796849608421326, + "step": 2960 + }, + { + "epoch": 1.8140030441400303, + "grad_norm": 0.23534800112247467, + "learning_rate": 3.0672841439401223e-06, + "loss": 0.08645985722541809, + "step": 2980 + }, + { + "epoch": 1.826179604261796, + "grad_norm": 0.40188542008399963, + "learning_rate": 3.0428647933973103e-06, + "loss": 0.08427774310111999, + "step": 3000 + }, + { + "epoch": 1.826179604261796, + "eval_loss": 0.2774485647678375, + "eval_runtime": 46.6449, + "eval_samples_per_second": 18.866, + "eval_steps_per_second": 18.866, + "step": 3000 + }, + { + "epoch": 1.8383561643835615, + "grad_norm": 0.35732510685920715, + "learning_rate": 3.0183909630321865e-06, + "loss": 0.07381275296211243, + "step": 3020 + }, + { + "epoch": 1.8505327245053271, + "grad_norm": 0.3167949914932251, + "learning_rate": 2.9938651089442184e-06, + "loss": 0.07289664745330811, + "step": 3040 + }, + { + "epoch": 1.8627092846270927, + "grad_norm": 0.4893674850463867, + "learning_rate": 2.969289692453773e-06, + "loss": 0.07124295830726624, + "step": 3060 + }, + { + "epoch": 1.8748858447488583, + "grad_norm": 0.3017306327819824, + "learning_rate": 2.944667179855109e-06, + "loss": 0.08125877976417542, + "step": 3080 + }, + { + "epoch": 1.887062404870624, + "grad_norm": 0.3442900776863098, + "learning_rate": 2.920000042168871e-06, + "loss": 0.0724608838558197, + "step": 3100 + }, + { + "epoch": 1.8992389649923895, + "grad_norm": 0.27901750802993774, + "learning_rate": 2.8952907548941057e-06, + "loss": 0.07104775309562683, + "step": 3120 + }, + { + "epoch": 1.9114155251141551, + "grad_norm": 0.35838621854782104, + "learning_rate": 2.8705417977598277e-06, + "loss": 0.0677955150604248, + "step": 3140 + }, + { + "epoch": 1.9235920852359207, + "grad_norm": 0.3615752160549164, + "learning_rate": 2.8457556544761687e-06, + "loss": 0.07164496779441834, + "step": 3160 + }, + { + "epoch": 1.9357686453576863, + "grad_norm": 0.5117827653884888, + "learning_rate": 2.8209348124851187e-06, + "loss": 0.071807599067688, + "step": 3180 + }, + { + "epoch": 1.947945205479452, + "grad_norm": 0.36082401871681213, + "learning_rate": 2.7960817627108965e-06, + "loss": 0.095755535364151, + "step": 3200 + }, + { + "epoch": 1.9601217656012175, + "grad_norm": 0.27905145287513733, + "learning_rate": 2.77119899930997e-06, + "loss": 0.07055851817131042, + "step": 3220 + }, + { + "epoch": 1.9722983257229831, + "grad_norm": 0.5642575621604919, + "learning_rate": 2.7462890194207513e-06, + "loss": 0.07278798818588257, + "step": 3240 + }, + { + "epoch": 1.9844748858447487, + "grad_norm": 0.2286670207977295, + "learning_rate": 2.7213543229129956e-06, + "loss": 0.07153088450431824, + "step": 3260 + }, + { + "epoch": 1.9966514459665143, + "grad_norm": 0.3228706121444702, + "learning_rate": 2.6963974121369242e-06, + "loss": 0.07440360188484192, + "step": 3280 + }, + { + "epoch": 2.008523592085236, + "grad_norm": 0.21613839268684387, + "learning_rate": 2.671420791672093e-06, + "loss": 0.0517767608165741, + "step": 3300 + }, + { + "epoch": 2.0207001522070014, + "grad_norm": 0.19271881878376007, + "learning_rate": 2.646426968076052e-06, + "loss": 0.03812239170074463, + "step": 3320 + }, + { + "epoch": 2.032876712328767, + "grad_norm": 0.12863056361675262, + "learning_rate": 2.6214184496327865e-06, + "loss": 0.04107579588890076, + "step": 3340 + }, + { + "epoch": 2.0450532724505326, + "grad_norm": 0.26679477095603943, + "learning_rate": 2.5963977461010022e-06, + "loss": 0.04673115909099579, + "step": 3360 + }, + { + "epoch": 2.057229832572298, + "grad_norm": 0.2743483781814575, + "learning_rate": 2.5713673684622524e-06, + "loss": 0.03674449622631073, + "step": 3380 + }, + { + "epoch": 2.069406392694064, + "grad_norm": 0.16999909281730652, + "learning_rate": 2.546329828668949e-06, + "loss": 0.03422380387783051, + "step": 3400 + }, + { + "epoch": 2.0815829528158294, + "grad_norm": 0.2931291460990906, + "learning_rate": 2.5212876393922657e-06, + "loss": 0.035878732800483704, + "step": 3420 + }, + { + "epoch": 2.093759512937595, + "grad_norm": 0.34781521558761597, + "learning_rate": 2.496243313769986e-06, + "loss": 0.03577531576156616, + "step": 3440 + }, + { + "epoch": 2.1059360730593606, + "grad_norm": 0.472351998090744, + "learning_rate": 2.471199365154283e-06, + "loss": 0.04281675517559051, + "step": 3460 + }, + { + "epoch": 2.118112633181126, + "grad_norm": 0.38320988416671753, + "learning_rate": 2.4461583068595014e-06, + "loss": 0.042955422401428224, + "step": 3480 + }, + { + "epoch": 2.130289193302892, + "grad_norm": 0.12197626382112503, + "learning_rate": 2.421122651909918e-06, + "loss": 0.04432957172393799, + "step": 3500 + }, + { + "epoch": 2.130289193302892, + "eval_loss": 0.3123805522918701, + "eval_runtime": 46.5258, + "eval_samples_per_second": 18.914, + "eval_steps_per_second": 18.914, + "step": 3500 + }, + { + "epoch": 2.1424657534246574, + "grad_norm": 0.21291697025299072, + "learning_rate": 2.3960949127875556e-06, + "loss": 0.03356837034225464, + "step": 3520 + }, + { + "epoch": 2.154642313546423, + "grad_norm": 0.44130077958106995, + "learning_rate": 2.371077601180031e-06, + "loss": 0.036935809254646304, + "step": 3540 + }, + { + "epoch": 2.1668188736681886, + "grad_norm": 0.49069875478744507, + "learning_rate": 2.3460732277284994e-06, + "loss": 0.0395690768957138, + "step": 3560 + }, + { + "epoch": 2.178995433789954, + "grad_norm": 0.2824937701225281, + "learning_rate": 2.321084301775689e-06, + "loss": 0.044693085551261905, + "step": 3580 + }, + { + "epoch": 2.19117199391172, + "grad_norm": 0.35114097595214844, + "learning_rate": 2.29611333111408e-06, + "loss": 0.03243565857410431, + "step": 3600 + }, + { + "epoch": 2.2033485540334854, + "grad_norm": 0.47931790351867676, + "learning_rate": 2.271162821734225e-06, + "loss": 0.04325798749923706, + "step": 3620 + }, + { + "epoch": 2.215525114155251, + "grad_norm": 0.12716218829154968, + "learning_rate": 2.2462352775732653e-06, + "loss": 0.03856868743896484, + "step": 3640 + }, + { + "epoch": 2.2277016742770166, + "grad_norm": 0.3522437512874603, + "learning_rate": 2.221333200263637e-06, + "loss": 0.041602414846420285, + "step": 3660 + }, + { + "epoch": 2.239878234398782, + "grad_norm": 0.234590083360672, + "learning_rate": 2.1964590888820233e-06, + "loss": 0.04286134541034699, + "step": 3680 + }, + { + "epoch": 2.252054794520548, + "grad_norm": 0.2972453534603119, + "learning_rate": 2.1716154396985526e-06, + "loss": 0.041756758093833925, + "step": 3700 + }, + { + "epoch": 2.2642313546423134, + "grad_norm": 0.3236760199069977, + "learning_rate": 2.1468047459262882e-06, + "loss": 0.0359495222568512, + "step": 3720 + }, + { + "epoch": 2.276407914764079, + "grad_norm": 0.32255831360816956, + "learning_rate": 2.12202949747101e-06, + "loss": 0.04322676360607147, + "step": 3740 + }, + { + "epoch": 2.2885844748858446, + "grad_norm": 0.2753404378890991, + "learning_rate": 2.0972921806813468e-06, + "loss": 0.04191597998142242, + "step": 3760 + }, + { + "epoch": 2.30076103500761, + "grad_norm": 0.23182159662246704, + "learning_rate": 2.072595278099247e-06, + "loss": 0.041278204321861266, + "step": 3780 + }, + { + "epoch": 2.312937595129376, + "grad_norm": 0.25987961888313293, + "learning_rate": 2.047941268210849e-06, + "loss": 0.04312986135482788, + "step": 3800 + }, + { + "epoch": 2.3251141552511414, + "grad_norm": 0.3683331310749054, + "learning_rate": 2.0233326251977426e-06, + "loss": 0.04236046075820923, + "step": 3820 + }, + { + "epoch": 2.337290715372907, + "grad_norm": 0.2520082890987396, + "learning_rate": 1.9987718186886724e-06, + "loss": 0.04433901011943817, + "step": 3840 + }, + { + "epoch": 2.3494672754946726, + "grad_norm": 0.16200299561023712, + "learning_rate": 1.9742613135116986e-06, + "loss": 0.04127628207206726, + "step": 3860 + }, + { + "epoch": 2.361643835616438, + "grad_norm": 0.37064701318740845, + "learning_rate": 1.949803569446828e-06, + "loss": 0.04586326479911804, + "step": 3880 + }, + { + "epoch": 2.373820395738204, + "grad_norm": 0.416020005941391, + "learning_rate": 1.925401040979171e-06, + "loss": 0.03624185025691986, + "step": 3900 + }, + { + "epoch": 2.3859969558599694, + "grad_norm": 0.17131179571151733, + "learning_rate": 1.9010561770526076e-06, + "loss": 0.035064518451690674, + "step": 3920 + }, + { + "epoch": 2.398173515981735, + "grad_norm": 0.5188226103782654, + "learning_rate": 1.8767714208240312e-06, + "loss": 0.042050021886825564, + "step": 3940 + }, + { + "epoch": 2.4103500761035006, + "grad_norm": 0.3398009240627289, + "learning_rate": 1.852549209418154e-06, + "loss": 0.038166466355323794, + "step": 3960 + }, + { + "epoch": 2.422526636225266, + "grad_norm": 0.2541758418083191, + "learning_rate": 1.8283919736829332e-06, + "loss": 0.040885674953460696, + "step": 3980 + }, + { + "epoch": 2.434703196347032, + "grad_norm": 0.4074256122112274, + "learning_rate": 1.804302137945614e-06, + "loss": 0.040162667632102966, + "step": 4000 + }, + { + "epoch": 2.434703196347032, + "eval_loss": 0.31583163142204285, + "eval_runtime": 47.319, + "eval_samples_per_second": 18.597, + "eval_steps_per_second": 18.597, + "step": 4000 + }, + { + "epoch": 2.4468797564687974, + "grad_norm": 0.2724802494049072, + "learning_rate": 1.7802821197694426e-06, + "loss": 0.04170995056629181, + "step": 4020 + }, + { + "epoch": 2.459056316590563, + "grad_norm": 0.21376508474349976, + "learning_rate": 1.7563343297110375e-06, + "loss": 0.03834344446659088, + "step": 4040 + }, + { + "epoch": 2.4712328767123286, + "grad_norm": 0.2933412492275238, + "learning_rate": 1.732461171078486e-06, + "loss": 0.03928310573101044, + "step": 4060 + }, + { + "epoch": 2.483409436834094, + "grad_norm": 0.46805083751678467, + "learning_rate": 1.7086650396901489e-06, + "loss": 0.03358933925628662, + "step": 4080 + }, + { + "epoch": 2.49558599695586, + "grad_norm": 0.45667552947998047, + "learning_rate": 1.6849483236342322e-06, + "loss": 0.03547535240650177, + "step": 4100 + }, + { + "epoch": 2.5077625570776254, + "grad_norm": 0.24512450397014618, + "learning_rate": 1.6613134030291217e-06, + "loss": 0.03600102663040161, + "step": 4120 + }, + { + "epoch": 2.519939117199391, + "grad_norm": 0.20636337995529175, + "learning_rate": 1.6377626497845278e-06, + "loss": 0.04347077012062073, + "step": 4140 + }, + { + "epoch": 2.5321156773211566, + "grad_norm": 0.4584953486919403, + "learning_rate": 1.6142984273634505e-06, + "loss": 0.02908192276954651, + "step": 4160 + }, + { + "epoch": 2.544292237442922, + "grad_norm": 0.26193487644195557, + "learning_rate": 1.5909230905449846e-06, + "loss": 0.03611198365688324, + "step": 4180 + }, + { + "epoch": 2.556468797564688, + "grad_norm": 0.20813828706741333, + "learning_rate": 1.567638985188012e-06, + "loss": 0.03758668601512909, + "step": 4200 + }, + { + "epoch": 2.5686453576864534, + "grad_norm": 0.3395022749900818, + "learning_rate": 1.544448447995773e-06, + "loss": 0.033633843064308167, + "step": 4220 + }, + { + "epoch": 2.580821917808219, + "grad_norm": 0.1472434103488922, + "learning_rate": 1.52135380628137e-06, + "loss": 0.036797890067100526, + "step": 4240 + }, + { + "epoch": 2.5929984779299846, + "grad_norm": 0.5788060426712036, + "learning_rate": 1.498357377734201e-06, + "loss": 0.039166563749313356, + "step": 4260 + }, + { + "epoch": 2.60517503805175, + "grad_norm": 0.7623679637908936, + "learning_rate": 1.4754614701873703e-06, + "loss": 0.03717599511146545, + "step": 4280 + }, + { + "epoch": 2.6173515981735163, + "grad_norm": 0.16205403208732605, + "learning_rate": 1.4526683813860792e-06, + "loss": 0.03962793946266174, + "step": 4300 + }, + { + "epoch": 2.6295281582952814, + "grad_norm": 0.11986076086759567, + "learning_rate": 1.4299803987570396e-06, + "loss": 0.035475924611091614, + "step": 4320 + }, + { + "epoch": 2.6417047184170475, + "grad_norm": 0.15573006868362427, + "learning_rate": 1.4073997991789078e-06, + "loss": 0.03256964683532715, + "step": 4340 + }, + { + "epoch": 2.6538812785388126, + "grad_norm": 0.25151512026786804, + "learning_rate": 1.384928848753792e-06, + "loss": 0.03712306022644043, + "step": 4360 + }, + { + "epoch": 2.6660578386605787, + "grad_norm": 0.20408153533935547, + "learning_rate": 1.3625698025798322e-06, + "loss": 0.041410398483276364, + "step": 4380 + }, + { + "epoch": 2.678234398782344, + "grad_norm": 0.3156696856021881, + "learning_rate": 1.3403249045248907e-06, + "loss": 0.03158504366874695, + "step": 4400 + }, + { + "epoch": 2.69041095890411, + "grad_norm": 0.3835665285587311, + "learning_rate": 1.3181963870013604e-06, + "loss": 0.03525224924087524, + "step": 4420 + }, + { + "epoch": 2.702587519025875, + "grad_norm": 0.45423486828804016, + "learning_rate": 1.2961864707421345e-06, + "loss": 0.03239959478378296, + "step": 4440 + }, + { + "epoch": 2.714764079147641, + "grad_norm": 0.15982766449451447, + "learning_rate": 1.2742973645777394e-06, + "loss": 0.031032082438468934, + "step": 4460 + }, + { + "epoch": 2.726940639269406, + "grad_norm": 0.2770426869392395, + "learning_rate": 1.252531265214662e-06, + "loss": 0.030566230416297913, + "step": 4480 + }, + { + "epoch": 2.7391171993911723, + "grad_norm": 0.3693839907646179, + "learning_rate": 1.2308903570149048e-06, + "loss": 0.041362547874450685, + "step": 4500 + }, + { + "epoch": 2.7391171993911723, + "eval_loss": 0.3391737937927246, + "eval_runtime": 46.9306, + "eval_samples_per_second": 18.751, + "eval_steps_per_second": 18.751, + "step": 4500 + }, + { + "epoch": 2.7512937595129374, + "grad_norm": 0.3229275643825531, + "learning_rate": 1.2093768117767613e-06, + "loss": 0.0388390064239502, + "step": 4520 + }, + { + "epoch": 2.7634703196347035, + "grad_norm": 0.6786078214645386, + "learning_rate": 1.1879927885168733e-06, + "loss": 0.032555675506591795, + "step": 4540 + }, + { + "epoch": 2.7756468797564686, + "grad_norm": 0.32371029257774353, + "learning_rate": 1.1667404332535504e-06, + "loss": 0.03606459796428681, + "step": 4560 + }, + { + "epoch": 2.7878234398782347, + "grad_norm": 0.44066882133483887, + "learning_rate": 1.1456218787914128e-06, + "loss": 0.032086309790611264, + "step": 4580 + }, + { + "epoch": 2.8, + "grad_norm": 0.5005165338516235, + "learning_rate": 1.1246392445073438e-06, + "loss": 0.033362787961959836, + "step": 4600 + }, + { + "epoch": 2.812176560121766, + "grad_norm": 0.22586822509765625, + "learning_rate": 1.1037946361378027e-06, + "loss": 0.03638745844364166, + "step": 4620 + }, + { + "epoch": 2.824353120243531, + "grad_norm": 0.2905796766281128, + "learning_rate": 1.0830901455674977e-06, + "loss": 0.030933958292007447, + "step": 4640 + }, + { + "epoch": 2.836529680365297, + "grad_norm": 0.10796497762203217, + "learning_rate": 1.0625278506194538e-06, + "loss": 0.02879139482975006, + "step": 4660 + }, + { + "epoch": 2.8487062404870622, + "grad_norm": 0.2545916438102722, + "learning_rate": 1.04210981484649e-06, + "loss": 0.03345020413398743, + "step": 4680 + }, + { + "epoch": 2.8608828006088283, + "grad_norm": 0.2986568808555603, + "learning_rate": 1.0218380873241314e-06, + "loss": 0.02593054175376892, + "step": 4700 + }, + { + "epoch": 2.8730593607305934, + "grad_norm": 0.1755756139755249, + "learning_rate": 1.0017147024449674e-06, + "loss": 0.03906567096710205, + "step": 4720 + }, + { + "epoch": 2.8852359208523595, + "grad_norm": 0.18288888037204742, + "learning_rate": 9.81741679714493e-07, + "loss": 0.03371626436710358, + "step": 4740 + }, + { + "epoch": 2.8974124809741246, + "grad_norm": 0.368429958820343, + "learning_rate": 9.619210235484333e-07, + "loss": 0.03090968132019043, + "step": 4760 + }, + { + "epoch": 2.9095890410958907, + "grad_norm": 0.17118144035339355, + "learning_rate": 9.422547230715931e-07, + "loss": 0.0322105199098587, + "step": 4780 + }, + { + "epoch": 2.921765601217656, + "grad_norm": 0.41911277174949646, + "learning_rate": 9.227447519182353e-07, + "loss": 0.035210177302360535, + "step": 4800 + }, + { + "epoch": 2.933942161339422, + "grad_norm": 0.3521968722343445, + "learning_rate": 9.033930680340097e-07, + "loss": 0.026842504739761353, + "step": 4820 + }, + { + "epoch": 2.946118721461187, + "grad_norm": 0.20812013745307922, + "learning_rate": 8.842016134794682e-07, + "loss": 0.03439584076404571, + "step": 4840 + }, + { + "epoch": 2.958295281582953, + "grad_norm": 0.2796875834465027, + "learning_rate": 8.651723142351603e-07, + "loss": 0.04011322855949402, + "step": 4860 + }, + { + "epoch": 2.9704718417047182, + "grad_norm": 0.20960687100887299, + "learning_rate": 8.463070800083562e-07, + "loss": 0.03800423145294189, + "step": 4880 + }, + { + "epoch": 2.9826484018264843, + "grad_norm": 0.2586495876312256, + "learning_rate": 8.276078040413879e-07, + "loss": 0.03839131891727447, + "step": 4900 + }, + { + "epoch": 2.9948249619482494, + "grad_norm": 0.37137141823768616, + "learning_rate": 8.090763629216589e-07, + "loss": 0.02721840739250183, + "step": 4920 + }, + { + "epoch": 3.006697108066971, + "grad_norm": 0.3677407503128052, + "learning_rate": 7.907146163933102e-07, + "loss": 0.023991990089416503, + "step": 4940 + }, + { + "epoch": 3.0188736681887365, + "grad_norm": 0.11811063438653946, + "learning_rate": 7.725244071705871e-07, + "loss": 0.01451514959335327, + "step": 4960 + }, + { + "epoch": 3.031050228310502, + "grad_norm": 0.3449067771434784, + "learning_rate": 7.545075607529104e-07, + "loss": 0.014327619969844819, + "step": 4980 + }, + { + "epoch": 3.0432267884322677, + "grad_norm": 0.3237353265285492, + "learning_rate": 7.366658852416788e-07, + "loss": 0.017832010984420776, + "step": 5000 + }, + { + "epoch": 3.0432267884322677, + "eval_loss": 0.39115142822265625, + "eval_runtime": 47.6431, + "eval_samples_per_second": 18.471, + "eval_steps_per_second": 18.471, + "step": 5000 + }, + { + "epoch": 3.0554033485540333, + "grad_norm": 0.2226281613111496, + "learning_rate": 7.190011711588101e-07, + "loss": 0.011674411594867706, + "step": 5020 + }, + { + "epoch": 3.067579908675799, + "grad_norm": 0.08729376643896103, + "learning_rate": 7.015151912670562e-07, + "loss": 0.013690856099128724, + "step": 5040 + }, + { + "epoch": 3.0797564687975645, + "grad_norm": 0.2745465636253357, + "learning_rate": 6.842097003920903e-07, + "loss": 0.011978642642498016, + "step": 5060 + }, + { + "epoch": 3.09193302891933, + "grad_norm": 0.06905842572450638, + "learning_rate": 6.67086435246406e-07, + "loss": 0.013893941044807434, + "step": 5080 + }, + { + "epoch": 3.1041095890410957, + "grad_norm": 0.07840315997600555, + "learning_rate": 6.501471142550194e-07, + "loss": 0.009910025447607041, + "step": 5100 + }, + { + "epoch": 3.1162861491628613, + "grad_norm": 0.19672124087810516, + "learning_rate": 6.333934373830222e-07, + "loss": 0.008863755315542222, + "step": 5120 + }, + { + "epoch": 3.128462709284627, + "grad_norm": 0.37645605206489563, + "learning_rate": 6.168270859649761e-07, + "loss": 0.010502541810274124, + "step": 5140 + }, + { + "epoch": 3.1406392694063925, + "grad_norm": 0.2069159746170044, + "learning_rate": 6.004497225361786e-07, + "loss": 0.012096930295228958, + "step": 5160 + }, + { + "epoch": 3.1528158295281585, + "grad_norm": 0.2584017217159271, + "learning_rate": 5.842629906658226e-07, + "loss": 0.013278065621852875, + "step": 5180 + }, + { + "epoch": 3.1649923896499237, + "grad_norm": 0.2050527036190033, + "learning_rate": 5.682685147920481e-07, + "loss": 0.013548998534679413, + "step": 5200 + }, + { + "epoch": 3.1771689497716897, + "grad_norm": 0.13838107883930206, + "learning_rate": 5.524679000589256e-07, + "loss": 0.013736458122730255, + "step": 5220 + }, + { + "epoch": 3.189345509893455, + "grad_norm": 0.06378225982189178, + "learning_rate": 5.36862732155366e-07, + "loss": 0.013177134096622467, + "step": 5240 + }, + { + "epoch": 3.201522070015221, + "grad_norm": 0.27431613206863403, + "learning_rate": 5.214545771559879e-07, + "loss": 0.011971819400787353, + "step": 5260 + }, + { + "epoch": 3.213698630136986, + "grad_norm": 0.529901921749115, + "learning_rate": 5.062449813639528e-07, + "loss": 0.014422819018363953, + "step": 5280 + }, + { + "epoch": 3.225875190258752, + "grad_norm": 0.19417761266231537, + "learning_rate": 4.912354711557856e-07, + "loss": 0.010663678497076034, + "step": 5300 + }, + { + "epoch": 3.2380517503805173, + "grad_norm": 0.044735077768564224, + "learning_rate": 4.764275528281892e-07, + "loss": 0.011400717496871948, + "step": 5320 + }, + { + "epoch": 3.2502283105022833, + "grad_norm": 0.057179443538188934, + "learning_rate": 4.6182271244688355e-07, + "loss": 0.008456526696681977, + "step": 5340 + }, + { + "epoch": 3.2624048706240485, + "grad_norm": 0.10396906733512878, + "learning_rate": 4.4742241569746407e-07, + "loss": 0.014539115130901337, + "step": 5360 + }, + { + "epoch": 3.2745814307458145, + "grad_norm": 0.32904428243637085, + "learning_rate": 4.332281077383177e-07, + "loss": 0.017625690996646882, + "step": 5380 + }, + { + "epoch": 3.2867579908675797, + "grad_norm": 0.20823979377746582, + "learning_rate": 4.1924121305558563e-07, + "loss": 0.007641100138425827, + "step": 5400 + }, + { + "epoch": 3.2989345509893457, + "grad_norm": 0.25470009446144104, + "learning_rate": 4.054631353202121e-07, + "loss": 0.011799700558185577, + "step": 5420 + }, + { + "epoch": 3.311111111111111, + "grad_norm": 0.3968588709831238, + "learning_rate": 3.9189525724707634e-07, + "loss": 0.011455408483743667, + "step": 5440 + }, + { + "epoch": 3.323287671232877, + "grad_norm": 0.10818332433700562, + "learning_rate": 3.785389404562259e-07, + "loss": 0.012499115616083144, + "step": 5460 + }, + { + "epoch": 3.335464231354642, + "grad_norm": 0.1818460375070572, + "learning_rate": 3.653955253362351e-07, + "loss": 0.01148865669965744, + "step": 5480 + }, + { + "epoch": 3.347640791476408, + "grad_norm": 0.3504088521003723, + "learning_rate": 3.5246633090968205e-07, + "loss": 0.012819178402423859, + "step": 5500 + }, + { + "epoch": 3.347640791476408, + "eval_loss": 0.43404534459114075, + "eval_runtime": 46.4882, + "eval_samples_per_second": 18.93, + "eval_steps_per_second": 18.93, + "step": 5500 + }, + { + "epoch": 3.3598173515981733, + "grad_norm": 0.4551874101161957, + "learning_rate": 3.397526547007832e-07, + "loss": 0.013325585424900055, + "step": 5520 + }, + { + "epoch": 3.3719939117199393, + "grad_norm": 0.35187825560569763, + "learning_rate": 3.2725577260517396e-07, + "loss": 0.011712662875652313, + "step": 5540 + }, + { + "epoch": 3.3841704718417045, + "grad_norm": 0.6071529984474182, + "learning_rate": 3.14976938761867e-07, + "loss": 0.01580573171377182, + "step": 5560 + }, + { + "epoch": 3.3963470319634705, + "grad_norm": 0.18844422698020935, + "learning_rate": 3.029173854273909e-07, + "loss": 0.012312603741884231, + "step": 5580 + }, + { + "epoch": 3.4085235920852357, + "grad_norm": 0.13131535053253174, + "learning_rate": 2.910783228521269e-07, + "loss": 0.011797953397035599, + "step": 5600 + }, + { + "epoch": 3.4207001522070017, + "grad_norm": 0.4402364492416382, + "learning_rate": 2.794609391588504e-07, + "loss": 0.012182456254959107, + "step": 5620 + }, + { + "epoch": 3.432876712328767, + "grad_norm": 0.3497592508792877, + "learning_rate": 2.6806640022349897e-07, + "loss": 0.013599888980388641, + "step": 5640 + }, + { + "epoch": 3.445053272450533, + "grad_norm": 0.2316354215145111, + "learning_rate": 2.5689584955816497e-07, + "loss": 0.009272868931293487, + "step": 5660 + }, + { + "epoch": 3.457229832572298, + "grad_norm": 0.3858301341533661, + "learning_rate": 2.459504081963421e-07, + "loss": 0.008165979385375976, + "step": 5680 + }, + { + "epoch": 3.469406392694064, + "grad_norm": 0.14734333753585815, + "learning_rate": 2.3523117458041865e-07, + "loss": 0.009182130545377731, + "step": 5700 + }, + { + "epoch": 3.4815829528158293, + "grad_norm": 0.03280401974916458, + "learning_rate": 2.2473922445144485e-07, + "loss": 0.0107998326420784, + "step": 5720 + }, + { + "epoch": 3.4937595129375953, + "grad_norm": 0.1505511999130249, + "learning_rate": 2.144756107411733e-07, + "loss": 0.014469687640666962, + "step": 5740 + }, + { + "epoch": 3.5059360730593605, + "grad_norm": 0.2366904318332672, + "learning_rate": 2.0444136346639333e-07, + "loss": 0.0121701680123806, + "step": 5760 + }, + { + "epoch": 3.5181126331811265, + "grad_norm": 0.1468425989151001, + "learning_rate": 1.9463748962556096e-07, + "loss": 0.014668506383895875, + "step": 5780 + }, + { + "epoch": 3.5302891933028917, + "grad_norm": 0.14534050226211548, + "learning_rate": 1.8506497309773885e-07, + "loss": 0.010488402843475342, + "step": 5800 + }, + { + "epoch": 3.5424657534246577, + "grad_norm": 0.15501493215560913, + "learning_rate": 1.7572477454386257e-07, + "loss": 0.010667071491479874, + "step": 5820 + }, + { + "epoch": 3.554642313546423, + "grad_norm": 0.26535800099372864, + "learning_rate": 1.6661783131032726e-07, + "loss": 0.011079683899879456, + "step": 5840 + }, + { + "epoch": 3.566818873668189, + "grad_norm": 0.24390950798988342, + "learning_rate": 1.5774505733492263e-07, + "loss": 0.009308797866106033, + "step": 5860 + }, + { + "epoch": 3.578995433789954, + "grad_norm": 0.3409421443939209, + "learning_rate": 1.49107343055111e-07, + "loss": 0.012319787591695785, + "step": 5880 + }, + { + "epoch": 3.59117199391172, + "grad_norm": 0.4800300896167755, + "learning_rate": 1.407055553186701e-07, + "loss": 0.00843576118350029, + "step": 5900 + }, + { + "epoch": 3.6033485540334853, + "grad_norm": 0.11663182079792023, + "learning_rate": 1.3254053729669564e-07, + "loss": 0.00938587412238121, + "step": 5920 + }, + { + "epoch": 3.6155251141552514, + "grad_norm": 0.29512378573417664, + "learning_rate": 1.2461310839898656e-07, + "loss": 0.011934128403663636, + "step": 5940 + }, + { + "epoch": 3.6277016742770165, + "grad_norm": 0.2641650140285492, + "learning_rate": 1.169240641918104e-07, + "loss": 0.013170333206653595, + "step": 5960 + }, + { + "epoch": 3.6398782343987826, + "grad_norm": 0.47704726457595825, + "learning_rate": 1.0947417631806539e-07, + "loss": 0.014534834027290344, + "step": 5980 + }, + { + "epoch": 3.6520547945205477, + "grad_norm": 0.10114685446023941, + "learning_rate": 1.0226419241983865e-07, + "loss": 0.011021688580513, + "step": 6000 + }, + { + "epoch": 3.6520547945205477, + "eval_loss": 0.44063475728034973, + "eval_runtime": 46.1734, + "eval_samples_per_second": 19.059, + "eval_steps_per_second": 19.059, + "step": 6000 + }, + { + "epoch": 3.6642313546423138, + "grad_norm": 0.2619183659553528, + "learning_rate": 9.529483606337902e-08, + "loss": 0.010764393210411071, + "step": 6020 + }, + { + "epoch": 3.676407914764079, + "grad_norm": 0.05733129009604454, + "learning_rate": 8.856680666647882e-08, + "loss": 0.012128306180238723, + "step": 6040 + }, + { + "epoch": 3.688584474885845, + "grad_norm": 0.19483673572540283, + "learning_rate": 8.208077942828713e-08, + "loss": 0.011729901283979416, + "step": 6060 + }, + { + "epoch": 3.70076103500761, + "grad_norm": 0.2111903578042984, + "learning_rate": 7.58374052615457e-08, + "loss": 0.009119105339050294, + "step": 6080 + }, + { + "epoch": 3.712937595129376, + "grad_norm": 0.04995311424136162, + "learning_rate": 6.983731072726818e-08, + "loss": 0.017101363837718965, + "step": 6100 + }, + { + "epoch": 3.7251141552511413, + "grad_norm": 0.5839787125587463, + "learning_rate": 6.408109797186118e-08, + "loss": 0.012368235737085342, + "step": 6120 + }, + { + "epoch": 3.7372907153729074, + "grad_norm": 0.4685717523097992, + "learning_rate": 5.856934466669212e-08, + "loss": 0.008782628178596496, + "step": 6140 + }, + { + "epoch": 3.7494672754946725, + "grad_norm": 0.17204681038856506, + "learning_rate": 5.3302603950119994e-08, + "loss": 0.008994438499212266, + "step": 6160 + }, + { + "epoch": 3.7616438356164386, + "grad_norm": 0.07392167299985886, + "learning_rate": 4.8281404371981755e-08, + "loss": 0.011286454647779465, + "step": 6180 + }, + { + "epoch": 3.7738203957382037, + "grad_norm": 0.3728208541870117, + "learning_rate": 4.350624984055196e-08, + "loss": 0.011785905063152313, + "step": 6200 + }, + { + "epoch": 3.7859969558599698, + "grad_norm": 0.25468680262565613, + "learning_rate": 3.897761957196877e-08, + "loss": 0.013624191284179688, + "step": 6220 + }, + { + "epoch": 3.798173515981735, + "grad_norm": 0.09725204110145569, + "learning_rate": 3.469596804214548e-08, + "loss": 0.011700452119112015, + "step": 6240 + }, + { + "epoch": 3.810350076103501, + "grad_norm": 0.07126162946224213, + "learning_rate": 3.06617249411581e-08, + "loss": 0.011029987037181855, + "step": 6260 + }, + { + "epoch": 3.822526636225266, + "grad_norm": 0.08542267978191376, + "learning_rate": 2.687529513012488e-08, + "loss": 0.010965974628925323, + "step": 6280 + }, + { + "epoch": 3.834703196347032, + "grad_norm": 0.2627331018447876, + "learning_rate": 2.3337058600575722e-08, + "loss": 0.012378603965044022, + "step": 6300 + }, + { + "epoch": 3.8468797564687973, + "grad_norm": 0.19707690179347992, + "learning_rate": 2.0047370436317437e-08, + "loss": 0.011792077124118805, + "step": 6320 + }, + { + "epoch": 3.8590563165905634, + "grad_norm": 0.47547003626823425, + "learning_rate": 1.7006560777798608e-08, + "loss": 0.01145942509174347, + "step": 6340 + }, + { + "epoch": 3.8712328767123285, + "grad_norm": 0.3591565489768982, + "learning_rate": 1.421493478897945e-08, + "loss": 0.011088228970766067, + "step": 6360 + }, + { + "epoch": 3.8834094368340946, + "grad_norm": 0.20619548857212067, + "learning_rate": 1.1672772626704909e-08, + "loss": 0.010828402638435364, + "step": 6380 + }, + { + "epoch": 3.8955859969558597, + "grad_norm": 0.2822403311729431, + "learning_rate": 9.38032941258965e-09, + "loss": 0.01165580153465271, + "step": 6400 + }, + { + "epoch": 3.9077625570776258, + "grad_norm": 0.15682674944400787, + "learning_rate": 7.3378352074163215e-09, + "loss": 0.010783226788043975, + "step": 6420 + }, + { + "epoch": 3.919939117199391, + "grad_norm": 0.34253379702568054, + "learning_rate": 5.545494988045963e-09, + "loss": 0.011295531690120698, + "step": 6440 + }, + { + "epoch": 3.932115677321157, + "grad_norm": 0.27684664726257324, + "learning_rate": 4.003488626848073e-09, + "loss": 0.013613662123680115, + "step": 6460 + }, + { + "epoch": 3.944292237442922, + "grad_norm": 0.69688880443573, + "learning_rate": 2.7119708736486615e-09, + "loss": 0.011696261167526246, + "step": 6480 + }, + { + "epoch": 3.956468797564688, + "grad_norm": 0.2617769241333008, + "learning_rate": 1.6710713402015577e-09, + "loss": 0.010873865336179733, + "step": 6500 + }, + { + "epoch": 3.956468797564688, + "eval_loss": 0.441041499376297, + "eval_runtime": 46.1946, + "eval_samples_per_second": 19.05, + "eval_steps_per_second": 19.05, + "step": 6500 + }, + { + "epoch": 3.9686453576864533, + "grad_norm": 0.23670868575572968, + "learning_rate": 8.80894487179651e-10, + "loss": 0.012961818277835846, + "step": 6520 + }, + { + "epoch": 3.9808219178082194, + "grad_norm": 1.036125659942627, + "learning_rate": 3.4151961369188745e-10, + "loss": 0.01224210560321808, + "step": 6540 + }, + { + "epoch": 3.9929984779299845, + "grad_norm": 0.1431870311498642, + "learning_rate": 5.300084932574612e-11, + "loss": 0.010245455056428909, + "step": 6560 + }, + { + "epoch": 4.0, + "step": 6572, + "total_flos": 3.545203061907456e+17, + "train_loss": 0.08566030774240586, + "train_runtime": 13933.4985, + "train_samples_per_second": 3.772, + "train_steps_per_second": 0.472 + } + ], + "logging_steps": 20, + "max_steps": 6572, + "num_input_tokens_seen": 0, + "num_train_epochs": 4, + "save_steps": 500, + "stateful_callbacks": { + "TrainerControl": { + "args": { + "should_epoch_stop": false, + "should_evaluate": false, + "should_log": false, + "should_save": true, + "should_training_stop": true + }, + "attributes": {} + } + }, + "total_flos": 3.545203061907456e+17, + "train_batch_size": 1, + "trial_name": null, + "trial_params": null +} diff --git a/training_args.bin b/training_args.bin new file mode 100644 index 0000000..8a9cd5f --- /dev/null +++ b/training_args.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1c8f0bbe96b1bd3b42d30bb419fc0c08037dc014a4a8cea153b553983982dd1 +size 5585 diff --git a/training_eval_loss.png b/training_eval_loss.png new file mode 100644 index 0000000000000000000000000000000000000000..0d78d957c29ce2499b830c8ca75ce881a303e80c GIT binary patch literal 40648 zcmdpehdY=5|L?6bGRjDb5So-(k*p+28YEo=fRa@?z)<1hUk$+EY zO-*o6reYuiQ~0JcJ*Li$^VDzOe%bokVKpuI%#)*H4mX}exz0VgxJ#aff*1c;9D1Fj zxAy;%F4Gf%0RaK*+o^mM@ZUZ9^>TPYtm18oolHzjs(h717=E2_--tMcUw^9;CU%h@ zaAh08jNcy?{r~ChD!vTF-m_0P2nnTI^u!Ck|L~#WTjIsO{{D4=3bzUzhFWffv*VXS zOGC1&iN7trC2=0RQP$YFxu&KjBPWNFm>KWlD0ZJuYj$f^Vh-GpW%eMCT7@tnJSuX*6d`-fVEj{CoS{c1V(nDzQp|H)Ikn4e2JCFLf6 z*}2_5N?4(3`ft%hN##g$GPiq-x?G~VabJbMor8n#lPBA6`T9-`#dt?(X~e1Z`EHi+ ziHYG>IeWHmZ0s)2iQ78`1=DU@>SftGIEa{j*phA8y6MP~BN^G*3bi4enPwlK)|Sk3 z$$IU;fZ%`c-@j+=Xc^87QE~e4^2D9VDwgxTQO?~WF3&J<{fTd)Z)Ij)Kira%@3}Z< zRC$}hRR8<@cvmo9+M*jrL)fjZcrGo^_h^-P93^_|?(==N$kEC&V|XfVnQf#AxD8;TcV&=T>tEfKl|+VCQiwn2M;pbzkmP2zAfe*+1C!11dW70dlpc&nh}-#ir98bDJUxTW3-dMdF5yyG;54zprfO^a`mdFi#wGM!zp89V}DVzqeqV}jHZ`n zT>BcYR*zp+R#e3DCu#DC+jc8BIvzS=+pRlLEBNT)!}Sgh)7rZx~&HYLgleA#qW-sN)Fq1Hy`djKk~ur->*01#n~+_ zT;D)R*{tko8C+Z}y)s|A`s<+kueeA<6BFOr(bj>7=VfjN2cPkuUu?e`)AuRr*keg2 z{+5)>7yEw4O_Z+6JUwDfd-3AMnbFqGt?33r8s?bs$*HNE!5l1#5&9mr|b_q}t68o&6QbV2^N*xk9U zqod->7v@y$oQl=up4DIBm-B2mJ94e*92^|l+S_@Ag+ITjlNAwRx)`r=Gb>As9PdmM zC99qiy}T8y#;{Yn@|{Mv&W(2^pP{a;s(Mt=GdXFuFI<_P{SXb2VcmHI`)+Br)!^|_ zhdsFPnUZkp@A*vDow@#jfg8Vn|33LaApOyT`S=}fug}ZHoSd9fOVf=5SaF535BHp{ zxXr*p$Z57J*}ia}ANNO1D0H3(p<@#y3rwqlcXMiJP|z&}-s3}`qGoeB4w*Az>&`}O zSDUEFWmZ*H9d6Is+wk#Y@bYp{M~UaLyLazS{u@mnjrje&fQ5pBBIE7bTRAz0JMwJP z=U*Q1@DAmY4%#keR#sk4(bCfLSlDRYmoHyt<_zvnONojy`%-Vd9TmlS;J|^;*qz2~ zzR8;D#y{%rXXNGGNle_oZvFb%?#5UQMX-$Lzp{n~dVPKU!n^jk4J<^*_Qy);6#hef z^F7|1PoxQ8<7|_#+i7KGWoTm)qF?ORR~5)oIEy{oTwL1S-R(ByblEyJs3p@*7Hbu2 zrPY+#+R94U_0JFH^rM4%>9lcg-jHQ(o9LAQo6AP=*{C1FhYz2`*9qJ8$~`~o5Z^W@ zySgNXYD>++!jf&>xz)(XsQSbAVs|#7k_F!9QBh8VB};$mjR*hy`IF)JOD)@~of*5r z&dI5Ivaj;`K+Qd4Q@V{j!k+&e>jv$|+Ow$Tsz0fgWi+7=|#6qAtbLPyUYhN~C zizZBZCZ~*kdo|?AgYKmA^w8;f_r9J54B=y0Z<(iuuB}6Xky;$fI;E_fbiwn+L^s0| zmD|6*z8Q^p^7N_khe!L-;y-)Z(8{tk$}Aqh)_b9md}e8EJHNCvCr0oVR+^QywaE2; zh3Chd&rKAsVhQuyKV{cfLD_Xfc4dKY7xTv3o4M2SN?THOOd1}GxYg4aY}&FSz4CX6 zd*;_y>YqP<+BrMdeh3#}aO3nJ(l>pY|2jUtKQ*tr;?H1xy{9BIE35zB7!PJMGqcIL zIj_fB8);}5W4!()qHiT)bl8p`KOUQKk3*c6h*b*ZGDbZzURnBE(cP`T@b6y`R!rcZ zck|uu^1Sy0L`-S|@X8=GfsIN^N_~TayRmTn80B|JyQve5J>ILu-MQ`U?G+Uj^6&n9 z5GXW7*_-wPa^1_kvBcm1$0prNPVV%d_y^i14#E9Qn#+I{X@_+PAz z4TOmOk5ieZ^;^*u*~wlha`n@5(NWZJKJ7f4gLm)UtA6J?U4<2T5?gNrk?A&T!Q;m` zH8o|7R$9^2q`I)U7!1r2K>kTdNfDi2{&Geru&hk3Yr?>DHk{ar&co+6JF*+ouva5R z3-9hncV!|JPMr$S%Cm`jeAqG~?ERHkY%KXWHP%k0NI|`-H!)uP=yAIm8X9I+y7!)s z-m<(n&qg>pb!D2|(eI+#v}2!Q$d3IOz*kK)IyyR29hT|8TD;K&^*(E6u6 z&&kQ@E-*ANcE{K2>gpbV!@%8IP`t70$nm6hr4w4fk-s5pG zZ!A=Hg^^Gw^;)jNs$Tnp%3hVA`YNbc9AKx_Gg{wukd-G~`DA0Xw7*7jbNJyIsnvl{ zSwCGtw&>{SwL2td&EaPQ6Auqhp(t*O85kt??6V{Nm78U^kQ0Sz*uosBfGbk6_2v&( z*3!`lMvJ23@mu)yCZ;N@QAU6Wz#9U|v7H`j9PQ4EZS|?Vvzd;F#bn368&!63$q%a^ zjJ0e{>&LfmC0FP0=a;_t8Lz0Q8Vyg?-|3<1{Obc7h$(#JNK|W{O&6Atrh(^fE=sv~ z)1Qu2f4X@tN;0so@=h?hK+$MviT-=y-Jd&wMlQXKI_6Bv$H#X+Iyz0~$h9xKDCE#L zPNnDjP7F7sW^)qrgNp`oGV9s#&2w8wIZ1!$olwI z+gMRmrLeTJI8o?%>Bo-4Gg-64PB|HZQIbx~AD@Y{aeFPi)GDx#vvYLZg5hem)@gC7 z2y1-CSlu{;N^k;8ln$G&YXu--FHcqFbRJ$+(bHq#@#pDoqU$KHUtjA@6r+ve;^KXM zeLUoA$m=Ch8HL;!##*Z@TsaPPm^m{u(-;f7qPqGdaA(lc(h_YkwLFc8!%vl_Tx*?| z8p%#E)vsUgDi-yhb+ET*2Ag2U(qqK{u@O~l1|B>w)cKwKTT?Fc5#+6m@Q?QP?ksft z6{}eeK(Ub+r1B9F5g`WwP1eBnZ*?%6Kl-dqfkrYbT8=SB(`~S1b;X@Rjx3CM2lnoL z({|mg;V};|!byGoXq(HIHu#`tW{tl-Xo1d7Zdq(4Iv(0v_aAgG#YVgS+>T4_q-|EB z;n7IfKS%`Fmi$%BQ;AaxOw%i(MT5fkz0_GTf1wr)BJfL3m{y3Mync%u?K~+F5mofj zsc#q0Z@j1YN~gqY*^NTZJ^|~>k*7+A`UbibRP`Y+~C7J z8XgsO5ZG&0Fcv_siH;Y&N09t;Mf6Eqsdp*nHtT95<`z6E@j|ixzJ(VABkHlBMojF) z=+ncNeyHFRC5vhLPp~;Jp%jc3+lKOcc&}f#ZpJ1|Kr`)IavtODs3BaZi@Q7B`SXp& z<)@oCoKU0`fTm|GA*Hla{i$(%{)iD zDfT3th+!E~?Dq4=4;our+on=)@2W_vd6ka(fl{n;U#HzxJVIyq`;>ibWh zCh4ry3zgCk{dqlJUk?iWmNcpxIvenyaR|ijk&Ep%_NgpIRR$LsGO(_*)A z)ZTe<+oo9mg^}d+wYsRCZO((9IesRNhjyZ#9QcD5lbZ{xPBqP!U=h0=w+57TGKf#t zEr`|3tB_RxA$eh8egUY}ss532oZa>7`?oft@_Mf>Vt>uEY}>X^P&eOj(o)C~sEOy{ zd&iibw`om86{ul~rmVNOci)d6yI!B{oSGSKS}%8eQGk$e_(`v(rY3CNacJ9iX)8eq zuO$~?{g+bib7MC(lB&91QPbHA(OhhTF6QjK0>dG>~ zm3!|WCQaFT%*xU-I6q&aSZT-Ymh$RqN&wEU2gQyYsli*q!o&T^swKZD0BpIts-|W{ zx`h7_uDR}tXx9YwW-dSFxC+3K*^Ua8XGa)-*V~Gmg?w&J@5YRYUH$aUl3MIa9pl2^ zzrQ|&4*{{eg~s2!_FHk7-zx7@dHVj|R}HiUu*;XFnas1SS1fIiik2Y6N` zTB_iiTdHmWb9%`?ewn4o6YpkQblgS(3hzfn=~+zV#m1kHNoy4x!L8iBf1hdYcjp@& zvpUxml%4@>jK{~EZ8DVz|IiPBxYvh2^W5c<*3OIh@#D(t*RKuBd?@m7{C?U!@E554 zCHh6^VXInzdDRxDLEIC5#~$rIeiSbq9(fcMb-;;`d8{sd16Ln)GWOR5qpgrqqq`PjovOgx8WGS^+$djm5;S2iV&LJ8zS(hFF zXAXBMaY>*5U_3mODmpZcmNhHmEmxlNa?;7-5IcFjPycOM;rwv%%ipVYmY(Xat@}@$ zjgirtSRxn4A$A6vo)YUx8AvBa#>Q$^u6?tzXyb(A{A@#mir<|(CQusx@L=3OK*I&~v(I+Yz{eOF2DQ)gr=E0I1YK*A`*6te~PH^sl9@Wecsdj&q zAhfi#`~MEbv@Q!q)-!F{qTb(=dp<^H0hD(N;HWn)QlHdZ;v{Vwo`_NVQZew)PU}3v zm!K28)GP%7+=)tE%5tjjPG&}`QeWVO>>-1`O$>kK^onjU$kSw-H^o-Xo)i)`skvJY zFfs*Upub=!`h1i$-lp*>mPloZgJXIuI4BZG#9 zhFd*qW~z>X);zG{4gjudl-AGb2D0?rGCTN`LcZx8+A63U0Mg03ZyybSNbSY*(*8ry z?wh&_odohW9n{X=*Fy8RNmBhAh!!bLm6eqlkG?A|4sNC(v~w)t@^l3wD{E_&@=CJp ziQb{1I|H?$9J+Y`&l1OujYiC(9$4hCL(`m`oRl**=5?DHzKK#d>KWFVeQmwF`yy>~ zQCLxtdm<;t%D+*ev92OEFz;`c^P1*A8=foxFo~Fd-ib0W02QkO^DAQ4TV`y^y)17SXUdxhxj4q46^QPuHZ6mezV3(pN6G`zyPH6NqUfwf>_P;w0 z6O`f|U-oUG^)0!gaj6*VR=}}X((~V;2M-?1tSjRXw<-Jh@vQU8rPqS~v->?l@#C=Q z=n#;&ly64Pw8w_N=U-nBHRH?T1O2C;KVP_FYHEa{W18D1II$yj_-gX5!Kl5aA~Vw( zPp1DijdD+BFm%@3{m zB_V=mlXsERDAnf8n~NnKC%X00vrf5UQ8Q?2YT~v>4hsD$`HB^4)%|WeSbHr{FWCQQ zD4u$T5~yA~j@_Wg1b*!@5wPbZe1P$`F2+KqdDA+V5tfyeHC}QAoXKOdoO-yYR2HDZ z%x-TO)?FW}L%!p$2ge7yzJCvL`eD`ncDIk38%3~^q=b5eMlvW``O5O*tf;bo&`6>_?~fmTph+aL zCExr=c{d91?W-}KxnF7zjymN*tN8Wn7eG5T_N40A9`rCjhoMh>WjEIY?#Rwf^n{`= z)3WSQ`8=9o5|AX-NLMeYl|ihhqTiRP(#Q4=d#evy({zqjT;X(#Q*WPBT*f(&-CL!l z-hrmv#rd*H+4^~=hm*u#>)+N-$F`-02Cf`>@!GKa3>2tH_I=KvIu*5?!iEGYYli)x zGQ{qp8uyf~m%80AY&{`k8!jDlrf?HERcp(-KUk>0%eluNy5MWI{g6>DKM!SQw`y-^ zEtZL#d*bqZzBCLJrkll2etx2d`U0cJmgT(R$ba(Z&YfGpL@~&io!lbBAc?`Z>3$ao z?GOs+I%&6=GVtU3WIJ44;lQYu*gOl;rS_?H=T-{Y5eP=fQo8gfSU)1|Kziwlq~bkx z$^zZAMR|?3z#sUG3yJE5^F>moUZyiMGaD|1ogDu9Ce)@tJ5>>iU;*1skP;f6-rv7d zgC*)k-Db?eM5)kJ+Hx$}-o1NAs@20S+(+#L0N>{}oI&|caeZX!%MZn!m_8(izod6~Z(NPWNCWBTk48INs1O9TO3=I(YeDwNKf z=--f;rcg&rruwVn^aG@&rLRL^0f;O~zr12Mj$PNYdqRk650 zPM(ZeU8062Q*mB)S>5<&#yf{>epwyU{gy6Ke`IXqq8h`uzr9M{%-1)Yfu-q+2eDm1 zP*WyXl6-+rG8=uB`1q}zxsz2wDxtFfv5ix#&^KSF{tE_j9vQt*_0a$Nf0=DHTh87@-oBUci}f=bhNb8 z^9#aM_6W@Bui#F1Qyt2wZf(tL8|s6&l$^`+#pf1%gyO3v%Xkp?sb)k0o!3Vd@H7hcZ*n$_SRNt*hp8YU3o z3VTpOa;@5L!7_XlMe(S9khBTmNZkY4H3mfr2n;L-i-WAKGcLX#)sfU#wcfdiz@!qk zY7^GFe7O(so|Mo?UA(LJzFav&s_XPmm+6nj@6}KLczC|z)2B@=d(TyY$)Npb zfMf8xUs+CoZx9P8O1_qfg~b<5uODJynNaBx>9&rDZU_FVg@Pjj=0bXN07-y63Q%jD zdSqhUM)v3D=f6O2v%7JlCR{)>0}2B4#n)OKl_VVr!SXN!fDV8s5*f(`)%zZDN!eGe+ zV2)mCrt5DLx7oqW%$xxk7qG&p`<-i{=T@&?z(8M$^;AFwMlh+sziboNkIn&Cl2+IF z_kve!!?UOwI=NOiBO@cb^ZjqP6!F5j3;_>WkL#)Np;%8&E{R4)z9pVIdzMt2o3gK6 z{9fY8f%*3>EiFweHiPKo4+><2`6>Jd(8Ylh1Uk5v%k&W#RiWwis2g+mG{hN7lYWrnYmmEmn z3Ou};{H*cu5WKk`<%u*VP^_qW`&|FPoHNGVqJGwZRbqeKgsN#=9mGmneK3Kp&kU<} ziFbYwM)9}pk$L>|>4<5a&n>EMp?8z(TGI4_(N~Om%WlGxC5!97nO_(C{(5VsXx)VN z$|oVQ4YPd$>TcN6r$HcR{n!$PLeYRaWiY(%L2(_mv4@sVc3v3Zfmfql6leRV8F*x zuAo9B+fAU#fu+CG7*4qyvwgn4z9JUi_8riF zw=FO*(7gNI(Olb}kjIZ78@GSiEVDpB10)63G?PKpP$OGggP)I+L)j%A_o3R}hIYiW zXAhNiM|OF%w7W9Hh&w0;##JNT_XUa(M4UjZ#uv^L-EP+W0_=I)`k&dBl>hwrOawb` zVPRn>NFVxWS&a8eAhwc*dDtIV=BzpzkEh`MUC1oWg9S>G^{T3>7eJ#ViU9`FWXxv! zEvnn`nU<6jfm%qx7d9?r)N=tSpx`ZXzJ&)?0cFV@Lb@bcT3R%+va(ZfCj*9uhmB2p z0AeWQ4%v3UtMY_HB2RYe3UJo^f`UNtZkH1~B<$n}My#cl@Z=yPLqA4dF|msc&&1E; z1shFFOh{QQ)1sNbczhZ|w5Dv^*x1MmM$$nz6|wm)F@-%d@&P^YBpU5S2?+@r$Pj&_ zqXPQg%T8U}ZkV1vb&3&Y^wyA&5WFoIOFB|w$)BgP40s9n_;ss)$%Vd20xq23^-fag z$MqXoSsj4SR}INa*J^=@$B%4&!`~7glCsv^+??^!!W6Hd;JNg>P|C5`ww`@@hzX_v z!3aE2QCFAH<`U*FDk=(ni-qK@m9icvUF&?vmcXw!fz}x=sdE1GgyoehuVJP#lb!>l zOcH02(FSHfTk=|gB?7^U0~Y5eMsxRjc&~y^ilBNLUA?;Zz`=vn3sVCpm6e0KR~V!l z{Tci5&sgu(m5L7^lrawhqzcWjm)DO`+~zwY_5mY&?~1lIEiljERU}i$Z<7zW_0nw@ zbKt2{r}oYL6Og2D08DwInU+2J0cz?&(yCijAysD-*I+o*#rC8c3WttOEm7U}ROs{P zJ4k=Z$i##XHd7VItdogb_#yg+SF=J(?oyIfDz2@`_xY59%6+NX+++OqNu+D0YVU@I zO?ooqs6j^{9p^$(Vy5QE!I@z@PWde!a|SSSNLd_Kmw#|@FrabF0-Uj#4}(8`thqr@ zTSq_K+i_G4#P!64?FETVhA%6w>(kXnrqmtf_cMZYdHbeowxIvS*``vx)2$nM6bfSU z-hLVaRGa$l(6|xy6os5--j`1XJhXV_@0?aYunb}B6wpi-u={G6^>GaAO^l3Gtz!I# zOiw;?iuz)Hh)lbX2HRsqHc-MznL84r_Nw)7MeFMDY@8J6qH*~xN>Z)I?hb(07m$&W z!CbF+@?;>e0qPzPIZ|ScQm>q24yj}M$VK4?Qw$x33AOGPNovl#`?O`#>FHq?|B2%H zXkur#*Wz#HP@_}d)jE0xnjBUj-yq?o8VjUJZc1{xv0;Nj9bT)M5Hd+AmW~LbRKt`| zXid{2?Gad$-|o3;WoAuK+*9Nm`~ABb@eS^xe)TL~K0Zp~Ix1R6nz9H;EB}!Un-gmK zK6=kc^{6TCOk_uzVpL>Ij1Uk~HLN#;Lz-g+V_vbAPZvz>x3s#zf4ShECzM}9W8>KQ zQ{w;+ih%6p7t2+<{*+U5gGm`Xz0Jv~MrmX4I&}6@ZBNf?<{j^A^w9Vab9q-gVtuYT zbVy9~?*|5A72y!FQ*JwKu?3*lZf@+d2W%bl)-+C#v`Z{b9#*&=@UytI$oE%RNBq@e zC=o4b56oxy{Yd^&`mk=ax+EM5<{_Q9!nl&CMyKCg;o;$zjHy#&;c%*=_-xU`CHq}sUnZe-bGh%R+Wberkf zZ89D%PsGfyAx0P~&L}9TUmWR>9s!8tclPK2zL0LYdpxudk(fYVJGR+R$0iQ72^j*V z9calMQ-$%$3izaMTKkAb^{k$`02yM@%CTVW9~dB!KisHKFI1W_k7Qkwnr;wVfEwpi zkD!^l710g-$?x{<5wFI&x-v`xBNG!vfo=#BpQ7Q%r-vElIH;JInx(OmOgE)Mi_jQP zQN8pFEnx86#a2KDe$G=;VCvu7V?hW<)Eh^W9z{glPDn@)ttS%>X=!PfZZt?T5#t>> zbY#!OJDGOuxZ9kpIRHE(2O^0SigK4A+7Nu1JE5VQv4nyteKu~~NNV(s^`ONw@saF^ zl8pD1h5}LTl$GU1-BpCgs=UMSKA8=CnZQC}XlEx(dQTj6>dDRU9@nFM)jwN!&*{ zPF`Jo%T^)%6rDZc%1q46xA1*xc{)gc;kt!Uq97opREdK6-vlQZ8you!qm9Im>^}9(-4lt2_-PtqKTfwojXDUbn0- zgDj1;Wc*O!7?i=Y+b@2b8mPruH5mTrJf8qD$9Q3~4=Du@FY!t5)p_r27#@7m&WGAa-B_(0@K~^|!>k3#04?ZZo^!fl5p!y8e-viGZzSq^!G&MC99(4=;gOJs$ zSA1y(r6Krq(ro~IaWnm9aTt5|jFccjA{rpV)@6N{b(eK?#K_cV(p7ec=~Qj+$%7D* z85w=hIJ!hrHOL0yh&+atD_SfGn?QNLE|Z=!$;YwosW)#9We#Nl#(eZ>iyqquWISGJ zX}$E$S)i5|%@$aZy8SX(X^!PwzB1&Lq+3X@ry)ps4N0A8+qQrgFLr|bu>3@w zO^t4)WNyBXj9{hd9V`BAZEZ~h?&ttNzJ(Op1G>Fwi22F%6nl54;)+8kvP7G z@5$T?GGCme%HZHoNh;BZn6d5Iv!~D=nDHR_kr>wF{Nbd@ih^SuUP|6qY~ky3ch_qf zkP}x^Lq+Q!`>nQIo(AL}%Z)va^AhzF6)HZySBhOWegO$3Vn0^8J~s!_8=i<$OC(j7 zA3uJ4yKyOj_Q|tnmCemL-3|U{6%=OnQTa40BsY`(BcdH-W>Z)2YD+>ii@aQ%8qfay z8xcd0hjYdLn~24-DHS17_8Ph@j^ikN<@xq>} zA0P8rxGae8#(oTO7A0?hj0Il`MDKv3Un<{823{dVrp~%yVIMwXwf@v4vP(XecE91D z3kPL0J~lZ~kzB3pR6a9WevD9g5DoPM=JwCc$;(TR*T$%;US6Pi9Hh!0{#fklCTwg& z3yUDAivwtfgb&1j#M?-(jE08B2Qj%2Zdv`YresJKpp)3)bg(iH=;kw$1T({^($SR5 zbHmu+pg)Pfp;_>vFMm$CEaoxSe$~xMC*O`oJJ)I>0)0Txy}$E%um-%GN;naHexM=n zqQ(5Bn&th-^y6vDzE*MUPh9_A`0M@9$dXYcra z`eXU;M^@jJKh@lOeL(koLsjX@u3a{6y{Kjs5kNfK>jZBGEtk^OT;Q(kUm0!QIz!t3{ zsp0ZJC%q>i4SncLx5!dGHfHyuj;YwwW1;U3sqT@Y0x3w7iCC|dzgIDSZyHRzm!@jT zq{tLms@jlgCHw!(NHZObfpr85p%E|zxsDMK5P>F7a(Y}r!e`4}ycT2G+BaVx*xmod z{<@GfTftv;8lD%da(Yks8a{m6#^qtCKDdo&*_oplJkeOyyl`EK`p97knubrGY9Qs4 zdB^{Pk5IUl(DYz)3w@3|!$y>WkirOGRh!SU>5?RC4VgP543SQ0Ljfw{?hsWawc?G4}HU>$V2Va?2x%=8K>+p$-$Hdwmb3}4v zH}Ased5>R}OHMuUtB%T#!jM7q!D$i6jBBYo@{%q?Rr)JKv1OF?08*33O%xP1)P!<} zg3TGhH3haTM}eJ!8KHSbFCbtfIopbLT*f>&@@Z<%xsDItXvS|)9yee)^H5Uk?6q9A zdk)TDu1oG=y=wf$?#K~_qx=jJOK%wT$Hw(Kn=S1ylNlp4HAxT9cYfF(J@;IK;{5sZ z1UWkAVq~l*AA2HHMQSZrR!vz~&l5h1ugbB@xh?OO01_bH#e)1tyiGgStw6CXZ|rUP zWY$v+>isruo3McHr+ynwb)8^o zSog8@(OYV_9rVQMP=9|)a<#sL*CUEf1lnd ze|TSg-Bpf@`LZtWEG-TsS3liKzkTb&2h%%?G#O{Qls^e-eCleUzo~zNX6CI@SfsSJ z#ASK@RF3T*Zlff!W*Jx0f_T20te~rFXi$K#i?{^=_~1LAY1b!@He!O7h4TDZ##55C znxCZnVS5QFjvbw;5-tNIK~m)+wRT8w*W%_+E2zg@5opeTrq4LD`^pyEM$HBD*6NOI zjTi~m+#Z33@u-l~C+s8C((fv)%eK#ze-X)9=+e&jxt?o0*DObUphk&TL2qPCuk_0Q zX*OC)e(&B%eJ^yPL!uVp*wi@o7EUnTRGqvUP`s(}qKTFpPF!@WxeDqJ)v_*8MoE0J z4~ux!6W!R+Fnm0saYsnRA8pGPcY{kQ(rMM^z+;Nn z7Gxtq&EXzyPG*Cb)sk)yN*Yk`gp+?K0nT32PL-24(7}F*<2hAjs3+#<)eNOXpTH4K}ok&j^-wU zLVZw$%rW!Un)d<{KB*=hrW}}#8|;tfWgAiIm@a609Ge)m{kxTAiL^n}ockNwxc&y~ z+#XAsZG>75?A|g+9~MRI3JOjd5o)RK+0Py)etYIw`AtiE(!8~M$0vPr z`3SMTzsz4OYE`PaisAvN!sGgved$?PV1BmlYuPW+jWrEgt$~&0v1XmqQKb(iQD?BahiCBkk+ZxMPqm)DNG#s|-TI^9CA!~#4{6^FT?wd( z=M$fa$XeE~tR;`%Bv5@n+jvqrf>u#&uGhQAX%5?<2V3E>y1Vug9; z6?5+QHju}6V%hCP&QTENeO4P|_C(}pM@~C`5t;7H%6}G3CuXu;W=A{4d1uGuW=YkL z#^kXc^^fE6L%fHJb}qJdU8Dbdd91OK3tK`R*BBaONL0*O|i&1r#`)Q>gQ5c z@bzB=?OX4h**0+@1K+q({hCi-oe5&OQv#fa)6)9D@g%(*@$>XeCZJ}_pfwtUi(R?a z9LS3#dz31ClVH8F`p5F{qBAz%t2_n6p3nE*qEM3{1hA*coQ}* zlwk6ti~_4hXM5>D>w~G7oLW1ca}+ubCRQ;!{51a^y1F-1F(lLn$C#2%Tbr%=F}BUL zGI%)dWY?z1cF3!VpQ4z|5KnJ%yUwHE%Uk<{u5n7{0JfApPB~rKLN*$-rO-VA8_&hms%% z{sgQao41UGO~Mr@5d$znAOtD17C%(@*2ZeKs$HnnY(6I>Cd4h|$>JYwL|NXG;1l1r z{N*A2%Befs%h-4M$=Q^|FWtJ-(4b{D8Sn?rYAph(WX&Yw-zZI0fVw-m8dM#jGm|$4 zYp@!tpQKXHrCdqHN&&VFwv;~DBh-Q(XoDjBbIRM95wynTnFL@vWT0AKa=etbtA}n^ZuI`k|6H!W#p>0R3wn3Ym z1RYoZ&FvGw9tSw4H`ka><=(D;rmpHPr)vI_=L`iZL2>8vs`Gb-vPWI^8<7cQJb9J- zbx>Vhoexs&TWBd$QfDtNy1KfO2R9HGF|x3rCF)cYo6O$dWKSQGt*0*2wb4rPhF*c@ zzY|0mimBTu7{4EoX)g$Mm@?giT7XD9d;1FLX%qy^8Z^oU+=a-EF)i{K7J?#G@B|Hw zj3@{I9Bzb;f*`Gyx3_P{^t&L{xW*$3O(1!Y2bJ0~jMjs;!HB*`+T1HEE7cEw7C#dBHcHR$z|D~-QFLT+6@LEy zJkrv|Rr3k2OAj(Ny{ZuGQnI)q(~*Cn7^yiCI_COL}&&9B}B% zCiBm4P0x@K5-&1sL&|}grM8-U=`iJ>yh1 zkyRwN@Jg-uiBcKI5v#JKOUdBmp|F;W(I04NY5ARtFmlHDK$o9Su1;d5ttW7@ZmLDc zmdx|zyH4*x44Dy91FL|>u<0$>{&whR#moM;ZyQ6lzVxqy_R048$;Lkv&2K0Egjnvw zQ3~1BB^KyGRovc7H-W$VK8p8{+lVM@)AFt(0u|Ky!IKvkSjFWo*M9o&;g-st6YA=Z z^uuU~h=`nHw6aWxts=*_HvCg%v0i;38vb~>Cu^bSqdHM`S6{|uIZ;FSfk^F%xMUWJ zUT|k&?-dU+Pg7oAu2t-I7-+%=W)+UOg%|3nsL*dcpyh)&-0;E1_wPw`Mh1*tVfrv% zmC5NPK_uwFxq~KwBBZYM+(n3tw@D?o7R-TFy-Jxmr+M#_j@ocbDhDzw>wy23X1<;e z@6g1IX9CY7y*>KYXYaY^KI;4@ksvX&wq}52|6Jnwx?;E4FV@ceRU5n5d@7E3v!AW* zIDcd0+eT~$hzr%9?#Ipw(A&^0Pk65;f4&|N5R~+OtOum1Vd>|Aez8A>Jdu5`R5?Ww ziY;@v9i%nxmo^^!u-q2lJ|E_Ncf z9A;Alsf}!lX4>;oQyWO+k&v*DOiPC6;BXTp=kYhguRhf5!}%pLi}C6CQC~>sm^1T^m56<81OUp;!h+fW}muN-r;fHg9>>X9Sg#Y;-8Y8E{+@nn%>;;mnfB z?D?;s;WZ8b`)vc?0xb?Ck0W6bi%>DJt0yi)XBa@nh%_Q0_X#73MV{0n69(aspf~+& zj0tVkcV__HSJKkj=CwSlN!EM9@X8gx9LrW?=rAO$geGn>IaT^^M1VXu!w9M$eL+7$Mbf&JA zhTdZ6mVud*lEdClLuYO$&+3=K@$fAwI);OviLDDOBzrdSUcNLowS(A!gQr;g)tI4P zgaA%I+=D8p*Ge_1ii%Gza#$9V5CKLaym35X_&w^0V<^!%B3gF`!D{i+b9MwebM{QCeEFa^$3&FLkNcK|GKS& zR6O&}BBdfFCB-$+%*dpALba>Cl&>k!#cAm1KKp4STb|l~*P|w{O#hkiRoI<4OMGr> zViJ8q58>9Wo`sni;+I;=5(G*{80(PPMj}|}{v8q(MM&t7ho>in6#WWfdiRj@3FjDu zfN~;LgAct8k^p~hH?Edz>rv{}cBy=S{kuGys<^cEf# z(gY#{kY(gtTqJ<^|xCldA+s%;M0|IDpCYQ`2z_ybI$3|PfgCoyeH*VZGJl`J}kkI|r zboF_%;G*-O7nT0MJ+yfj?RSbl=r1<IiMMmNCyjyHco9LGA^V^ntqQPqL}OW zYYor2iAV?M%*k51y86}%5YT5ao8=bck6e0d%ll%}%(}Ej`X_X4FgDE0tll(*{rLHF z)RMv zW{d{{|j+(c@f;(`aeB(H_X`0JlL55BQ@>m9h=)n%|KOM>&F@an%5nU5mr-e>Aw z@96hW;#yy3!Taz%B)H(Js#~{Gk}fLv*XDJVX^I{8sT*-)w`|^ z#M#E$9hPAcP28|5o_|<>>}Penyt>JMtmpHvy-Cmk&)3Bo2RlY_w=}%^PdkJa?izHdirSH!2R=>LT z#0UP$f$U|0r^|HLeyiiNlO)zYC5}?UwdSP z`YrzYm01&fG1uB#cJFIg*{&(N7L8QLzJ6a+8y!T&L#(O`Rki2vz zptgRcOFl%|cW(R$E-|Wl?GgiOKdyLkGOhQ?^19Gf^WTq>KCH~HeT(4Qw`>+(RJupG zcK_5|dl%>1eWOb`pNcLjIdPLeqyphN5m;T^e@Ar7AbQSqh_Tt+2E7sB<#<-hh$w^O8bsrlu*np8*B}Xb&F%>J9 z^U3lnHwZ&e#kJQ4v?cWZ^1$;__0s=dGn(nPvi1GStWERlrxvHwrAJhVe4|P#q>0KA z3kI^2a2k1Zypuebjq}bh6u;^S`jw%u03p*N>*eR?*NZahW=dME04!+ZByq*D^y3I7 zTq-7&iOQgm#i1c%hJyQjc~w`6FUvk&e)aFoxeY$3iWP{kOo2J|0{RTMW%7|AvZ|^| zG!LX30zTbm#N~`fHPUng$wm&13ZYCCGfMJx(2(?=z!4jOh`V-*0QHo_5RkUq)_jg6 zcN7o#`#OKe4D?I3a!Ol~gK-*(g&e&=KzQemNdhC@59OcN?aL1(E6|Y^yyO@JC*e;KkE#G4q zod=}bvv)6gQdxWZW!nP4Only%?>~|JASH7IL|d3!*=#ghWCjKX`hL7#P5lTf4}q09 zF7ZQ$wgCASa>HZ$V}SM6yg&5ooDgn@b9EpeYqHsm&{BQ!DZ%pfwP_iwA)y$YWL-E3iAsI3QA9PW9Q^ z2a#sfN8;F~2(JQ{ANlp!@2L$%;#&-Nc4L#^h)E)ypMB3xp`7)>A@6-GViNY~k>Bn!5hU~gr>CK# z`|8wBvUhdu%qxTB<2A;{tW;uTNMn%3tw4(XGOp2$dG^xAqJPziTZmS|P zD0-KRHf2+gRl+7S@o3?g!I`0;++~5f6(wgTRB^|fylPYO|qMtD5}#U zo=-=zA6YBqF94hGhrRTJTyFO^<#R4B(`2pXu|FCOq->WJ*AdftylwkCy{0klT6}%T z+ShM`!XWdA3F{B46j*i(B!!2n|G$J zuG0ApZRd&zu4=8@SP0W{ujb-QFEHl)s$)pN%_;h=eLBti%gV3KhED9Ee8+i^zB>eW zdH27&v^&T{M|^JRUKd{@>YFFfu;Mg@mCU}G{7Ur+5F*BB|6WYr$?A)e?PtZU46OC0 zk>Y2k+drFAxYLQZPsUH7i*7tCr+(3_*o73 zN=4TS2>!VR=dbriv6wWmULQ+4wkmZ+c~n&zT@4zD(yr~05Z6ckRjIX z^efiUC+r;TtgLjMiV>!F0OM~Rr}L(ZaBKSLLTbjPmDh0+jfpL>Tk3%LHL z@I#Ix6ShWkLT#N~=l@A+T*ZV+{WlZ|-?YCgr?tH+o&ORH(N#mID`14~zh8Yn_x`pv zpP{wchO3qreRKCeT{dYn;9dJ&f-R0tWd!3I|KYei_$jBG=`QBefwB-(zJyQx`NN;<1=ij?#~v6kJhjX0=kfc` zHuEKJ=nmORWnGCHdR-BZ?;^-NC&|Ha8jcLgqN0%rVn$9*2IQoS2Y>(mJ%w~`1&XSD z!4Bw%$H-#0ckf>MZNkLblaz|B$)iL2FUA{^XThO*8-(D=0l&QR-imq%t76(Ow{l%s zmEvsR5$Bh^Q;siwKcYY^yx$vRi1_e-Neo~E@}wy=nKK8pvlF!b;*7)S@(Sb-W1M)! zVcL8_!V!gAU>d2*7hqUO9XILNn3`TnIY3663iuf?(FCC_!7*p*IpBL)HhL0MnUJ9E zy&**{krB@YLB6g_6pAPQ1_XXoRLr;zAwrzGQpEZBGY1}qh50~IBp+i^-IA)6xen*n zNbwyHG@#7U*89tLLP4SVr=r@{>~(e-k1D(6^=iJ(1iTMA9Rpx+c6P?2dvL&(41(j47($t2h?yFhn)>%u-Wfd@0|`VD2NPfskkUDh z;Md?qQy85``P?9L2yHA%^YK-Fjy$cP5PCe|&h$!dirk!#0pH_gKK|pW)IQ;+_$>4{ z9)wnKN94m=5aSAx22w@_o*8Fy@FblTJBqa+PARxL9yNh3LIt~X?YyWgXs%YS)#dQg zt-QSaon14!R|F0Vi#&*ZQe1F+upOi5n6pCD_J1n_AFQ734Od>XOBWZ(=OI<%!9{3& z+9}`R4Pf=$Lx4L%ClW{o!IUaI#3|H6x zX4!@n>GGrAfkDtAE(k{Q?c7NQp9@VP`Qq>4j-2H3#U_g!x6rF}p2$&=f&tcieOK*& ztYoU+Y1NLNf z$aW?s(ocX2MGer5WLX)`u1^ATR(|@F_RruD6VvwBcHIV=Iv2H`ut(~64czB|{3P{j z)W;C}TUmYYx-zIXJCU%#!wHUicu4(^|1-LcnS+BaamMJa=&@rQIDD<3tGfdaS0N9b z!XPJ4do^`Vz_;h|D}y@|#QC+VY6FJ0>Py@iTyY;%f1IHBl|zRD*%8-beqAgPb@_y( zcU9bA?;SCJX>T2MOVlK>Kt}m*H+GLC-@QxU*475Saq_w_lj`C8Cy)GByHx!B1L^e`>dn?ZZFEysacWI4x4;@_ufj`IDP|lbyNQV6AbVD~t%+NSK@1 zZn|i4`HAVo>z}yF>jYN0=$ajwQdXu4~0S%0b{L zhefJSYJte}6qWh|?t2eY7rjE29{YRd3e8$=9oq6~sj0%}W(-}HH=K9guy$Eq-!&#B zxm*AKod&7HcdszJ9MyhwIig)Q_QP6Va{7={b?XziOvqN@h;Ywm+TOz`_GQ3HyeHOL zk0hT`oFWJ-@xuRNyO?)X{13zV%X3S+lh)e0`c-6kZZY~i4Bu$gP#-<-yxeE`*__fX z*XkB={!`yMb*BGr{j&w7%`WNTCDl)#h4L=Gof>E_BHFO=ME|=r#>j|Z-SDmPHEJ$x zPa-ehSbjN5eoBtjtxq2@K1s-Kp`Y*Fa${xE+IVAP_Kd5y$%CxSAzLgkF6y;@$XKQj`@DMBUf=q@ zz1R2c{T^!_@3EH0<9Y7;y07a$oagUva6w;=@5=3{yPmBYD zo%D)!Ym_bt#Kq<0GSa-Uokf-Bg`TM?le@h}{L?pn*(sqCkpWf zBVDB{K;l_K%_HO{t+`Cw(sOt!NNeTM6J`XTmFCJ5J2#wSi4&~Tmn)E;7tE#o_Wf(< z#n%fQYWez%L8xURVW6P^U15iJPz%-QabimCt2%FPp@~>A?#BhU0pCx*`;>^D%UnD3=W;h>j8 z>zfDy!+e>%4v5F*-7kJx;4sg_XGaT=)Gl4Wz7f74sK_iWP2b&haJ>K^O=gF9uq%Xp zy(7Wis?U1i5T|e3b@kEFSyXXiKI}T$-Sm*bX?s()xhl0J2m<}V-{@9eM1xn|;w&=X zURj~ps^mzi3kRhNbP-Rc%$hxA?fFMHV<13O&2J;BvhUzHvMF@EuP>vgqr+-+%_Qp7 zz)9-WonMLBUtP949m(}Lf?t)v-{>egN$x8}1upzrxiJ6JXsEkPlsKr6sHZ(Tg91u6 z9wS<+@MBzgTT@}v`m#G^j6rIFk!jjVqBJ!%HQ#B=VMq{0wV@g+N_x=CNDlY+)xm>u zVha;xtsIL5)_EVxei20=4)bvRBh)aYZ@4(r);)XTGWp<=6pVW4=tw^9tN13NRn1m$ zb%$IF6lEX3o_t}jZz?(`V7YlTeh-e6iv$`#v_bm%%xH5gp@=Xa$Kg+zbW*MnA|cPs z%@FsunKVGRBRkp!s1VO%Hl!%=nC8QZSsl5>bkj>rZ+-|e_TFt z!Rl0s*qjoy9BofYXtGblNr+hdiDRwMy`PyPbq;_u#IWZPJ&MW7ra$*uDUtA`>cK(n44&*w8_wk99g95lA=EBpd3kxh z{LxQ`_f;zX~miPaVg#<^mbl_y((?p&@~z<{rJjs0qqw zU;|`+pZog-O0xd%bt{IqvRDg8U^-j^1C2aNBKCfMs*qlO<*Xx4&VNYUpjNGVE}1Zq@E_TZNE;7?awLrj+W;?cfiW->ep2??H`m}p67_U}T~ zt4>>{V5qZo^z;(Jv-0~6g$XSZ0MZ{tfVm#n8_)09&WKF@KNYJOT2K;tH8qN3<$B__ z$O=2p)6!FFwF~vVg@HPX360S~$$*|h2KWiL%NLz98VW7pa`db$>T$&FCj_qECf&-E?Yb3+@{Me4x04K!{;sba8h{rKx44E!uAJkTsbkyT%qaUeZC3RS@{=$8+m^=ocX(MZIuxQ@X1XrYINY>FMtWFl*()W|d$+0l9}0c_&RsIGgNKF2;n#$Nb-AEm8Ve=b|LGGQ7-}|U z0m4wkhMYgo0zt*(D}>{KCSLRFKwItmAthV{&!6I49GBEHEPDKUOUm9zb&Gxov!kPy zofxIn;6n)EsGrnM@gmDGPwio+py{NgrClx}l4<=As(TpRp(YS45zVkjpZ1+H2H zjvc}2>9W*4I0LsE86iEon)&*_i1g9%&zMF^B~^Uu&Uv9&25DWBjX6qd^A{XXS0gvA zK~!VR^NJ#!f6g5*J(Sc3QDbHp^#QFhPT5TYsV^Wjp;!@;2#)a(GyLJpLI#7LIQ{Oi zRh!nWTMh~U?lV!!cn^IH#naoga4rz_sRnG z0k0>ak3vrdB7)RzD^7^r*?*Zy$4rHXnSxo=LN1p7-;4fZ^hKp*0V5%3wii`Z6=0b{ z?lXhjdL7tWR1mzHz=dS{&v`#cP9{7XBehBHF8}F$rH`qy-&5AFZz{5o`v?FqYzf&A zusEKK!s6YbYpXsY%N$1MWUKWxT6TH^sY?IFNdh<4SP{l|yBDFA7>E3VuzywAO=XoY zq7(cQ+UnlH-v))PHxqo)`?Nz!=MA~1?9SE1uw{N6v-xV3BCdG2B+$}$MfnpnBop@l znIqv?_$LPBc{67{-!P&vy*_)<1T7GAvPQP$F$Oi?wntc@`o-Ik&CF45jln_JoYm^1 zE)DsCaytiv4Ro5*+ek3#Xv6&VH^~$ped_5y#T>ftFcy3MpKw~{%H<>-~tOXYQ2w-)hmFjQB?L z=S5O~QUwxH)>oVv>fTL0;68jngfVFDWsX$h3dj}5-e!&B&EBA-%O%TI>UDiYW~@eo z{Yl_&5>h+I-eLK;8+wJb5pD8fAj(t&b6w@Trd;Yn@&l0(X?v_BZ)<&)b}LcnCZE}9 zXqaI};-iJgx3$`b?+EIyU%Ewjf^nhD(=u3NaYv9D?dy>`(pu_=G4~ndLpB?Pt(gQWbsLb zrlV8dq+H$<1Q`N6Z+>-x-#h%UBt?f6B274U zgdXlAHPAJJF+NWz1nq5U*Z9$Jui6{O%csL8?RoW z?D=guVBt$pk3Ws`31tHTBCms*#qy9af>E|twRJVU>5;pgmfX%TeAo2Gz~%Q%S6rvT zG<^-)1#F|_wTb9qle!OqBVlJF?gV0Zfh657c9N{1ztD0e?~<06U9e6!n%7V)R$WM{ zPs`@1{G8G)q%5k*jRNIOO=fUkP^*rPS}G6ignR=o@rJZiVHAW^^z#_Pv!unXQ-fnu zIs6LT&va~G^fpocUb8OHX9jIOv#;1(C&tGsAX}y=@*o14Lc@KK0S2IS5*C>iq$pYm zMeaHyp(J|Pef8{&3eT;d)ARA3p>~D`-`XXpK0+i1L%Z9;{7I@D9yORo7>MEr?nK|; zlP&2aa5YmpnnzocDJ3bjlfLK@x4*SYW#a?ioe9tPYJ_>YhZiJndpxOQHYd}`;7aWO zi{WU>d#P4s?CM4|s^w;t#JTuJWhe@w_d|y# zQPEawCrRhi%tjjff2fHdtvUc_&e6sPOF>*0KrS4Am@8ib>7iJ3(ilb097_L7}+x6rS%QL{=iAIP>;L&XxM~yOjC2A;Y*X|@f zhe%s9gU2;bv5HHb4QTJAYj2HH#}~R}sUVK{8+f+Xl4lFuA;N+ z&B{XYOtDrhdOKD8Hu^N(>Yg?})+$p{_%3gsBG@+8_8!Hg7&J)ZoHLPtF#uoE}GqbDe9wahHLzvzlo z!0BH*a#uO{@=K)M!n3mu0y;>J*W`D=4u_^!wCPGL92%>P^2qNm;T(CsU0PFAW96P( zymz|`-rY}M8ux2>@8^D}hSVE!&u)LDN601kNP)vV7OCAn8yh&JCJ)p_i)yfmir%eB zXR^E^xjJ)jz9zEVZefoE>>CRX%=P>n&;;CX9XZqpO%g{Q$S0m|%A7WTd? zeU8U3-}A-cfJE9H7Kgl5E)!AW62cN3>ZH3(&~`vRtvfX}#60SP&OR9p0O%MNec7|x^NEtH~*Y%!JG-{CP(|SKfF5}-~ z99j`Itpq;>N&*3E;gl$EY<%*3j`SC#4dR0SrN01-z(2&MJhFlCo;^rPs`!uog6*kc zMZ(1ccPvQjemUyO#7{;PeaDU+^F@A!GP~=S*8-gZP}LI2ITYW91+M|X;pGiuJ$Uod zB^ppnfxCm8!Ey$6{pQ>iJN8DD++e20rn9%LD7c)p@0Xw)@5nRM5crmjPX>nkURRpz z64iVqMRj~(A3cq|ITITj@g@@%3>5?rTzEJm;n~18Cy*PBvJ{!xK-_RUZ31T}j=|T} zix8iv5QENOU+ySEf_)-su$~Kh=oQHOh`&OM%Gj6jE_Lrq?SjuKO=JlQMasu@Irm4f z@pHV(4RhLQ=I^_hd8Q~x6g(#4ScVYWvkskFBFM&8Ix?lj1e)XZHrb*Wt@}Yb62Tc6 z8C^e+0TZ@9$jZ+lQzppu&Tx}d2+_j2*g1^-V`46aUr}N{qj6W7H;Ew-yT^saEu4A@ zSL_|vRDI>56`1Ex+?v;QPDw>Y02*6pBK+#&E-)!DUBwN9EZz>Rib{0uiQA5| z?n{6Fevl5ZU){ESpmF4lkmpE;08(o*#sZqd3#es~nGLJm{�V%2tZQcz8#>d|bC> zBK^&_8{)Nf&tACuy!TM(J-@wU`>U|18w$^EH=U*1!lk$yItGhtNVag9$#N$EOGNzy zGXwM0UHBp5psxT?;+C$&{yC-rBaLQ~yTyG%737 zr%y6Z^PF_eQ>bcL%itDtB!S`42fD4uoY1p@0QZ}m(e~{@m_L!)z9Y#5se%yI1HvgP zH)7yWucg-fhX!))wgx(A;6H+xe6%^u{P86E5mF3cID(BY>PJ;ta@{urzh(ag5xJXN zEwsk?yf(A0J+YY;LPgYy`o8mpkVBS5Q99wK$#-W8oUMPCY2AwE_vN35+<}!qTSG#$ zi{_MlMuZ-i9E((o#6K3<51HU=aRtGL61O>WIqgc9 z2R+{UY$Q-~9$uoi^S5^$oNC_|WIy^1@dlk~RpVuykbUgHWUO5^=Og?Q3W$bJ=by%h zwXar*(TBED_&j|ewx_O-{xlkH-s}mwMDk}$2QOA|AlDO$+UGCaWT)w_Lh} z>I>ve)p$|m-=J`e3W74Oior!*wKzQEWW|t3{LUK;7bE^|_D?pIPT0(k9!BLbDMzYJ zDX+nL(S6xg;)ePLEB%m(m-bSHI>jwNeaj2x;96cbEHww1q3K+)SXv){lvHvEmz{1*LN&r$10x5&dmGb8g1~KwQms9D6M&} zqqB`9uW6dN&kvu7Wt(oP9XtfTx1=R&UjD=l6*+^0C}xIb+)zOLT47`B`YXWGJ6egc zpgQ-Cy5bKjWQ>1qA1ku!^*@`QiW!ZI)z-bP(>vyN$BcH*O~DF$@(&NI=@)MbRJJv3 zpH3-ANys^c65k$`Cv;&xUxq|<%>F11qcu}Mh)G&zT(399-?vU(?X?08{||?C+0PV* z2j7W}ykr}G@yP@AxXw?=Wa0$svd4eD`S$)?&Z*^>_LHkDAZD3#@~>N#V~aMi`AyEB zbsHI5w@7}B(^xk3y8lUy<6YvlTYH)#@)o(_GLd8zErnP?Vw0jC-%!7kzRTSs^4HaB z`ve9KO&CO_2h7CclNPqd^*Oy6(bS10GRbcoWQj!XYP6QVL|v1nk9PLjTgn)0Yq%`$ za(bh^Pm}QJiJCI<{vs>Lt&xCv9Nu1vLJM_9)#`cMYbJ|_)iDAe)w5)h4z3>edf_}Y z)9w7=a5@!)FVf&Z+Ky%~$|7%UuX`EZvHe`8I$QG({+8v)B=8JMd+SvVRMSjkD^)g) zsPn(CO8hXmO^20^{3%9vGe?^UQ;M2bJMJV^tGeP=*xQAtk#vhMw6`4f{s<-1!|90% zl%l2B7`1iGjLCWyiFg{%fFq^BgsoeeSJB%h!$NG{Q9^yQA6>FTqi5Y5Hn0IT|MAdh zwIh>a2{J@|%0#c74$uEt9oi))TMWu$5tqLW?5_IYpyVdGdIe>_=1Px1ppU`mK*+T> zL!CQynnQ4*75yBeELz4BW6zuC`D9|1e!5lLJ~ODZ3`7Bnk~NjQz&`cj!OGH}0}fUh zH7oyY2~l1Brvz#(9PQ<=-!CmT%oF~)%aQGwasW&B-J%d7^9lp;%uNsD8ar1wJoUv% zQP_6TP{)iLm%_0LPeP$PY)#91j#@M30 z#q}4l3dprcG2YB|g*k)jIBwXx-^KRO9FJB{R^XMf)`nZkCndKPl$`R=BVP&v^47;8 z5ss|n59OYaIwC2qy%Y;I*mlbNi4ii2?l;|)?~!cjT_}4_dr2@XMqSwU(1evbdf?ERmz^c+)N8pmeHr?w zspG{?KBp%xbK$iZdWz}Clki+lk9pgux8+ClT>4r(rtby3dz^^GMMYd7bT_U~Hl2R0CcZ@C!~^@fhFde=yZ`F^A?9-G^|yf#^^Dsx zJO6C-oRhEml#{6BI&h*QM!{t1y6kusEXT8D z;rmg&Nt69^RW2o(d^&BTN|kq4GCZt4r17xcB5nNljmFe$sPsm`jmAc=UE@JR6pA1zE<|lCE)S_26$k9i4`W z!7uroBs70shi87VuGmx#;{YZcA!M$G=0jhu_{duk$RnRaNsNC8Ll7D5#4E^UfkJ`C z6NI;A=wgD$e$(~_YB(1}HAFy~N2*c93SOy3wNTk{F^)SS3rSF=2%3BhwD|@ zuHk4d#PRWtxxD!Ux5$)~QO&iihQ8R*HxklPjL-yroY$fl-K7&4GQezlg$4mJ;<+NL zD_LBaBVsDFyG$}_Y=A4pgrX7Lrp?WMTO|Cp;5gY{yIV_|o>pyGUv9cU6|MKQPa5fE@o3m-aqWW*r$itJ0s(U_M^afw-f!Hk#Au)O8uRl-b9J;A)E|QQfZo) za1z=&@)l?*K}R-diIjvd+GL>e;2{XS2^6%jP#HxqQm6p8k`-?AMA@I!D_q3TfWjmq zJ+y4e96h);<8G?op81VWTLlXnnLe;R$%J&A&<@Ke;@hQTzKbdNJqc*m>Up3@&ntTc zO10JPMv`hPMTKv*+4pZ}o7Td;i|1#|e%7a~rQ>CU&M8V@6?>#B-d#LV#o+qkDE&i! ze}?c(X;hDvl)V|WJrJ{9hm~)%+v^@HPGj?z=Eq#s<%N-|?ziq65|*vZ?0eU&6UV>f zv6q%uh=zpYRdQT}xPjl|7!?$fBvc>2=6puCFi*Igo(nCdF4)W=e6#I?D=Pwa268@# zq4w5?yymaVSai|(14voUEw+1bQ0tOiJeCdDzOmK$qsC3}GfLzkHm@&sJCE%AXk9z` zq1*6n6@2u&kLB{e`yk+ftGq`B& z_nWNLEkpg!oCGAce+MKs&IYJy+PHM06(TX@z6IEk{F3EC552!64V{fhaU%k#$qyhf zy^y^cvmrFMU=ls#{hv{TZ z9Xg+~xDD(Xt3)GU;*L5$&o^UPFXRL1e?GFOxV&(YjNBuGsCR9Xz1tU-3Y?6aniN28 zF>$PU2@B(F^8qoVNw9Xt%m0>Ih|^}8Z#i) z5C8MN5~1aQKrzwys>D@RRht!@?v3ViRsTb@IIXU*d-O3#WPiLygRxGLd$-rW34d&G zW>>w4_xwlS&kO-7KJ4ItY`QqQ%!u*4)gu&W{+OT!qikXszK%YiBO&m9>FYGqLZMK! zIb#A1Rd?su`IY{{PU*M~leFYID4__g{Yyf{9&0oRra zy7uN;c0r7G>W7|w80wIC*s^^=Z}JHLOWz%m66=>_UDGl9y07SNFqC_*Kbz$G6r5d7 z>+;)kDdKMXLyn+@yb}T|gXOaYkX*mZ> z-CgZ2G2Y;28hcU+mC);7BQ8hj*Ga#lTkHKZZCHk6#DCt=BsDMMoZPpWhU#VpRHmTB|{`HRd!RO=XDo&7RSxRxbuotrZ|W3Z(?j4=s7i*A4A^kv>ePAF9cR!2$=-CD5 z+fGH^P(_g(DQ$zhx!~>MD{Pc&?)r+3eP8c2FfaS*Q(0Gf+Nl72GghWQFZ##Ic)jNw zF88|Hg0P>rl8cvkMh&?beisw zTW0l-wNPulTG>sK9ui2D#+g<+y|>B~hI)@UL>xO~=L8T&d;%dYU~`}-(2yw?iH7Z( zlP=Uv0?pp^5h*8@Yi*TfUf`iZsa&lsdqB!Cejv0&OiX0&wCmp9MXx#T=tp`L-ncX9 zNTXL)J(8kKmER7KBNw66jgfT;X}&vjyuy~#Dy&u^c~+~w z;u2~LXJPZhTp^xkn|WCBEbJjdc|F{c;4KT4R(Uh5!8n;oM1rxs7a%1fEJGB@|H0op z7pvj^nF&SOaJbDm5s=RSA3u&+L%nd=p^D2$LCs|6atrEm2Z-GH6%K@U;V0J#!~1D> z#-m4Ry*!-FY~o8Gr0;v)>3Q&Tnrx|Pn3QXuw$SprX7JA@vj+?9sqr&P| zNHv4?6X_`!TY9eV?d&<>G_|KjQbL9uC#M))%HLK2qKx^E*!TH8f=ndDQq;JmGEogK zhdbdh#y!B>2*NmlHiEKRNoK+zqN}J)`OwXqpkJI-TlH%*Gbu|@)$5ne+dOJ`RmJR{ zI5;pZ6}r{r8mGcb6?z(5{Y@vN4oONU^{iZLXFZtjeuG?x!*5suq4^i&~1 z1@F++`@`M7mwy`z6pxA`i3tUl8?6C-n=)udpg`IWo~U2lSK#9jqE>cwi4c|ubgFJ( zK9aGCQX_h)W?HFce<-o*QUY`o#T6vkzkcH0{%Vb~Uep85T+heY5X@tv65%awWEnXj zrNQ*$vPVqPn%^8b0GqL&@*syzT?1_wdq9cM8WgJ;1Vn#fi%UV&F4}UAN0#vPgfQ9U z;X|otz9}@k2(_Z>!`r(WcGHVsPH&aJj%Z5&A{~&9c0B$KR>6VRCx^`EXd^^S-LG*X zQ{l*3#Wa7=31azde&RGZS>oU+}(4n#{@>+bO-x zdPyJXeYUlKfpSq&WvBUU#dj79`z75F3C5jCYXuh^R4P7*=NO`n^)&A@oM!1Dw&Q~jo&KtwG0kjb||u+S;gw+ z7N!xAC4TI*>T4NU;$8}ThQh!HT^CyV=mU_PL;xm?j@Ev3fNFtYgvNn}0`)ftYz$aN zXei9?6eqDVw2EwK8b(A%Gm`0G7+>n{d`p7V9XNN9Ruw=uIJ%KZ-2zm!Hauk*1 zeQm7yJ5;R7YZ_?puzksvITT2gtN#|4pc)2EftHN_$AF{A5TK3WS8;#P70`9^tMHnA zPtB(1@QD2Pz6rHHsWg@+v5q#Vd)0qF-WgOcGIbasDxI~*NtlfDXc^^e9DetHFFc)f zFXu9D#xGnr) zes;ks6<<@Q+?nC$YR9*nrfcEMp{OtJ0@aL4mANA7u21BP61h2y9iIQ)C1}tlh&T@o zyl(xt{e^=NL*FYcmb?S4X;h@24M0 zb*6);#|^=OSsh|t*dkw$v$lnMNdO8$o)g)@>4q;iI2`=a|LN9ZRp0qfN?w)uNNMc- zjO;G3{8;6~&kwm*hwpWMy**@7531i(lft z%PR+&22cktlsF15Wi(P0EJZ!BUG?L+Wh;)fLOsQd@ZL-tRpeVQ_ZE68(W$VXQD)&1 zXe(|nudASKfPJG`+_n&X|zm=GE*EDjk7MfqgCz2+u z#e`Z`Pv1WJ_3kXR8m11b+n%;!i#rtAT1*tg<6~a)ehYU_43(`sJ)}_V!UBI#d3iKg zbnw0;IJ$n-?74o&^c|cQ+s}Q-{BU&O_|inqw(q1UJFDHo;jLm@@CZ7DxaMRPp#7X4 zb>ZnyM*Nhf_wjGb1);l^jpo+I={#^dFq1_m-8gr8n4TQF>tTha7}K5OFG zMemVKiWV+JpGUrMupXU;(J%;joJx z@yU+Qa&Z`cg4Oq%!d32>UqyTF$L@R?Y&R3D^Jw#FVUka*s$G8ZTL<^r$scDS1~#;W zG+1yo?ko>zb2e6)6~T9PWwTLSCiWo9Pa9~sbC(M7H7x@Qr@TJ@bE`UxRcR~;E$PfR zj@7jsB{YtYzB8&Pq>s>1Yd)_pdDqNh`t(#{WbBTpG6r1sAQ~~_A{oJ%q^~Jsu+z*8 zzWBTTdHnSSg`!)0(rN?rc#{9THX#AuH#R{c7_=m$4rAS>k1=OW-HiS4uxsvB2S;|% zEsLQm*BH*Y5vO#p&tD08W`B+Cth3nvw%cbMug1zuvhHeY7JN&uR9XAR5P1{NW+2y% zolyav2)h60jZY*I>(t{>lM?y%BZt>iz8CIXCtoO_x+$;s5xBu-M--?Bx^Lf=uSLsF z^lf!7UW{gdb%Fma#@1&?<~xzm*vMuDLn9*r;AEmX{|rz3li`MPBc@>3v?{Q44=h&s zrfj5dRrmN6a@=nB22cBZte`;whM=Zz{G2mMhtsnq`!*rAJb!E36MyGU<|qC}J_rB2 z2A0>RqgVN6ULG>h>C&(Ax%8N+_`%)bQ`Kc+?JZsLt>8|lE#l9HSlaen&uHrTex-nX z@uxJmg2VrOI+ais*dA7uq#K8rTlBO*ZxvW0Gd9c4H}S+DImw+VpBfL%`ghM#4V^7; z>r>j+Q4biBAC$k{BdsaRC%f_S`U;n?qx(nRc=)wGLpPEi_ugo(SfQP`F+@f^z20Ht zQTCVRG0O7@JD<1L*)q9nmE4%q!12h5j?>50yjd9IhCxg?K<(ihR`q9W-&Jk@GV6Np zVW)FJhrPcwr!586E%cTg-E+NI-Wjkw`|vDo5-AF{#-$Lx$$M}v%=^!!5M0^ydr-`2 zgCqcUfV;9!H~w^8`&&Jr*cDiU0}kg_mJFknLwBZRxOezy!_iYo4-coi0%{DD&%Ijz zchyGBDI2YXF7B$hyZ9Fj-4r@px=-(v40qhX$4jU-{2O2~EAuZwZGPsD-=Nq&C%@5m zm>Ik5bN_AfxzY1^LUR2u$q`WOnUck_(o^h?gRkeP1w*E#!Tef<^vh{1zw@M=d9vEs z=SVM)bN4=%_Pl4uOOJQQKb9@MAe^3`lh7{HG&1tBEz^5=jaPEMcl+x6&b&RmQEmIq z#hPDo9A`Ll&6Ve}>kitJT^cconyE7~3^zxQgbBF6S-bM>FT*dNHygcCSz4`J$uv`2 zb~iXXy`4TMNr~I%UT1?`M$YD9xBEq`56>}}Ddz30h4CU>v zO5I!Qo!QLdpOFjLI$?dxaroZJy=r9pq$VN&h>x?RyTDO+b|YZ<+choshb zoprHRbxlp8^SZXnvp9%NPw8gcP+`ZHtw*l2uv%=^m${`()&$*is+RJ#-ZIJH_@vdi zv*V%%>n5`}=M+arZF7ee@*4{(8~WYT&F5zAU#*JgW=hs!)XbPDa&jm&*J9e(CZufX zQaOHJF6H=fZPQzHhMP7EIzlTvs(pC1naoo@$Hm|(27^&`4Wt|KlW?*82;4$ zRi(k9%a1yG^k|tH@gMi`tdv?J_(+YW;rPId8>+JSs0CkN9xPuH!u$Ty#P-Pd4pz5C z#08s$S6C+JtWc~UCf~~3bcM`{cH`!cVLzibHSHbh{?)NHZ>?@PgS}@2gPY|U+vBd( z0c~2}NqG*A-({>xZ(WQ8*3*m;>KV|Yv^lL8UcAu3;2e6CHp})8BXJ#XDO=#_@9(FA zAisc@xgjIqMHa^i$uc%_tFfwyWyW3P^(2ji4I8e2_w##` znMT>@LvaoYJsW@c$(aBc)rL`v{uU}*? zG)rdNN>{ta*_%1WIAspNf z+cO!<5p(fW@vRb@@H@x7VX+gcbOi_=6j{?Zh^7mEKOv(0GieF`_2Si4t$j%Z3% zSie2WHAD33x!zM=>p3`B?J9?Vo?7QSuOky)8H{`Z^>JM*>Faf*-2m1OjdNmYsqn63 zF2B+)=c1PfF)@lCXtuE{cjuRo$yul+&`?!XRqqBoO=jqVfcC*K^Q%{{+Cy>YV?P?k z3);-O4AJMg0oyMw)%qR#lP>*M+x}7a-s7dLTaEZ{E4&O^^tazgdsSW8>iwC@*m`A~ z)(aolRvtKW>P^jg7Pj(Lr5dzm%pZ%N8n?GaZaP)a?!@qGPC{mT)b@D63L5>a(qoDo z91Xks#wj?ZBSuC}K%7=Ahq1La7#X_Zo-@(jBH@AQ<@&Yy+bUcTacjf1O_Xee0|)T9 zmYbUbOi1>dZe`8+?FC&itEKYzrtUGXA@9vYQ|Wy2d)8bF9{Cc3`4G!9C$$Unzb1e2 ztF$-tFyh~0dZhA(>WcZ(rw%iS&h1}jG2fzwEB(#Y^HXE&sh^XjeD|Y;m|AA{AB+*? z2zKnc2rt4>Cc&w@ZvZ|bDoo6q0;upoKl0#|i9k5bF@0_D_#HZmxS#1fOE&tHxp^I*Ox@gkcgz@q!HS zA?i>2b@!e<>&keTp3YKVU;q2r2rz%yprH>ut~#LU={e!^Y{XUadfjr4j~h-Y)l1pu zd+Kip5oWvY$*}`F!K!#cV^zg7v{yw$aDO;DnWGsov?3Dw6gLmget5JfKY7~KGBR2) z=$F04AZD&j$3GIYqxF_ZdU$K&QoZXeVOuq3ezA4hafzcvEI{w{mQ4o7u$ol~uj0+I)pJ<@jM>FK?R|yX)2Wz0V z1Vsk;va+&bdWhWX!y86Qr){~pxwrF$nORzD%a{4twgQ;h>#974xpf`Rn=u(>Pr(Uc za{nwW!r+{~my|@01x;`fwx`Fp6$q3NU@$-vSTBuH_v}s$=N0F?2TwysWc^agAEU?O-Kjcq0@-a4NT@k1nowXuwje5gy;_d_ZrA@O7VGKjQvbZo56 z5eLUf0R|X^BqY{Wq8|N!`DJ5YKdN%?=-XA+Yk1kCL-CsoY11CX(;giw6sRr+2$U(9 zJ*URD<0s!N_GVZ>h~VePFCxy>B^2m#rLO-u5o2R662>c@^ivLbbKtQU)Z>bZikKXD zN90XRwJodADKDpwS-}oj3~$=~q(Xz59k)Ri@Y#KI%ZiH|f(kTI!h!?+j!5-BOaUtG zquf?zThi6ob_z~8^?XbZ{`t`M@Q=uSSqMI^=fWRg3SNHcNh3?ufcw5Vm(x!FN^|bO(F9|XL&zV<|n?NT~6*Y z-T)e4vY`SQQP{juNl6l?92gp^@obm|qI`VQEo;NQdj&*%duZSxC-7N7Ed+=56@S8l zTnD8!X;p%MMy9C9$jEd!!o~g=Ojw8)z!tNI6gMS3-6K6UGqW1EdD-85%KiPXV!sUz zx~D7LFd!ov;Uh(vF9a9+R5qHTZE$B`vw3i_HMpoU;%~G)KuK`)C3fBE9cqnv0f5SwTC^HWj zhI&Uw*P?VR4%X~y3{I861E-^@S+TFPyPc3iS|Kg0xz!Cmc{v>=#N`oy?&iV z{A2QhkoxI8C~lntFpaqScGk6KiQu_CfElF;`S}*JPDhW%e8@4fR8&$r9~Z~}7@`|v z6BCIycQw`m6)KCdF(9_b25Zzmi%}n+xP@q-+c&sPmPwY3|-!b z!XIrA2^1dO#9S2vsG;2k8l}WTmj}+izYAzeOm_RQNj)^@FNlxwWHf`Nzkex3MplT0 zejA9&%F2REg#E!ygd9t$gu=o-ObuLx{q9BUZq=LAW81-S=-Anx7`GP*pl(s=2e|#Q zoU83e8}BhQGjqC>PV&w*v8Ol70MIAP%)(L?9WWuentB4O*^wh)g}zvk>2S9LtXchz zz{TIcGrx|<&-PiC|CRdaQE2ONT|d94xohA8MX0n&UY>#C=Ht^&elM<~vNvsqkx{sf zyxxu-k!C;f5m6J@SHyhol$4ZmG(<)lyE;xnT2E2nC{wHCk~3(W)+M2U5e*xJFen;KvssD3i`5qOHC6T)*|=7^RtR?r!T z1!2^sU??FS`xYv_`IY)4al8>6Js~2a5rUbWMm?BI`5cc2MHZGR_Gq2T3iOun&|CzE z@%ejgfhVLfh2OZJm&Zd2$iycA#DlNj9_?L*hOBo2W{+|6@`5t;_I;&5BSv3I{k{t# z6GnaEM)$Y`&aG&2%wyEsG=cLNP6%fx2K@aFK_cp(14+`~Le>>zy@k96qnEY`&~e}6TMFu*ix8qD{> zx9I@~DMaf&;TX>~v&ZiWKQ&Us2vvOlTp9{7Oa`}=8=PB1mvTMGdVTVA@-u#F2Pafp zAr(Oh2@=O{Akx{fW9bE5Nvizl#snO^Nl8g2^-$UOxEIA-zAVu!w8eM%u4L1+)oa#} zM-GsF&+AiPrk09BGj|pme8MP!o9)E+E0-^0*7ElcS?A6%!ShGJlrNA&cAnsR*Vrh5eYnaYT21;@U0oH_e4%A!i`m77QjW>VN%Ar)P(XNb?_n3zyshG)JJeBR3hmH&pRSX7 zX}#m|nN`%cLm;EArEI1PFcw-|=WPaUz+8LxIkbAP&FWZL-NcUL+OBJAdSwqlDR?fm z(womAsbo%X9<=b7pK)_r{5{PQt1pFf08`FKI@_u?f6?`)75nky$Ag;}FS3o_>Ymp{ zuVyJuVKhr^jIFJ|=f|g|$tYm{1!jJO;1J=98)jfWrgq=!n?r@REqbhsNNpS zve{VRI5{xp64&ZnZtlj_^Ghq#I}N(2+JMPD+4iV6Ml8oqL5h!rhpn))<)Xhk8Kyn`yR>pl=zoY~{-;3jha-?!6}Y%t)Rsvk6rcNN zB*l5jl61E*8j(HEDhjP|SmqrNg5b6@5=H~gD*PwTK$_Z$%_U&`yxrF1_nKKP4p@CD zX=#`9TQ+-OWQZTWGcGylK)Nam-GS{o!TR3V1cj*5+Syr-SKcN9)PViSF0-vNi1iuh zoY28R$31I!ArGaYCj8WH?^=n79DBVR;GyuGl}t45j8pNDe)Z}VhB47l+qP{R`}s2j zxvaT44*|gA!(zo$Tq)kE<7tW*c%szO+WLHEraX|{?C=dyL4Uk6RvBTPC&DZ2V+Om? zv%;oV0{%IoC5nJ^zj>3?v;MG z>cfzG;eq)1)C;=C5H%Z#1kD;m^tK4_#1$2JP;?=&>VX3XV(#8$N2-QYx}Xi};Xd$E z$P|EFE9bKIEdldTPO2iDrJ?Kseo1W2*e zqWbJN$&DK~+G9u-c&OVD^-ok8* zB7yU6gun+&r6v@AI3Ro%L^)Vb-lq90VN?wRDnl6A3Bx#h&l&=-6B)Z5?!gB;h>)<8YlVL%oEXc{#JL|5KlVc^PCd463#hn+iJ(6)E zciN355@;?j6%S2j)YRG7L99(UhcIKWAURx1$Fpumjs8DnPV>w_!y;vJW%&O|o&HVU bH808?ZJA>iTz;IwKRdSV(#h1aKl8r;h%=xU literal 0 HcmV?d00001 diff --git a/training_loss.png b/training_loss.png new file mode 100644 index 0000000000000000000000000000000000000000..b1872823cf14c04a3965756a180c511458870d0a GIT binary patch literal 37015 zcmd?Ri943j`Zjz&W+J3P%9tb>3n6oclp*tw5TVTTJUm4*Bqbz-C}XBXWQY)vsm$|` zWF}+A?^@dX_kQnjeBbf@0q;KcK6akre%8I#wXW;D&huP%$aPgkDhdV)48y3DuE=R% z7y%x`a3f?S@QP6Tz)$!;F&B9~7fpK$7x!CE=9tPY7Y7@A7n{4LEN@Y*|B@>Sn(ZGo$hJ6zFs#xUwz=zq8jsr0)T_JmMLPDaZk zVRq2nM{{CKYO(dJYG2ya(EB&D+R2o(G^=YA_c;&#%rcf4py16tKq^ul681*qy1&ST7 zcDeIA@&3P-!*EZ6gM%|(-on`79V<(EECl}UJ5D0K|G}XDTi(|CFyM48b>3sC?%4QP z7@u9&=MG(2SptR|cE%2iBQl0=z2;<;l!5F@dc`*S*N$a&UUHkIXIJ`combKIUH-1X z!H0wqlAD}kV`Ir0nf!JGA{d@s$>Y}xOM~||In8{Zwh5FX7dv&0Uv}5uEcXzAZ^rQ4 zXqg!&$k}r)VjY!r_P^06v;kXP0IY%(YOmM z#w$JJxYeC|6Ag4}Dm}@x3XgLgn(VXAtNY!a+Io3Pyg5mVIqVo;u4__M)Zqc|g^I~^ z>DLWSO#uq2)&6^1##q%uX1`!vW8Wj)H{BT}cQ?$$SK2O5=5QzZ?}`rjEH^K&c~Gsk zTv}k^=cjQit9TRkouhNoIfd8KxV6xtd5pQf`FBUAX36g6%&h`F6XAHrPHo-MhNq{D z)|(p|s&2XuK8TH_r>7)gXJ;QN?zg+tdRlUCZMHXHX}mtwto||o3uQ7Ysyg{&Hv{)} z7W`Et*TVQJ$&@17vUT$;IPW%FG{-%t3p+NR=sjPZp5zxAZ(mvZ#aAfoD34V~z*~c& ztbz&a?#9N(e8+JG@xMQxt*UOjl`+DXzs&X)wmME_#Y_21%3i)~{r0$s^DoL``{Bov zaINj5j~2%xq+}!6(OP17xZC3b0s&CHsv@$^7i(2P9C1RZe#!Ob9v>57Am)R=gXH{ z6d2cketi}zWPNmddpoYQwsQwYUNUn30KfCEZ({46f@_58=|6;~{pN1!=p4nWOba@; zBxkzw6xG#fzdSmwIWBqn^y$foF|mGXsckHv8h?$iVx?S47;AV(8QshwK3qY-W6@-3XMI?=Y@y%&6Az4r^^rnqz8*K*>C^H)YrU3@a}ORo z7`sQxM6`2I*Xq86pNQ_a zY)Lqad2H9#*LQmk-2IUj!X)9v`AIEKP-)}u-;m1>4~8`HeKOqG+6v3g{dkp(RnF0| zAXeBmE;u~=h}cV2m#Oda<(t#lvN7fnW_@#RO9RfWA&(g^QM^<=Gd4b6w=(rZ-qW-6 zotlr2Ptl*molQgMW+Bs+-|ezwtV3QiJg+n|<&2CHowgvWabUp@nY^y_{&Elr{PgLQ z$4a|mwCyU-MNGiF@$k~#?(iO+gx3lUL%0I>&5v3qCA_pSn!L)*tao|bO=FKQ&C?j< z==kiccEhD@gyT^rluK8A8EBMuth&0|6awG0GfTU(Wnr}DVI6ET4rPvK!J(mtS|zt^ zivFORFl~6k$|fSB2brY(lfVDc#B*aWUq-2&6-@z$5$TD(6Ouk98%y=f5^MSerVr?a zZDJv_d>$S)hA1+a9jYknD@agHCp&O}F0cGg=%0bsUHvk*9H$nGmW0fKZeKx$R0Pj;d>Dj1>*+Y^@(736LQoCJrH|;|PBLUT|WK_LM@0$IUV+&Bsg< z(~ipPlMl)s&5y6l_G&`XQ8e^fs{4FLF!cBDJ4=6>_;k-{!Ys!Mm<3(Adi4;jOYN60 zp^8kNwd0$vTk~ZLP(|1zByRS`Jb!+M$!Ae!!k3AT?v!2SUyG;F(G%*Gn|_HqaAT%F z8=ii)5k8RG&dK}nH(SB2njovKm*Sop170&ZY+PJdf5Vn|@`Po+a?9>kk%2@@_pj#o z=&hf>euds6Bn^cKFR&k$WR5>n?$$j`H6hgQ{n(?xl!xB%S^2P6g^`?`T=3UtXToM@Z6fNI-EMxV zuMZP2`x3l5(;WssMNwhIcZiOzuAbT7aL)DZHI*dIpv|1GEc9H=a!67(9+VjXE-m;G?EqXY_tUOd1ZnnJY;C4n?e*)|!54=-E{iXJyYTt; z`4E`^63+qm-R-{wczTFr52*EZ@9+MQb8yIqS+LT=l-1OZ!RXjnSfo*zGi^x_WfK+E z&(PvpYQ8wkxmB?~D*FM4ocIuuA#f?$#LW zo}*BFkL)e%?R+w5dhfk%H##|aCM6}McDYrG&&X@)^u*=g5mLJ}vL%uULmuN)IOze0 z8mh4$YEnn8J?G6?{_8f=r6La5xN+ebhsq%c3-;5eLjaD1JUDPHR88`)T*EtV-ij3s zR;9?2owfeDfl}u>KsChJ(ntWc*GcE^uaz20V=vk##)>$mU@VaE(q{`@rZi|zoH+N< zb%sL{wl;z^UOo=}Ho~`r)VS{BvcGPmj(tBJ%1g?NgH~MJ{SXBcf`J8HKqh=e4SZ@MB41~*E31z=Q7k& zYf(rT`TaFVdzJg>wIXa4HVPSLx;{UcQc|MlvpJQXa^r&?o3OBsXU@RA^fas|&sfT_ zS*Sbs@A70A6quXXBH4s287f@pqJiXewcCG}>k3;X&jHYpmzBMLXhk>2;PmF&9C{(T z-p#n1iV6?VfsL0}ReZ6E&=dk~?E5voBUo+dk#kWykQonoq1>|s`6v_+Cq=kI>NcxZ z=b-!Woa4@sBS%7vD!j-ci04bj4;GkI6D$F~K$|^uuK$B~RGws#rFGX^S_m=9o!u0AQAvlMD113prB@L4|hp19l~&J+f>S zNLDaaknd44N14^M`aba(;OU&B_ov5P#2!G!S|4y$E9y`5nm&t@K78~j1#6(k{?Kug<=NBwnf}zyY9zM*D;$vw#J1?~5<)yOwiqXz{ux(-4VJwZl zgTupszF)qZgL=MFqQLM+6q9B@bLQM|t9|9)wBJ1isZh*1cSUGEuPSQl=){P){C*O9 z!SeC-?9MUR+BR*=$3Cx0bfl{>JHmCyS>X5J)eQ`|KDsaPzBkD6`2Ff(Ednw@)Xeh|SXx?|SGhiu*KgKW zI@$7mewFWXASTh;<>|8;7RhQG8)kQM$;!&s0;+p3Xs5e&quisQn}Nf=WjEPtv6>Wc zmfwz-`3-gTr>KIElaqg*a-m=~yj9!(HIIv*|GM&v^KV+tC)f^@I3jRV^N{*nziN^s z4*;TCXbfK4EbC}$$DdMsthH6;@bCldmjSb3^QIVbZ0x}?qs1P-LDz0_rN}70CybZe z$gr`se*5*F1<2}0vD#Lty|*?qy@i(Hd0*p%ti^{KRMRzMn=MYG{DcO*u{8d`Am^yO zygXnN*wEwZntH=d2S&Y8A-5J-wlYQdW+&>F4f1j*$Fe+9Fe_px{`vE^fW_= zU%!R21xfRG?k3Z3u>vsqvqf^UWH6c2j>hrL8~|@mqaFM=sw;;XV%vmcbraoqmTkN`FzuL0^F>{oam9=r9{xKt+ z6?6_8V_{5NbA3hE?g%nO%$a?8v}6!J*oKiw`~3NOrcoHyg6-6)x}t8QY@4l>A8PQc ze>^@NV6FZ9-~c3FSXWP*g|=4#^~*^BNwATn`d9H=s~HQEFSwghoF`il&R|M<@#f7N zHURWF8?COt~URH&CiDgL+2!5 z)xo(B<6)jC{Qdo(Mn(NBdXUU9zc7*IPh@Uk5n5Fx9pRkx;>8rL)$;N(T7>cXi1iHD zGAM)+Rum`DW&`XLF|z}e{Sbx{v}VpMp!??<0lyBWl$MsR`pw(H%pAFQ@7~pLPurw+ ztQOk!DBJo7W&{lleOEiFyp-Kl+_cel#YWVhniuv4b~Al)AM!lIuB73|Ea@YXpn3-S zxOrQ5>St%v)_yaKK#IbyE1a#C-+OW$H6*3BntmTcNw|WHx0VGtP z^KaEe+3xP{mMphia-8iEK#31C+?iKR6c1^Rzu4`;9aeW=YHyn^T`ehhef-i(RcMPT z#=nGLhKPxtQ&(32_!4wu@|Lcd88yHUg8%&c+Z#CkFOs&rz4%CGrqgEu->x^ zmGy)y8I*PEMn8<-w(q^wRw;9#Bx?Sk`CwsX#bLl`IE+3xhSm?mS9q_pojF4aNI@i! zrl-X56sDW>1!~1vU*G8)b{`N>?y0Y~D7>}3v9tj{QD9m}@#b=Ae4S}^qJ%eaDb-?0 z#+@=^^S3sI56GLROBw}qi_0RIurtB~eTiHaVFt?XJWc|ZHD6OYQ|IuyEVEj&RU&oqg zC(Y1MI@MV3$T z6{NTD{QP{eb>{Q8$uu-HqJ}TI{KiA`N+GqoaVkO8GJt8z=A6!3X;>Y*l2H=#c%AH3 zNh)Ok2I7A1q=n_>Tb0q%b+fF+NaRm-2B!1_wT=)Z+Q0zSQx?S z(o#jfM2S$%)OHqO;V!d&V5(>t8N=~-yp_4b@cgds8x3u5DwtOwyrx&H=Xv{knfst< z)x%n=>*xeQA9RX~ivk))MJK28U3tdTSWo_~fcl3vzD;v6v9Wq)FDzOU!=ZqUX6IG@ z5mTFRZk2erz3l@HJ2BkC{%pUz z0s@nawGD8sm3bdDSBfvsvTsO zH2mwQ2Qf-QNsq@7lCX$~&?B1b)-6`}E?>k#c#zLQjLO@#%ylLRf>${~gm&0}PXg#w z0k>Jb60Vu)$;|XbS(sf&9+h5m3v-69uqsdu8%Nr|IDI;5sVnXA@nAPcNIxe%4%@ps}mYT8DC z9FWSJ-~m1l7~wJ3l@y~q`;{mvJ4^p|;oA08MQGS7t;|md+tAOtT@tn%Kpplud%g7& zK7f`R8XNz3oVuvh`RPxqu+z^gZw*Ti0kuKFTe&f2l{$|&0u&U(ep@^`IR+2bhy9uD z%a=%w@)`yM2B?skK+}|4{^zQpOS^n5kfhN3C3jjawJS0U{pK^T61qx*p&m?5K|!ai zVwxG7f7%TVw^9JpxQ%OtuDN~?yAtZ0%jahn7K|s&I$Xj#SB}~GN=n_>JH3$aVws9k1o;H3yQ$v`is9{A- zTWI1!h2+AAFGj6HlBB3}-iG4g@$;!-8&DF!cwLS$f5aek)icNHN5Zi2P-C~@!owiw z`h$$}>R(*2%o_YPbd-^CZpj|ViXx}Lz(7GRvhI2)rTHlTi8{&s9eTn}UK2!Bv$;A0 zyPHZsMxyc}0PhjMgF_vYMHSFUI{=m*1x#y1INJ-XU@-I&)TqEKTAO-$io%irmqt?9 z`)FijL@dIU_htXr>T-vR<|oBm8pf#>97G(TSOdqFu3kR?j8)Ox*4o^%hcdRfyj%l4 z8~?4(L_Idbh;u{iy@2ng%V};odHK;H?*-IghJE{{ys|LN%vZT#8jj=V=BDI$migs; zhE|ryrWLWpH>J<5~e2Zp=iS?kEGD+T&5rP5V^}&*{ z&~yBL+aAE@y*HS<3p;{CE%6cXKB*-o^v7bCfjx55?25xhmB5D&PYDSXbxoF_(&Y5@bP>tu zuP0GaM65(W9Z9qVTok%_X1*^4l48W%YL0nsCaPb@c9$ajI~ZpFEH5tsub~@nf8p#| za$th&R>qTjxiS9u=AufwTeoh(KOh+axkmj`%=I(uL>hYfU}zi4`y^LO72=mGwB!4X z9)29M7rNO>MK$N?>8V#}L6zvcrVk+ga<5U+Jkw-TsfSdD{2Pr|?h8Xky3hXtFVbFM zrZ~~(a8V{v%(c->D!w^rc6QcF$oykneZ9BD`cUPW!D_qJ>CT>V&qAjPW7tp1m%^L+D)edtDrT11qms_f140Lv-NekXG6^DomhL zyq>A$F<&~F)hS!Dw+TC%{hP%F5Q`>UizwXb@ZK7WX~KJ5^+Mf-gn$i!3M%e1{whlr z;XDOvv0Bs4_+((m8`HI|2T82^x6@4sFA>-2!{sWb#!$i|BM)KG&pML^saKGeX6(Q7 z;LWw?I%WQst-S!ZK%@qC8+oftNohxKxjOSTuYcOxX`MM%@S)xp_?snw&t{t|Q$j7b z^ty%m1{QMSI#xv0ySH+RXG+t|x@vFMIIoqiyg6b8G(0rb5<}E@>%>ausgEN97@th?{{r1{!d6|Lt z-`E(Ba0KZF2?JNhD&XQY$B%~q0|a6-0YEv#_@PFdh=_;~yiV#BqEP^5QMvBOz6JtPi^5O2j;kK{vRfq$C5Ww%DfkK9Wxic%H&$YK%T7aQ8bq zC6hS4)b=7_44?7iGl?Fuf30mGT>%et5Q^)A6ZYk_0KI@wWkT4vy%lIYRS#HTMAex= z6IAjm&N{h!rU2A6IYmV?&|u-W(S0DP0acp4&F)V$M~?C@2j8C5|&Lp|uD6BiHSa5xdK6;2@2L!o7+1L2S#>vnwyk{Zy=RD`4lK`4^?@|ZCU z8oe<`F4_DRon-)j#- zLehJG2g$MlAkEVQHw7B54DglbH>e$4dAwjJqi8^|4W-VfF)=|%;`lq8m8Zz;dpOB& z^DK%7QYH@!^c9bViuI_zj6y1A$&(OZ#9r(3IUP}Ju-x#bh{FhGSH7eWu``2`0&pJ3 zC=MKOY?nXA%E3W0H8r&X2sR&NBgE*gPOlMQuU@?}(#?0=6BwH-hBp5ekVPfHqkuLN zEk!Xf_#I-b89BN7uXb+$ zgK{!5_h>!~P1Hhv$`<+TS@hS0XV0F^NKAeQO|=$G2oFH31nLy+clBSnVp~2PLR8=8 z2g`tx1U=R2L+Kf^)wGO^EBjipn3w@5^Qf6%ICI|*imSY-X-b<_5wPaDeR6HGC26e3 zFIYSxBO|+>HuV?63rJ(=0&A>~LwxdL!4NqEW@X;FrPPue$?X#gb%8G|v!}{GPA0gt zrfwja8E%9;td0%D;on_3@d)-wa9mmy{%QxS8!PH;QFPZ(a$|%LX%9$k`d~L$3rqT1 zS<|r?w%7F)5RPE75S^^>h@Mp^&uFfo6@=_MxKnu(lVqfYojt2G;MBy^xPrkq^6OuZ zXbZV8#Af~IB~xFH;(pmebQNN6x^j5=!y40xktU0oewlMf#algAS25`eVQ zd!3fXMYk-uR7+TfeV_s+x8$vvvL6hgNddX}?dM zZW0UqAJovk0HT{Bc~h><()g2X2}d7fi|hw=rnJD{p-ndiiKugkutYBtg8QTO?dj06@WK42ssZRGeSK%s!7E-l!Zs`~6Xglxa#&dZlAPIuS<(ufjaNBk0RrsKYe7EQhDxZkCfxJeZw%@Jq zf=FKlgKz(BFtKL(BTa=~+=Cw~!4Gat#|;MKN`({myPdZRJ(f(B6UC1!Mlji~d}Hnq zZ}Fz%A7LO*?Vgdi-+U&?=l;NeVV#uSzi*|A40K5#y!eH(~Ab(9#V83Y^52n_y69;h!70nQC!UvP&axCE!A?iKoS=8x~psK4mnj!mlWE5y&`$CxRmnplko2h z*Cc2+b#5Hb;HAVvQ!6+3%`=HC4%~Uc20CFYIGc0OUqQ1;OG7jAEY%fkdl_P^M4_Rf zG|&bYh#Po+#(Ic=A-3ya_yphX_@kO2a<8YSCx%xc%mlL`SSRQ^qndE1!vMYOvU}v{ zQ6v`rWwMW;$KtC7XRU?fk0Udn!Hcy4-)vuDA39)#ip>5q7D_@E_*4B-jUUFk7{=|&IcfB@G z%#XurLaOJvdD4nQA(a-&CeZT_n=JyWP3}yw{XMB*3ctOsce!J2Q7DJ8_29qpVz4?L zsz`}ad+|-_~9->MVQQx>(JQuM)rxY z5Hh_0cIjJ{dAF(75mzdgqhi z2s^YkKNu$IyQy{Q3lzIY;Lf3B_FbX%+nT!w+Jb5CJ9AgbHE91IL#Jv2Ag%-kXDT=xG5X}ad-hYJQFZ`Y0F+9YUvQ{@5wKU9g z=El#0JZ72$;UE20&x1z|;nkyjMvvgAhsUG)_X%#_ zzHI{Ja-%S`oq%Q4MJD6?X?*0YQQ?-Kno%+1-zmbg{9T4&srEs zq?r#}I}%sjKYxj$X^#Jd$YyBpZ=bdd?Gis}n=lDCXArQg*UShqgt_!0*-s1F4V2Jc z9CV>W%*CUVqKBZsfjY>y2+NrC?A`V8eClNCKNs(eept1qcWUVQ;*mJc+}g%>?yDD1+N_3*ad&Gjjwae`L6j z2439}N+}29vp8mJ%=gLFNSf)q%LSeHQhQJTrn9~~#YRi}lzIuME{XMl0B}1ILB9t! zFG3^Fi6c=lF=Jo_tpPNmoi~np##jk&Vdz={L2ns-={_W5WW>a-WaZKe63ruMM^L{2 zwjdVN2S6V#yy@X$$B%8YkV|KLVybt#>KG*idFN*xq;Mkr7-ZW>&B!2waUv2bI5|1l zi+Q}Dp6RlemlyyHGFTbZv4Ef)Xpm#)Qi@nmgaBj3T)E!roI}VPtfmaD;>*gZ5lYQ! zhh&3ec;&3M_4I;4Oa#JP(0a1#@{2l;x%wTw(ld;@tKna6>`iyu)N`<)Yt+b#-?4iqU<%9A2As9(#y zS+)ow%PVM}d2R}d)$0-d4q(UbDRk3pr8tpZwLKW?;*$xb!4t5a=OgyCwE#4L(@JgD zT5Wd?`P6it7CpFzs*5c-VKQ|jd-vJMMwCITaQT3Rek$)Ths${ZO`i=WB|s$jmcHKhvQqx_X~R zS~Vo+!8Ligjx}=D>sHS18zN{@h9*FSr$nF_nN)hp-1+?eP8>sIa-)FX?qACUNneJU zuAG_#QD?9KD;ql29RXSSGZ1_s$Y6{*J3)@iC`EAwsfeL11+vlx&W;R;+jxS1-eH7ZK>oK@`^paKWg*2;kgjnQ{&Ek`F2vi{{9H&@$_&{BiLb`e5` zg@=arFcp~Gk#+`VoN%D!yrRtDWe7I=mlJa!5yIzK zzVo#9cXk+<5Q)d4w;P^_F+aD``6MOuyWEM7_)MU5 ztiL?Q{ZpC-QeJb0_V~i2LUE5-_j6#Rr#rI|xo-x3V8o6rf+{oyCLSUTdh>{#(@K9n zs>HOa?fvOvDudz)5D|LVRk9FlWMRSIsIi+#0P2D}votGy5z{RaQY4_Ged!&^-dT3M z_>EFzAlwCV8-p8{1i04%P+S!lv*ET8Fu8j5YLr102xe4V(G+k?4|;7BTwDYZx&e|% zwsaVX@*r4R9Ofd%i0LjnU4}l58k#bo-N3Zz5kBGk@WuyFvQuRuE+-;HGmwfWmW$>v zLHW}ODF_1AUEDTc_yAgb<5#@>lj!INz(!o0_CQsfl7tZ9yTLMdenSwHxsr&JEAIVW; zb8*-wAXPB}wR=!!i@#4U0C!Ojq~a2A+oP~Vv6tL)>&b3gi9j}go%NlRpBwKBiMin! zP*`A3;0L-FNz-Z)tA~+M(hyQjxLD%X+n-T&g;o$%A9Q*I$ulPN1Er0Upf*ZDf(QZ< zZy2KNA{rYa=wSDgV2GOqM<}8_k**H%f7VdM-hcSe4WQD)p^TXV)r#m{fVCKgm}>;~ z5Nx~x>}-~H_F{*on$UD<e&tVkpt98c4&yqM z)u0U9*PKOmCy5O&bwPNnv(RAC0n$DH3P&Im4y6obJgYVV%Qt>=V=-zHY9SVN@7z9vs{O9F5=()yiAn^e=1AOpTGyS+2cI~@_ zT~;0ZCk!d(B3CYqoImjZY+zw3Z-l3k^NRStdsZ^)MGM(BZ9Re-_@j zmElS&oBaO$Vc0lOfjhg7aXfcBft^5_2|B;<#~X4EwC^on%}Uem(>{@(aB?uDjt$QE zE3cqHfImc#X$xO#w7%9^)Q{1FWB_3wVSD`eaUisftoJbKyl$|GCPj>$eI|22>*m47 z0Npc6VfRCy3EWhT-G>h@jv2=xp>%C`z8WN{V!a$>0!|pFexTIC;g9#%L&P(=`tT1h z*>iFDLQ%57;NJ=A*`1VB9d_?u$glu^aJP z1Y)g-*fxzEwF0@WEzI`WKIlk74|qLAxwXeLxeg`M5Xf*aYDJ!-{1wwfV<-IHVG72P zA!L0)2yYwi%|L*Y&WWU%)fG4ty93b^oDl+;Ez0{SS80mQg$`pP~~u1`VGl{;GZ0z$fjt_{0&K4C+QP} z+vPg6;5EREoK5pGEVGC(k!udXwJDaS-*YrWJvf^ckRFm(UAs8X9;ae&-*zeQf>+|8I2*y58mZC3DK+(!okOsFJ>@qOFm*!hkrYl8WQ81NbxiGNkondqOFWx z(C#J8KaX?zw>b}@@f{~NH2?CvEmRV(kHQru1P^y{))%Yj!p{G{eoVb8x_`hI)8D^S zm1-DSc{o9C0!a|cy1^MsXtJl@z8S$--lmX?e5j?c>>RCzn~3;qlFmV@%~(}^94xbxt_^5OCU6E#g6U+K(z;l^tu+m ze%-`m8Tx340wg@nuh*ERlN$~3Z=n%kOHlv}sy=_$bdm%^NYQqZLI_1#W#A|?e-r+WB7PnK^)E#M#gPUTos4xSm!7f#c9KjMF1i0Z zT0-2!r1{MI-N=6UWm2PiJe@NTYv#VGyKzbzNm5`U2zI1UgB5b+D=3$havqcTq@1Uw>*=S~JT~Sd%N9ZbsCP8+og*T$BfL5hNmRcyfl6>a5Vcrb% z`X2TqZtJ*jT<>ImS+SsKz~W+frJWmET7rH|!QX-PYhY5F?f=j^C8^XJ8jEpOjuIe-59j4wJ=F#r@L0uk+i(vdM4{T8?l zsY-*suBWLbJ%&bVVzM@bu6fsRAnWHRhAmu>qYw>B4uQh7NX7>_6`6sMr5%}Kf+&xV ztxAIb?RutoQl&u-$@S%qbHGr7+jD%<@F{y{%RLs_1d1TYb_}mnR&Ye9c-;bx2CVJiC|!L4hOD9s%Mdc|3dP;;DfB$UG>UQg?icve@3TT)+=L{3YhDsB zL`8thdQ0+of2y?9Ag?HArBbt8ca`{8FSD58nmVEsCBHEGTxH^#eQsv|#k5>ACTaZ4 ze&(xzgv_7$SiMR*S`u$ag(~_quqqEbg=ht@iymXqCH$Y?oDS7ZFJ`D(#5bhL_)o;5dJa4HOK*NaG=>_4Pn|C?M~*o++kZ1JBgD8#c!(KPfD8Gd4r&Lu^x(JEy38m(2m1N0e` z_~ZX7pO*>zLY7w-ra9pot7j=%N>{)Ix2C;RiEO{$rMdl`08^caemP@` zZu1;kp6N@pf>sVZ{&>-zQ`Nxq~nrh?&r{QI&POj>?pwDk1Q#XgOK zlZH=^=sa9DJo-P8^eY!fG@VOKx%Q(v*|j3U@=qxnivG?=Xb@b>#xHlU(@Gj7_Y+Jn z@T1r7qt~<9t2R|g=B7y-{zcRYm^TikLjyTgtMp`#VnhA@0}}e+$3@m}x2?XQ2-+w2 zw3s3GT7dmq#iI&bir#mZl2*@Sr~*{>_U z8r)6!Gty;+iT_1tt`HE82B)#E4+X ze#+!^rvizh2U>+Em+j7-6FsCfBZm8nUj{wGb8h|nD7%8Z*i&qK0t^oVVEnypVc|76 zY*9E0e!DY>ZbFnN5I6N;>!hKhb56Wwbjenp&cSyJBS<)ac9m87-6;aW=WwEa1de7` z!D(C^2G*HcREI$88iO*PSG)ZzlqfGM0lFB z6Of1;p+wP_t3p%-$BIQ>s=`Ei*hB!a=H>OVdkg%1M8>|WM?h*ryc=TY!DlT74y*?Z z%$6Wv!_zpl{GerwTlMt#(RSAa-zbSc^ABJfy$`{9=-+Xl5*I%SW*8 zINSM4|5q6XBXt{`7tZLkGQR<*I-e#c9*l4PwJP~PRnxf}kO5BLuE&Z+UtTAw-`egw z#7D|nW*y-OxROi+KN2(ndJ(W65-$h76+IsV6mzXSGtk6=zcgu%6KWLJk3p^pIBr=x z?VoD&ac@WF}Em7hGSP$~n9T9mWA9i#UyoFzeD)vRZFgpWL8R|-Dh6i%et#3Ny8 zj`ng_W?lNdC9ry`svbn5B^*+?)%PAAdIOK0c=v9z+eiiM1N!>8l4iv9TVETRt4mA@{12H^USOh(cDuMGho$kmqvflzaC(HH(So(lv ziwHXUxBN$zHfO*<^3y8D`!|K~_0Id!Z-ZvoMO-TY>dQftGCDd(D|a`oVUJ)KJk=o- z4$!}PaBcEL)Ys&hQYL<^_MsM$^nRbvu5VCI(q>wKh_cRNDWFSut}_h3wyrhAJ020j z+Yq2oC@k`z8UbvBPHj!B16%Y8jshSR=;`z4-SHb7*;^Q#ELc==kF9rL{ zY73TC@=wF5#z?C&x4F+?NwtTEA25K6Lz^VNg>q3 zLDa1dI!J2=~uQc(<=;3z_kHGE( zJHXLAT*s6PCy(o@<`ziJ4!!2jaoGKUfatjDHP!B54wGajcK%5M^h;TA=MI3Mf#YF7 zA*&KEd<#6Jm|?aPJQJbVVag{UjaH=!e-5(~kF42GFb$W(x-mF1dyFBR1SNo@+W77}Q4V8o?>&!Oy3csk~Iit+4U`rVtNI+~qfoLIhJd zK+9Lw-MJVZVRc^KgAn(MkWvmj&!3oL5TKyXc#HeS{VwI(uR!&BX%z@O$u?5tMT8F> zLY@l%{kK3~zjE!`Wm8kOd1!TFcoWs`eA95iOa!J(giFrvhKTR))2G+q-a#eEgD8}u zIeGG}%>hu9K;uL#cCpD^pv*fG_3;he61@Y!T<`-9Fj`B5ZCcEYPF+?qi0#SM~l?qnUMW+Pz~{5Rp#c5u+& zXiMo(O=$dgqJb)m-c~?1HPXbnv4N9Y@qL{KZzv*(a;}6CRb>hJXi0_75oYKp1yh|L z(^EbBcrxlAN7F=2@M^PN_#HVtCKgzdc3-sG4^?0yk}92=LDKwJ0n%9V<^1ff1rOeR z%)kb3q!y@DNq2rtZi*}VH!T8K@gcA9GIBk^jmiW{1aGoTF`LtIT7(V%$O#Jg)@KQ_ z+Fvfh8%cZ=gveIAaoBe!y#tWLI)>0?=hdrJk2^F{ zEs(}u7dY~5^zE1h4|+62kt8KPCs2o{x`IEnvwhKGq=j}D!v`GSM?Z6ngeK=$l@2M3 z31P{#eT!JnX*LXVU`fr;Z`ubf)bRdTo_o%Ku8k{oNf3{D~?SFGjmlnImK#L8k zRDs-n-Ok>F3Iu-IB1ZVh;4+f@jPg+&|2JdmFr@>m_{(pJVYgd#!%ficadco;BeekI z@1wQ83vk=D!`EpsTPkWi*7hdlxQD+OQh|f_-+B!MQEx^mR;JT|%}Z$Zyq$pQNa=cj z%ReG;r?V}?@EJ}x?P$KGG7Dz1o=HH~zKAKo+Sn4p=u-s=FQzb?L~vf{;35}2Eu3(c z(|$e$MyYTAt&BXL)vls zu?iZjjrj^g_caUWhVYO6NqV~2fM+CXEix+5TP`Y;_=6 znqz_yjpyQQY%sw)osFR^`;=9OBrM?ht91E=jyM|3t3DQomGw}gYu?=dIje5YkuPgF zA|Uo@E{SiUsl5~tH<3jB8Fkg0f|ORD$@Hvn-$%Fb$DcO`s@tu zV)5~wWbIVvI%%2~#C&$_zR$!&07<~}w_t0Uz=>NpETUO4zYfBqbR;`Cx#tIt=D!eh z<&{o$VjX;yC$9 zYh2a8TUQ6B2z-du@Nf-exd+0934j)>G6x-cdK1IKxQm$M50#6faGXKD{xM^#+1>cG z%%*BwqBVgt;#<|XzW@IbI@aaG=&a=8qQih&zwKmdY;&4u9KKARgp*K4_1~AvQV_U( z_`jpbD&W4$hb-Z|7icWhAhtaAl6u@}^`O&A+m8>J8!gZHEHs2@LYcM1v$eU;h=U90 zAmB4S)sAgz5Aj)mt90v_Iwvzds5+#5+ z#dtD2c%|V14Gj$qi#>@?pWcIiSZ#r+mLBf82IK>9v%1&zkWS=J3+*8EI~Z!Bj*gRKmI7@UH6e-L=`ECDQ<661aUL}3kz!TzT-9nPze4N ztD+rrv=P)xPaE&)%yc*w76|U}>@RT=6-Lw*M#Hlc_MD)9dDB^ATkiP)R!%lOXlC_R zK95S3tu7239s3qU&l*Gi88|S8vUtkt*FsyxMT9zx!4yw)vlJ@5=pWRYC@!6@g_5rVK0W^%-LN zFfd;mdeic{Q9G%FUix(xG6A7!J!3o3dy18PAwPSheJ;7`IUjunZoD%^9qS+X?w-&# zoqHsc{Y7ZUjP}IOxQ}(y4#JyyHJ2XZ9H@xCo+}W#_7~E3!w1;9rX)Gpd?ucB3WSOz z3~fSem@M{*slh52WtpeckOt}~4Z53jhXNU-u`6@52Qj^hi>Z$#Q0?P^BvZG-pi5<< zIZ3EIH_yPiJhS_jgNJXwB%%BT`Mxh+hcNb}HNY#TIvvto@wgu+QTxlr`~~X?1u`JM zhv=<_(v`V+D-xr^i-liNc4QkyZ%)PERT4>d5?uUJv)^S~%|MJhT-y~f9O6UDi*F;;W_5!L|ip|&@$j05p)+tHz5}#$c!6N zy`B=n2hd$k*1jh4^hOXE!&D@hV50tVz@>9VV*;F)e^~Q#3eaPVO@6!wU>-VjV2+)l zRjO3-fJY1~EBQsZ{a^+ZeA*&?xTv8yox?KssGFm1w-4U``k_!xUB4uolRwCt0Y+&kyVLoN%qVt>;3-vkK=it z_kXVEb4x9v?*R~jFJP>sU2tt@TEj{02!p#wKL)@ zT&}}|vKY?U;bKBMKlh2SEf_3h`BCT!wHQNa${vo}tPM|SMH zV5};V;xQaX%dglD@9Kl^ykliXZO*5jGi5+NXB_Ny?9xYQ(o2m_C|6QC&x_^XJ+#t< z-{Uf98LqdSo=8vrm_tHyFB}@|qg{H=hkx>&ZBXbPdR3}(N+wJQ!V5jOiUV@?mckyql zumm>6v{Vy}*a|4t(OVJxf0VIM)Te>vMvsu3K`;V7f#(RyIMWd}_=Kk=8Z|S{RJ3)N zziKBV1n(_7h#2Fl3AH*}KG%@wWFv$Wg}K%|^?@82#l4_Ke*EB@LzKjAC@N$F@A)a8 zp?4RpB))w|=36-c0oeDZ z*PbMN1O!6)Qejmb4tax8kS5LBuPRgeabu(#dPSMX@aVHWs_nbnX%! zIF;(>XG{2@AWe1AXg}F0;9^3`&Hl6Skw4zrSSB*GDkH{7bx6P|?^*M#{emXFf#10} zm6(9LIk6wWic=I0E3oYY?n?h=ypOHu1y|3%HD8_f>%X6mJ zFWWKI8flC)w4+8J!a=qbR&~58nD+{0Xy;|s1uMGahqtZnLK2uo%Eo1DoW9U}%8+q& z{-&Tb@}ZUBMY29j61(`^#7)vMrQ6)TZicnF@F3&h(3?#(rd%?4_%(UEU%aYfm!_Nz zrLo8OcR`+#Pa8-p?(a!maKjUNDXnSfwr|Mvv}%IXh?@Ck>$8`geRK|krCYOt*yP`f z+ADN$@9wN*ubzdZ?nfc4p5$pX%FxM+r#@)CmF?<0Ddo7V^Ky?%TxG*`{N46|b7$lW zA6MO9pu2x`Sgpc>Tnhj&ZjtDJ>J{E(Ri?CbO2c3iqA#d`&sx}0+nm!_^1534d#LaO zZ$DIv*Ys)JaFD&&5|ROZIr@){%gzThg2rKxvvBb6-~(p{`hPEcPQR*`VAV+{>X4(_ zk9MolVQO`e3Tw~`dgbM$4PSp_g_8kDf&R`Jj7bES#eH=F8>}0@?=ze<{>I4+h3$4< z_5U$u1cc(uLk71Nz#3MAWoQaK4~E0^LDFLPLUbtB7~{(F+S`)1Z%q*8lw?f;g* zDpejL#A^gZjCl3?y?ZayN~lXg1$_wYHN!}`zd zRp>oBI`RS(9a=5U6J1XA1yxmpb6q1Px)c=cN6FJl$GfOJ9QjLfO)5(l3_q*RP-J4S ztG^v{A}G7Emwa7%wvXUJzP_5_n`6rLV;6?tjGXi^?c^u+f zLQal(rM2wds5O9|_V5$!Ga>tflq)6o_3PJMDTnvf2zL!mSiWy8ANu*C#GK5;Loo}V zJoKni4czW@dy4=(y0!5%4GmW`?5UZOXN}*uk(vj$9vZW{2(17lD~Nz_OoFzV<6>^PL^&3Do_=X*Dt=qs8CUN4We+Sc!2?Qqp!!PZBs5GY!F^`AySy2xuI+)YTii@rYC z^}h^yy-B^}6c7Cf!zX!@0|PN-)WT)m;-ar#a#IRGiLq5JZsP8iFE37#N7g4Il^s@f-qm z7&Z#9?}*YZ(&a^M!WGaWg^u2>|Uc)=i}k1zHt<+Bg6)+Le-FV5r!9l zTZ8hAhldCA&Bbl$yO?_QFjk>H$*z$?1ym51#^2tGe(@M(gi?|TZ?XEqHmECAF#0ng zQOocVZ45bIN*Hi7UNQ|R4bWa}!}W43?5x4C?+i!);Ot7a~7Vzuh`b zIzLm?M92JK|3o??e)mjm^`%MiYv22meoN|TK3a49Jg}p7=9<5NdsWui0`faO3(PWMyH1tR1I##9#EbnzlZH)8J9ChqM-lkZN; zKKo1`FVtHfM#<_jjRr_I)^H6hySJ^EPaWn>tSj&HTzsQ~2MJQtBAu&>Rs;4c!Kv8u znV+BhT7n;qG&poQ_Dk8IN2+(iA|89_o+TdFTkT@tYGGRCyuHL>be@ODYZ#qs+!PPQ zH9vX$;8k{hG_kR)_;ZrvqXn67tI;ZZDZZLpd1-(sE9qDdL&|0?`b`aOg=c?Npi z8+{kLpN5(jhG3k-s!XO0iCi9;?!?v3#B)gTR7>aUU007YugaxvIA4)RV=H2z+C@g0 zX3tvreA2!N* z?x8PXYoyR@FXAwBPtC6!9H~Q(F1DZ$%_tHsdef*&qw6o6z=-!f#n_Hsh}*hExb1EY6d< zMRHp>#;>26NCu8&2Dgcxd1RNygMF(2O``52U^lLo^^w1QXqAGoPpI|uuzgsp{ByDJ8v{;e)wPh(SG;g6{;Gq3h8J0Gq@M`^mBDrIN{i?)=DxaRd z9w$NvUa+L=7}~P-FHTfY*ml1*qULC5WRy$P?yjGL%x z!5d3EK$!=r%AEvhQA=;Db$58{q;0bLFPuT8mzLjhk#I;bv5IKakRB|)s*u4HZd-ve zwDOsWFtcY>!vx4i2$+8Eh8R>V&#_KFM80n}M!;}(!Rm80<7hTIyZv~GTY@Y|Kdk*D zJHKx(-+P!67#_n3)t5{00f5f}NND+Uo;aRr;Usvl#t8Vnc!T#JL9TTKzKFU_b zET|c81v(q7r!>hI0oI^Z){atL(6h;FEqpl8tgchu{LXVzwXOpt*VIXc>NUewsfjMH zcz2HqYv@*`idu(*)&7%?jBx$MdOko#c=%`vpjdf7o>iMY(Ja%YosuFBlopp@?CI=m zmy*meiC*L02HqJ_&(_F?``zZ!S6#06wig9G&A;Wsy53xOF}ioHYAljquoT1^e)J*_a+!8H z!n5(5T`09jt4ch11AJ(ho?V3(f}1JqPh-f$(vc`-@>rsd3pNT0TdgNnT|19L$k31! zCfZ>`C8!&4qgIP3MO`trBXUma8IM3&dHuzuFSKMV#3uZ8Gw5=~8cXOGW^&qevWZyxM@Z5j3=8&|A1>?RM0O@I6}qFfz6Mdr#*hg;I<+P z#wPj<7M{Yjs=u2R%2c{EgHV`ibwP98JaTvoT5f(;f95AW?hWvie@nwk;k4|nHccM4 zO_s@$%rJPkYQ;Zg=7+OUArn&6>D{%~E+erl3-^5Xbz;nq&wskk%-&FdLv7mEY+R31 z+j?lrU@5_Kx7`j_ZFa~ppf;3cx5GV+itqlId(m8f^W&E?DYKYaNB!0x7T=at-cgv$HR zd*kHk;~o0Sb5ht9^luNy&K>PIdB+pBK+lJGjk0vL@^dj6-6EkEJV)mJn}om0JQ=BU z^?vSOT>hf|V(vCKA@qcr$E>fV?`oEDa)v$G>?e8Pak#0BYMHye@xEtri6rPdT;&U7 z*sQ_tJMFgU(+TDc9p^V}|DTk`GvSA|(p_Zm`02tyzY# z%vQXN&L8S!eMD~7Hdyp2&6@{UwJ7f3!QRZ-Uwmg3I6Fc%7fdy;5u!^I!z4RQt^Weu9THpMJgNV^6`m1 zSX4KM9^-`JE|!T=jn7cFT&D4B8~xz`*NM)R>4$gxvYfQPA;}C8Hfud0Y*r@48-vA} z8;|u$IBlaE-k|dWg8ZiCTEq?5P&gTNgQa>s8*UO2v6r}aI_i(DXOnT6O>3}g4OTm* zE{6B*F+%!EABvTLU*T>)s2`?a*7xh6;gAen5zO_$BbYXs7LAImPM3AU?_DJ=)}t_Q zsyOS$IDw^zLiSJZv~m2k=08LY=U;m5xWG?x>O!gNwwe(wzBrp_GHiTBE*tN1A$AHD zI{?t;{&eI8y~RZ(>#6JV5nOn*65eoWuU#%c8HGA&oaYpJa7=Um%@%Dr=|aO@X%Z{H zo`W?_iBJ7cxFVn>%7rIiKuDd$-;JA5C%$g|$x*n}fKz4v(GD%|>T|uapejF%1@y+?z1)V(?$>d2Z zTh_nrqJ-i$*lMv$OyTae`CpGij=j|lcRB&M(IOhK>l5sUslQ1Ur<`ziM@wWkT zre}R4mnU|eDar?OSnqMQ@`xUn&yc1|a<*DOOL}v3kMj&&xW2;| zKHJG3Pbf{8+;s_}ZGE}XU-&4Kzvb(cIWbXsz5B)}npNZQRW!wr3E5SV?tovw2*$&X)>Dnwydh=DUP>2A&q_| z12Mq^672C$+e4Q{QQ2+BNT;s5+bw->pu+Ij5AVvTJ5FZ&{45ZdHv3-c;BS3q>$Fl{ zh(t{UfH+Xw5tKO$q_%T5=5#Q>4MW+mCEO;LK5I*D`FThd62_3QS|^-I%b^!vek%J` zlhmMWidJ6JM3X5Sl$UDj+%(c^V9L0v?@hd0=FxDarQS|-#>rF8#8`v(h=ktO#z(Jc zyI0fjC*ckNu<9{n}7Yw+hVlUjNAn{V*;XT8CB2+hOiCw84h6UNl?6bk{b zEg7``%@c2NskoPI=oI9e)!j8+WX2zeT<8XMbo{d0@_wR?z{T2bnSKvT*OhNNArS^K59HT}sqyX*`n}w{92SQ@z?PMSb8H8u@ zx9mt_F{JKdGZbPym6YN;>)k-C;j+T@~c@B*qsRgjHirj-tzg-%0UMsl7 z?^ncs&NHBl>3qSAT3R;qs4 zu+>EOB$`--=9-6QSuvE8(&E2rXq^1N>8(>?m26!kS4i}rc+uwwsD z_nw2mqOuHewl87FODTQXP@3dtD_M#-ZLI_g#6SN5iNp>)6{!tnAxx(5&f^t+`H4|~ zT(#sN1Qisi#MbAs6fVa&V@1mnrZX2~W)pDXd#s_pgwQ||q4RK(SGe4Z2bS<|@)k@e z;N=%7X5*VRC}&{%yO7e&lXAyOivvRd8M7UUGHhR8tCF#Oj|UuTd%~VXxXS3M1RL-& z)4iP}ZGr*%MOqZwEsqV+1sWvmjibhbPtg_cLN$z9%(}Sw`nBL8Sd|riwJ04ahICm$ zJ6$g1rtzwjP9$91N+?+8gnx`E6#}K^-A_L@q-+$xkBAIJ`IDnj*eIIu<7meLVsq^ADzKULVlj6JOSa^t(0pnrEV!i{78v zt9s=+t2vQlR-gQFS`?yM59U-!1-O{}RWi3ePiLrB6mKn0x1y8GUnE~G^i4Tj72S)J zu!h3|3*26;Q+VeLkRF8j$9v#>d-^Y}!A z9G_g|;TBNYjSe#}>x6Mu9CRD%=&2srrp?GdEQ^vYdjAl<(!(VB&*D(hB(Q|(u-{2L z{;bGsLzzE>r>Zx%y#d)AhLy0$2$RqR3KksJ;;+X$PkI#2RvZVvw&3v17HHXGG;l3w zRtP0ZQnZk8*y8r&A7MzvbjuIiLn|Ha_M9fop~Y5OrJCh0XMxVt z(N{rt;nC%n;=fm5@?IQa!&k}f5$AnH=H7$WkBBwn{^ju|j}iXs`kRNX&`2|}QZg?g z>Cop^FDGUEz}E?>(8Tr-*My&=J^ao66^}wI6Ofxf+IQ&1@i=j;6E?I z$fU3q@o%6RY*r#uJl=&oDqj!2x~p9u`@w@6=WwZN%bk?5M&7il?1mW!TGPs>;94CW zUkXlWsi@>x4dT|>aw{7ulSC~|(iG0w#}%{4#1zmDdD_<@5rI`9L_v-YePQdBEm??V z`C5%n(!x>YzT8N0`{Y5!-GDxY^0A~x*HnBv&YQJMnGuuM`>$dHOt}M%U%wIxT^p7CC~bXVM@L*+UmMC)4&IN=Q-uLoBax15d0$9V zzh%MoNyF{g3mH|)k{6(#uFm^Xrpl1nO2|{Am%`_v_rwsaBFI=1n=!qn^~ZcKTEzP6 zaMKX?S5KiFA@b}qaBfj96z1|@9)m;(;{Y!Nm)l8PN)@FO>tQ<7=9xj`(MQ@0XyP&B zb{j~FAcA9|pjfOK>7tM1?L9iV@!m0qx-sifqvw_0z>hdG4x^7T{WKW z;a|IpDGRdIPo%mabjr}{n}vtMSrku~6~lT(dOfT|*DFZ^hWUX(7f})f7g!5ku0WD1 znPTwpWQ*E6X6Js;9m(`KWcGH*uR_OA?=+$-t;PnmZ0x!gR*!=VOzZ#kL6lub)^%`^ zM?}EJWje}RH?~$IusN6G{kHz6Mg5XBtWlabNjpG3MBr-uh$nIG+;57&ATLT`@A%gkTDnD>nRdb1h<7)SArmBL#_4g!km=wH zJHTL2DNl{4$rv|OBTlszOXUTa@G zaR8Jo5VGtR5a=>6x%FMumjJqnGdBY1wtH83>ZLO4%b!GV7U`AE6d5QLYAaOgDF~`1 zaI$tkRl+D>*DAyuXv%rxOR%+-cMX+OFn05({jXb-hq@A8S(p4OXfR=REZ$fSh{Wge0QkIv7 z|DB8!yI#M*(=~7?nle4P!kZaKAG=P43oNUs;o#eQ_N~hamJ&`gFpu{Rl;R|KEzFAp zgKhR#@6Fdnq5lmHKc3o#)JFb#DJ?6nypKPXGbRbj+M`4>iHihSiLlhrj5QdA!iP$! zYziU9xBvwKh*PMJ+n;L<>S&0>aA5(WAw7=1yT-YP%?O1?lNMpL%2et)@ME-MtTM41 z&P#OH$d44Gya|ag9Z#<&s6{x$l$8Xmy}JPc2~kmkzA@y3NZJK+dzgK|bEUgFkN;~# zsc5S_TsFfX+UHD7N_|BWZ?$xlr=K*WMbNQ)VXRn{%S_`#>gqoFbT9bd2hcUiHW>P+ zd607E3O1S+~zBrdu(Q|k(Wjq^FQNjoTxMB0EK0eB#V1V^@m&DyMrV=H3;Vz~W z=|Tu|5QiA@A!=-NoH67v31&7O(6oaNCv6N?wSo#hmJPKL7QrGFQ)GKa2OPhGypIn7 zKpWiyMd5h}SDiEO$>7v0XY@qi&+K0U*a+uaOyt;=Or?hy&96>ZVEt}h^MW3gV0+O> zah}$7C^T*uP4%0BCnQ`LJMm{j%nRH6g0f-5L_r`%iUqRG#aC%2R_)IVr4|{l!j0QEt30B5QR8PCd)Z+0r)c_*6lK`q z^A%ORsDq_1_*)}|qsy3KHte=~p+v$cKH?<4-j* ze3ZNQ=E<9YE}Q~C7gnSK7+sc6i^Mo#n=@+A!FBW+$re)ioDg~ffTCRw<;U99=1i&CK~$s zfr|LBeY3fLC7NqKig^=ukHil8DSJgU`6f2<^;{cJn8N5z1(HOlPYk%ODZO~?&e@1l zcG?e>?Pmnl6MwiU&PS|@-v)b|Sc6z44SxhfP0HyQaB$rBkNq8aKTU@ksYIsdF*pXN z-DX9EkTMR!?N1jzk2JE7Oyo{PO`?{=3t!?|g;41GDdW(RxqamjXu&{I$NsS*sJ+iW z3|uyz=Oo$&w;Of9P-C)i`E+u|y{BhK`W=kbpk#t3!Ptb8#J_{B^1HstOGB3pi{39k z3G#rW0N|`|dEpp&{i_G#zU-hK_@VXX+&vgwnvdsGJ;uSw^hR=MC}Mr9si4JE$4BL_ zoJnpe(Ne6_o~NY*1RevC;YUi;?vl3Y-t;!qDRVazkU@nv>3#%^U35|tZ#zXan}>#J z%0Vqrm2P@2LdGT-K5rhH%wq7nyb3uS#zAq>JmYik8dd6-H=y5U|eCi?%21Webz{&RbCaL0UvyuU=ZahwK-B1*woDZ+x*g zigutq<&=UG)bulJ_}%tlV;pV^Fmq->TJC;as*Zz8=Vl2(a~M%BH#%g z;5CR{ub`b5Y8@WXL9;5!r<&y(QKNk$%7}|!%c17|)E+}3Equ=b4LGrCqXD+&=J7$} zmVasjRJqS8A~bG9O#qRN1Oa;>+`S$9%jHLO$`I6nfvE@XmD3|HL_QoK|KpW#b~16SEWI*{#UhX{C5Sl30fws__<83vVOy2rp6Cf%A}{D47D6AdI;n zad~8y!gI3*;D8_A!i0LeGoLnD+&1hT&Fd5}poVw}BXi3| zDrd*9&Y0oiw}YY_d)u#&xX8$V@DaE#U!Gh!0hx92D}Tud#!~9# zRSCnNplu(;sphcy8XHUvL8?tUju4uvecD3M$1e`r4W{3Hrd9Stw(Yf35wm3G+3 z+4?6J!Sp%}3}zyWTgQ}9#3UqOZ3qsv_6w8$z&-gl-2#2m(7iJlKs*)Z<;nk+0zT(w zqFXR3@Xi4|R`KVh1TfnccmB#V{trl-uuV5vBm zgYUqG1^SoRBJmFJ=0er=jArj499&RT>q6unKy?od(qf$;4la2HQ7fsx=W55?lG<_) z5iu}k2@HCb_?OPBtQ+*-6^s9J&A#;iW$csJU;J->{Mn8x#EoE`my(Ex$mBIAd|ut# z>lL%aTiot3gF|$k zro^b+VwoHxN&8X@oS z2np?t+JlZPNaslcmjsq^6I9HzR*Jql)Y=bpf`bfbS!00egG3A9v+s`DX@ll&V3@K z6U6~PjZn{k02vnhfTd??2nGgRaBWQjWVk{7m6%g)`(O!_`u~218!#H%PLHQg*+7gL z6sAE{v=gy`93P*v7Z_GZ7(lji7~m#Z0uS;*(mNipbPNO8c@QQOG^^tRTf%U}kD~(= zQ8-YMpkwPlaRN3WQJ^VX?ea}@3j|`L_$%LPb8>M-*VKsG<^sPp3cN-ko&^0h*Vf3E zjzXNd8=Bzc!d8p~g+Dd$Ha9_xRmi~$53xL1SaFo-2BKC6I0tZoX`vvfl(7MERS>w= ztl)E&@&IuNC#Rv|{#tIt0UGh_h~tw794Ee4GK{1{z|#L~O@C)v2&ndKpgWHk#iu*~ zmnU{`Wk>Yc4nUu=0~B;QzBttbmT~^xgLk`IK=zNC?@fyVT6H7@{}@CG6BO2}!EcXE z`A~*M56mRhfP@waiysNXh4SB~rusjCTF_Mxr0f9DUve&-_xZg&-z#Bc?3XSXfbBhE zB5@UQ5B?4x=sPR}B|;nBbBxce)|G)hDGIg`h;+idU(sh?kk0G?VHuMu-M)=g@TIDW zjE$xI>p?FlCwCjP`4B^*Yxr#M!G%f?6$x^u{D+9qrQ%~wkL54CY++9xD5&$SK9ug)PkU4jn@VudwT2E{@ry5 z1R(BIwLNjkZ2}+IwtxiXIRsG##b?^HAf&0g*RWU+11@065B`;>UAV9zdpWa{pN9v9 z!Z~vW8)2{$Hcqa4n1lKw5VfW6W~Bi|8t_Sq-|Rs$5R?MngQ-jMgHq=9rufR+2C(T7 zZxImCglLiA```*9g$ib!y8M0tm$}cJWmbm!LE3i}*{}p4)`J?`_&Ge-kbRR`7=2)^SVc)N+Fo# zRCQgMdZe{0az90&_b#YTiFhw+?ko=zVah%N1eZFd!+N@Q;JVpB!}A9qCJP3QzCt+~ zNvmV^MBoMhLcV{!XTW$Iv;YbxbK=2o4GpjtFrwK$2b68R7ILfX*kd`5&eV zsmH90)$_xt3(bfoDM+-9t(wN8X(%CpbQB=5eVsz?o)v@xAI?!3ZMERC%A{; zNVw!{XS1v?7>s+x$Hpq#**))C0|jz$xn(}QE2A>pwRsg7?j0cD8DH-_g1C@>W2U1c z0IxH|i|u&e$_1ax3s3e$GW)Ll4lntPVFFP!3M`M5 z3Wwwihi2ez>j2Gjk~o!6h{(Qv8vS2=2P9Of@xr5}TTm zv;Ke_@%;akkR(tGd}~NcPmk>GR*hdUhj}X~3G3WjN0b->1FyjLEnWQu$pN?oLQP`$ z%^tjJtH#%Qt|01(z$OQ|WAB`to5uuf!amEo2=W~M|BIm5ecA1nnp!ZJj!IrYq;HLl zpLE^(R_~nFb!H!QX~Bd(bo`;4l2GpatLqo z=B3G}N6k&yNwrioG;?4G1@X`+Z3G_$4o7ukl-l6(CV&6F?B?lsL*pP-)|C7zYpSP=TY2J>ENl;RQSRksVin( z_(`6POZ8bX=S#&F7J-{pFSVMR$Z9hu*W*`xKd-$#`-A;s*9WNutGiN3zw@R81dhWM zo8i~u;v!jSD{I)ThuCzYR;>XZw$qErge^1!&~7;?}BYmW2m z)5erw_PMmWntWGQ=t}OL32IzWK1fsLWIxMtCAp@i<}Ed@l9CcAA+VdgbV&rK3H58& zBGPX>R~s)t4A(_LGj2#su<}_4*fMyD$Hm2IgX8%A#&p`$l=;?O+3#C-Jx7{f)fJbP zKIXoa(+%m|`&swc@lWhz_}!2X`~V66q?gaGdU@5#-rba^hzff@=i%t)MtxQ5X<6BI z<(*e7bH8^@TG;=LCp~}8H6(Tuz6FfRRQdS%JKSFNW?ljiZO|dx_KT~UU_=gvuibFn zKWl0-gP>MdUqA1(b2t8(To+36L*2`l>W+>j#ie}U3-&lCCr9zt(rt3WaGaKnD{wQM zOT6PsNXjS)Rt@K^glXTe?5cwu*%a-@bTdu!^aZFoZps1B? z{Ku2typhfTSzq`N;CH^fyuAHqa>-Th9gu@l1&nHxiSa@OOkZpE=gQ^!nZ@rL{_%>A zRJH79{#!ec9sCvDWdCOx_zBWK$Oiuhzx`DI=a0Y3XjT5;XIAzbi_&FYY3fm-+&88!gSrjV zljm$1HwyCeu|PA!0eCh)Jfk8{DB%6yyjP-2m@5~g;_51BDnx^7S@Xq4f%vk@wQFE_ znq~UH#Dw8p;Bho$n~GYLD0yMpFrb+~cZ6+?MirMvgSu0zji}|;_YTA(03D1%nDn`U z-?6FLpS-;r!Ct_7XX(?kP5&nM#!19ZTgdb^1`gPfY((eZ1;{enh|&?0l4>maM@F9T=AbBsk}m<5d!i|TCZ0u3#lV2Z*ug>BHDZkJ?}(!KEE;!b3hZ{WLO?(w{