From 1db71364d82aa6a324a714960e19433df3aa0df0 Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Fri, 22 May 2026 20:13:12 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: mlfoundations-dev/oh_v1.3_gpt_4o_mini Source: Original Platform --- .gitattributes | 54 + README.md | 67 + all_results.json | 12 + config.json | 36 + configs.yaml | 41 + configuration.json | 1 + eval_results.json | 7 + generation_config.json | 9 + model-00001-of-00004.safetensors | 3 + model-00002-of-00004.safetensors | 3 + model-00003-of-00004.safetensors | 3 + model-00004-of-00004.safetensors | 3 + model.safetensors.index.json | 298 +++++ special_tokens_map.json | 17 + tokenizer.json | 3 + tokenizer_config.json | 2065 ++++++++++++++++++++++++++++++ train_results.json | 8 + trainer_log.jsonl | 85 ++ trainer_state.json | 633 +++++++++ training_args.bin | 3 + training_eval_loss.png | Bin 0 -> 34292 bytes training_loss.png | Bin 0 -> 36834 bytes 22 files changed, 3351 insertions(+) create mode 100644 .gitattributes create mode 100644 README.md create mode 100644 all_results.json create mode 100644 config.json create mode 100644 configs.yaml create mode 100644 configuration.json create mode 100644 eval_results.json create mode 100644 generation_config.json create mode 100644 model-00001-of-00004.safetensors create mode 100644 model-00002-of-00004.safetensors create mode 100644 model-00003-of-00004.safetensors create mode 100644 model-00004-of-00004.safetensors create mode 100644 model.safetensors.index.json create mode 100644 special_tokens_map.json create mode 100644 tokenizer.json create mode 100644 tokenizer_config.json create mode 100644 train_results.json create mode 100644 trainer_log.jsonl create mode 100644 trainer_state.json create mode 100644 training_args.bin create mode 100644 training_eval_loss.png create mode 100644 training_loss.png diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..b491dd3 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,54 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text + + +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zstandard filter=lfs diff=lfs merge=lfs -text +*.tfevents* filter=lfs diff=lfs merge=lfs -text +*.db* filter=lfs diff=lfs merge=lfs -text +*.ark* filter=lfs diff=lfs merge=lfs -text +**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text +**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text +**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text + +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.gguf* filter=lfs diff=lfs merge=lfs -text +*.ggml filter=lfs diff=lfs merge=lfs -text +*.llamafile* filter=lfs diff=lfs merge=lfs -text +*.pt2 filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text + +training_args.bin filter=lfs diff=lfs merge=lfs -text +model-00001-of-00004.safetensors filter=lfs diff=lfs merge=lfs -text +model-00004-of-00004.safetensors filter=lfs diff=lfs merge=lfs -text +model-00002-of-00004.safetensors filter=lfs diff=lfs merge=lfs -text +model-00003-of-00004.safetensors filter=lfs diff=lfs merge=lfs -text +tokenizer.json filter=lfs diff=lfs merge=lfs -text \ No newline at end of file diff --git a/README.md b/README.md new file mode 100644 index 0000000..46da4cd --- /dev/null +++ b/README.md @@ -0,0 +1,67 @@ +--- +library_name: transformers +license: llama3.1 +base_model: meta-llama/Meta-Llama-3.1-8B +tags: +- llama-factory +- full +- generated_from_trainer +model-index: +- name: oh_v1.3_gpt_4o_mini + results: [] +--- + + + +# oh_v1.3_gpt_4o_mini + +This model is a fine-tuned version of [meta-llama/Meta-Llama-3.1-8B](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B) on the mlfoundations-dev/oh_v1.3_gpt_4o_mini dataset. +It achieves the following results on the evaluation set: +- Loss: 0.7332 + +## Model description + +More information needed + +## Intended uses & limitations + +More information needed + +## Training and evaluation data + +More information needed + +## Training procedure + +### Training hyperparameters + +The following hyperparameters were used during training: +- learning_rate: 5e-06 +- train_batch_size: 8 +- eval_batch_size: 8 +- seed: 42 +- distributed_type: multi-GPU +- num_devices: 16 +- gradient_accumulation_steps: 4 +- total_train_batch_size: 512 +- total_eval_batch_size: 128 +- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments +- lr_scheduler_type: constant +- num_epochs: 3.0 + +### Training results + +| Training Loss | Epoch | Step | Validation Loss | +|:-------------:|:------:|:----:|:---------------:| +| 0.7434 | 0.9973 | 272 | 0.7440 | +| 0.6845 | 1.9963 | 544 | 0.7305 | +| 0.6373 | 2.9954 | 816 | 0.7332 | + + +### Framework versions + +- Transformers 4.46.1 +- Pytorch 2.3.0 +- Datasets 3.1.0 +- Tokenizers 0.20.3 diff --git a/all_results.json b/all_results.json new file mode 100644 index 0000000..f506dce --- /dev/null +++ b/all_results.json @@ -0,0 +1,12 @@ +{ + "epoch": 2.9954170485792853, + "eval_loss": 0.7332214117050171, + "eval_runtime": 93.8096, + "eval_samples_per_second": 78.35, + "eval_steps_per_second": 0.618, + "total_flos": 1366411632967680.0, + "train_loss": 0.7035682309491962, + "train_runtime": 14220.3782, + "train_samples_per_second": 29.46, + "train_steps_per_second": 0.057 +} \ No newline at end of file diff --git a/config.json b/config.json new file mode 100644 index 0000000..c3a9daf --- /dev/null +++ b/config.json @@ -0,0 +1,36 @@ +{ + "_name_or_path": "meta-llama/Meta-Llama-3.1-8B", + "architectures": [ + "LlamaForCausalLM" + ], + "attention_bias": false, + "attention_dropout": 0.0, + "bos_token_id": 128000, + "eos_token_id": 128001, + "head_dim": 128, + "hidden_act": "silu", + "hidden_size": 4096, + "initializer_range": 0.02, + "intermediate_size": 14336, + "max_position_embeddings": 131072, + "mlp_bias": false, + "model_type": "llama", + "num_attention_heads": 32, + "num_hidden_layers": 32, + "num_key_value_heads": 8, + "pretraining_tp": 1, + "rms_norm_eps": 1e-05, + "rope_scaling": { + "factor": 8.0, + "high_freq_factor": 4.0, + "low_freq_factor": 1.0, + "original_max_position_embeddings": 8192, + "rope_type": "llama3" + }, + "rope_theta": 500000.0, + "tie_word_embeddings": false, + "torch_dtype": "bfloat16", + "transformers_version": "4.46.1", + "use_cache": false, + "vocab_size": 128256 +} diff --git a/configs.yaml b/configs.yaml new file mode 100644 index 0000000..9581a05 --- /dev/null +++ b/configs.yaml @@ -0,0 +1,41 @@ +adam_beta1: '0.9' +adam_beta2: '0.999' +bf16: 'True' +cutoff_len: '2048' +dataset: mlfoundations-dev/oh_v1.3_gpt_4o_mini +dataset_dir: ONLINE +ddp_timeout: '180000000' +deepspeed: /opt/ml/code/zero3.json +do_train: 'True' +enable_liger_kernel: 'False' +eval_strategy: epoch +finetuning_type: full +formatting: sharegpt +global_batch_size: '512' +gradient_accumulation_steps: '4' +gradient_checkpointing: 'True' +hub_model_id: mlfoundations-dev/oh_v1.3_gpt_4o_mini +learning_rate: 5e-06 +logging_steps: '10' +lr_scheduler_type: constant +max_grad_norm: '1' +messages: conversations +model_name_or_path: meta-llama/Meta-Llama-3.1-8B +neat_packing: 'True' +num_train_epochs: '3.0' +output_dir: /opt/ml/model +overwrite_cache: 'True' +overwrite_output_dir: 'True' +packing: 'True' +per_device_train_batch_size: '8' +plot_loss: 'True' +preprocessing_num_workers: '16' +push_to_db: 'True' +push_to_hub: 'True' +report_to: wandb +run_name: oh_v1.3_gpt_4o_mini +save_strategy: epoch +stage: sft +template: llama3 +val_size: '0.05' +weight_decay: '0.1' diff --git a/configuration.json b/configuration.json new file mode 100644 index 0000000..bbeeda1 --- /dev/null +++ b/configuration.json @@ -0,0 +1 @@ +{"framework": "pytorch", "task": "text-generation", "allow_remote": true} \ No newline at end of file diff --git a/eval_results.json b/eval_results.json new file mode 100644 index 0000000..d04003b --- /dev/null +++ b/eval_results.json @@ -0,0 +1,7 @@ +{ + "epoch": 2.9954170485792853, + "eval_loss": 0.7332214117050171, + "eval_runtime": 93.8096, + "eval_samples_per_second": 78.35, + "eval_steps_per_second": 0.618 +} \ No newline at end of file diff --git a/generation_config.json b/generation_config.json new file mode 100644 index 0000000..eab5082 --- /dev/null +++ b/generation_config.json @@ -0,0 +1,9 @@ +{ + "_from_model_config": true, + "bos_token_id": 128000, + "do_sample": true, + "eos_token_id": 128001, + "temperature": 0.6, + "top_p": 0.9, + "transformers_version": "4.46.1" +} diff --git a/model-00001-of-00004.safetensors b/model-00001-of-00004.safetensors new file mode 100644 index 0000000..1f6d664 --- /dev/null +++ b/model-00001-of-00004.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ad255bcedbb511599e1bedcd8e8cf9305c27d989004329da6b8bd3aed7b4798c +size 4976698672 diff --git a/model-00002-of-00004.safetensors b/model-00002-of-00004.safetensors new file mode 100644 index 0000000..2f972ae --- /dev/null +++ b/model-00002-of-00004.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4dc401136cf31c6ddcf5d956214a10e191aa405943de5d19951137c896f0cb36 +size 4999802720 diff --git a/model-00003-of-00004.safetensors b/model-00003-of-00004.safetensors new file mode 100644 index 0000000..4c0330a --- /dev/null +++ b/model-00003-of-00004.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4f60161d27bb92605fc5a38fd8e339f080a37c4089e970a937871f8080e728e9 +size 4915916176 diff --git a/model-00004-of-00004.safetensors b/model-00004-of-00004.safetensors new file mode 100644 index 0000000..7e2a12e --- /dev/null +++ b/model-00004-of-00004.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1f67b28bcfcc5ca34f746f705c84dd049735f917a0b64e79b8a9e9ec9041f33f +size 1168138808 diff --git a/model.safetensors.index.json b/model.safetensors.index.json new file mode 100644 index 0000000..0fd8120 --- /dev/null +++ b/model.safetensors.index.json @@ -0,0 +1,298 @@ +{ + "metadata": { + "total_size": 16060522496 + }, + "weight_map": { + "lm_head.weight": "model-00004-of-00004.safetensors", + "model.embed_tokens.weight": "model-00001-of-00004.safetensors", + "model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.10.self_attn.o_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.10.self_attn.v_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.12.self_attn.o_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.13.mlp.gate_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.13.mlp.up_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.13.self_attn.k_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.13.self_attn.q_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.14.mlp.gate_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.14.self_attn.k_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.15.input_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.16.input_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.16.mlp.down_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.16.mlp.gate_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.16.mlp.up_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.16.self_attn.k_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.16.self_attn.o_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.16.self_attn.q_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.16.self_attn.v_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.17.input_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.17.mlp.down_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.17.mlp.gate_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.17.mlp.up_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.17.self_attn.k_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.17.self_attn.o_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.17.self_attn.q_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.18.input_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.18.mlp.down_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.18.mlp.gate_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.18.mlp.up_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.18.self_attn.k_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.18.self_attn.o_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.18.self_attn.q_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.18.self_attn.v_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.19.input_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.19.mlp.down_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.19.mlp.gate_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.19.mlp.up_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.19.post_attention_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.19.self_attn.k_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.19.self_attn.o_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.19.self_attn.q_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.19.self_attn.v_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.20.input_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.20.mlp.down_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.20.mlp.gate_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.20.mlp.up_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.20.post_attention_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.20.self_attn.k_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.20.self_attn.o_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.20.self_attn.q_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.20.self_attn.v_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.21.input_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.21.mlp.down_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.21.mlp.gate_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.21.mlp.up_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.21.post_attention_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.21.self_attn.k_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.21.self_attn.o_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.21.self_attn.q_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.21.self_attn.v_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.22.self_attn.k_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.22.self_attn.o_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.22.self_attn.q_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.22.self_attn.v_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.23.self_attn.k_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.23.self_attn.q_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.24.input_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.24.mlp.gate_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.24.mlp.up_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.24.post_attention_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.24.self_attn.k_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.24.self_attn.o_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.24.self_attn.q_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.24.self_attn.v_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.25.input_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.25.mlp.down_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.25.mlp.gate_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.25.mlp.up_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.25.post_attention_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.25.self_attn.k_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.25.self_attn.o_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.25.self_attn.q_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.25.self_attn.v_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.26.input_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.26.mlp.down_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.26.mlp.gate_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.26.mlp.up_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.26.post_attention_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.26.self_attn.k_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.26.self_attn.o_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.26.self_attn.q_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.26.self_attn.v_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.27.input_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.27.mlp.down_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.27.mlp.gate_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.27.mlp.up_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.27.post_attention_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.27.self_attn.k_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.27.self_attn.o_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.27.self_attn.q_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.27.self_attn.v_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.28.input_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.28.mlp.down_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.28.mlp.gate_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.28.mlp.up_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.28.post_attention_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.28.self_attn.k_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.28.self_attn.o_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.28.self_attn.q_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.28.self_attn.v_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.29.input_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.29.mlp.down_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.29.mlp.gate_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.29.mlp.up_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.29.post_attention_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.29.self_attn.k_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.29.self_attn.o_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.29.self_attn.q_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.29.self_attn.v_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.30.input_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.30.mlp.down_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.30.mlp.gate_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.30.mlp.up_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.30.post_attention_layernorm.weight": "model-00003-of-00004.safetensors", + "model.layers.30.self_attn.k_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.30.self_attn.o_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.30.self_attn.q_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.30.self_attn.v_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.31.input_layernorm.weight": "model-00004-of-00004.safetensors", + "model.layers.31.mlp.down_proj.weight": "model-00004-of-00004.safetensors", + "model.layers.31.mlp.gate_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.31.mlp.up_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.31.post_attention_layernorm.weight": "model-00004-of-00004.safetensors", + "model.layers.31.self_attn.k_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.31.self_attn.o_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.31.self_attn.q_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.31.self_attn.v_proj.weight": "model-00003-of-00004.safetensors", + "model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.5.input_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.5.mlp.up_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.6.input_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.6.mlp.down_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.6.mlp.up_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.7.input_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.7.mlp.down_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.7.mlp.up_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.8.input_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.8.mlp.down_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.8.mlp.up_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00004.safetensors", + "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00004.safetensors", + "model.layers.9.input_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.9.mlp.gate_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors", + "model.layers.9.self_attn.k_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.9.self_attn.o_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.9.self_attn.q_proj.weight": "model-00002-of-00004.safetensors", + "model.layers.9.self_attn.v_proj.weight": "model-00002-of-00004.safetensors", + "model.norm.weight": "model-00004-of-00004.safetensors" + } +} diff --git a/special_tokens_map.json b/special_tokens_map.json new file mode 100644 index 0000000..b43be96 --- /dev/null +++ b/special_tokens_map.json @@ -0,0 +1,17 @@ +{ + "bos_token": { + "content": "<|begin_of_text|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false + }, + "eos_token": { + "content": "<|eot_id|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false + }, + "pad_token": "<|eot_id|>" +} diff --git a/tokenizer.json b/tokenizer.json new file mode 100644 index 0000000..1c1d8d5 --- /dev/null +++ b/tokenizer.json @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b +size 17209920 diff --git a/tokenizer_config.json b/tokenizer_config.json new file mode 100644 index 0000000..3ae1585 --- /dev/null +++ b/tokenizer_config.json @@ -0,0 +1,2065 @@ +{ + "added_tokens_decoder": { + "128000": { + "content": "<|begin_of_text|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128001": { + "content": "<|end_of_text|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128002": { + "content": "<|reserved_special_token_0|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128003": { + "content": "<|reserved_special_token_1|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128004": { + "content": "<|finetune_right_pad_id|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128005": { + "content": "<|reserved_special_token_2|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128006": { + "content": "<|start_header_id|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128007": { + "content": "<|end_header_id|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128008": { + "content": "<|eom_id|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128009": { + "content": "<|eot_id|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128010": { + "content": "<|python_tag|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128011": { + "content": "<|reserved_special_token_3|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128012": { + "content": "<|reserved_special_token_4|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128013": { + "content": "<|reserved_special_token_5|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128014": { + "content": "<|reserved_special_token_6|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128015": { + "content": "<|reserved_special_token_7|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128016": { + "content": "<|reserved_special_token_8|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128017": { + "content": "<|reserved_special_token_9|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128018": { + "content": "<|reserved_special_token_10|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128019": { + "content": "<|reserved_special_token_11|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128020": { + "content": "<|reserved_special_token_12|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128021": { + "content": "<|reserved_special_token_13|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128022": { + "content": "<|reserved_special_token_14|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128023": { + "content": "<|reserved_special_token_15|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128024": { + "content": "<|reserved_special_token_16|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128025": { + "content": "<|reserved_special_token_17|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128026": { + "content": "<|reserved_special_token_18|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128027": { + "content": "<|reserved_special_token_19|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128028": { + "content": "<|reserved_special_token_20|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128029": { + "content": "<|reserved_special_token_21|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128030": { + "content": "<|reserved_special_token_22|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128031": { + "content": "<|reserved_special_token_23|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128032": { + "content": "<|reserved_special_token_24|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128033": { + "content": "<|reserved_special_token_25|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128034": { + "content": "<|reserved_special_token_26|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128035": { + "content": "<|reserved_special_token_27|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128036": { + "content": "<|reserved_special_token_28|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128037": { + "content": "<|reserved_special_token_29|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128038": { + "content": "<|reserved_special_token_30|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128039": { + "content": "<|reserved_special_token_31|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128040": { + "content": "<|reserved_special_token_32|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128041": { + "content": "<|reserved_special_token_33|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128042": { + "content": "<|reserved_special_token_34|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128043": { + "content": "<|reserved_special_token_35|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128044": { + "content": "<|reserved_special_token_36|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128045": { + "content": "<|reserved_special_token_37|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128046": { + "content": "<|reserved_special_token_38|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128047": { + "content": "<|reserved_special_token_39|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128048": { + "content": "<|reserved_special_token_40|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128049": { + "content": "<|reserved_special_token_41|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128050": { + "content": "<|reserved_special_token_42|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128051": { + "content": "<|reserved_special_token_43|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128052": { + "content": "<|reserved_special_token_44|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128053": { + "content": "<|reserved_special_token_45|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128054": { + "content": "<|reserved_special_token_46|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128055": { + "content": "<|reserved_special_token_47|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128056": { + "content": "<|reserved_special_token_48|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128057": { + "content": "<|reserved_special_token_49|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128058": { + "content": "<|reserved_special_token_50|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128059": { + "content": "<|reserved_special_token_51|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128060": { + "content": "<|reserved_special_token_52|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128061": { + "content": "<|reserved_special_token_53|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128062": { + "content": "<|reserved_special_token_54|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128063": { + "content": "<|reserved_special_token_55|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128064": { + "content": "<|reserved_special_token_56|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128065": { + "content": "<|reserved_special_token_57|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128066": { + "content": "<|reserved_special_token_58|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128067": { + "content": "<|reserved_special_token_59|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128068": { + "content": "<|reserved_special_token_60|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128069": { + "content": "<|reserved_special_token_61|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128070": { + "content": "<|reserved_special_token_62|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128071": { + "content": "<|reserved_special_token_63|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128072": { + "content": "<|reserved_special_token_64|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128073": { + "content": "<|reserved_special_token_65|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128074": { + "content": "<|reserved_special_token_66|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128075": { + "content": "<|reserved_special_token_67|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128076": { + "content": "<|reserved_special_token_68|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128077": { + "content": "<|reserved_special_token_69|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128078": { + "content": "<|reserved_special_token_70|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128079": { + "content": "<|reserved_special_token_71|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128080": { + "content": "<|reserved_special_token_72|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128081": { + "content": "<|reserved_special_token_73|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128082": { + "content": "<|reserved_special_token_74|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128083": { + "content": "<|reserved_special_token_75|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128084": { + "content": "<|reserved_special_token_76|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128085": { + "content": "<|reserved_special_token_77|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128086": { + "content": "<|reserved_special_token_78|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128087": { + "content": "<|reserved_special_token_79|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128088": { + "content": "<|reserved_special_token_80|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128089": { + "content": "<|reserved_special_token_81|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128090": { + "content": "<|reserved_special_token_82|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128091": { + "content": "<|reserved_special_token_83|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128092": { + "content": "<|reserved_special_token_84|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128093": { + "content": "<|reserved_special_token_85|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128094": { + "content": "<|reserved_special_token_86|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128095": { + "content": "<|reserved_special_token_87|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128096": { + "content": "<|reserved_special_token_88|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128097": { + "content": "<|reserved_special_token_89|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128098": { + "content": "<|reserved_special_token_90|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128099": { + "content": "<|reserved_special_token_91|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128100": { + "content": "<|reserved_special_token_92|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128101": { + "content": "<|reserved_special_token_93|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128102": { + "content": "<|reserved_special_token_94|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128103": { + "content": "<|reserved_special_token_95|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128104": { + "content": "<|reserved_special_token_96|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128105": { + "content": "<|reserved_special_token_97|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128106": { + "content": "<|reserved_special_token_98|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128107": { + "content": "<|reserved_special_token_99|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128108": { + "content": "<|reserved_special_token_100|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128109": { + "content": "<|reserved_special_token_101|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128110": { + "content": "<|reserved_special_token_102|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128111": { + "content": "<|reserved_special_token_103|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128112": { + "content": "<|reserved_special_token_104|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128113": { + "content": "<|reserved_special_token_105|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128114": { + "content": "<|reserved_special_token_106|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128115": { + "content": "<|reserved_special_token_107|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128116": { + "content": "<|reserved_special_token_108|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128117": { + "content": "<|reserved_special_token_109|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128118": { + "content": "<|reserved_special_token_110|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128119": { + "content": "<|reserved_special_token_111|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128120": { + "content": "<|reserved_special_token_112|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128121": { + "content": "<|reserved_special_token_113|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128122": { + "content": "<|reserved_special_token_114|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128123": { + "content": "<|reserved_special_token_115|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128124": { + "content": "<|reserved_special_token_116|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128125": { + "content": "<|reserved_special_token_117|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128126": { + "content": "<|reserved_special_token_118|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128127": { + "content": "<|reserved_special_token_119|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128128": { + "content": "<|reserved_special_token_120|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128129": { + "content": "<|reserved_special_token_121|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128130": { + "content": "<|reserved_special_token_122|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128131": { + "content": "<|reserved_special_token_123|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128132": { + "content": "<|reserved_special_token_124|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128133": { + "content": "<|reserved_special_token_125|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128134": { + "content": "<|reserved_special_token_126|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128135": { + "content": "<|reserved_special_token_127|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128136": { + "content": "<|reserved_special_token_128|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128137": { + "content": "<|reserved_special_token_129|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128138": { + "content": "<|reserved_special_token_130|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128139": { + "content": "<|reserved_special_token_131|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128140": { + "content": "<|reserved_special_token_132|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128141": { + "content": "<|reserved_special_token_133|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128142": { + "content": "<|reserved_special_token_134|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128143": { + "content": "<|reserved_special_token_135|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128144": { + "content": "<|reserved_special_token_136|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128145": { + "content": "<|reserved_special_token_137|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128146": { + "content": "<|reserved_special_token_138|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128147": { + "content": "<|reserved_special_token_139|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128148": { + "content": "<|reserved_special_token_140|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128149": { + "content": "<|reserved_special_token_141|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128150": { + "content": "<|reserved_special_token_142|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128151": { + "content": "<|reserved_special_token_143|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128152": { + "content": "<|reserved_special_token_144|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128153": { + "content": "<|reserved_special_token_145|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128154": { + "content": "<|reserved_special_token_146|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128155": { + "content": "<|reserved_special_token_147|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128156": { + "content": "<|reserved_special_token_148|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128157": { + "content": "<|reserved_special_token_149|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128158": { + "content": "<|reserved_special_token_150|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128159": { + "content": "<|reserved_special_token_151|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128160": { + "content": "<|reserved_special_token_152|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128161": { + "content": "<|reserved_special_token_153|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128162": { + "content": "<|reserved_special_token_154|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128163": { + "content": "<|reserved_special_token_155|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128164": { + "content": "<|reserved_special_token_156|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128165": { + "content": "<|reserved_special_token_157|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128166": { + "content": "<|reserved_special_token_158|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128167": { + "content": "<|reserved_special_token_159|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128168": { + "content": "<|reserved_special_token_160|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128169": { + "content": "<|reserved_special_token_161|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128170": { + "content": "<|reserved_special_token_162|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128171": { + "content": "<|reserved_special_token_163|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128172": { + "content": "<|reserved_special_token_164|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128173": { + "content": "<|reserved_special_token_165|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128174": { + "content": "<|reserved_special_token_166|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128175": { + "content": "<|reserved_special_token_167|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128176": { + "content": "<|reserved_special_token_168|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128177": { + "content": "<|reserved_special_token_169|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128178": { + "content": "<|reserved_special_token_170|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128179": { + "content": "<|reserved_special_token_171|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128180": { + "content": "<|reserved_special_token_172|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128181": { + "content": "<|reserved_special_token_173|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128182": { + "content": "<|reserved_special_token_174|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128183": { + "content": "<|reserved_special_token_175|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128184": { + "content": "<|reserved_special_token_176|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128185": { + "content": "<|reserved_special_token_177|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128186": { + "content": "<|reserved_special_token_178|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128187": { + "content": "<|reserved_special_token_179|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128188": { + "content": "<|reserved_special_token_180|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128189": { + "content": "<|reserved_special_token_181|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128190": { + "content": "<|reserved_special_token_182|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128191": { + "content": "<|reserved_special_token_183|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128192": { + "content": "<|reserved_special_token_184|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128193": { + "content": "<|reserved_special_token_185|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128194": { + "content": "<|reserved_special_token_186|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128195": { + "content": "<|reserved_special_token_187|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128196": { + "content": "<|reserved_special_token_188|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128197": { + "content": "<|reserved_special_token_189|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128198": { + "content": "<|reserved_special_token_190|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128199": { + "content": "<|reserved_special_token_191|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128200": { + "content": "<|reserved_special_token_192|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128201": { + "content": "<|reserved_special_token_193|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128202": { + "content": "<|reserved_special_token_194|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128203": { + "content": "<|reserved_special_token_195|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128204": { + "content": "<|reserved_special_token_196|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128205": { + "content": "<|reserved_special_token_197|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128206": { + "content": "<|reserved_special_token_198|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128207": { + "content": "<|reserved_special_token_199|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128208": { + "content": "<|reserved_special_token_200|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128209": { + "content": "<|reserved_special_token_201|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128210": { + "content": "<|reserved_special_token_202|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128211": { + "content": "<|reserved_special_token_203|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128212": { + "content": "<|reserved_special_token_204|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128213": { + "content": "<|reserved_special_token_205|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128214": { + "content": "<|reserved_special_token_206|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128215": { + "content": "<|reserved_special_token_207|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128216": { + "content": "<|reserved_special_token_208|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128217": { + "content": "<|reserved_special_token_209|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128218": { + "content": "<|reserved_special_token_210|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128219": { + "content": "<|reserved_special_token_211|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128220": { + "content": "<|reserved_special_token_212|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128221": { + "content": "<|reserved_special_token_213|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128222": { + "content": "<|reserved_special_token_214|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128223": { + "content": "<|reserved_special_token_215|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128224": { + "content": "<|reserved_special_token_216|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128225": { + "content": "<|reserved_special_token_217|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128226": { + "content": "<|reserved_special_token_218|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128227": { + "content": "<|reserved_special_token_219|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128228": { + "content": "<|reserved_special_token_220|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128229": { + "content": "<|reserved_special_token_221|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128230": { + "content": "<|reserved_special_token_222|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128231": { + "content": "<|reserved_special_token_223|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128232": { + "content": "<|reserved_special_token_224|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128233": { + "content": "<|reserved_special_token_225|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128234": { + "content": "<|reserved_special_token_226|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128235": { + "content": "<|reserved_special_token_227|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128236": { + "content": "<|reserved_special_token_228|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128237": { + "content": "<|reserved_special_token_229|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128238": { + "content": "<|reserved_special_token_230|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128239": { + "content": "<|reserved_special_token_231|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128240": { + "content": "<|reserved_special_token_232|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128241": { + "content": "<|reserved_special_token_233|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128242": { + "content": "<|reserved_special_token_234|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128243": { + "content": "<|reserved_special_token_235|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128244": { + "content": "<|reserved_special_token_236|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128245": { + "content": "<|reserved_special_token_237|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128246": { + "content": "<|reserved_special_token_238|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128247": { + "content": "<|reserved_special_token_239|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128248": { + "content": "<|reserved_special_token_240|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128249": { + "content": "<|reserved_special_token_241|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128250": { + "content": "<|reserved_special_token_242|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128251": { + "content": "<|reserved_special_token_243|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128252": { + "content": "<|reserved_special_token_244|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128253": { + "content": "<|reserved_special_token_245|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128254": { + "content": "<|reserved_special_token_246|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "128255": { + "content": "<|reserved_special_token_247|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + } + }, + "bos_token": "<|begin_of_text|>", + "chat_template": "{{ '<|begin_of_text|>' }}{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% endif %}{% if system_message is defined %}{{ '<|start_header_id|>system<|end_header_id|>\n\n' + system_message + '<|eot_id|>' }}{% endif %}{% for message in loop_messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ '<|start_header_id|>user<|end_header_id|>\n\n' + content + '<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n' }}{% elif message['role'] == 'assistant' %}{{ content + '<|eot_id|>' }}{% endif %}{% endfor %}", + "clean_up_tokenization_spaces": true, + "eos_token": "<|eot_id|>", + "model_input_names": [ + "input_ids", + "attention_mask" + ], + "model_max_length": 131072, + "pad_token": "<|eot_id|>", + "padding_side": "right", + "split_special_tokens": false, + "tokenizer_class": "PreTrainedTokenizerFast" +} diff --git a/train_results.json b/train_results.json new file mode 100644 index 0000000..fcfafbb --- /dev/null +++ b/train_results.json @@ -0,0 +1,8 @@ +{ + "epoch": 2.9954170485792853, + "total_flos": 1366411632967680.0, + "train_loss": 0.7035682309491962, + "train_runtime": 14220.3782, + "train_samples_per_second": 29.46, + "train_steps_per_second": 0.057 +} \ No newline at end of file diff --git a/trainer_log.jsonl b/trainer_log.jsonl new file mode 100644 index 0000000..dbba6e5 --- /dev/null +++ b/trainer_log.jsonl @@ -0,0 +1,85 @@ +{"current_steps": 10, "total_steps": 816, "loss": 1.033, "lr": 5e-06, "epoch": 0.03666361136571952, "percentage": 1.23, "elapsed_time": "0:02:55", "remaining_time": "3:55:13"} +{"current_steps": 20, "total_steps": 816, "loss": 0.9011, "lr": 5e-06, "epoch": 0.07332722273143905, "percentage": 2.45, "elapsed_time": "0:05:44", "remaining_time": "3:48:40"} +{"current_steps": 30, "total_steps": 816, "loss": 0.8764, "lr": 5e-06, "epoch": 0.10999083409715857, "percentage": 3.68, "elapsed_time": "0:08:33", "remaining_time": "3:44:10"} +{"current_steps": 40, "total_steps": 816, "loss": 0.8446, "lr": 5e-06, "epoch": 0.1466544454628781, "percentage": 4.9, "elapsed_time": "0:11:23", "remaining_time": "3:40:58"} +{"current_steps": 50, "total_steps": 816, "loss": 0.8204, "lr": 5e-06, "epoch": 0.18331805682859761, "percentage": 6.13, "elapsed_time": "0:14:12", "remaining_time": "3:37:42"} +{"current_steps": 60, "total_steps": 816, "loss": 0.8104, "lr": 5e-06, "epoch": 0.21998166819431714, "percentage": 7.35, "elapsed_time": "0:17:00", "remaining_time": "3:34:21"} +{"current_steps": 70, "total_steps": 816, "loss": 0.7968, "lr": 5e-06, "epoch": 0.2566452795600367, "percentage": 8.58, "elapsed_time": "0:19:49", "remaining_time": "3:31:11"} +{"current_steps": 80, "total_steps": 816, "loss": 0.7836, "lr": 5e-06, "epoch": 0.2933088909257562, "percentage": 9.8, "elapsed_time": "0:22:36", "remaining_time": "3:28:02"} +{"current_steps": 90, "total_steps": 816, "loss": 0.7781, "lr": 5e-06, "epoch": 0.32997250229147573, "percentage": 11.03, "elapsed_time": "0:25:24", "remaining_time": "3:24:59"} +{"current_steps": 100, "total_steps": 816, "loss": 0.7732, "lr": 5e-06, "epoch": 0.36663611365719523, "percentage": 12.25, "elapsed_time": "0:28:14", "remaining_time": "3:22:10"} +{"current_steps": 110, "total_steps": 816, "loss": 0.7711, "lr": 5e-06, "epoch": 0.4032997250229148, "percentage": 13.48, "elapsed_time": "0:31:01", "remaining_time": "3:19:05"} +{"current_steps": 120, "total_steps": 816, "loss": 0.7637, "lr": 5e-06, "epoch": 0.4399633363886343, "percentage": 14.71, "elapsed_time": "0:33:47", "remaining_time": "3:16:00"} +{"current_steps": 130, "total_steps": 816, "loss": 0.7617, "lr": 5e-06, "epoch": 0.4766269477543538, "percentage": 15.93, "elapsed_time": "0:36:34", "remaining_time": "3:12:59"} +{"current_steps": 140, "total_steps": 816, "loss": 0.7562, "lr": 5e-06, "epoch": 0.5132905591200734, "percentage": 17.16, "elapsed_time": "0:39:20", "remaining_time": "3:09:59"} +{"current_steps": 150, "total_steps": 816, "loss": 0.7601, "lr": 5e-06, "epoch": 0.5499541704857929, "percentage": 18.38, "elapsed_time": "0:42:09", "remaining_time": "3:07:09"} +{"current_steps": 160, "total_steps": 816, "loss": 0.7561, "lr": 5e-06, "epoch": 0.5866177818515124, "percentage": 19.61, "elapsed_time": "0:44:57", "remaining_time": "3:04:19"} +{"current_steps": 170, "total_steps": 816, "loss": 0.7549, "lr": 5e-06, "epoch": 0.6232813932172319, "percentage": 20.83, "elapsed_time": "0:47:46", "remaining_time": "3:01:31"} +{"current_steps": 180, "total_steps": 816, "loss": 0.7525, "lr": 5e-06, "epoch": 0.6599450045829515, "percentage": 22.06, "elapsed_time": "0:50:35", "remaining_time": "2:58:45"} +{"current_steps": 190, "total_steps": 816, "loss": 0.7507, "lr": 5e-06, "epoch": 0.696608615948671, "percentage": 23.28, "elapsed_time": "0:53:25", "remaining_time": "2:56:00"} +{"current_steps": 200, "total_steps": 816, "loss": 0.7492, "lr": 5e-06, "epoch": 0.7332722273143905, "percentage": 24.51, "elapsed_time": "0:56:14", "remaining_time": "2:53:13"} +{"current_steps": 210, "total_steps": 816, "loss": 0.7446, "lr": 5e-06, "epoch": 0.76993583868011, "percentage": 25.74, "elapsed_time": "0:59:01", "remaining_time": "2:50:20"} +{"current_steps": 220, "total_steps": 816, "loss": 0.7512, "lr": 5e-06, "epoch": 0.8065994500458296, "percentage": 26.96, "elapsed_time": "1:01:50", "remaining_time": "2:47:33"} +{"current_steps": 230, "total_steps": 816, "loss": 0.744, "lr": 5e-06, "epoch": 0.843263061411549, "percentage": 28.19, "elapsed_time": "1:04:39", "remaining_time": "2:44:43"} +{"current_steps": 240, "total_steps": 816, "loss": 0.7431, "lr": 5e-06, "epoch": 0.8799266727772685, "percentage": 29.41, "elapsed_time": "1:07:26", "remaining_time": "2:41:51"} +{"current_steps": 250, "total_steps": 816, "loss": 0.7399, "lr": 5e-06, "epoch": 0.916590284142988, "percentage": 30.64, "elapsed_time": "1:10:15", "remaining_time": "2:39:03"} +{"current_steps": 260, "total_steps": 816, "loss": 0.7457, "lr": 5e-06, "epoch": 0.9532538955087076, "percentage": 31.86, "elapsed_time": "1:13:02", "remaining_time": "2:36:12"} +{"current_steps": 270, "total_steps": 816, "loss": 0.7434, "lr": 5e-06, "epoch": 0.9899175068744271, "percentage": 33.09, "elapsed_time": "1:15:50", "remaining_time": "2:33:21"} +{"current_steps": 272, "total_steps": 816, "eval_loss": 0.743977963924408, "epoch": 0.997250229147571, "percentage": 33.33, "elapsed_time": "1:18:04", "remaining_time": "2:36:08"} +{"current_steps": 280, "total_steps": 816, "loss": 0.7593, "lr": 5e-06, "epoch": 1.0284142988084326, "percentage": 34.31, "elapsed_time": "1:21:12", "remaining_time": "2:35:26"} +{"current_steps": 290, "total_steps": 816, "loss": 0.6885, "lr": 5e-06, "epoch": 1.065077910174152, "percentage": 35.54, "elapsed_time": "1:23:58", "remaining_time": "2:32:18"} +{"current_steps": 300, "total_steps": 816, "loss": 0.6893, "lr": 5e-06, "epoch": 1.1017415215398716, "percentage": 36.76, "elapsed_time": "1:26:45", "remaining_time": "2:29:13"} +{"current_steps": 310, "total_steps": 816, "loss": 0.6868, "lr": 5e-06, "epoch": 1.138405132905591, "percentage": 37.99, "elapsed_time": "1:29:33", "remaining_time": "2:26:10"} +{"current_steps": 320, "total_steps": 816, "loss": 0.6885, "lr": 5e-06, "epoch": 1.1750687442713108, "percentage": 39.22, "elapsed_time": "1:32:21", "remaining_time": "2:23:09"} +{"current_steps": 330, "total_steps": 816, "loss": 0.6871, "lr": 5e-06, "epoch": 1.2117323556370303, "percentage": 40.44, "elapsed_time": "1:35:09", "remaining_time": "2:20:08"} +{"current_steps": 340, "total_steps": 816, "loss": 0.6935, "lr": 5e-06, "epoch": 1.2483959670027498, "percentage": 41.67, "elapsed_time": "1:37:58", "remaining_time": "2:17:09"} +{"current_steps": 350, "total_steps": 816, "loss": 0.6943, "lr": 5e-06, "epoch": 1.2850595783684693, "percentage": 42.89, "elapsed_time": "1:40:44", "remaining_time": "2:14:08"} +{"current_steps": 360, "total_steps": 816, "loss": 0.6938, "lr": 5e-06, "epoch": 1.3217231897341888, "percentage": 44.12, "elapsed_time": "1:43:31", "remaining_time": "2:11:07"} +{"current_steps": 370, "total_steps": 816, "loss": 0.6856, "lr": 5e-06, "epoch": 1.3583868010999083, "percentage": 45.34, "elapsed_time": "1:46:16", "remaining_time": "2:08:05"} +{"current_steps": 380, "total_steps": 816, "loss": 0.69, "lr": 5e-06, "epoch": 1.3950504124656278, "percentage": 46.57, "elapsed_time": "1:49:04", "remaining_time": "2:05:08"} +{"current_steps": 390, "total_steps": 816, "loss": 0.682, "lr": 5e-06, "epoch": 1.4317140238313475, "percentage": 47.79, "elapsed_time": "1:51:52", "remaining_time": "2:02:11"} +{"current_steps": 400, "total_steps": 816, "loss": 0.6917, "lr": 5e-06, "epoch": 1.468377635197067, "percentage": 49.02, "elapsed_time": "1:54:39", "remaining_time": "1:59:14"} +{"current_steps": 410, "total_steps": 816, "loss": 0.6871, "lr": 5e-06, "epoch": 1.5050412465627865, "percentage": 50.25, "elapsed_time": "1:57:26", "remaining_time": "1:56:17"} +{"current_steps": 420, "total_steps": 816, "loss": 0.6829, "lr": 5e-06, "epoch": 1.541704857928506, "percentage": 51.47, "elapsed_time": "2:00:12", "remaining_time": "1:53:20"} +{"current_steps": 430, "total_steps": 816, "loss": 0.6824, "lr": 5e-06, "epoch": 1.5783684692942255, "percentage": 52.7, "elapsed_time": "2:03:00", "remaining_time": "1:50:24"} +{"current_steps": 440, "total_steps": 816, "loss": 0.6822, "lr": 5e-06, "epoch": 1.615032080659945, "percentage": 53.92, "elapsed_time": "2:05:47", "remaining_time": "1:47:30"} +{"current_steps": 450, "total_steps": 816, "loss": 0.6879, "lr": 5e-06, "epoch": 1.6516956920256645, "percentage": 55.15, "elapsed_time": "2:08:35", "remaining_time": "1:44:35"} +{"current_steps": 460, "total_steps": 816, "loss": 0.6804, "lr": 5e-06, "epoch": 1.6883593033913842, "percentage": 56.37, "elapsed_time": "2:11:22", "remaining_time": "1:41:40"} +{"current_steps": 470, "total_steps": 816, "loss": 0.686, "lr": 5e-06, "epoch": 1.7250229147571035, "percentage": 57.6, "elapsed_time": "2:14:08", "remaining_time": "1:38:45"} +{"current_steps": 480, "total_steps": 816, "loss": 0.6856, "lr": 5e-06, "epoch": 1.7616865261228232, "percentage": 58.82, "elapsed_time": "2:16:55", "remaining_time": "1:35:51"} +{"current_steps": 490, "total_steps": 816, "loss": 0.6814, "lr": 5e-06, "epoch": 1.7983501374885427, "percentage": 60.05, "elapsed_time": "2:19:42", "remaining_time": "1:32:57"} +{"current_steps": 500, "total_steps": 816, "loss": 0.6855, "lr": 5e-06, "epoch": 1.8350137488542622, "percentage": 61.27, "elapsed_time": "2:22:31", "remaining_time": "1:30:04"} +{"current_steps": 510, "total_steps": 816, "loss": 0.6875, "lr": 5e-06, "epoch": 1.8716773602199817, "percentage": 62.5, "elapsed_time": "2:25:17", "remaining_time": "1:27:10"} +{"current_steps": 520, "total_steps": 816, "loss": 0.6856, "lr": 5e-06, "epoch": 1.9083409715857012, "percentage": 63.73, "elapsed_time": "2:28:05", "remaining_time": "1:24:18"} +{"current_steps": 530, "total_steps": 816, "loss": 0.6841, "lr": 5e-06, "epoch": 1.9450045829514209, "percentage": 64.95, "elapsed_time": "2:30:54", "remaining_time": "1:21:26"} +{"current_steps": 540, "total_steps": 816, "loss": 0.6845, "lr": 5e-06, "epoch": 1.9816681943171401, "percentage": 66.18, "elapsed_time": "2:33:43", "remaining_time": "1:18:34"} +{"current_steps": 544, "total_steps": 816, "eval_loss": 0.7305116057395935, "epoch": 1.996333638863428, "percentage": 66.67, "elapsed_time": "2:36:33", "remaining_time": "1:18:16"} +{"current_steps": 550, "total_steps": 816, "loss": 0.707, "lr": 5e-06, "epoch": 2.020164986251146, "percentage": 67.4, "elapsed_time": "2:39:11", "remaining_time": "1:16:59"} +{"current_steps": 560, "total_steps": 816, "loss": 0.6313, "lr": 5e-06, "epoch": 2.056828597616865, "percentage": 68.63, "elapsed_time": "2:41:58", "remaining_time": "1:14:02"} +{"current_steps": 570, "total_steps": 816, "loss": 0.6281, "lr": 5e-06, "epoch": 2.093492208982585, "percentage": 69.85, "elapsed_time": "2:44:46", "remaining_time": "1:11:06"} +{"current_steps": 580, "total_steps": 816, "loss": 0.631, "lr": 5e-06, "epoch": 2.130155820348304, "percentage": 71.08, "elapsed_time": "2:47:34", "remaining_time": "1:08:11"} +{"current_steps": 590, "total_steps": 816, "loss": 0.6351, "lr": 5e-06, "epoch": 2.166819431714024, "percentage": 72.3, "elapsed_time": "2:50:21", "remaining_time": "1:05:15"} +{"current_steps": 600, "total_steps": 816, "loss": 0.634, "lr": 5e-06, "epoch": 2.203483043079743, "percentage": 73.53, "elapsed_time": "2:53:07", "remaining_time": "1:02:19"} +{"current_steps": 610, "total_steps": 816, "loss": 0.6327, "lr": 5e-06, "epoch": 2.240146654445463, "percentage": 74.75, "elapsed_time": "2:55:53", "remaining_time": "0:59:24"} +{"current_steps": 620, "total_steps": 816, "loss": 0.6339, "lr": 5e-06, "epoch": 2.276810265811182, "percentage": 75.98, "elapsed_time": "2:58:40", "remaining_time": "0:56:28"} +{"current_steps": 630, "total_steps": 816, "loss": 0.636, "lr": 5e-06, "epoch": 2.313473877176902, "percentage": 77.21, "elapsed_time": "3:01:26", "remaining_time": "0:53:34"} +{"current_steps": 640, "total_steps": 816, "loss": 0.6342, "lr": 5e-06, "epoch": 2.3501374885426216, "percentage": 78.43, "elapsed_time": "3:04:13", "remaining_time": "0:50:39"} +{"current_steps": 650, "total_steps": 816, "loss": 0.637, "lr": 5e-06, "epoch": 2.386801099908341, "percentage": 79.66, "elapsed_time": "3:06:59", "remaining_time": "0:47:45"} +{"current_steps": 660, "total_steps": 816, "loss": 0.63, "lr": 5e-06, "epoch": 2.4234647112740606, "percentage": 80.88, "elapsed_time": "3:09:44", "remaining_time": "0:44:50"} +{"current_steps": 670, "total_steps": 816, "loss": 0.6288, "lr": 5e-06, "epoch": 2.46012832263978, "percentage": 82.11, "elapsed_time": "3:12:30", "remaining_time": "0:41:57"} +{"current_steps": 680, "total_steps": 816, "loss": 0.6381, "lr": 5e-06, "epoch": 2.4967919340054996, "percentage": 83.33, "elapsed_time": "3:15:19", "remaining_time": "0:39:03"} +{"current_steps": 690, "total_steps": 816, "loss": 0.6332, "lr": 5e-06, "epoch": 2.5334555453712193, "percentage": 84.56, "elapsed_time": "3:18:05", "remaining_time": "0:36:10"} +{"current_steps": 700, "total_steps": 816, "loss": 0.6351, "lr": 5e-06, "epoch": 2.5701191567369386, "percentage": 85.78, "elapsed_time": "3:20:53", "remaining_time": "0:33:17"} +{"current_steps": 710, "total_steps": 816, "loss": 0.6363, "lr": 5e-06, "epoch": 2.606782768102658, "percentage": 87.01, "elapsed_time": "3:23:40", "remaining_time": "0:30:24"} +{"current_steps": 720, "total_steps": 816, "loss": 0.6355, "lr": 5e-06, "epoch": 2.6434463794683776, "percentage": 88.24, "elapsed_time": "3:26:27", "remaining_time": "0:27:31"} +{"current_steps": 730, "total_steps": 816, "loss": 0.6351, "lr": 5e-06, "epoch": 2.6801099908340973, "percentage": 89.46, "elapsed_time": "3:29:12", "remaining_time": "0:24:38"} +{"current_steps": 740, "total_steps": 816, "loss": 0.638, "lr": 5e-06, "epoch": 2.7167736021998166, "percentage": 90.69, "elapsed_time": "3:31:58", "remaining_time": "0:21:46"} +{"current_steps": 750, "total_steps": 816, "loss": 0.6388, "lr": 5e-06, "epoch": 2.7534372135655363, "percentage": 91.91, "elapsed_time": "3:34:44", "remaining_time": "0:18:53"} +{"current_steps": 760, "total_steps": 816, "loss": 0.6364, "lr": 5e-06, "epoch": 2.7901008249312556, "percentage": 93.14, "elapsed_time": "3:37:32", "remaining_time": "0:16:01"} +{"current_steps": 770, "total_steps": 816, "loss": 0.6421, "lr": 5e-06, "epoch": 2.8267644362969753, "percentage": 94.36, "elapsed_time": "3:40:18", "remaining_time": "0:13:09"} +{"current_steps": 780, "total_steps": 816, "loss": 0.6379, "lr": 5e-06, "epoch": 2.863428047662695, "percentage": 95.59, "elapsed_time": "3:43:05", "remaining_time": "0:10:17"} +{"current_steps": 790, "total_steps": 816, "loss": 0.6414, "lr": 5e-06, "epoch": 2.9000916590284143, "percentage": 96.81, "elapsed_time": "3:45:51", "remaining_time": "0:07:25"} +{"current_steps": 800, "total_steps": 816, "loss": 0.6361, "lr": 5e-06, "epoch": 2.936755270394134, "percentage": 98.04, "elapsed_time": "3:48:38", "remaining_time": "0:04:34"} +{"current_steps": 810, "total_steps": 816, "loss": 0.6373, "lr": 5e-06, "epoch": 2.9734188817598532, "percentage": 99.26, "elapsed_time": "3:51:25", "remaining_time": "0:01:42"} +{"current_steps": 816, "total_steps": 816, "eval_loss": 0.7332214117050171, "epoch": 2.9954170485792853, "percentage": 100.0, "elapsed_time": "3:55:44", "remaining_time": "0:00:00"} +{"current_steps": 816, "total_steps": 816, "epoch": 2.9954170485792853, "percentage": 100.0, "elapsed_time": "3:56:59", "remaining_time": "0:00:00"} diff --git a/trainer_state.json b/trainer_state.json new file mode 100644 index 0000000..1f6d305 --- /dev/null +++ b/trainer_state.json @@ -0,0 +1,633 @@ +{ + "best_metric": null, + "best_model_checkpoint": null, + "epoch": 2.9954170485792853, + "eval_steps": 500, + "global_step": 816, + "is_hyper_param_search": false, + "is_local_process_zero": true, + "is_world_process_zero": true, + "log_history": [ + { + "epoch": 0.03666361136571952, + "grad_norm": 10.920966734668648, + "learning_rate": 5e-06, + "loss": 1.033, + "step": 10 + }, + { + "epoch": 0.07332722273143905, + "grad_norm": 2.445980198452297, + "learning_rate": 5e-06, + "loss": 0.9011, + "step": 20 + }, + { + "epoch": 0.10999083409715857, + "grad_norm": 1.5424232386159482, + "learning_rate": 5e-06, + "loss": 0.8764, + "step": 30 + }, + { + "epoch": 0.1466544454628781, + "grad_norm": 1.1797649838046136, + "learning_rate": 5e-06, + "loss": 0.8446, + "step": 40 + }, + { + "epoch": 0.18331805682859761, + "grad_norm": 1.0295589020655365, + "learning_rate": 5e-06, + "loss": 0.8204, + "step": 50 + }, + { + "epoch": 0.21998166819431714, + "grad_norm": 1.2160434357225554, + "learning_rate": 5e-06, + "loss": 0.8104, + "step": 60 + }, + { + "epoch": 0.2566452795600367, + "grad_norm": 1.2135493715768004, + "learning_rate": 5e-06, + "loss": 0.7968, + "step": 70 + }, + { + "epoch": 0.2933088909257562, + "grad_norm": 0.7594488712178804, + "learning_rate": 5e-06, + "loss": 0.7836, + "step": 80 + }, + { + "epoch": 0.32997250229147573, + "grad_norm": 0.8913076302913621, + "learning_rate": 5e-06, + "loss": 0.7781, + "step": 90 + }, + { + "epoch": 0.36663611365719523, + "grad_norm": 1.126183659145103, + "learning_rate": 5e-06, + "loss": 0.7732, + "step": 100 + }, + { + "epoch": 0.4032997250229148, + "grad_norm": 0.7476760341544976, + "learning_rate": 5e-06, + "loss": 0.7711, + "step": 110 + }, + { + "epoch": 0.4399633363886343, + "grad_norm": 0.828783632948725, + "learning_rate": 5e-06, + "loss": 0.7637, + "step": 120 + }, + { + "epoch": 0.4766269477543538, + "grad_norm": 0.7005369874659794, + "learning_rate": 5e-06, + "loss": 0.7617, + "step": 130 + }, + { + "epoch": 0.5132905591200734, + "grad_norm": 0.6781356553576761, + "learning_rate": 5e-06, + "loss": 0.7562, + "step": 140 + }, + { + "epoch": 0.5499541704857929, + "grad_norm": 0.6643060954517749, + "learning_rate": 5e-06, + "loss": 0.7601, + "step": 150 + }, + { + "epoch": 0.5866177818515124, + "grad_norm": 0.654862572470797, + "learning_rate": 5e-06, + "loss": 0.7561, + "step": 160 + }, + { + "epoch": 0.6232813932172319, + "grad_norm": 0.7126834121476828, + "learning_rate": 5e-06, + "loss": 0.7549, + "step": 170 + }, + { + "epoch": 0.6599450045829515, + "grad_norm": 0.5845932549413623, + "learning_rate": 5e-06, + "loss": 0.7525, + "step": 180 + }, + { + "epoch": 0.696608615948671, + "grad_norm": 0.583642927450063, + "learning_rate": 5e-06, + "loss": 0.7507, + "step": 190 + }, + { + "epoch": 0.7332722273143905, + "grad_norm": 0.5759630428428489, + "learning_rate": 5e-06, + "loss": 0.7492, + "step": 200 + }, + { + "epoch": 0.76993583868011, + "grad_norm": 0.597809207757354, + "learning_rate": 5e-06, + "loss": 0.7446, + "step": 210 + }, + { + "epoch": 0.8065994500458296, + "grad_norm": 0.6520665055230834, + "learning_rate": 5e-06, + "loss": 0.7512, + "step": 220 + }, + { + "epoch": 0.843263061411549, + "grad_norm": 0.6521761800994458, + "learning_rate": 5e-06, + "loss": 0.744, + "step": 230 + }, + { + "epoch": 0.8799266727772685, + "grad_norm": 0.6083361886529014, + "learning_rate": 5e-06, + "loss": 0.7431, + "step": 240 + }, + { + "epoch": 0.916590284142988, + "grad_norm": 0.8966782629847545, + "learning_rate": 5e-06, + "loss": 0.7399, + "step": 250 + }, + { + "epoch": 0.9532538955087076, + "grad_norm": 0.6584181334872885, + "learning_rate": 5e-06, + "loss": 0.7457, + "step": 260 + }, + { + "epoch": 0.9899175068744271, + "grad_norm": 0.5614900416740534, + "learning_rate": 5e-06, + "loss": 0.7434, + "step": 270 + }, + { + "epoch": 0.997250229147571, + "eval_loss": 0.743977963924408, + "eval_runtime": 96.6447, + "eval_samples_per_second": 76.052, + "eval_steps_per_second": 0.6, + "step": 272 + }, + { + "epoch": 1.0284142988084326, + "grad_norm": 0.6377526876986616, + "learning_rate": 5e-06, + "loss": 0.7593, + "step": 280 + }, + { + "epoch": 1.065077910174152, + "grad_norm": 0.8312923337011684, + "learning_rate": 5e-06, + "loss": 0.6885, + "step": 290 + }, + { + "epoch": 1.1017415215398716, + "grad_norm": 0.6499984381614756, + "learning_rate": 5e-06, + "loss": 0.6893, + "step": 300 + }, + { + "epoch": 1.138405132905591, + "grad_norm": 0.658519279927457, + "learning_rate": 5e-06, + "loss": 0.6868, + "step": 310 + }, + { + "epoch": 1.1750687442713108, + "grad_norm": 0.6307182099292118, + "learning_rate": 5e-06, + "loss": 0.6885, + "step": 320 + }, + { + "epoch": 1.2117323556370303, + "grad_norm": 0.6191143311988347, + "learning_rate": 5e-06, + "loss": 0.6871, + "step": 330 + }, + { + "epoch": 1.2483959670027498, + "grad_norm": 0.6735946598593434, + "learning_rate": 5e-06, + "loss": 0.6935, + "step": 340 + }, + { + "epoch": 1.2850595783684693, + "grad_norm": 0.7213451984916242, + "learning_rate": 5e-06, + "loss": 0.6943, + "step": 350 + }, + { + "epoch": 1.3217231897341888, + "grad_norm": 0.5841901070016948, + "learning_rate": 5e-06, + "loss": 0.6938, + "step": 360 + }, + { + "epoch": 1.3583868010999083, + "grad_norm": 0.6609752377099979, + "learning_rate": 5e-06, + "loss": 0.6856, + "step": 370 + }, + { + "epoch": 1.3950504124656278, + "grad_norm": 0.6004672142282963, + "learning_rate": 5e-06, + "loss": 0.69, + "step": 380 + }, + { + "epoch": 1.4317140238313475, + "grad_norm": 0.7494020947088555, + "learning_rate": 5e-06, + "loss": 0.682, + "step": 390 + }, + { + "epoch": 1.468377635197067, + "grad_norm": 0.6711006066177567, + "learning_rate": 5e-06, + "loss": 0.6917, + "step": 400 + }, + { + "epoch": 1.5050412465627865, + "grad_norm": 0.6517430215570676, + "learning_rate": 5e-06, + "loss": 0.6871, + "step": 410 + }, + { + "epoch": 1.541704857928506, + "grad_norm": 0.6180564693914907, + "learning_rate": 5e-06, + "loss": 0.6829, + "step": 420 + }, + { + "epoch": 1.5783684692942255, + "grad_norm": 0.5764324092377354, + "learning_rate": 5e-06, + "loss": 0.6824, + "step": 430 + }, + { + "epoch": 1.615032080659945, + "grad_norm": 0.7134204082562298, + "learning_rate": 5e-06, + "loss": 0.6822, + "step": 440 + }, + { + "epoch": 1.6516956920256645, + "grad_norm": 0.7630512170385407, + "learning_rate": 5e-06, + "loss": 0.6879, + "step": 450 + }, + { + "epoch": 1.6883593033913842, + "grad_norm": 0.6285437172539765, + "learning_rate": 5e-06, + "loss": 0.6804, + "step": 460 + }, + { + "epoch": 1.7250229147571035, + "grad_norm": 0.5968789313484854, + "learning_rate": 5e-06, + "loss": 0.686, + "step": 470 + }, + { + "epoch": 1.7616865261228232, + "grad_norm": 0.6425175740435289, + "learning_rate": 5e-06, + "loss": 0.6856, + "step": 480 + }, + { + "epoch": 1.7983501374885427, + "grad_norm": 0.7614365266625939, + "learning_rate": 5e-06, + "loss": 0.6814, + "step": 490 + }, + { + "epoch": 1.8350137488542622, + "grad_norm": 0.5496379357416068, + "learning_rate": 5e-06, + "loss": 0.6855, + "step": 500 + }, + { + "epoch": 1.8716773602199817, + "grad_norm": 0.8494093367270151, + "learning_rate": 5e-06, + "loss": 0.6875, + "step": 510 + }, + { + "epoch": 1.9083409715857012, + "grad_norm": 0.6756166103000668, + "learning_rate": 5e-06, + "loss": 0.6856, + "step": 520 + }, + { + "epoch": 1.9450045829514209, + "grad_norm": 0.7228484772895967, + "learning_rate": 5e-06, + "loss": 0.6841, + "step": 530 + }, + { + "epoch": 1.9816681943171401, + "grad_norm": 0.7786774729146112, + "learning_rate": 5e-06, + "loss": 0.6845, + "step": 540 + }, + { + "epoch": 1.996333638863428, + "eval_loss": 0.7305116057395935, + "eval_runtime": 96.1532, + "eval_samples_per_second": 76.441, + "eval_steps_per_second": 0.603, + "step": 544 + }, + { + "epoch": 2.020164986251146, + "grad_norm": 1.0425759476709966, + "learning_rate": 5e-06, + "loss": 0.707, + "step": 550 + }, + { + "epoch": 2.056828597616865, + "grad_norm": 0.8473344095764829, + "learning_rate": 5e-06, + "loss": 0.6313, + "step": 560 + }, + { + "epoch": 2.093492208982585, + "grad_norm": 0.7205628261028438, + "learning_rate": 5e-06, + "loss": 0.6281, + "step": 570 + }, + { + "epoch": 2.130155820348304, + "grad_norm": 0.6604987014823058, + "learning_rate": 5e-06, + "loss": 0.631, + "step": 580 + }, + { + "epoch": 2.166819431714024, + "grad_norm": 0.6774961015973217, + "learning_rate": 5e-06, + "loss": 0.6351, + "step": 590 + }, + { + "epoch": 2.203483043079743, + "grad_norm": 0.8519809292040578, + "learning_rate": 5e-06, + "loss": 0.634, + "step": 600 + }, + { + "epoch": 2.240146654445463, + "grad_norm": 0.693823740633704, + "learning_rate": 5e-06, + "loss": 0.6327, + "step": 610 + }, + { + "epoch": 2.276810265811182, + "grad_norm": 0.6448705487045298, + "learning_rate": 5e-06, + "loss": 0.6339, + "step": 620 + }, + { + "epoch": 2.313473877176902, + "grad_norm": 0.5865817788059118, + "learning_rate": 5e-06, + "loss": 0.636, + "step": 630 + }, + { + "epoch": 2.3501374885426216, + "grad_norm": 0.8116556137845999, + "learning_rate": 5e-06, + "loss": 0.6342, + "step": 640 + }, + { + "epoch": 2.386801099908341, + "grad_norm": 0.6231657257473445, + "learning_rate": 5e-06, + "loss": 0.637, + "step": 650 + }, + { + "epoch": 2.4234647112740606, + "grad_norm": 0.6250913266909794, + "learning_rate": 5e-06, + "loss": 0.63, + "step": 660 + }, + { + "epoch": 2.46012832263978, + "grad_norm": 0.582068921531117, + "learning_rate": 5e-06, + "loss": 0.6288, + "step": 670 + }, + { + "epoch": 2.4967919340054996, + "grad_norm": 0.6912367969819871, + "learning_rate": 5e-06, + "loss": 0.6381, + "step": 680 + }, + { + "epoch": 2.5334555453712193, + "grad_norm": 0.7147652107920064, + "learning_rate": 5e-06, + "loss": 0.6332, + "step": 690 + }, + { + "epoch": 2.5701191567369386, + "grad_norm": 0.5792260811836798, + "learning_rate": 5e-06, + "loss": 0.6351, + "step": 700 + }, + { + "epoch": 2.606782768102658, + "grad_norm": 0.7963438662743851, + "learning_rate": 5e-06, + "loss": 0.6363, + "step": 710 + }, + { + "epoch": 2.6434463794683776, + "grad_norm": 0.9276380358330181, + "learning_rate": 5e-06, + "loss": 0.6355, + "step": 720 + }, + { + "epoch": 2.6801099908340973, + "grad_norm": 0.9313823270809661, + "learning_rate": 5e-06, + "loss": 0.6351, + "step": 730 + }, + { + "epoch": 2.7167736021998166, + "grad_norm": 0.7304200587600748, + "learning_rate": 5e-06, + "loss": 0.638, + "step": 740 + }, + { + "epoch": 2.7534372135655363, + "grad_norm": 0.6212966397528322, + "learning_rate": 5e-06, + "loss": 0.6388, + "step": 750 + }, + { + "epoch": 2.7901008249312556, + "grad_norm": 0.6720686482466423, + "learning_rate": 5e-06, + "loss": 0.6364, + "step": 760 + }, + { + "epoch": 2.8267644362969753, + "grad_norm": 0.6438467896193539, + "learning_rate": 5e-06, + "loss": 0.6421, + "step": 770 + }, + { + "epoch": 2.863428047662695, + "grad_norm": 0.6043416931907646, + "learning_rate": 5e-06, + "loss": 0.6379, + "step": 780 + }, + { + "epoch": 2.9000916590284143, + "grad_norm": 0.6496494693588303, + "learning_rate": 5e-06, + "loss": 0.6414, + "step": 790 + }, + { + "epoch": 2.936755270394134, + "grad_norm": 0.8144443719589332, + "learning_rate": 5e-06, + "loss": 0.6361, + "step": 800 + }, + { + "epoch": 2.9734188817598532, + "grad_norm": 0.7037764123768507, + "learning_rate": 5e-06, + "loss": 0.6373, + "step": 810 + }, + { + "epoch": 2.9954170485792853, + "eval_loss": 0.7332214117050171, + "eval_runtime": 94.4197, + "eval_samples_per_second": 77.844, + "eval_steps_per_second": 0.614, + "step": 816 + }, + { + "epoch": 2.9954170485792853, + "step": 816, + "total_flos": 1366411632967680.0, + "train_loss": 0.7035682309491962, + "train_runtime": 14220.3782, + "train_samples_per_second": 29.46, + "train_steps_per_second": 0.057 + } + ], + "logging_steps": 10, + "max_steps": 816, + "num_input_tokens_seen": 0, + "num_train_epochs": 3, + "save_steps": 500, + "stateful_callbacks": { + "TrainerControl": { + "args": { + "should_epoch_stop": false, + "should_evaluate": false, + "should_log": false, + "should_save": true, + "should_training_stop": true + }, + "attributes": {} + } + }, + "total_flos": 1366411632967680.0, + "train_batch_size": 8, + "trial_name": null, + "trial_params": null +} diff --git a/training_args.bin b/training_args.bin new file mode 100644 index 0000000..b6bdf26 --- /dev/null +++ b/training_args.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8baf9eae949940237851325d5f9dbadd1a80ca1a6f07f4de82e70f739b749ef0 +size 7160 diff --git a/training_eval_loss.png b/training_eval_loss.png new file mode 100644 index 0000000000000000000000000000000000000000..a50e877b22f6aefd4a3e6cbe4deb69ce97383039 GIT binary patch literal 34292 zcmd>mcRZK<+xKbjO*WyFy%I@eml;WRW+Fn!9@$$ep~yveHm{ z@Nwa9MEidJhW}h~R?v06ZGYd{)%39?qGsysU~BJeYi)M+iREJ_Yx_qc{NntVc+Wm? zc6M-*5D>8Y_XqgxA6p5qp676ZgAh2}&~ri%GE?*mBU?Jl8bN|gZ_3MPyCp6Ea@W-tBDI`7UxLpM{2oc6=4YI0t_g7EwjQ;Ll%)sjkBx*+p9Z|NF5`-_F{2jO&;? zo}Qi_$;Z=;>GqcP_AK>L{(JRIlwsI+3k-9)2Cy_MHCm-9J~KW(x^nygOredUEWx`^%M)7sG3_ORU)E_&~z$ zs@I+aleh!&=+UEXCK^#ustf9YI=KoK7UxKr#QloR+lMN{cd4leciflx6T6YpoXukWGMgZseEmUpPa7%q zWQm~Dqd951%eD!c$1zM-{B!RXURM4yNNu*a{rw&V0|TMG{o3DSO)y`#DIRm5WB&T~ z8Y42Xv_uppj|q#AeAN{*`yx*?oe<{IP`=#v0;4o-?v2%&HB97W?ev7~_+UGrq(mey zKc9k{I#5GuAN&0I^SJHVAAWMg-Vk(I)IM5oA;ZVV_lt|8sdirQS9^1rek=C__Rj`i zUu<05sg78V`Oyl2yu7@Lxw()SsRR6(nHk*C6e({8xnSa_@LT8&aov7zvN_v@JLe`w z%Wv4&aC&Sne)Ve0w{Mgf7#MpEr-#zV2aIBmuBY9a=n@;xD_yB_nmZeNV}0=My@f;d zx03YCSDyq%MUf;WCvOz`9us-)EawPCS1aGR(Rf?qnkD(^T#B#pf&`y_DZWv)^Hcbw zt)1QE+FH!_d%i{uQqt0_0u4vPUf$kt^OF-3e!ac7j<)(WL_aw!4sc2S>EI};s0h)@ zk#}$q)XCFLyXU(!URhd79dU;5n=jdr#mY$8d~X_-gv;VP`_UR#0bEihd;B|?w z^73*zNjE`=$y+(&vt4f}n3#xFR8;m34yHt>SwpM%5O|X2oNG>zp7oP8lPgJU$R;K1Z;upJ3D6}@SA&jG`W!9+eD#w2YV^Z)kYdIWzSwQ{}*C-1oE&IXg8yy=}KU?%Uwt>`V<$fNI?5Fh4Xr zoD!Dq4AL4-!?Sh#{Kbo)jEsy2zpBf0>%FSbBjJ;g(Mm~4VfqcjE;K6aTIkOT($7C5 zBO_xr^CiygaA(DAC{It8J1!j3htl|3pAsvLv9WP_Ufx%e!@I`r3KyvfV_&`E3?!fW zS>-epL~BeQ8yjmj*PUzzi*NSr-A(viop2mKx5Jf+HHPuz+Pb>G}CY)8x0RFfs{AmjqI>;CoQ8q&w-kWkutUYfS!^+regb174!_6;;})c#%f z)vCF`xVX4ISn}3r=Busm)!wu!v3j4YKRH;c`Q*I%^NQ)ihdd*t)*LzohFq8J$9N%? zTtGr&q)&8N=bmEWVIv!R(=kg8cn`mo+Zk*eEEO&_j8!>h6%`ecv$3(6b-flfYY8SX z>v+Z{%N`rr(Q(7O!C?G0+`8FRYnT}%2eXZ-w&ou{^xWLt!@ho1Ws`4f@P&OGn46pX zuehxk_c1`onI)Xb*XnlPqR*nR_D?u+_Pu&C_tEij+ueKK=hM^ETLTHGu#w^?EBePf z8yg##etRc}Yb4Pnu4~2^*OR1ujmKwKR`?{ZT~jbIN!?q>N@;z?XCSYun}A-ceckq} z`TndJ-HJ!aXxu*-udj1(bfnnd-#=LKJ=##hxK2zjEN?qpB$KCCq8Jz$h#w@&wzK>r zV{I3*0AZpkCuBFN;zz$EHaZ0wLRWv)L=5E{L|m0-Io(V-#U~;n5_&kqTWmLax!7ad z29kt;p`oF?vT}I+y*mw5p|9CF{RYy#4;HfGp640B!wpj9q;hn0bp55Ir@kn)LW$5yr4{P9VRJ|xv1qrNC$nd zdG6S<#olOzfFwB}VU`%k#&&?L(%j#FJB4^$7*Z`Hr|r2)$7w~3>u7|GXO)&}SJu=- zw6l2TjU30ndKLcsIi--p#PyB&j2MQz+!*ZjvhT*Zeep>zT}(ii(O`IoeYr zWe?-b9y>Wvxx2d`?2s^C#z)MiKSjxghNs+hb#;YWWU(Hi0vF-%e}ds0SR}|cV%h{)5^-qu+F3Glbw-~ z5hT(z0ayq(D*p&=hP^1i_4h495TkrWE2f+rV*3r?q$j3J*Kqj?!mXL_Z zaeYDtX)H2rnOIy5Uu!ruZa8N3J$m~4w}onwc%MlFBr!e!2Dq781XYHl5R6Aq$i*Qv zqJ##AAn`@PX>_X9r`uyf$T59SkHwHwr5KiuFJGD=FxDYou<`Nbf2;S{ntz&aP%#D1 zzok3*Dy)i$Gcgt(X<&T(-AuYao13}3VhrYxH6hW;R6P2HAh6Kl&hz9fzDJJ3!^883 zpDly~LqewEL4-y`hzz&C(2%lTcUc>&n_O9mf}kDX3LcH36KbAV^7BJ(oTChwW1M+> zWu)Bh;lQq^3znLHK)~eWq^Sc_YPy%5rRANRn&C zD_(mnGcW+KxI||V|7Tzz_SVWM?aiAvb&E_eV6z8sC`Hew#E5Xpz+=rDxNmK(XFvKr zE32igEmN!E!^rB(w{N2%c041g$O+MK%mC7f3)n} z3ZRFrLUHCyaCY{^n|!ob2M_6rEWf*lw}M9V**LR_}JRM^09PChX) zkz96QQMaI=Anu^Lx|)HxKUwH!cT-c-#KRz23?3eyfOCsFo~EXgMlnu%>!z@0wcYbR zL_1aN7_?WrL=PGj{`vFR;{JVh5fLhYT7$;9qecxr%(D2Em6h5DiC5g#P2zlY$wgh5 za9|UB!22U+JM0HbpJ!0fZK7?as;bJN_{23jI=a&5*duf2<;#~06=f9_-^Z`L7PeH5 z^9GQ#eJTz0+6z)(Xk=t-f{=L;4Q1YH>-^~#C6t{T?0n&g#O^-L}QDABEU&Z+Mj8)MPRDb;EJuy}a({COO^(enJ; z#3Ysw*08g$VPat|*`-vR_0$5y8Qa-hzlWSnL1lJl~7K!os3k=OG$M zez1dvg;9-;wD8^#{*=qo5dF`bub)3R!OcA?>Y?CJypI3$-Me?(zQ+Fk{)^j8p$u4S zh*rdtCE87Y9HRBFB0Y`G&6ofO1l^?8)M3K}%ceD=Y3d74VJehzfX}k`@835MGRmMA zxo`60qvhXw?|Op=x148Or|;XoxMP*_;iE|t=1v{QX+zvMXPdr%r_IdF z-1hu~7<(+7&oiu|YzZN2uXW#?fXvqQM$CM&Imm4?h<1yh&-Y}LHCvl|YoQ+x0PT&N zH)Z7Iu?K8Kt6kSf5r7IUp%fgwRZdnAThkDgdy^roTVb*;V?Kv=wj-qk5Fd>O_H{UT zcz8zuzMv5h0nkpBJ-Dc!^1ZO(BwRB?@lLe}GX=#{$bsfiY25Z_;?r(jEOMBXO_FdS zls?|Nop}NYz761b1XLUjAt4GVTtOirc$^w<^VX=KDT4=N+Lx|?PeAZgn_J<*g9l|o z0~HAgbcV1;#;C66XosV18+@HpjHG)y_tukCK|x^;+MC$8xFG1rZjA5N?T)ozkxBZY zR{`kvA`eeXJF748!-o%16mz%@_kVi4=8M8~=pm@lrm(C`v{avOXZ2S^YU9{;1)U&9d;v?iLuXuAx>$9_O)E@omY;I|h zyBMe3mv10eY}yiRQ`l1%jd?~k6gFzO@9E)qb6=mPuC8tz9$_Ml016?BfHe!n&Q;_#`AW&>5lVX>Wf&EZ*4TrgU4^uhvhW!b4a* zZe$PIz|)kE=QrYxdR*tyEgFSc@3t{zq~x&nqeSUw>E0zIH>gEWhAp4^TtD zlUv#YtV1}!1R_An`W27(qz>jej;qG3VMns_@~WQN_r6zS+<=IQ0L;T+NEVIxe)c+h ztKDu#Shn8bn(t{uiq8ST!otEq$aIeT-tQM0>p!egWO4@?B=6}lLJ!&wh~MFN2Ss3T z@auE9<}S;_6wqt503IJpZD&jWAcS$w<~{VEcu+rK9ofXhXy076Ylb}^A~xoR1>LcT zi<<@J?rVfHc7H;|PrmCV;4~@N9fmbeNMJWV_+G$Vzc)psnj-aUAdNo%{Tx=NmUUE* z2qlH)$_D9KibPd4ERwhGv-G_8>T4bS+1Q0SYO!v}b2|O*h^;9o>@@U{m)8~<`xnjT zRT=!>Xg4q1b(WX$@f=sK&_l~iBK1%TXu3f%HtffbA5W~VJ}MU>5yYs)`~3Oy`kz0- zE_Cv6Fubs^u=TC2OD=SrP1x%!KRr@x&CJ-BmzPzUNZ_~WIj&x1gzn1KTfj@D9sL0J z;mD64uoCR-Hut~oZ}4sMdQqWlpb)1Y98dm!Fu_^)V{GCi+`qiVP9QnpN(JaSa^%uH z&c673Homj#fWG1NU#hS~$Iy^CUnk<|$m@29>^9&-LP%f`!tY5lZ<=A=fWzh1)-pl& z_3X2SgR5(ZY}$lT#(j1Cr%-l&RytzAPhd@xf2i+xiJ_!phb;{)Z3g@bTyoLJkHRkE z+8V$z@bdCHY%kuSrlpn3(S9xQjx&Ty2rl{B7@Ljzp1p^w+SDSiVI_HeeLZYq$WToX z=O0$C`W(64IQLvx1xo6NWmbB60$`$IIh1s2dO}0-p%?;~m}cn-glCYwAmWZ$dB2MR z#TStI6rkQ7?5sq<{)MhJ?aK+#8D2k~+(7|YZImgQ@5|uXuJhRDHul-K1WZ^lGR+F0 zGJ;k>5nA-6l_WUk>vWVp1V~jj^4|CKWZNg;9lztPV_|p9Jpg$o(89az{IH@RC-(!2 zu%)w;2nyAQj}7|y!O-gZT=CetC@doKm$LA&w6t{l^Z5cwm;v2U_{Do>c6RFXg_PFk z=T%JX?eDG4Day+ayhO9l9soR>vcIe!R3*_{@)q2jY6v72$K_!mXjxlRy`lFWZt!i; zJOp^dg-gPK4R9!62WnQ@om`!SOV(Otqw0ymIB2;_l?kw6^}?c(O_0EbfMI*o<@CVZ z97{>5UEgkLVgfNR7{v{bRBd}lV4|tO;3&=kVz6ADGVd1&Yil!Q-HZ4o49p4)?W`|7-7&Wef}~6ciX``uX>2YilFN$}iVPO2h<4Vtzf?$NosPhb1RJkf7Z8MeXCj5Km#P zf7!$DW^Lg#;yy>>(9}dg55UgB(UJivZG~S9uSjG`}a7A^PSVhV$dc z{;Hh0&J_ZpqGjKVnywPC71{nkyD-MDf4mpIokxjjvTIw<>ho)Br|?wx<2m^GNsEh% zZ>Xs$+_({1uc4<$|72|}3Kj|%N<8fBxaccZ-ArGTuG~I#dGh2*z`3U>(tS8&U&jL& zs}vH7IdMBbxfbr4lT&C~W?Ihd?azz8U^)!DrO{#e=~w*c&d!!}g@_`r{l~(Q>Pw{ z$KUe_1G2e!e4G_o|HNoNe|P}IJb(?|lKUja?lUwQ-7ySrZ&FQp5Ha;Q{hEZUf4|abrs1_v~ys1UxKxnUsc&IHxaF2T-^aEG*H$EN?9i5~2A>Ru=QE z?}8?iWw z>bwOC?uQST7&Nr*-ulGJg5jar$ z?67&DiEA0my}M?eqJ66BgGY{_c-QyUggUCr47}IL1m+xiJ^9J%PwTA{hKQ(4 z94Y2sIFsYd%~da%ylPjNW=Yo`EEa)))(Y5YqnSv+1X?~Q6W5_T+JH2D7MTARoNCRG zc_K%8A_VFo?ctZ&jh;C5&bIy0s?4v<#$(q6}xZV&%4)9-+h=7 zZ2IS_pR?;VFQ)aBdKxM{-wH975dN$;;z0A+5TZCra%b2a_y8p!SsWlTfI??2^C)i0 zEvyfBoc=Tk?51L&ah-{|c>tuCA)DaHtJgqpa9hZXN7WT`9ZtZq4Nd6H&L*%7W%T* zRq+V?6$0M6O@-Y*-HE9=9=$v?=X(3oRlhDjS^hSq7@Vdi{ilpq6x{ab=<2zk^sfGZ zb{2`6Yb!Ngv_mV24V)2~UaPd@X^(m)Un( zT(1P~MFY2*0b43GDyl8T_r!YZI;Bnh>(;a6!rfG9`HmA>xR={)Hg@c@`dYqI)T@h< zZCqey{}3J@PrI`+YPBfXf$fG@M6cPA>2}`CZl6qPc)@Rban(o!dp14LBTf?brj zQFM1QE{!?${04QByXU4t(auGjEb?XFoZq8f2FfjQNQCwWSvA>_H^A(@S@I6~w-jnS@--X@X@zWF9s;a7kZ6KBs#Ozss zqBpg+4x5-TLA8vAG3cz3-m-^x0l{spjgOn0rg-nr|NQxLVtzgpB)-&l@8lZ_Dl6$g z6(E$9lr*uki-Kgg{Ijaxlpp)C7 z|8oRA@kHa`Q^|KxVf+m+u1z=rNT-+y6^JJwNZfEC!t>DXuR*FFrd!n-Or10!1X;VKyPFiA z-7tvY0GP7IA3^Z~*o23GbdO|T6!@+P4g*aco#251?Z4gG9x03DgWe}|-?gvuVpLog z&cUu55jWKqBOB(Lsr2F?+w_q(M=N;tTae4}S$+1{qFJtm1K$sPH7O|B*WsV!on4?D zWI;Acw0hf=b33cZyq&Zbq*AE6ApkTGwCe9cOHvGdWNTa7B&Z^Cs;b2AHBv|s5RihE zGn*lt9F7lM=A9)nqJKfpAqnET8!+6cPT1PsJ`Gg=-y>LdoNaFp;h0 zA5`PsyUZXQKSa?55J2dqEIJoxKh5x4lex{RpsmASw4zPuBlSAeXwJS-ovXc^{&=E5 zGg0CadphUjBMyZe_vwohHPg(EXH+@kQnOO9a9y3BX4Mh?2+Bme3d@9~z zrn~PGr3{NrB*C_SreS!?2EA_ObUL~r+1aVgFdyJzUK@BYZ5t3$t*u&g<+PAU{E28u za6VelGSMG%D>p^sTj$GIX^4LLg7aQUk)x*Q-E0&bJ-f{m^8qJQ9u59huZcww9#bB5 z_SAFr;;&0#JG-`xU6*;L{+&1upW9hN3N5BGMTLehX9!%dpmn5xQXal zmSzd6wEgX}Zm;)G%A;vZUY;Zr&UM>BFSR5Kmt}{Wctv}oXt`D`%TY+Qy)_m;~~i5CIi`;p)oNbAeYzIfkzE?yfn1FexL=N z<>4V>ylk8OL>khgA9%N*RSN=B8a<8YlP8x!yl|Fa1{N}MlV!SKVtTV)DYy%$MB~}syhhd3 z040TQoFFE^20R%2K*Oo_+G4;5&Vhi@sVVz}FE05TW^&y?(X*REAKqM`e$4eGx#8)| zVPOJf6M*{{IK8QE{0|hBPNA_~FX|~k&m|-zq}aw5i29)%A&IdG3Kos{0M&7k`yh+y zb#k)Zz{#68r2OSKzFtI-NYahgm=MbKz9Faig}Kn^)!F@cN(ep`vp-hI({Rn~3MyG` zo7^v;@&eONjO+IM_wR|Ethdd;-Aw?T8Q1`NG919(6y(zChYb)v&?+BK&CFP$r-9HU zeWr4Le*S*QTOZPk{5v6PlM}Z}ICc~Nsb%LY zHc3e)!Zzu(f;s?*fQtMV;Z#te;K{F`(k%%6z!a*9bQ?Avj=wMfs9? z4X6~5N9vc#Q8>~TLCbS(dJZgb!^66*t*!I%z`)U=f~Kl!hHH$FZ-Q(*gwN-b8B>?X{t2vmsZn>}?T&G)C%ux+p2adLkP@%ftng%lP(B4X7o z@8%{hD<{{Ndsl$hxRwt3XSb96ISlQeE-K2(&0pepNXiX>=WK3n=Dt>vrOnM#W_^hP z5qyv$Y)J@DJqoN4B(PZzYMw!}%N_TjUZX_s`Hge)lS_X2#as>mo#x3kjvg;F7#p}! z8r2Rs+}@OceJiTO>5JQAwRGLAJne16kk;SH%udghX?}Sdi%Oa_4dyvRLx?4`JyHfKn`aa zjQu=^VcY%Hl7)>8`$=miyTaW-VK(yO8Qwn;o$Ft)m<6kR4bZeIugx?2XWv_lvjOXd zT8ccoOhO^YZ(#xpN`kiy<7CKs+s|CQq_@un7^_e>V zMLyaD7W$~4>014`a5VX{X6$E?Lqk8#JhD0eB?!AkqN>>_qjhl0t9E;o>!18v)`5I%=7uIoz} zM!)}_I0_zQWGfscUyX;c()#5^9y|$88}e@>ELRjV(aZVv@$uiAlNU9?yx~Imq=Y&t z?w&u_sSTEa%9ZE~7L&OR0O<|9{ypccrdKGK(~3Vf=Df4@sz0s`%RnpCD7b?? zrdJ-22!w=dt!I`>dP#l|K}kUUB^$S_#E9p634xSI*trd)#9s;|HV00HJ3V89!`cq_mA0qqwWVt% z{zG@_Xxq@MawhMeQyHMAa@j1nh_ghfoH|G|wRxc$L3CIu!=-x+#T=bs8KJvBE1Es3 z(gcNl{`aF0tF=uWGl))Ifc-QYubcdABmKK7@@rvwM3X1MC~c?-t_a^&c8v_kq1|#?U3Nf@p2S;kTCN<9-5aTqpAx7$i2+D_pEn$?$r&&~f7`_joBDS^2AK4td2i@c*C7m1F&mIgZX71C5GYO2(S> z=~Tj#1)7r?nlhk)xEeK|L|Ai112vbu|Nkao?!JvtiIU@gk)Cz04ZAP?(QmdZxdS3|4v;_T%Wc!IB($U==3@p8RRzPxcvgYVB$7rK2c|&DV z*k8eTRT-tVOl015v~Vq^4obAVq+~D@A>@((u$_2gbF|gLKxoZxNO6jNBtO$3Bp8bO{;?GcDdTyDo`Eof?nGH?b|IK9b7O7 z1w4UoU_TnDtON27Xt#~!k{*WAwb2SP@CUX5h+EvQ1Z)Vma#=Uqw8Nf}hAHJ$$h$mD zY>(^>sL%+qW+xFq^xa97o*km3#1dd3hBo&M0b3;Z8XruPFhqSa@l*Qmn&P&{&;P`M zcg!QTUGr`ECfDv0A>vFa(dplWRhOqDd{zw7vP+o+PN<`rp9@nd-GqfXO1SiIh3`8b zs_^})`cH*VF74L*sDM#*AV9#0v=A15L;es${&oOK#o(>$n0btvdAi$^MXX4jC6aw^ zs;cI-=-sw+-nrGHzOBKf%N;jS(w98>Aioh+K1#QtNkTW#lmA-?NDLb3qL(4ax5Gbv z(DM!yJ&N-`PdTBiFGVTx2PI^!GHU({!~7C9WLLz`0Su5DslY!8wipFaHHzFe%%E4A z0Gb2umg9FmE-nJNze~f9KtcrflpNT*Wooo?@&?}bJwO^$>)uam^-o%uhaqBDlWul@ zishk4B@K*|?u6cUyz#Xf{~rxWtwZ$QZ~>QGve9$;f2L{_GXHe;+BIe{i2w@>mhrRy zMXTz{`m0sx^Xe9*HbtwbsXPl|xS5F4OY?GKZ%}pX(Nf@D*7f$MDhp5m1N5X*8p8>$ zosE0^7sJ0!@M3A4joic+E!g_WZY%A?EQ~HYRY( z3t+$3k*-zcu-+7kb}AMkb6U`6{tXgm$+SuS^$WXDo@04vGEF=m;?#Sw(@_$jJ-Thw z;A?SZs3z|tp!}*lU9}k==fz=-?m-ZAJs^c*A#Qd?yw6Jr|JfKXpi;a(ztE*sq>X9V z$=euLDv30HZCdGTyG>#wLtv~=74Y#f>0{7ts;M9+moMi7C36c4i8k(%mvN+r6pij> zsFI{@RwjMj5&xvB1)u=B^p(V}it8GrR+cg;6#$g-PRbKniw&=+_u8MsVbbWhWEl!i zS&bA9JG9D+(+(FZJ=BFo=IW@VSJk-H@!&W0s|0Z^^kxF!W;_{kN0poI3g5bAhzpLA zT~?AnAf>|Saom)4|8xw$(xR@@3o`VO0P$FU;Ze%6_@bOm@_OhoSDy2y@NGtPmPx zG6{w0>8{V={Jj7ReZU|JRifOwoH(U=L6r&mnuI&O+2lgp_{h5NRe`CA8Ry%K*dNnhIQQ@>Z^r6pI9Ge(^)CXL%Zi12v%nB%amtPqy$H$lU(bt<6-gxxm4g!q+ZGTn<>AtXw~9#zje z`{zb_7{_^tHZSz_A8+fYM#SSH4w7s{Rd8D}Z*9J^j9?(^t7}=?Jp|~fiQt&&E)K;C zJO(N+UlA}>Clv_nOL?ZSu0C7}Y<~V&^^l7}A=8pG`X!XH4RrO@*ke&<@wXNMz`=#J zotv4IUK{eX;<`Kzif^JcD%ukNKKM^C1F0`L3eUZdK(B*~7XuL=<=Xgco{WKvuSB(X zS8Cr50zq&f8vu&lgEccOEA#@0-~!SFaL5+t{j)sXu@K{v8SF>DsT0wMl&%Q9##eu6 z)rPyY9BkAny6>G@O|&nO;id*e@O^t6A$0hiDv1FcV)~MRvM$@kotYoe$3nzo_aGBn zv$fovu9jtsH?#Ul!aMHU<}3sEFNz7D)ESW&ZEPz@r}kPe*K)wSYfvi$y%6+_Ox-pm zYV@42%>w`(-Oo&?UdFFd^#^<(TJtvJ9_?5^Bi3=UrlC$Wt#7!Zj8IlE8+-aIK+lQH zj4D^Ef9Vi**Xt3!0^7~+jczn$M&wkv`EJrvDWa9f0ljmZC0nAqW#_+qc^VLZl6ZCm zSlN4$eK^QE^zKKF5^XO{(W}IVt1NH!0+eK3)g~VQM}w=r{Ndbu58>Qp<#Z} zKoKojXR^2u3vEF3yUGl+-`DIdBnRL!jH;m;UC33#px_S=5pe15_;m zKX2*tqXKW0Lgsry$kN$k16MGTgv@clY(bdHzF!00VAK-|b4{k;UfQN%)y!CbXo=+- zMLB~h?qI;B%N{$?q7*QZra5WjAvJcgQu;Uct6g2x5nqs1ofou(j6Q)CIeKr;4V*rt zBKN;G!f@CU-}UrQl(S3tuxS9Ii93zHxjvwCX7918;5$l_m5qmce4L;E?#y2`5*Pfe zt*yaJf`i!p_=p2OOGRMjg{;4w1FOrBfuDSoLgq8Jfp!c8$_K2l*>N>XJ!!&0;S*r2 zhf-wC6zgxALHhqd1{Y;yXy`Eel^$kTP_Y0kwB!itbP!I=8^b(jqgt;%v|e;K&5$md z5ZBN8)F$FlzGw#*ny4PUd=>Xb5Y4C6xlWkkVE0AkIecQ`Kza72iKYMy1mpz&h6dj{ zoErnf9(&@~+2Ywb3R_dDdE;*uo7mnR*8k!NN#NeGci-O|8#GF$YDC?3sN`^RIDTpd z6lzgvDK^q!AzZsN@~~{=dvg%cB+R3N2^M@ms9S8L9vyFiG22q-#etjSRxn@)fFcO2 zV_-UlxsnYK1nHJx`iUA`uDGbksg;9Hm!d|w+9X$(ugS`6a`Ro9O~^F1#%7sGUQ>d?XRYSUN&%U z0RS)WE*ADL+=B6<(`q;~*qED8zX*D7NRiW=I*5~~w}~(a6LqeFTvG%t5n^TuYUr== zSSNrKMn@8Rb8cTi9n@5us*NB`$ACGF@xSx3HZEfWTPsq&Sl@+-OzbV6rX?G)#VKzs z-9Prid>C^tphiC|zyDCi&eF8`Mp$U}<-L7xGJ*-zto`@f+mAADiNXL;NsF4NK`47$&MG{B2Ha~!| zFWP^1Mj>+sf2&B&N+Zh02>6s>-!i~p#uPi<~ z4F&T^at%JmTPxsSVukTX0o0&^I83z?EdQ#ZMcvYx?U}DMtl8vt-(;Zd499f2R#8+Y z6Vaz9L5AE-voG|4HGG}ZjBXcEcO$U@VDYgXKxG4S%mqOpx6uIuUf!?S-tRIq(|}#) z=jX>=fp0JXAjBOE{zNkna^r>;oF%}JFFPun>j`5IbWB9?0BaK8dDHw>re}@|B z@mRh5!0B_xvliTx)HE~;4Is@Flnay*=QHQ`O0*CX(C>@8U165HS|tlS8q;!S(^s~Y zjnC!cwIv&uWb0NYzkm7EN&!~j!Mtu)WMyR~HX#8Y!R>FvXM39U75C?FZf_%aZZJA84f8^Ef!RCe$ zhDq>+h~bj4L~d<4|8)wIQ&9zgpvq821_M7G79W*e54CRJ=J`ca`+^)3OeJ!_3{w*- zWNvHaP=iTK0miEDcfAgRApSX#*bHL<)9Bn94Cqq8cmo*WYq!3sxO%TT{EPkmdxdYl z$p*931K1d`eK&XFM61ggH_<87)T^sEp{;dUYMjODgCJn+NS!(Cb#(KA9Mo5At^~fq zrcdwFw=xH9gxvpFWbA-HpPZ7?AH?;59r(7j`s=V@bakYR2punZck@N0#4GaVRIfKF zDP%bV>syw>Ka8@T7MGR=LSx({JBxM$W6?U~nD7X(EGyK=S3c-WO}BI7UOTqmwOp=y zcK@Ht(3-PC_;`6gFfBv75~;spHTyLYwS_%+aM5FXF$m~*4mLK79_dp_zwnwz=pb9w zqNM+7(L=xme6XrnK` zdQbEU*iZh&Bm}b&_8e*eM7=cdDg+q92S*_8(a}-lSUddRGZ>zOWD5p=c0ob%e-rCB zZrl(YR?O-*h2#TscrtN^U%^5OJ{QN85m6YU**iQ$owx^F58(v_g$MF^yWtpzMLZ1f!waQ_n^UWm~-j%qowux*x}nlTi-n`==& z-;M=IHU4<>VpHO8+sLB^Z&bahRg#!oFqV2x1PJs)E2OI%2nZ$N;L4ij&nMWxE={?p?4jfq>CdYeYQuE_! zG~p_pLMvGhs9y~{(uX{i9t?3@1X~x}K{*2~R8&~tcJ&Mk%l^DsTHz0bHZlPn!OK_2 zA|8f-R!aJeCZLb5T)GlE{CG$k#dddcx&yACz=ZPzcm)LDWf_s+%amh}HGz>%)Y$+- z&v^%tvDr5XH_DZipJ1Sr=G1Fawz1Qf{>iqy8$af|=}wO+^28lzyhqELQ=?r|Mp4SC zcwnFYIve(EPf83#FB6PFAP7uxps$&Lj4Be*7vmM|2fbvG$SN*ka!bide3YAHsG(wk z_|xQA4<>%7r1^`b(;$(w=HftqS zzOpRp*5dyQ&aK(n#v-iu9<5pSV*^5VexrX4YZ!cM87eiif+WNg$i?4Uf%{iX+N&5t zRXzvB*U==|^K6&9lsAo%xilxa6z^@th;RLIl6{sn5(t?&wkr1yCfMIQa0vsl-JNdd z_P&o3<^R>u^0afoO|XOHOhuf3`;*<~f=b5WClWaQ>>0%iZHFdy?ivD9Y*2(sJlt{D zANd?CTtEkrVKnDaM@s^B3R~(ZP5dkKB`&@{;a_^Y+Z3hmPVnjNx4D^|u)MRNodTn_ z*B>`pG`=jx_V$f5;MTjGdai}Q*reE&X{f3?ptG;vDG-NA4}NH(81laBa-qcMjdP7X zN`!$IXzr(%p-LK2m~%L&pkQtvV<{Z|@V%mW2ZBySPQ z#5X1aBzb(69X#A}&ayh0+x>WDaz*jDSAsI#tksCWeA!n*8w!4yUD^IB%c-Ez0Axhl zfZ+odoVX!h(Q+PJ-Wl=7=c!IBrqj95N-P2dM>a;~FjwEneXczRFWhhy5|t(>k?Ug} z>WCkXuTf)hE`1aW2Q-rca8#0W!zx&enMe1z#7n*%DAjnFSrlDGEh5L%0n5#lRPu0R zv)N5l*U^zuUENEt9jTM`iGnk9N48~<2 z8bd|g6l-=&KmNB2=PYU1fuV0NxStMd7_j|Ie-zc>)_?450vT}|f1^Ak=62^B@noQZ zqV>=FZO--BrwWGZ7Z_wtNcX%$H3YbKXgTP&+|zswmg}Dd1C@kIZ};A=kpv~h%)?|; z%Y*g1O=T8dam0h4HUd?X`e(0aKAUPfE2jI!5_joE3E_I<(XS4E-ZrA+(TPF4BQOGH7xxSST8tQ-wljHyQ7UAw48F(`(0i5S^?^8NgBL&DH7 z8+x1BPM8Vt@gcA>@Dd36%aR6 z-K~PczCe z=d>QLZJgSK@Q!>7U1cB}<>TW6UPlG*Rz(XPiK@}2u`ifyu3R|Mz|;m@{mayP{AFqxuK=xT zL$G=U1E%O(FQj<+gY^Kl96>qSn|))y$Z?sygdMC?xoW@@>fgsGcz1^%N98|x+;4w* z+`PZ{G+WS3Facvwjxb#WFSx=3jNVA4A^8*#p(&JX)Q!I=qR3v7R;+clHv>4OEt|ni z6Fgvg-TuY&f+r)>4t=uOiRg}jp|dmy57fn@o+1?m7E)$-56(6nL#+Ga{CzyUrN3hC z7Wd(nHL$=3Wf(|p_2o04^>Ct4c|_C+cm*m-%D-Mh7)F+ZYlwaM5*tBho{QiuM;=7a zI)46?;uimhE7kd7KRdBAznl|XkcuskA0|VOl2+^-gI6#ND;_ZilQI*=DF;EHzm(+* zm;s&t(zVB+Oyr1)!<2bTlR062S0tBdIqjLs#^7*4kyeMpMJcTD)#`y^$hn##cOZ7) zOALXF7-al@4tGc%LZ%+Jh%bd(;(P0S0qvaj#ID?jdR( z1t<*;Jt(#)YV?8IP(Fm|vSxT+6b{&bVdjhtUVH@YXzrb5d6M&ugQU@y317Y>UN|fEov2iKeHfdJFG0D6fD`8>TLWJ){9wH9=o42rn=I z^K*v%cs(6V_)}6+epqgRDX<4+`p3_qp@SLmNS15*gID%A{P}%T6>US zL>m9ZNyFYF5wcg2h*C;f*=0sXN-E{DBP%jWMwdiZ&*#*AUHAPS z&+**v{k+HPxQ^?n{KxtG{m$_npRwh^kknbyj1BK>y0u?^rgu5&7#~5wmv*1nY};Av zi_%U`>s-{KEh~)I@CX)YodG`qYhiG#bukb6hsOrky45_^ZrQN$uyL&H z5!W95sS=Ju>t@)+ODqWhJ`<1_)X5+_(`>2#zpWSmQ@}Voh50=EKJR{&BvREDI(aI} zU!6;(h!;IAtmx6I>ur5_d$XQMbRVh&xmvFoqvHZ{7N7i4i_VC;lYRZF-lw2PgQBdU zsd-)DBJi3JlJIJ?aRYi1Y&|0ErNXUBdoMAAfr7d22Q4-~`Jwv6oFAtYc~3?k+Cn?& zp;6W&qp0CBhem63Xv4TYH8NZu~+6U`)Df)p#jQx+Vr$}`J zsdFXhCG>ZbzRrSYuha3*7O6?H3oa32TF)>+93b>XRw}9k{DkT1l?cg!RREYiCj?qD zPLy_Z%iP6M)Z!cMx2ZZ`{d%2sxG$NCqO{GNr|71}MDV+@4cvWycZP}?$;3`MX8VOw z6VC333JxLtK@iQ?*O$npEgnnI&dAb?ZCOIh3A66dJux6Z`RAXaD4r0a29)7Y>e4N+ zWWn3-k2fK90yS=<(_mc{QPdb22?K|^A5=T*;%NSdD@P5jBjcj3HYf0+0dvLsr>xG; zy05bIZqI;C2|oU&+REy5-f+6DikvDIIdl~pY;Nobf z@(x~P`ye>?!l=mRWT^RZzN;491PQtMdD~_DCcF>j^pj~W6EL6Hs#7iH z3w*UFBW?saQz&I~bMcm`kvNwo0Srrvw;Zxw&L+Fm9x6-l{bD=#D>Y}+Y`%+ldXXZzK%y2No@tVgswnF=BMPo2o2Pe__N((qY zV!d3dfa%OYW!0OO8NYCivm2el9JKWT zyZda^yay(l|TP{s9J^k)J$JUXu7F3w_4?oCV>dj zE{iAki*;CAK$O$O`5wr31p0W7M3i+ug|3T?@y4vA*?>g;UhOxz{(Hqv&OzxNj4 z4s$cM{X|OddE`^8{%(R$pBoeJQoW@u=|24t32S}P%a2NC+I-4*t|N^Ld6qdRA;K4UE`!jZ=(-qdRaxbv`JT zo=8N1_t~6Vd`x5Y7%;Ty%R81)2D8JI%~USke~wW`1g+W}6+hvcuVS?_gG057B5GqI z>ui;!a11cX%n${OQy2JDP|8||DHX-OYAP=X-;Q2i!jV?35649^;y7wrHLf6T`!?0X z_WR4vlP?d(mygNQQGA+SwI8Y%epr|TKwr1_ZU)K(UTFX6h|1E}uLHO(D&AuBJPr87 zv{YPW2@Y*i-g}w8`l@&BJt5-59wi3Y#%{9*IM_~r91wqx!S3RA;RpD~&64S}8lsfC zi(41~e~i4Y<ynhsXGBSQ5)W6v|1j}5>6Q`q%+tbHZ;+22(3XO zcH)Aj!RY`bkyGL1x__7P3XrcB9Zu|O7WCW=tZ1)ysmST^fczTLw*0~f(O*R=yLG@| zk>cE=K=mqm>L-0Yrdd>!UEWtpMhq_F*6bzbPaO>z9Pak(lptiuD%PHMGo~#Z6G7$_|KP@Zqp$}3cRT&3$!~~^9 z_}S9i-FDI(AMW>C0n5ju-l_vHi$JWznmXg6-kF?-Pxqu zw=;m=%BcTtX#~z>1nT^T`X8!0{zS<~Y$V%Dzg@2ng-!}Zh|b`J@wpr+MSs-tb}qee zbqTX6*=;NCgpEjCzdYw=U-MwjlZx`#Tkh29&j{Al0DfWAdZm@6B+CZ;<9el@EatV8 zoV|ZgQQ5lI)(X{R8XvnmFEY-$m6x}P2&S~G{?Ygs3bOgc>cZ=9CZ>u_F#p~=@vKvw zu8}$xNdVkOd8rKkpRy0jX?Dm{0JQ?TN%N2eqr^oJi9KdP*x6}NuN*oymfThNuF%~| z=|z3tHt&Q3Ukgerv_R8VtNygA_ojrE=;@lZb+={NAUZ*GvI5s}1_JpnIJZ+bm!g-Y zT|o$&NG%YNw@y1iC&DkZULB~FDWll(^T00zRbDT6L$=J-pJ z;v+M1j8|Y}>@ez~5*wp$`1poiYZ&!si0OFv2!b8wcXI&s`KWOZWQdmp?$>c203Xo% z+y%r}oFT`0+F?DUjQ$jv9uN?m19r-<)|E+E$t!_7)aWG(8XH@Fet?bc{Q)3BrBLiv;2^mAhKxa zlR`h~i>sp2*~)h9Ro+til**5o?J?R5E1QJ7ue7?`nZIZjU9mCvkn42g3||;0{2`@i@JWwC*hd}SH;Z$BO9*qj$aMgciywOE$1}5 zCMGAoLO*3X;C5qDR1yMG+4#LwT)WP$ZER{1v+LfAngSF=%+P6_Re!3NRoIx|gO;>a zTlNb`r9A9BXJoVyY8v7A2OwHN)*Za=XDKH8XGz3cLJaHCk{=&-}rCKGJ4ovV;NNb5p@TDIeNF01EB$POl*H`%*~z|)8xp0gy^O`(I#{8Bf4JF_iAiV zWa`HGp5(#zR7Ox&kcw&7YI}*NgBC_$(YjAjdH+_Ncs7*~2smo(*QJ*Z5Gbe{8id0! zYWI&+t)H7ovXkA$9DDt=o_VmWHaTS~#$05~Pwqd(|z7E|Jt>^P3I?d9%YgH>&YaUbhg5vN!({AgWG?a{e| z#+ZOW-=696{#k0`X}5_|`PKMXugWp?1A&K>j_MvhB?CZll#Zt5mlJwh5IZ!RfDv0& z)v*ku6KU(`X(qs;#_a=UF)CI+oUkO9s>kLcpkX*HdeDJ+Wj2~;Mb6+=DrIKZED=O9 zO2lD=K=}qsCHYl~w3SSC4T+<*FkFr^k z0Y&nN%qNYI^lIya*F_%S6m*m zGQ$U1*Q-xzYiM6#HsnD=DKbIJ=h-#Kg*_n;`WE^&a?OUS#!Dn??D{!|6+sxrCR;FH#_Lv?QyXyUIQ&yb)d7qlJlHq?9-)XZ3~7{B$u zrFZJIztr8l~l{7isVqNV%F)L&U`D^P0GaFKAIE4y! zEtD$r+(-h{a#^Q!1GqliT`d!CkI7#5?f7#wvYu4d_nr-VUg!B>h_(NZfwgNVa!Qyx zQ?u6J8EIRA8nM4lTTK#db-Nx}17s}QNvDR^pJT};lsK7UxgW+ys$W^{ zAppmH@*mcVUs=1w(pe^eUz$157acn3+YhQ+!aW}huU6N4VIvnp4H8l7HV^JDp~Vqv zt5qTX`zlh<=?vzGA-v-&$7-K_^T$3?Fwo`ABJgPmDP3LB;pcZ_*TiVHYMl688$1v5 zOa2UpT3Pz69VN}8odJP_zLB*-eYS_AD|lmz*@eJc-|v50dJ%t=kRMb?srF{)wRFbc z4%O8j)Uy?%dyxuqyNHBs|A={n+omzp2kty)Qy*$5jo^DySZL>eK18n&Svx^ju-iyt zBK2oVa;6sSF=x{>e5RGL10dmu{cf`UvwQA74&uO(SSHj%Oi7Ydwm~7{aRguCij0areGiOtl4578D`fR;(i2{XElWK9hlu2%+S->9e;23PX6%dg&YX<@FW*vGr7E+{$nF)6q^MVzT&|U^@0ktiMVG!c%{=sxHk!_+T z@Z>O9Pv`)1@R;#7J&8JLK9SjtCnM5R`}9qxha_f|@8jG2DDLIvUFR&*d#fT46r@6P z{3459yY8cWD;4rJhuZ~(89Md+?(2v%Mn}>}6*Q&|&8ETA@sMHmue!@IMP zQ6I_6w;J4{7%aGGyZJO^0)GbpRS4ahKQc(@1FYvI!qWCQtk>Eq#Ft8f9DYzb2o!#g zR)j{`Oip2(#DfD^pqdqqVMiQ}S()M}bF)^+?! zV$Tk$_3G=vm^;|k%E47~Eo6PjDKIl5Jm`l}JtZMn1y^Yxko6`XzF#__pO>ke;xgCi zQ8f1DcEi~NIwv&FcQ|$(hLFE|0dckh)Nv?7V!{X~3r$CD?6l z+eF62Uu8CAq%(Y1xtUM3YJ6|!s)IRaN%!BWHSzhcg;Gvq3FU|sNvDxl$xQh!g<{)3 zt|-R&sHZ+q?i6+e+j(W9NQ*4ld(A-z@&CEU%qf!<6-kmjFJ?>JIET8MuBhaO`>bOx zW8X8+d+ugD{iMj&KloKbv#Tu%qh*bcJw^?z$8M%!U=aaiU;>1JY-Mv5+Zo6s1r7t5 z@9uDv?af~<-{C#)0Qo7J|MMsHKK zwD~-V)}otQ+gE^8Oga@4j(Y-~oyETReXsCZ$Q^pN?rUk>6%@a8rAnmUm}%bcsLwig zaq^TLc4VFEp0}-OLt1t>^ff~t+*-9swbXny7mX*qbZBte$4@LV5LRL3KeG%~8fRyz zv2$NNTzNLeRd9kBI&<9ExQ@F!&zCQGu+M0)b;SK~_{fmCVd~GVR)V7)f&&w|IfrI- zJ8azN=LfkFt0XPeCT*@BxUq}j)^!^Ci~X;SMO=>%Y|Z?+0reg2>#vrN>fJM*(~e*e zli;5cE4uSIYd~(@Rx^tmH^rCkeSwA6w`>y|ZBQ@leZth;R3M0=;C|e0-hM*x?4swg z4`;szL`R(AQ&Nk_k{@c#px-g7;Y7l|oB7}?TFa*^d)SJ-OEJG6UD}a0I!H&4((C8q z$eUq0m*hf{>-`-qv%^Q()6-up2<5j+fC&=D=aYKI|A-QyX;C%s5!N$%3GpTu*2s~&xb z0_Pnfd(4dDKNOgsLCO%YiXOo^sFZIc2+`(lARHh`&OynAD2^vOrf4DZqGw7(o9b># ztO4@bk8R~8t^S=wrWlz8D zmhbeSGre?$)_zz0%JN$epS;U*o9hj~Ir#fdsh<8*y%K#?1>iV3ybS|#P#+nhO59rmQM+12>8_lm=%1X`dpe5$YXoum1bpD~q z_eXWHMlQp$XlB^+Ub20g!)Me>JeQj^OVEgpq8EcESYNM9nJoSaXp`s8~ zfV!9TJd7@mO8ocE>4}0@iND@O z5R!IjM*0*V58o!_1opiedjeg>vzfml;i*69R@L-|u0P06ry<8nXzkc=BQ0Z*r^{{q z@gGA|U-lFkd8MW2kK9XlkG+~~XlVhiV#;55>%fX&kQgGbUDN3Qf%!ncsSe4#;Aei^=jD}WetfUd&nfD{W8*7zAU|;5=G2&r?bBnnuIF8@;s_Gr0VjwR zY}?3WLvgF4W(>8rE7tlms4QHDm>U#l{|}^f> z>a|B=z~^S(WUir~5p2fTKCDD>u+w8YG}$oewc2dc=YLaxQU7_P0)p^hfKX8&#R3zN z9bljd^7{ZB%qs~KD9F22H#JRWzsv`mIhlW3G{3ln9XS2=W-3&42FG45SBbsM;wZox zJ{rnsJo;i$^|Nl-kocX7ilcl=oz4<3N~XS4XEoYtXv=Fn4%6-6f8zJA@~t1?oYYcu zP+9pSP(3AotBw##%l#|-{-W-eaP*Qz3bY~-pZhrS=4Vy#&1kd=4!v#V6{?_4z2<^s zLd>!h-Jpqg)!g5{bm)5iILPHuyNt53$Y(6~J@AH(c$$pOv z=bazgD%-Xl_xCU*p)S*{%NF~TPfzBT=N9h%nN#i2WI#4-i5S=0Ho`)KL#Hinq%lSR zs^R<}3pGSzbj1hM;o!Q!3SsLs$HWU4E)Wf2)W}xE9HD`-ZKhtqY9PoPrb}YhcAsK` zi`F<)h-YSH6?iR05itJ6D6kjcMw+E~_W5mY6qSPy%JOaHS)H zMTt~A!UM-l55F*!k97?h3t7@Mk-o|h^_eC1x+zZB0R91@%z(P<{USMM8SegAE~OF* z>7RguX;-{|M8PRRf82;&zhkO7t@nAO2s;FWj`l)uz2RxG*6MEL<>&I-C-QZw8fuEG z^svRp?YS#lK0C(?NdRS?o$JcDGC8FCpt({qHP({VI98D9dH(V0Qyu)46mqz=Pdesa zMqU8ic0Y(>Yu7mMBKuv_io$9jc%|<|1`vYdpA~YP5)Kb7ZJz@1ynROr3C8ZUS*>;& zX<3Qu;$R~c3C1TEKJf9EGPKhqOJ5LPgH{4(!EFO=F^Rfdnu{bHE^l{W+_j=0XL^RD z4B9^nW&Sb0#Is_*-SQ8v>mE(+brO)7z>uP&l1+B56!M%nzd7Y#O>ahpNN;)JG1{s0 z#eztQ^bvgj4f|Vt|5&Jmo612$6GinJ1#`*Zrq>K{6EUw>;U&UW`~xP%xbAS{b1d0;>~Iw1nL z0UgUM$5MpqF3u-=5*(8+{66sbEiXya_saF|Ra83K24XcT^S`0n5j%kd`88S`eEe5% zMsv3YIooTlyUKmlve$Z?uX>bzU&V!$k#Lp+0(J?oopKiDo>BxUnlgxQRdPAc6h(@lAcwzoXinapJP{9_dyteq(2>f^Y`Mm)sd9ZvH`Ssw}evP$u_5t5JX|aV-SuJOE#pR3)RU% zT-7X2stK(1zhyQ^c8#!Z2@!r4oK1lp3ed*5hGUa2DA|zL=_TU9Iz{9h07;V`2dXGf ziY7EfRZEqei3oq~4qXUrOu9Tr)ZqX~ciGph-iNlQl1(t4S}1jd)LJ9MJwIedL=7Ca z%X3#R>$>Sx5{=J(2>z{-x%TC!yAQpaO=a$^F1iqN8LKR577fkdy3hc%P@_v%W+tr# zMN$Tzy}8#nrh8J`K#wjfyHCXU(YsuTQB$&PV_oHT3%-Apud1X18nx2dmk_R{%a#O2 z{>8mF?CzpzpMQU7Ammtr>onMoueNulu7+0sp`-|aYe}D=ftD>pc{Us0+?;QmIAmO#;#9) z%Nn%axE%K8b(CjOW46jF`gM4?&1GbKPmcamn*jK7fLT_IkS7#N7JGw@xyC9K8A&M^ zzad{3MsD&jP>Hb{HuQ!kU2_C>p1htFZG2S9fm%F4Ha*ZT23bT?k9b^;KSulyD-Y-xR(5o7AhsYN(6P_OO85~DfFuK$ zlqSFf#mTyfB;CJH(Ak6+hSN>!hcOJg^7XQagoQtGz`D9IYrG$pn?%kOFhS`He{LZ{ zb}-89g?C}vqytD?!FsDh;EL6m^%0Gfdk6+~^Up_>3vb>`fz22B(ufasFx7%{m?Bhf z3G8pQ4TRpco8-A*L}>Jsi;V(t{(Csbn?9RB!mL>m?=jb#pO^f`T?tlq;Sy*@w`I-PV#A|7*HY*4x(EqXP z2N$C!je8I~)6>&~`=_uq+7vezc)tlbZYl7&0GiGgOC_R-&F1m*?E5LesQ$;GydgPm zw|^q<@@h7aTXmBoy8GV?jcDragb5BzASwHSd?gaMP&9%5S;f1TaB1*|n@V0PQ5tLB z0<@auUzj-Njb7i?y$CLDwTQ&tcw4@~Y=R!}=E#6cjrSgfb`ww;5eL3XcPwjxlKOF`1g0 zYK={T;L8l1TQ-OUGeKRK1?u4?kUBMcb&X$TQ`-i3eJmsCDqL}%C9{j*uLX5_dUUvO zYM=j94dk`g;J|<>^p=ZVM>*hR0?Fe%%N9Y5t!B8Nqq?h$8y^efKZ!5#FH(4aN9>2x z)OLUPI6f87_GocJua(~Df z)(=(=W4#CTFtlbN+`~W3Jr%;}vs#z;2#CeB=m|M!F{S)Uz5+OBKLBuL0E+Miv(Sv5H@p zyq48kS(Fi*RcBzFU%TM;PD1`S6q#>M2kyR$l-H~?NonB6`EEp4wK|BN4K&Ez=Q$-T zuagJn&oEzZ{Y7_~tuJJ~8Q9rFVDU!$DDc!kZmt6J`n%T-VPQf#Aie_>ALu;ad+^}- z-K(XgvS6W3dDXqTE^v|nm)COauDyG^?U5_iAz}Q1MMGRAiamddQ;c4AbU?Qf`IVT* z_w(Rj+{eTMk|X4jT+X>Jl5c?r8Yk_*3lqMa)56de_aayf`^EVSluGEJgHT|!7k~xI zj8}W}tE1eMmP9;wGEm$nNNQI>T$-?X7wDs4prhaoB`AZZ9xN0T43@zq@#4j~BVHOB z8XQB^H;I!i!B!l+!j+ys)>_b80Lu~G4RFZ~fnIe#KFA#x+W_tWN?SME@?r@!63fUG zz-KZK9bmFZVA!#-u|yC0g51b?Xk-z?O=x^mO$?P55;=6_Y+$z@*cwgf;+s4tr~o!! z>C^TYp~d^l#8nS?1M(h-Hh+UJo(WX2XQ!A#p5LutNYeOlJN$FgH2`$X04#oA?j?Ku z$eDr#cVR0@Xw5GCT1i8554y61_e%)da8O#->#|!Lcx>F?Z#a3EQJ#RPis^kCieF@q zQw#2LR_6!`9!!JIuC8?*7q}xJ2F*c|M`BAzSi<0F18PL}g!;Xc;pLk!U!ODCuSq(w z{STZCINq^4Eh91qzzlrn+c`dOB?trKUpL}2GK9cChBd}1n@vtcRUB)Gcgq%P$0ZF@ zA}$IJJWYu~b_Hu$Yym(G5pamZUhUfxT>xX0Kw^}d4?)|_VCqoZ726K%X&O5%#Zw#Om6 zUj4EDp&MY&jBs&h#d0G0{(Vp@U?Zy;ZoRbXV$8;HiF$mhfk?W#wdMs^!~u=@BoSnAUH5IL0B54iB1*vlb@Mv)Be7&gURA zB$0jCS&Yh>2dw1n#L!sr-jbId?BSZXc`ItJed&#TDgfSB5bhw}f9v4}vSM!g_U#PN zh-HEHF)^^i=z;+_tC5e{I`ob@5{E|7;M^nF^5IZ4gIaMPG~{tWV8g<$8CF@jAI=d1 z#1xuXJ+)4HaEv3|W1wLBH%P|5Rzo~|F{RmXezC$Ykhtq1L<|L4!)udKeTpitKSN6S z8h{81aJ3#GHsHVn(TJbFozh6)jzI4$iNOE;ucJC;M!K&TXKmU5v0QF}C_g}AOQIp@KDX`@rIxa+|7cwe&@J_|^Cr-=6g_f{|;7v{;bQc;t5a6tO0WOMz(JG!a z;c~?N=^PSlNzw+ry_kvcEF@xp-QC?3A6y>A=SZ?53abW&hEoeVS$Z^3^hbC~hpVFy zE)Qr%#N2BGLS&-mPjFt415w0QA_~Y6;d>+2-uOU+RTrjheRkV_*G;Tl)3tVDOb&}OTPRcF2~Z$J)%`jsk&yus7-TUJ^JWGbnVRsndsbn zQC+6zDGDjd>qAN$KvkEo!osJb5!^4;P)D$PJ_j zhzBesZ@qq`rmQ~fDL?Yl^I^|;hEdL&X)l{1{n6=*Hjb1|R%C zkcUD=p{zYX!D}}JQm82u(X~{x6pG1)Weop(w>;$w-}<=E?rBkY3+1r#F{Nxplgs}F DU37hT literal 0 HcmV?d00001 diff --git a/training_loss.png b/training_loss.png new file mode 100644 index 0000000000000000000000000000000000000000..da4bb30a8417020e460a50038ba208300c757de3 GIT binary patch literal 36834 zcmd?RbyQaE)-Sxc>F$s&QKV72l@dWhK$MhHLK^81L`q6UP)b1r1q4I{l#~{wM7l$| zTk6c^v-dvl8Q*)(H_qS3G4|Mxa^GuRF|Rp)F)t!@uBwp|F%lsNLV8(USr0)l!3csi zPJjphBGxrL4gV+Qpz11hqIJu|#nIWr(aw_no{gKkowL(Z1U_pxc8iS(Hq*)yly(q)~gm33*9l;Ka{rOA0D z0{-NBiePbba&m5**F+-W?}sGRittxXg?gNL_-jjt@c(;1o?~d-7RYTHcg=_i44}Tz3EV{6%7%tQ`NR%S;Nz$r0^Rhwx%Lj9$qB$#-BZSS1LEJwpfdn znMX$qho6K*-c&L*W$Cf^%-VGxt3DDl=MxpBN=QiP^RHBp@m^y`#^-Y~RQ2`svtsv; z4^~Bdx8`_`{EcrqH0l@`MQEjnPlVB%xh#4{v>X{>aocA;KYx32e>MXjdHwpeA_WdX zNbp$T0Vxunn)*n@s>x#DgU#H?7YY4yw6;H9;3D29N4u^^+kIT;&Jo?ee}8VYO2#bc z$TRteZIZ)<9-K1-m~{V*%l_+)^y*ixTvAj-e0P>2PmWfD1P{b|NMs!Qg*_H~NoBlN z8aiKHqNJb*zPCI$VSe(5-cdZIZ9SZfP87 z{VT(~gyiIiFRRr|rKP3UE31Fl)-M*;#I~o&Tzad|3G<*nbG4zueKz#~B4rA~0)xzX zd{z|`SlIT>H~m8AH5O3jo>FH~`{>m9_{WbiVHfY0mrJ69+kV#=+tIo))7F)xMxLvG z?xRB=*=qH{d-n`<36BnT@e$dL7OwV=!A4gxn$W3|4VX>lwVbOudB5iR3fVs09m3P4 zS#M%;Wm_e*$qS66 zJXcdAUc6wWp`o!DE_Wp$BvjJWBstzls_8k%PfARjSZ`)iGcjR?s2VJF2&Xsqr&P@< z9^D;ujd}T!eb{%gKr-;qJJdB@(iKB4yM|%vm+X$fUc`^Uq> zChKo=(y6c_+MWqempP4`a~k<DZ5L_k$pIU-JWBce@=HE{2i2rqA|4HF_FC3S;Y z*dlmuxoj+o@A`Y1a8f3oZ`PvgJ#VkIAITqWU84+6PD@)JzE@8w5iv0_Vc8tZ(nv19 zr}+K*cN(^!Ls!b+y@A^qQoC(!ZPfMk^^FY;$|77`mrN?$vL$bQ#X(;K6#|DV%yX`$HIT7LN0|Gshm{8& z`MS<`XHU9!$jw&T-n;iHSMP1ZM16QPpJ6^l(803AdR}2Q)fvrX_Gd{+<|6NTuf2&t z8qQ?+>zP#fmhRQzvqiwsw7hsR#K^>iKPvO=%0P))s;tik0!8!#<(`X%A`%k2QL{%I zZ9*#9db|*Gx-=?Xwk45VYOypnqR;o>2hk3_`IKY+n_42m!a7SMU&`(XGo&{2oIV}t z>go!4CA0ia7pLpYx5UX%a{03uiZSmgQg}-7hY#KRasl(%NvRp7(Nmu<;OTx0GJFjC zwzcKq3I}@4{7<3e$T_+7V zhWg`NF+;b0AD`8w+N;+GuIWrxlRt>^kq^i-!SivcuHf3 z!imXKjcm&sH=b`gBwax~rPJC%t6#oUe5%epnJVR;IwY8cU?|F*$5!0h1x-Ikz}9$F z2SMbhC{Z+A3MZ>rl5m@DanL{@231OLE^i3atLT1jZKX`q$Ugu4EH5u5R>W%MX1h(t z(b4W$G|~V^h_A`zEj$OJrI{^g*ydnoc?bqQcDy4o`{}xwj|$GXCQ+g7Qh)IzoQxI! z8sBYuh;#_!FH2?CtyJ*e6X9&qWv3wsLfCu0WS~LG7^9m1_AMnGPOp_=pGM`GZU*J9 z;~SB&v2=NPd1*t2`9>TV3cu}zna4|2o9!~3i0jGmQTtukTn!Bk<>#6_FuR_LAM2Ze z3vY848gUf;50=YJKkY93y0Y{$CvNiypUs~gLs4_IW1MvKyFHB$L(!LigA3yN{moJh z9QO7Ui4L2ZpySPqZK~P?F14uj_4Rgz7G|_2=w|CloumO`32NrC;FOr_%EH&Xc5U(Q z*cUIMmGUXF2jStD&Lv(HOtON)Sah=>>2CYy{uqhTi1BfYS--Ta;kmhIQ|0`2-ZiVL zsae$qV{K1X?)7gk_GKBDIn8xvN4Et+@;Wi-7>h43F3WuXp6m&amfJvje0*?<(`Uk( z!wq7D9cst&P+2pRbLIWAu|Ouj-IapA3nO{gh0f9tX+9Ul%?Q{zbu~kNB$pxZX!|;n zOJ5(1g&qIw+3!cM-rC#SuWfcHe7R5sDeZlo!-(H%6#)rJzE{C}(=VB~JJJP*SiQ%w z=C*y821;gMSh|!$%uzBh%=u?LdxnQFh}qNNDCL$N*=M}sBfW6yyzQelzh)`tFUy~N zhlHxzKYK1xFfc^;tyDDIC^YYOtRJ}@lCb%&DJ31Dk4@Ue3&lbwvGRFQ5C;dR*0NaA zJWDO^ArvO4Bk#R4l#fFQ*%O#9T);@2IfD;j*|@(sE8;vV>o{Ca2^H64N9$c)-j@X* z$dZe9uboR2xaVkR$Lk?&T6M%Y*vpWeosBRa2R1Y{N!JJxU;w?)KVp`mDy%t{{7IHibSD`<(+pKfqw{XOHO$WyN0)w zT8rMOQj!c|XP1`#JX)HWm9?1GS6m>1mcvU%M#*h{VHaH+m+?X;qWb=zXvMzgEKKgg z;!S$GpsT%y))y3?Li7PchFxVm^{$zz9`DwR`HixabE+iN- z8{g=egqrcQ5MSk6_wf+O(u!JIq|XG5AHu<(-Qm+(?HUA3plf7g6#hS)YYZrr;FaaJ+)W;?Af!= z{^o*5$6*g2UObm*`Sk&z_hjT5#!6c_w9%v335Axpkm1fIY9KLrgHn>LF0ocvewJ%p zYHX#BE&9#Ew=c{mzPRH2d7cM75dfTAr&|)*np{ppNJ7od@7r3C?91-E z31Aft$%5%3V7vY@=Zv;b5Il@^b#-$;bI60+f_eq%LuC&4rs61YUYT8Ub{~OP=DvU5 z6h+Ht@(InS?Sv{X+%1PxFQZ{*JqA(0(1O+qi30wveJbsgMikX?20@@t| z0KufubOK*sM`3&@<_8?9$tSF3h2t(s$X&e9l_IkD`lj>VlE)SO@={%dvk=)X99 zSr+H!;rVJIY|)jef{dp=d(Dw1?fLa7?=>`l)VZl1F^UlZF{x`4s(JL_0gmj!Qi;~F ze1TaNBLG$gnS1$dJCNP2V9oay%}?LOP?DPH=?d5qP*l+iq3o3^{Vn*wkpd>V>NRvp9e#KCG!^V&px z;og2V9Xeb1dt5kMAOnm^X?JC$*zPA!<<6i31C;2^S!v4#0+XPlZEv6vI@hj+1|I#n zwfXzU-ek1ULYn(%rB68|C}EE|okJ+#s%~zg@7}!wNP8LG_V&OR%Ez&U>|J6gHn;tr ze7{5ffP^hzi_%e?2~D^pOyWx)nmtyAPhCk8Y`h{=^+4fpf*hDvGd8(wFkpSFo?P7m z`a@bsQkF-5wv)y0GQPQ*78M+f414yOCd>JW1Gbz{6^Oa_77{EwfP28>g3tJGU?ALY z)jyml?K8Us#UHExz^xS4gqV(B(aA{=NE~3mwO(WA3BbFM-*7(I{$Q-!_{Mkk>{&$} z9STU!57`3uu((tom0OYkv{i=8-7p{i zn;!{EY=5A1$U;12{TpB5I6z|3rp|6=Y%MxLlm5wB!?*f-+kNKiu=JBqC>32@MJ6XF z*OF=u8~2;BEKcgDU>9q*=6|lkNvgLNz10%WVN~IETH0%6=`WD6=t!6I2Bf#%eMRV1 zZY}|Y{T#$Z2^%{lRtVtuoKSYoRdJhdG;mHO&I$;4-do$f&hvy&S63JPT3YLTsruEc zsz3}Dw5CU1h+Jqw6DD8;im;`8=v9EV93%3GNrG&uDaIkufl_zz7DioJ5EwZCLef+jFEYAlTo##o4MWZ~ zF}Krwntr~bq|_z8MJ)_jJ0WA@g79Byq5%1oegV&4ugk^pDdWNc2H%nE_F8cwn7$h<>(t9?|3qkDc?D|U`=tlwN zC*kN&;3%u9nY)|#`1q)+tC#zn&ERKGhpOyXs9PcPcW>zd{f33=hA_Pt5g$)z>lxrys5_Q(phM(aQmd@ zN@Qk4em+0U;iGB<8X~My)U>q=3-&<5y7%Q}WSDN;xIsWndi~$jK-fSfg*F;tT3Xs@UK?#v{2?#RRX6 zJO5m>rK3ab?%jf%ORwl?Ktcg_4zN}`d@>QIOGG;4g=S&jrMzF6A3i+J3XY8Mj|U+n zlKKpLq6X!uQ-om`>)s{AQL9kG`EG1%1ON_`v#M%M-#^?*Ac9ksAd{@g)9)zG&CSiu z&R+X-LWdY=7?fN|za4%}o@-AG_c!S8x=|3oZd}`2bhEg7D(KDird6{4c_63H*tVxe znpOFV%_l=LzBTyKjx{dyCfxx-MM=3IS#vlee(n5 z5CIXRS5fpejG$V3Rc$Gn&a(4Z=hBp_m+%i z<&(s*&VVKaMWdeWWKRtTAD>G^gc1tDt1FH{M~mhOfR-Rvqq+}ZDbeq?B{DTEJ<6I$ zRp;*~r~6Zk@7wsb`=P_Cu__*v-LQX0Z|04K*0$Rj0S|%9yaY7KAmzsW?b|n#9YC67 zplXG%%K7%lG!Iv}um8GkKJimqq4D>R6gYRqmW@$)W(QDeF2W3oyjIO*kN2kbHZx8h z0TGa{VP$511i6e^-k$*~a=jW`z=NQZgE6gYH8(bKEWcM>a+$@Bt3FFV3KRovSS&x< zYllOQq~|wesvhxd=jvna6Jiy$n_3VR6B9Ekci|8f6%__Nj?|)00y6IpaA$6;Mgg(d zSsGxF_usSWI1dQsd$I^ND##(@RlB40-xB!>eUAXDO@Ri5Z&A2&bTD?(ekj)@0E(#A|SGrYhQ_DJ}bB>7$Q|VZr&7noEVO%+A zsz;0TtHlly&#Y+st|S#ut902ITUd-6bo}a+djGpyw=RUe*0w~#%6aG$DPVs#&_=_6 z?_SLNWw}1t=nZnmdtWnqIrJ^h^+Vnl-^4llTGCuMTUR_yQcuz0Pvea)0QpnJo*SydnJG^R6dlx zw-ixwyF+|2wPz#w?|gzy&MBQ(A9Nlx${2~C;mjPv#lxHI2s%EW9(3%J4}db9oUO?! zPVV90;7!eR*@~`NA2@Hqx7(Rfg|4yKs(J1E`Hq*Bm6c`&MZD4;bHS;p%=~7R3?Pbq z1qC&FrY*^QbH7CnecgW8HZlI|2N*<@oIbPs+gvM{4lQkm08zgJ8;T*M=6R`|Aus8( zfr@OmA3S_0{)ZVoG_D{aqa2?>)>}9vBt+P4T0K|)okQ$myUc1sNr!wiq*Us;hiX*|}fc+XZ%kZ!f5?&ZCSbh5aJh`|V;0)R6He%^}5FT%`aGhf3b7jUjD zW)L7oK1B#^DUbF;6)0$T5g5biq`~myjTwq~5pE2grv zB*-1?X@*|{jL6#2ib-4}`o`)w!k0;pFPriOBQ9R%2bdX!mA3FY$8B{J_~G;c#0Ky4 z1`(pYT^!-6036GY=k$A@ueT*|YKMUY+;azHnXeGp#)p;Q$$)fix4!v@m?zjwgQMg+ zBG5aA@2`Fuv$DW~j~IQPO;wvjK}rkcMs@( z&y0b_OgXv*ESL7Z{5elSMHK?oz0R_D99$7KBO`{%si|VnSCckbBwY}R5sxRE{@31U zDtUUAZZ3+akirR6_4k*3s&U$Pdm-#s-gTLpbZ`j})HT6Tsx7)zZ_xt$@yo$gUoC ztNcq`6;ZiIhg3VEx^hp5vaWJL>tw?9Yk?WFv$InG8Jf{j#VqB91J7eRUJF-{>?j6) z52zGsE-A_|M1YY1>cV^V>Q#SrfDFo@AkA`0^vUNb-@Pk<8c}duEun1gt$bN60;xxK zfBH$g1IT;i&Q;rVC`erzt6>BEvVp~G_&$(E1i81=kA)b0xhDX6O0naB=xlM#@gKgc zY0_cf4A7#&8XSiBgoKI1KU{ILAwNGVeF3RQdj`HLk^FKj;^QHmc3tdwCvpOK*}JDfG^aHQfx3O zQQHSJNR>jfs)a`Pg&tn@Y(1p5+;xgQLA}U(-7;0$lL#d}Akk1TF)hljJTrEx14VLe z(4lYwba+J{A1T9vn=^-h0(M8jA@;HW_0+9)Wh)FtS8VPt^Z93D%5T-<4F?`9N5sU0 zu@e{ph%kt|0lwp(on_)gjWE=I18pm}tfM>oN>}z3deD(P=DTs*l7zf2ZS^&YP=0K^ zS%b+XjkofqXF|l^Epn^_7BIsP)ZfR?)kLS2#GRYunO#l$c?Uy(3 z@deFegq??{$!8gmk|K=?PzKJkXIu7f!ssEcW*#}Ojn@gzR&L3OX-ysXtQCPHwzV`s zjdJRh>VuWMm5j8s7!-CAQdJnA8ObMNS#Vd=WMy7c5Kv`+}3xl>{fKU(tqhL zy(`Z2T95bM>ge)gZVc!Q@te{;AAln;gI25>!}x3NNMUar8`Pu8G_O%C!;Qi zi#Z=HF^;^LprkCl9KwF#^QqpSkfUJfg#ln4f=#QYr`Ii3X6*?PfF4(lhuiy0Q0#eDsGQQuJ9 zY2*($VX}MUp~;f2oTy?9`C@6f0u|620n}n4C`&*;-)A1gg2Dli|7S%p&tJUP;t9sZ z#ce0g551Zb3Ucq|%dND#yl2mrFZsl%0za&3YcriD<;V5nZN4+^sTQqs^R9<6Khz>q zaB{^R`-34wxnOaHsPFWxITb0`N0X1nI z{%N9OHXS`=#b6GB{7H_|QACmF$pql9MsQ)+Ky;V{ROPs}tpHA{W~wA*huki+wCBPD znU!+vp)zN+ix)A71u)q?$haZ^=26W6jbhN=JhXN-?#W9K8ByTyn^zx3wS9vFIGLcv z>MFj9+0okp!7cpo&>xWXrB^!R0Nm<<|E>91{+aP zQS47zftv+31_%}qnK?qdgKF+k&AfT0A~Xv_fY1%ixNm(q=uf$yv@Esy$p~OT?p=^F zP}Mu(`SbY?cfleAz?|XE-958&SzEhi$tO|c^aaKk*ccoom-l#X*|y_DeoYt8QKCa? zf7(+nz>2~S890mi@87E$8a{KsKMw%`ET03+mEbB-3D-&G*<7$2wqimf(QP%xa}F7a=f z2UQ`Gq;P)z{8AOTXUsUd)bEQxp zYE)vs{+pUsG3a!Xv>|qd8?G?7;{8`FTnM7DVnrJtk^9z&*v@b7e3dAGyCwT4O6HE5 zu2sXIve09lR*a?&p~Mr)ARTkAmFJ+rGEF4x_ZRs?JL`Wt}I2>aM#Xd z1+z45MHHV2KG<9vj?h1*y`tPh0CWz;StWd8Ou%Q&?+4b88-dT*az#@;6-!>n7zCMv%FNHg4sNfaIKks?CYxXy&} zaB^n0SzzWGQ)z2eNieQ1kMNG|g5=e(sR5mb0*Dr=c)cE~$n3O46NZG*%zj&pPc)u* z(}IHBBy_+v5?B(|m|R13*DEi`kfetI+cO`cKZwQY?0G=MzIHHJ%SS$4-`jE=iUM;5 z^qDGLNg-c)NYgke^Ph6mbO*~x&8!f_z%%_B&_#P9UPaE1oVC#-S^Z{ffgS<$pMNJ4 z3|dmB3_tZV_VPQw_Ab;B4@S><9dwT5EJB7mhbF;feV&gLi}W5|tfKUDA0W|$7uHz8 z)m{aMlr1{g8+q8IEU+RD?z3TL6k<($o(*|8nRp8fly=~Z-W zWBB(?{BU&&;wRt0z=JH$0|iSMgN2x}{;`BTC;WLRtg9AZ5*>5P80?kX2HhMs0U~iP z_JM|=($yORWHa-|oEKnmH+11$sU?JajE0yBqu`4VnPiIpGqN!?EQy+dUMb!!tXOm_ zHg3a*P28Ws)@r3?y?lDv<;~N-3wrzr=IF4Z=R!}c|IMW>?DDJYOLAjNo2Vw|yzv~z zr7i?l{qp7e8rc^?JwX*Qz7^=KB~Y)1!}KWSr$br{u>`~Ajj2*;1d5u6+oHmz-e>YTe}5Frgtc4;%iv4 zh!vjvpE<6r)iE}idrP&bCehicgW}ZoBdCf{uYbGcYh=3}99;;r;w}HJi4#!FB&~8W z>+vfc=Z2l1+3KqlU7WM0VWq|C+tY z@~nW%>Oi+CWQgv%^T4do%cmkGi=4evlIyQoLE{SO#A$^O9z1yVrHjwyD&ENV3WNoZJDfu`?_y$I$XG0GX^Yg~ zee({$%&`-MwE5}MEZ%iTtxwd5Ctnnv3jERBf`Y#!?8%eR>1k_JaOKRLP$H*_!~xn=V~Gp44G_Qb z%A1$2#NprtI^&sn9!2%5;;e}6HThX=2*^~2DxC6r%)R*0W?9N zC<)a9&{_}HCBK`So3Q7CKC<&>02W<(S0V6F67`2rHxCVKG|)ov$tOWTc3(N~(93@a zm{c<0&jk=Ngnc%w0ZvGD%xf_RAja3tS**n21d;_MV!x>i7Lj6XuKU!MOSYI{_v=e2GFOp-stz z>pzlP*$)1&+dG`w8uneFso04K_@#ZJcdA<{nem?gbU{}7qiJ+{3S2rj-O(qiu_@+}77Ywv$91GX z^5UEm<ONW}ixhD+-o^BLXErSNp@^av-mKOl z=z+OlJZz4o-@;v;+S z$upr&Ve-L9r@+pvHfzTsxXdlzKMbzH|{yBYx=XiDrFk3!v~Lx^*A{waT9v_cA&4 z02sgzfCOC3i&$m0xD-g!c8LVG(2)E4UxjF%Sou3NHLS93c8YK&88r@0_4|+QsRQM1 z_Ppyuq0KJ(<(m$?sb;^ikFESI<+{eTFsF~dVdFpVe!Vkp9!L8s_gm*eAH;6v zGjyf?{%j$aShes0h0$G3L#Vd@<)_}3jLFe4i%CfBC<%X5fq59ZPR}6b)x7N~BI=XV zMRZzE!)b`As-R_!ypOIC9;}h;x}HmVZmpfZ+oKgaY-G=g{yH89?6{5m6GYIc2;t@B zg}gX7R3;?uH0<4Y1g1v-Xa$f`vw)!NLC#rRE`g>bRM&wnY17XhbLK8S^2V2U%rc%F0F$aa^aldf3g@D z2M$$Lss~Qj1-~0omTgDeHx%$X*k%+xNuu`V+myt6^Wau%2A&@`9&F9;JIk|W5*vSr zUbb3)T^acz0?jW;%g_W;3jzV7Y}1_!(2!o?crEW2=vesQZOgGi>77;m$N=xKZ3Jb;5!oC_%PSieHN@but9zK$smG1-!scV zqzOXCH_T`neTSIM0{3muMx#(X(hw8DX3&!&LE{8XNpD^(jT(W*S?mJCMgP;>+$K{a zCMkCcs@^>}pC9-o$wPJ(5Am-#V15^aDu2q;4lwpd5fQ=q=L)Vr=R0@q-Ss+mXlOV3 zc!v^b7_{J?nx39!my$XKjb9MK2m%U$#C$3uY}G^px~;T%MMZ%Oi^lgboFXIV2fr?% zfao(BrFtZRa%oq{%-_@7G7FF-SLbYJnX|dIt?g&Wj_==NO91w%LWz9WIeA;n!oGEM zcs-{EH*gYleNoKizvlwYZ2u|o-Ty7y2REUp)Vc67Bzs)e75rR-Hjz(W%*gm=!EISa zV)S&!Xn)6^E07h5i?4HR{2;dz=;gH?8+wzY>Ko;X`$xq%yvn(Fa(*pK*hfud{eCp4ND$WfTyg5SWRMfN69^)@&tf4yW}PJdW5GBr*_fSa{qgaA z4ejttFox@BikDIy_Vtx!Kq80?Z|RpYAc}Z-Xn%?sl<&AG!ipk(lSE3oydU*lNF2#! zJmgSfvGO{1HhjXv2MYO9)zfM(^@vgkXnb96NQRF_zG8@{!YeUo$Vn-YV?hhHDHe=` zuE6yX-j2V5B9iQTNd~FCKIE=*V|Jj9_utPka*VK&S6sd0#`26YOy88^T@2k-5-3B{ zjj$F)XJe>Bkhw+vT#=iyEM&l!gFaWwAr@V~9T~o-GQ0rbzyx*fmd10MiTgyz=r!ER zs@weFcYvN-Y`N<5&umGdUQJcn!BllxxRd=&iIDMn(wti*KKHVzAQ`+vC;ZcHnffCA zLKPPlPVfO9H&CewkISUJ%z|4UlITzgVeJKFbqFI71_5z>KP8wDf!;UaTSWmr<>b8? z8+*Qu@UdwWOFYrQd4SV@fc4DKFHl}F{PH|6ytd_AL2c}k=syFZzlG_?Q^pyfu6=C( zB%EvC)VhdjcPj=TIlNtON3MqM5WN+Acq#8Q0BA(kR=-cN#y*5_VopXrVKF9>xV=U4NORXdobFGr%I4z6 z6#aJL*qJf(mS%?8W2kGzm7t@_+wQzec&^3puc1-;wI5>5vdpBghgh1(&?6TiW`arK zE-cF}TBUWLolNDKOAEXw(na>&v>)LXHKFV5;5T1Yp>j|Zjcix&D zw^nx0N4pq0%2K!g#!O&1p#{@aGI>!0@hrB74&Uo!c}`kHIB0}WzsJ}40}9XS_BAo( z7Im0^S#RL*^@G1fyL$pXzMUnvwQ+E8P<0LUY&5O}nTJ739X4EO@H<6<=QS2(93}(d z;ic#4mEr9XOZIToqVCZ1%s*RWz8;C=()#!`+5~FmzperoW9ZSYchDN*pFH_Few-qH z_iIBWwFnR`W@!%s$l5ITR)!;?o2AD_Ax1S6#OtIajcofEV>c!>+@i{+>0j>4mrrAk z#idD+wf)}nlEycuCSWxp4#NQqK@W>Kk9PVa_&yDE>!DT)=#9aEPsGblpambjF$b4U z7y*r;J+Nyz3P*%c!QBS5|910zsq{}%JsoOtuGW%T_r~oV56SBu0ruPv?>!10I5)YS z5$E&I7ENpu{`Y8uM^26fELpU}h68%ZiHX-iZ=d=!QK>2N5?v6MA`7=01Kmd z^AXx!{tU4?xOB9#OIq)k^e=4c&jfy<#aRLN^3mo=K1f2 zOw)R+?VDMp=VX0NWh!?UFMl@;ppnQ?@d@}2iLUzlV#bI6uPPzotqEp3H&vhU6A)Hh z26$9;X%Z3>e_6QAIrI=guGl%)PCPf>b8DHB^0nf<%cRxAO|7p$uNiJ#q=sT6A0RD% zyk~_r3aiC3*P)#=(5-X($qJ}2!hhWf5XEFrCksM!654?KD}BzV2mY}m0TUWrJR}(U z940{CYXXxSUK|FJ7?|#D+0Xlvr2b&hUH3Urq(Do++LFt`bfD{-vxwX&(4>A`5Ej^%= zSwVLKgRq5S-dKi=vfa}b-aM68qrpzDH?_lmp55KnZYYy+FJ3h{ZF{mRJR0;eF6R<# zqWu$`gnGfk=Vpb_g{@*^gSt97^{%=$!c)Lz;t5dj9q>kbq;4~@e5y*TC{nSB>qsR3 z+Eet~nc__;8Dj;F1vl!@emuCi(z<^Pw__C{)FJ*|clZx{Dy~6(WKSd!r#zdxmG-*_ z#v?1rjf<@9p0I{M(+ld@DBq1UrmlsXg{WqV6v3sw%H2^vZ@5zc_2oW1x;HI}2PwXJ z@Zlb=d~e3ez{J~#u6Oa*45RlK1r^SLXcF;02A%*N1RispSV)m&BN6KOq3xb%V8$>> zeBSJaJlT-$KWotABl-9ZA#BUHWm7DK%}FwbI?Uj2*3P}_>VRQ@=6<-VfYd@0tV^3f z8G^vw-Gv2zKHhbe&lab-F*;TsFFLQSc-|Kh`;bRg_FARRk#>D@Bh-WgrOmh%Vpz~Y zD9IYIr7QbqE(dUY~+%)n7KdJ(SdgF@VAO z#VH%*jg+2TIqf}_o*u7(bNEq#Est#4Pa3^XpD_!AheQx=R)~)mxnySXJ#()oJ z1(XyWJooRf9>PBIz2u#*{3_{!s$a<=<1;6$ThF&+(6J}NZh~Ei5^kgjm^a#O13mwY zbo@ra26=`RFceBa$L236{^)6s3}e0h%>GnAMR+s((mz=_2X;aKEDflkxCrVQ0_8-I z2Vr4Se4*2UT{mfQuyXUyXEohf_mJ#U=6y2x!*5xuPGPC7egQ=bFoKrb@PtbADhBYI z%0CNve|rfTlc$>Fxa8$W(~{mqpS7k^#xYGMYYNW`mYcjJJoD!s5}ahntSprh0#A4! z33GrjCWq#0w2Kn%97MpqXQcL>L7odVI6_2v5UMHE`m?2y{kH5F9;Rf;4AIfM+mO#T zKm-1dhkzuh=J~yD;f-#4u0VNxf(m^&jFTJ*1nFe+FP4AsHjfaO)$(B2uFgeMnr6wN zakEch?i=_e`uhkywEIG<3NA1*j1mQ0z@V2o1_mN~DsxBXOsYaovd!_H@ywiF8Qd4RLrG5R=4>x_oWS?xI9jMb)Yg`7(T z<7~8W;$tRy<%7|V0QhbkoLji^hmUshT)j%Br>D2J^-~+dttU8{0uD;`C^-cJ=ubYi zpBykeb9>*^{_5`is#9?88Ji0T$Ny|EqE|Gy`1!wA2*5Ep`O{ZZ1Xqu}%w8~#GZ5$* zTB+I;4AggBFgJ-{BxYU0ey&`5vNJmsx}EcP*%Gu}958VIT-U(Bz}e5!Q;dcvOh!nH zRHrpNg1GcLR$01W*pJ@_v?ceKzpj5{fviz@;!jhkdZFZ zqsJF@ZHmT=JG_*f*N%9c;3Rt4CHf%R>9Xi$XlMXJj}wcuhs*A1<1nEPu8-@=B5rL9 z6jbcZ(0&>JXRR`rOvM0iSS08?(H0)+9;!U~b4k#1aSZs)bg2LS{)pa9S*%p~0N6E%xS(x|AnMLpK?lwIY zx_4{6-GuuA?sz2AkZR*|NVC~4yydoch=Y_Ir3^yOpDraZY!!ZIe8Kwe>((t`=}=t; z`-sxL%!P&r_}c5Cj3kuZA?{fHZO^3b_t`e!$wkVS@tNN&>W()P)5&rgHJMW%sTE$q zLS#=y0v#1h7yiQ5N)T8Ta-I&PgkX((E796((UGh5xS{Yi>$6I|b?L2d^j};*MEtA= zb`La&P|-1hQMXQ`UwPccY$%zD0s%I9!a7D=MSV%<#_cAqQ!+X_c55C+5dW;rIPpk7 zzBW?Znz41iC9qTkkv zO-Y~GH2;`UcCYK+Q+*~m7CM+-^WOgjPcX&D@SA?70U`6g-dp7)LDM4teK)VskUobz zc#Ap#ajirg>BWz#Vbm(WyyvEt?^@DA4}PY@1gI^KPhSzfJ9q(YqV9*Q#}CO^qToIb zbamv^9Z5j%7Z^E~@GBT3J?1E&hfD@-0fGkLT=$HB^@}A7-@-swQaJQCqrZCr8Yul7 zMxOQAwV`h&A+MFVhR1QvC^8Gz0DW_At_>9crsL)_zp+#-Ch$=VU=fH=f}_p z$X_pSPUC;q6u{=Ba=DaHZnAW5qw9WW?sPgCbkpCwZ0sK>k=S<~(c+e4`4eFXfj64tVjdnteC-YYW;>ZFk0)u684JP`Cb z&Jmc)NfX?6gE8Wzatbc>#Y6{gyQAPmzQU8kgP6(DznuM`{+S8 zbH(thU3GhI8;TF`T8dY{{;5ml1ipV529h0F(t8&Es7aeXYPfD5I|Yp zd=cd0;r2!9T4)?l&+hh3tbk5RUs~C4^l}s2oAZX-snEkij^2odOM|r_%f@HG~1WuRk z*O$@8WVF5Q72LEc2i$>{X@28R-z&7Cde8}(J%VaW1HV23e%%fpT<8pe9?egGj(qsi zRKq~5Ch}hM&j#Qxb2pgT{3>h2qO|hMUh77-^Y` zO+gU15?nu8zr|9tRwoZO&)%|;$jXg!+SzXoax9E6yb=+t*Lg(5`s^HZ6|)%6NRP`9(xt)n%trw4ZsSE1PKxcN}B7<=W*rC3EuWYyv+y-i+I$INXio zn4`*4vV;2<@&mEEy=lQenpR4n{-q=%dRY%}&c-vdB1nJG`>!Q+-e|&aW7Tfn3nT;| zGQ}ZwDlTJrn(-@q{0aJT&*;936v355*kCh2nfdj0TJL*R4!&7U4*pY7znWT+O$iRZ zvaje}OiD3AYDN1CF&eQ?BNUO^>S3aqyBL-}A*?nwwvg#HB(*TNzKYGU?zVAz@cDC) z@Cz+4>RR0bSBWDLnuVCXZ6it&+q|4h8xexVjE8GIp6I0m+q{VDfszl-)Ql3#uU{N6#bAV_$J9_(yo9b?nmEj|90ZtSZ&RY`lozq(6k_eqdMC+=%f{R8 z-X7ov5Nj4q$ZkpvF~#*|fg;PG&XT%#-8{eV${CpxsNJe_~Uj_y7lTC}izBS2D zV{i&ChQM~pnGlzsDu2e#9>octcj2xfLMRFhEM2}3%mRgf`{7RV)AXgXRuKRT|NiduyASP;1kPO13A|M`ezi+!E|G%h zt@4_SbvkHBXj2^Mpe~g~j0gp7e#1v&J(235kg-5>u()G7oOyzOd?nWjFifesyk~U3 z>arxaVl2lp*Gtm1vESNxS`9%Z?NiI&rzEeN;n!Aw99@$6lIdlc8=l3*xuM`6cpl4< zE-+Y7s4$ak=xWkei%&|Nh;$6m`YZc}uy+7oe57ULP(Rph=*FSUrsTUvbYS2G5_(Vc zUIm=sD=E#%c~UnkL0xM*fj{?+#Fp<UPGV5n|r}xRH%7<`Mo=Y6J)mQtCF&QSI&WCEz80R zjJ)>Wlh}t8Wz8!1pw97+izahD zjEr~@`tO&woy`-ONa6|;Hdbi}f9Gl%PN1HQgFzs`*P1%i(6fQCD{?eWFY2w$c`hIN zI;nt(`tMlmaf{l*JM_&bFk}M$MnRo<5$Ft%*{7l>%S>}c`__bJ<^ePZ?FJZxMvw5J zXH)}Am8GluZI+}YE>3%le9d!2oLE6oV9~@!0AeAu)zaOG8i!p}J@P>j;_--RWAWR| zvr(Y*VU6DK!>9@6f5ag7iFlzcWCAW&mA}9AdTcj~oAWc{f1cD~ORWoxbw5ga+JGVT zS{8!xzb&$f>-hvxnGrXil>X2b4gfcd?Zd`eeYNcDhFfsJe3kAtdvn``niyNzn@_()#F~+N{KWi-#61`Ff;nbvWZjbjAO5DKASp@3-`O#f_)n7#l@UUzpk> z-7Gn)g6+kWC-=G>jb^3BV;uSItW|{*S5Yt7Rqbd5lydeRRg||TPw9Z$yEyuSclq+F zuE?D6zptv*EzAO;Tix2;mu>xVNS;OLlR>!vSMrKuzs3%h>GVK%W#jLzJmyVeUv9Gm zUSJLIeOjB;e;*}5XP^IEJg_XBZl3Fv5(_<@?*9?dysc5$|RPd@F z$yEAuOH`MjM;I%e)S;ePrSc*6Z{sjMa=Y{5GHve#<4ew-=vjzw$RpIyvhz4P;4gPs zB5G%TXCJF|>frvkKg2P_T-m*x?YHXspXa&mRDIuHhiMI-!W-38EZOG8OD6TvwJ&tc zSlkSvYR@!WoAonXNY{ULQ!0vub%`})g1hMdHq1D_a992m8@zoMCVPvrJyRgUrCpHY zX3RaS*1m6ABnYpfSO0+M^Qw_|S|Y5mTW*^H9H}qYx?JbxB1Ku@LPT=$V)JpR znPuD7cSS($N%MgRN~zLZ^)8WIck&CAt{GB)8k6TobDH|vsVZ*pYe1!YenYlAp z8UAf*wt);nFaQbeKQZiw+*1!;AZ07#Z*rZ! zk_-LRMI{dzzv7Caf6}bh?Sn2DiyQ&N8Xabu+@}ckLVZ)3GnA|VFH=8cVmB+g8o;(C z;<0dq<}-_{R&kuTs6Yd0*mw5?_`Kls`Z4a9D5v`NgFPFoiPX$h4Sy;+W^#&#Pl5Yq z?eXWJX-4v^ae0otc`-Q)r=Ci|)fqaJ@1hbC>-@_omC>ZZ#n9T*`m0Xs1pqdn7UR*5 zJz}x`X~(Y=T?2nU+_kj8gm8LOn}ga;Pr3`XJmLem_p;&Ck@K1nbNqNb--=|Lth`DAVHFweUvv#pU?`al5#tVDKy*Yr|P(`t&2Y z1K{N-KgpNkJ-5yc#%Cwz#{cnFsBLI)gz6dX<9E(?j{nSA+7)4)=#4J)u_-?v*RHAa3N4W#de{H#^n3#(qmKAyd zlw8oJ8gW?Y$Rwm5(l*HM|1hil=TnT?QCa9K#Qg!VkO=7%ikMKMG5-1E?&7?05Nqnp zZ`%!k3Zq_?Q+{zj<#~H+M3l33Gn^)Xz}H9fPd)L0HpT19{V-xwNVqS-SWz0bVUVgU zH{Wylx9fs2M*f6NV{zt)DG&$;yHfYa3D#?+f9mZm9~qpx_CZK5J8+*!YRT(xb2&%>jkBb&iyFNZEMmLZf#+9rMz z1=sd$W+SN1Adr-N)c^&HC#rr_cU4~^PDGI$B`_;oQs~K{w-6p+lpb6s?l6AagV-h& z^wv-Av&d054abnFonf7u(FbAh`j}0sT3!#9_wU6s^m2}dSBpc}tUbxjMd*~8jU^Mw zfMJ7sUi#O>wN?`h`fF!M-tvXpId-fgioK!){UrX4AR2A~qm`3^1{%O?pvc~uzJ2?) z0dz4#g|Js@z#IrO(~)c&eGXkh@FE_{K4kg<A==`sz=3LCQ!Np72;O;0 z4%7ZS4}&aLuKaKZg7&h$cg?^1(nG!;F@-K~A3Vt5$=QzvP;RS`P1yfAB+Jz~l56B< zUvie3E}dG8Nz?S&TR2P=N=XQ(-{Fi%bLej>qZ>#FYg_ADBOg}JLn8i3v#g4eMV zt{#j6ySQP&ApAd(Z~t-axTx|i&RCi1{Z>MQuAJn}9$JX_XChyr`xUz^s18hjRQO5gmz?9pFR{>;S- zkyt#X2NsvOZI(W>4AoG&i|W&|LmUE$Pc+!*;I$E^MUc$mh?R1zU%Xd&tn?o%5Sa#$ zr5s=;l&EyZCTKk8zZsYL_k9oRl|kF}`RF-i3(JuA&g|GEgbD&i-rYBL*b0u@qiZg) zhfMr_=HdrNdwMKVzf7BT2?{@yuBTpJCTGVQP6=(=wH>>yh5e$ljtJGRGID zM(OO3eUhoh4o+oS2d`Zm!C>Ixe8g$lJqm|iy}a6(UDc09I7KlGu3eHH(s^Oc$GW8P zZxuJ|szSj2mtY;Fl=*vkKLFbfPt=E^& zM=;m_;up=6SfcvG$7+4@@!*D5EncC9TXA$E8fHo=(&L07Y_blt?BAkr7Yj4ZU>bg2Fh8<0HeXS8Imu+AD(F^7R_n#`LprGv|gtd zV_>&l9|vwaCu~*3=Mz`mDzhWAXWYYx)~Cr#mT3>ehT2;(%w%kUe^7UKpDg9!(?YLW z9sv8ll$YleVzc*j&u>_tnd4z*6c(8i z!}%?n=}b{5bw=GypQ#h1(@ei%ztH2+|D%fQPHDji9~PP2FmLglT+kBg>I_yZII3K} z=BWze(jCdXDnTySs{5~A-(&>Uyg!&YZV!2izFL=T;93E~fF$(Czu2Mpm0R82S1>`| zV;H)JOm;)_-)Ibi2FH#+`Tk>!Bd9}Wr`us+Cd1dEw_c#BJ0Smg%IPe)I5jfCjCo0} z$WzGR$3xGy%#p@LN5B5Iom@Z^Hr^7fjXeG*U$qHlAD6(d71{mJLf%I*vDxJHgG4~? z%fC8XSg1cG^S>xrt2vFjA;|~B2&$C zs&S-K1T6Nnn^ic@n5?7{9s@_FDUJ}qd#{)=-~(an>|j_Q(cxL1cDQB(Vo)9v@#$k< zsGQE#I14nf23W2xvP6CpU;@c3M{vXN^NUIj&3mHR#6{-hiHn(&g)45JY8tvA_$8?4 z;)M`Bv6{9($z3zE7xpcAZ^YYGGMa?_^!(pLbyQ%vDCX~9X8WnOs>yO~IVw?Ec^w${ z3SsNzmZzB#pU%-sPQ7s}RRnkwiF+XEG66{+8&75>I(hq!ojOE~y3c}l_33B%&}A2X zqC3mz&?|1xB$F@v{e<*&-L9PWg^FinD(mAh{-4oiMse__tLBoM!pk@p2 z)50-#3dNLd=z>rMA9Nf_Jr+5YLdv!JR;YXx3mtI}T8=`=n7!I$|0RVl)N{0G(V_3~ zwt?%{`k$ySdycZcvAsy?C(ua&6!ETTBIeQ))FvD%3Tq1Qr+$IQ3FT_AUOL9MmKzvl3ZU@O9%fh`E8KU*S4E?QN=e;4f%X|BJVtpF) zjHI4F#y*n(B&lCXh;_Gdslb_!ee7q2o$<~PvwZH=`OC`%+hYvh;NG%I`M<5K5X$_! z;_>$!M~#QNXsdCUr@7uW&Oz6A1gFZx$`PIw@_YL`)C54((+|rO+MsVh9*F*@^=B!f zA=GoLmU7jUqa{U0B0(J^Ho?>?b({Y;-hdKdWMomXLdnCIqJ{4p|FQ+pzNId ziaTX+A1OIRP{?`xYv+BIJ_RJ_5Ngm(7tSf~Kk>|RcB)OpyLgJvSIy?DD>L%}gW1;9 z&AU|PX~5{;pg*x2UJe%gZ}bcHh9;{{fg))Vw75J5i_eOMD}tK5WRqZ_MT=2A48c3y zT{bNpY5K=Ls+Nm**xZGfc0-Cf5#j z0pH4*MDo+)d;FNfMeJPbIyCz+`3Y&rI}@hx&L+lq%P;+uS{8I5-i6cKLd#}(SLZ%z zpc9`gqO$1?^c+3kvbON+9lObqvHzE`6w$qbtTryCq_olA-kw0m@BVT9_>vPdY@#Y6 zs(Hdyknpy_XzIw+^YB-JWD!lV;x)<4okdZ#2GCY?t1uX@-HFZ@2vT_Bk4fF*lskAL+QUyDG9SB|xdO@7dK?w~T-OchjpD^B7wU z>It6_A-Ji5g+}rR^Is>w%dz9JdCST_W+FrIia2G4R!;MD-D9eyt@#RQz6^k-tMf-6 z`Pw9*Uq809`@%~OUGrb;<#~lXgiN+ZRrmfOZTegHN@-~lXu%8}@%954QtQ+=Hh$Hc zH)T)*(h=Aif-JvSvG!;8|6t~yk0uQ9iu*mT!D`|wXC*)vmUZR0u#||0&h} zeiep*+*f)}@>Ok3rsf9hPO!^fFP{57=nOAr&=y)Jto7q2&Z$CmW&LXrw=Ku2hyfNT zMa}bYI}%U%;3Qw|v|Q$q+BL=Nxt?reJUSYuRlFH22No`W09oUi<2Rf|I|Zfc^BW=x zK6}Zb3AenS5XO1<(t+2TxjQA;(n}i%EFdtE(dK6QT8g%KK$F9dYwy^v=g-|9Ja#L3 zpX*nrI<$OH{@J0&d#f=HEubt^`1LgVMs)b5CVx*+sfid|MSnE>t$en?@-*R8W*g1% zd0dC4Q6{VET^Ivkb!`wt7!9Lp!}Hoo3J-3s4vN3W{L#g%Gsy%5B1sB16thnBjp>mAqjMIvmXMy;m~FG_Oo`5iZ#7ffs)i9 z9*yD$6S0ri&%Ka1XZ8NY`nMG(JPA+tPUia^U|FL4Us@Z&7PN(EZ%UvD%?Cx~*VI|K zkrL)t16{DejU?J1Uw>R2EyxRwzk6gRdEabqyRY+=JL`l5y9sAdqNBmU5OQ$7ru^#3 zpe4&I59q^w{Ob1U90;32iSZCD+54JV4GUk!6acW77lz%@YUcZM}2k{90132+BHqq4SI1A%Xt^lUE@7$y!ed1f={A$>|h4L=@l#WC_HvZ2Aa}yvZ3d~HDkP_hjcm!I6%^}LtK9& z?;DGFv%aq5(d*6AhpOLu=aASgZBH@ye^w|?@O31oq}ZM@s)SV+F{uaa;4*Uu17^!t zUVnW*UoFoyI}Mhr|1-&Zx#AoCD$0|^K&pi*@rcE>k0+Z=`hs0TX(Zh}tvE(Z-x=HU z@TS%41l^dy=FpzBt^3~5P>eMSCXPu&k4&9?mdXHN$2$7@9%}S+EF06HzdFF^dh(3( z{I3A9OPuqQwrDsg7;bz1%&)3Od3I!}c9;J^#1DFtBlgdGXFle6lKijysVh~lQZ=`Z z!Y1hs+rvFH)RM>O5A0>ljMb9i1iu@cqdRN4a#nVQSHm}$J32Llb7fQWNLz4BM1_}S zPCR+BH#@4^<1xn(ql6)|vkJr}bKsPJoyA&Lu9-;cksMmo)9J=R`{iGC$G%ZTiCM8F`_x8`n4V*A}X}+#~M=HN2;77t2R>AjQEwPY;>Z5%) zY7)M&Me1<)rDFpIEtxk)~NimW>S-UK~AEn{vJRveCfZCAW zzid-UuX8}vG2K*cJeM>@dQ#5&xB$@9I+?xAGvHS~bd_p+`l``*qTSCujHiDVVz|m^ zu&(C$hvUj#HXG2;j)YfU{*%y=Sny-trfn9i7*vq`hvq8A=2P;?hoj0WlFC~GQOdR{ zSOmEh_HyNyqcec1Az84%Q;X{(?PuaXAGWG>Nm?1qNQ+Hgm#VH;21JG|qyd0gtM(hsq0M@xunboDbMO-ITw(lMC-M z)wmQM>(@Qc_yA=~A3eup(S+M%d8NDDwZ}hRmSchmDB;vDH}^m8!PHwP`Ma#a+p;|w zpeJ4G>T#m5_ki_jvTp`ONCGX#B-7TMCw_kXgX6CvRNJjH)*e_(YG&WmNx!Kh7N;%4 zd8fk$&sK<8AVwBidp8A|?Vy&LUduOWX5~rMbV{YXPvH?vE6I}MsyuM_Ospp%D13;PF#|V5W?cli?g$9WTkxs$wdNv^xs1`*__0oIo z1cOb^3r7dbDnw*Q$|jx;UvEDA+weDI(l*5$)x;Q{bWTDb(yzfsials`4M&RCRfv^f z5TCo+_RLbD^;jVRy<*fuboGj3R_~-EMCK%!lx+@#03cSjv5dk`jrln`m=%uE-dG$1 z=af487`R#6m%CQi^yhxHaB7k?ADM{2*N%8gNr@Lc_4o1K>rE?UQP}ujWv(HhbKpq_ ztPQ$l+@3!Lz+tf|_Na9bO1@vTJLv<F6C}IuAm3>?sK;}%h!bG^VhZB zMt2-56#n5ZjMfYT#+DJY+ydhvK1{D~IA2N8%Fgxm{Ly(Z1KYG#s%eDDM~Bqd6&qps z#9UyjWBZ)toi`B*|GghxNm@Wn2aMf-0)}{E1%StC}k@9~b zHZiE_J(^vv+uCdTn=*jG(I}K*$Tn(MbQOi+{a5wWxmE*TH%~g4)9&o#lMHTiFkV$9 z)1Vj(H{?5M=z>w91j$Zw?^gV#=%Z&`$woIdS^S}QX5s92-v@y@CIS5_8GH7Ug)VOi zwLa+g91}P!v!PY4khp0*mf`;M&Rm=0GoQ+ptkNI2@W)31w7FL^!CZhhHapyJx84v%0uiByv&;RI3#t<>wOeVI85 z=}%gMe~g~TISfC1wiOWcl*iY+of(Oxzm?Hn24QyJ6~jaxd{Ch0+YsJ=?cdJo`pEl;y_a>i{HctO(>2F4!A!zx7-9DD``aPFWZmb|r~!dsu#olA z&N4>?-Tj-5HewxmCr5Y6Z6jfDcGpGqNRA@GQN`@fB3XUHka2^iw*e`w=rOIMQey7b zOnD*r#tcnLDNWfrYwqP%CBLw^uwm5P;av~Iw!td`S3F*`f|0p!@Wa%!{>Zw2hl}Cf zH8&560YRMd7UbI65H8ae_08`PhOg+?Z;c(fYG+K(FO>il-!Zmzn*)7n^T{HWuJDVx zi8(XoHvJh*o!9SBR-We{Nt*EV(3I4cP@iZoX&ldrsSXMPxHUolR=IHk`dpLD8x$Lc${skjJ$4h1{?5Q z@FPsmFxSQD&c6_-p&VVbTRDXf5<-PH8uO^wNi6Y;)xYCjkvS%dioUGxnEYZ1sC{sEFLI)IEjIk9W93%=LcWY=&Xnes7I^RkCe|FV!`)%{b+kiGvL^Rw{x-M4&8`Aj$p zCyuT2X#RRNMTi7SXRFG~*@}Z?lUfY*aE_JE>}?XN+#iRY3EbWM@>y?q$_7pm%<{Vj z&NfQq)|IXG1Y!b*vGC$#o7R4#>Z?`^|1NkU&g<}`tlO3?hvD^%i_wAoT)a7oZ9N-X z(}+AW{(y~w*iBsX zOHPcz+N~->=_t zUwCplcaJ2i(5>F@4RQ(0qz83Q_y`Jmi`RHBHm&{~bvqeoK+X*#EaG2;q%%0lmWq3U z_&zfsx^n-fVIj7)dyCFz%6!5b%YYuTUO!?jg9(anll|hzXI}^$QTMZK@Oj0b(NW*Y ze7zBRJOHZ22h`XrLm$1}^PMFia?c8D&JFvn9*@YQ1)H`oWLxbdi&P+g2*{VVSn+O$ zy!Kbg1HzQeg2ws|mLzbOXTm~`6wH#(x`Y^v6#w&~S?d-ZK&3An0+TC+FG4biCQjqy z)$PT1Pe-v_U4})3h=|8}=+*LAsdiOUauN*BlI)9oIf|^?bCttE;MdH}nss4gHyQrt z0AJ_z(-qub85HPZf5&7@L-KwhO(#0o<1hVaZ$Fpl^$G{VEZ+@K*-MEZ%YeilxqSdGm@Pu4&4nqcM9Xnt;cKL5Dh#J2+GI`(RhRB^&GC4@e)A zr?TCqDO~eL?d4FzmV6x`F6~JaILtpC%6e7fx{4iB>Vu>Y-_uw9&Qi zj~))^Ub!NIF8G6(LSl&%oxmUOjYgb@iW?Qem!ay#vGbm!DcH2X@~_1Qq}x-g9#rDC zwovddNs>CVC5P4p`L}14HF+=&b}oGjq&-a#l_FQ#6YxdiyZa&6H)}=z&lW|fg>x& zQN-Y!1Yv?cq8L5;so#X+RXd8mZ7ZIt8eW;(dv$5ka_WQqa#bU$JbkO)2r>&lde%8D z$Fjum7cEVD-rPo4AO{l+YAqIbmtv>>8(bj(5*EFNPw`1-YS-V~x#&H$37(KqOC4(JMgPWNr(RdRbZvNWr8X#kVCpDYwfm>H{{OjG!G^ zDaeSNAuOE43~$=O5#F-Ejv|zCQ{dY2_--OIiO>Yr*4EO-Y=Su_p?XAnOxUJxm6Qyu zb2!_hpWgNf;ue`c2><)O1;JMoR)|oG#MJI-?*&?`YfN`m;lH6OWEHPXc4QNggP)3-~RIK%8+o}!re3Le23DIA~B=i+<0r<=6I6F%c z*?*~dc`e5){tpgRtgLOL@H1)LJYRkxw`0SMGvP1?sE#w~6Fc*PgMId~u_>Lbx-s0cy2vFIxk8*97yuK-*ifSHtth0GVygRkU|%px+$4q$gMk$^_8r20&>4soY!y^`iEMLf}MD#sCk6 z^{-EQo;BM|{gsOIU$S{eIYBlz*9t>oLv3B1O5E^S+tP#lYagLJQj_YOU#@*1H!$UJrKlDMRQ{hnMi@jf!4~E|DIo^H#A>ZX7 zn!)cHL^-ZwnMzrF;adH)x`UjMSY(3M+&XXH!En4HVg|w;tL*>8f-V*+H{^{f3LVcHKbWI4BfWtd#Axqmxo zViJJJNaa84=D&U91ux#|+?+ndWy4OCZqrSvM}qcAR)Oj4ezbx9mFja|kRIz`2hh07 z8qFa4&1sbw#7n3xY&c+WA9O5CzQL!29-vJ{FX&k!NaGSd9_NAP)9kqVX;~%i42WW> z#Y46%Fvqxs1{qld_&2pPGDt9RbK!eio4tX{ZuqLh$RVBsr=MBRCeCQm5DX?U5tg&U z6lk_ZQ3Wp^P!I`erX>w-FDp{D|iF}Rtz7o$#^RG1w>EAX_kOi&5%Ba2U7nj3o=}5_E+v_&5Re; zmC)Hswkyl;xSazmAOwifNPLG#eziY(Iszi4ETcg0gd#ZPvLn^WJY(TF>l<4oQIFn_ z-2>`WtRc057+d25N@|LN9x}#0i8=GP>Lf+LVFYu5Vy^@x=ML(v8L~{L^Uy_*D+i(58lrqd~5H_Z2boV`l3Us zK?yrB_yan>^#R)-zwOI@nAJVI#^~H6NCOH0ird@ilupkhPY_&1Y;!(k_s3S|oq8EX+VZRasn`-s#Y4y>G@cCHUqVFfnNCQNSr16b3~qnx zQ1ls{3$8z;s#USftl^!CKBwaF;IusaP^2G0n1TViYo5Oww?_<6vs#z<^IkGRAj#ZW zdc?_ zK6n^`-JCy>T_y?QiUYfSQ3RwG^rlTvJ<#9Pqsg^meqjc`g=&YV_d$&RF-nW(DU*{8 z+B^~Tzg3c(0^@fWP*Zf!?u_1OJ`7po)5Dg+PaAApo`AsK`|f0s$|Kgao1;E!3>fXc zHNh}i;jl?pBy~Z{u{u0mxGzpJo7vw^j9VOb(Q~^b-Eye?{nVDKTW7@bG z7})SJ6HZY6DqUB6`bR>C>$p2r-e*+wbpQ&w)7%_FBYD8WvAXKVWezi**a6|R0eLya z_7lPOiKsyA{n&lc{Af+ydf~^;}{IZ8j69s>4e4-!YLfobx;TmAFSAgycu|v z#KGmSnZX!g$R?e{MBT1?h_RWAo-V;fS{t-zrTi-&UFs0@$bjSOse+P|n`pe}uW^1UrFY(6t<@ z_AsFoPayFm@bMJ!fx&V=fFYP>D^Jc4vX~^(X%dP~a2(<#rfnUeQZcVNsf8@Xg37-w ztiT-wEN(sf1PZ(vc#qsB`9i`|v-8=oF3(^d1#k*N2ZGW!aoTk z9^*n#6+ol`rcV#VlTAoJ+$z8>;mv8$l%?eph%du0eWH~;;|@_tyypcNts*(~m5LGL z6c*AKrW2N@=L6`XtLQ3h4Fg+AH27^GWCVH8_T&YN0=@@qfD47e5Gsi!8En?PJ`he> zroQxehh=L1Q(#uzEDEZ6G%(~06rwjMqP1U{b@(fWabhLNeqSX;trkYiQlGRiCRhhn zfzL!w$yP(f!u6o?CTQAOihwhJi~0ixN>h*wrUmYjsu^xIV8_|vT7-}T-P|V1^d8zD zAr$;4vgx08toj%;Y;ZEId#@#)AoL=jN=jN7N*cVDZYoMg1^%`qr4Nm0kDh$uaK=cb zCBBoAgt^gWvC!0`WFcxT|E&=qc0maGyu$hI)@+6Quk8Y=`e#V=S+xTXxbe|{+5F~Z z8NJXP#Il^oKhVi7jpapn+X0j9pKt6DD?z(hAt0r9Hi302n(Omei^;9Kotul_3|wR( z(H3vGcw1ft$0L18mG4blszC(>fon~xE}kvB8A_aSg{@wGcu)f5jnRCI=H2~FZ9~< zOtEEaMOqyFZVfvz@HWBgVk7}CxxGyFhongLhlPa|N|W48Q^p)>e?B%npaeoGENw-F z+XqkwT$X9mqV?;|QvE!(R`a%1w8Q}j?c|px<{1)J;;gAmv9@2{JtX$8c#C8ffd`c{ z@^=sAeffOmUrZfzj(cy#MWFb?1TwRv^^CDd_amDuxzz!AKEcO&k7X8<9$LL|`2De*+L9y?9dQ`K^^xM`Vz4ezj2QQBJnDo^di{iVygklO(0z8OmEyr{t zEjrQ;Uh|3o@!*QRRZ{KD!~GX^+LQja#trf(R(FXwUn*Z4hZ65UVlk)#MRtRZ2IXbr zJ6WPg5SiPF(Gh;V%t$x&*Q+3i5t%?On| z$sg%D8}tdYvi_=J*&eK?4k-62*M>F6yy7CPFBj= z?_8!9=$@@y8Ch8>f9@OeYWHAQ?*d8?$R`L94VnV2}x zpO!l)*gF6B-}L@gpIh}#+1;OB{8W8xAK;cSRN{EN%{{pT=JU|rfofCgl;Z|J&I zI_dx2Bl62cEYE4!t&~6j%8G;5Vj(O?Zso%e7j}*!$ee?b3I4q1zD}6+yc|yy5zB_l z7=WZ2B4ZRXVQZ-Y-5SE2WOe0_sIoFQ?9mZbx(!s@u_rOx2L(xls1$NdccYi8UNd$e zLA+i%Z(C7eND$7^m!s2fIbH5eq@V-~(aYq+gL%BmXK6s@!s_y+T|ZlOkou{Q5F8>@ z2c&$pif_#ex9{As{(f>RJ>@+ZO_Iue&JdxHL}~&o7g8;<+wzX85mA$h$!}j6A1N_T z5xb!eyI;%r=FuVa2b%>nye+-QZlA2hSnu(nG3>lNdaYdO}^c}uejOSMG?+Et^rd7&f%dUBHjEf4Ec!U&8=|mJNV3UTducK z9?}>keBr%6gpRWg$%C^nXky2Ib?D2-HOjsH(zZtIUY-eBM6TOaB$2Ym@T-ZhPM`5( zAN6?QHSNi%EFVWz0MXcFWVEDc=NQMK4*~s)~Zs(5?mlb6Rr~3PnI2NiSAm6xsY-xV9@uXw?@ji zV``MiRv{Rin($~A zokSpeJVHgzjPY$Ezv< zjKia9o@0@wj*7`xF4P`Eu}&uvh`b>pq$w*ai}WlmzpRWOKp6L?O#yWdRrl1d5TVSY z`^~49pld(7ym;0dIXjMH?d42JJXa+Ak^bhH;7$`kzJ&Xxs;a74igapr_Ie^!1}Ix0 zf~WY@E-(@{D_H3cSXyS@zC$8%4;dL5Q%g#?{eRo0P(X4~a&CfH0$8}XXmR)mTLt%u z)M{i@k%Z~gpZ)BC~9zMKja%u|kn3q1P7M*z0)MVm&L75LXrxMFYq35^}VOURUPnNbj zcaFp85j#3~5{VFD@$>U{KRLc0aAksO(Kxj~LY#z<8={yj?PNXOLwGOX&b)#@5s_eS zb>s*k*Fm->2Pdb*dFOXo7Q3kVC=tpEe3AFZL`Fr0-ou|1soom^eXrlvP`szm><`+g} zRAaTD{gNId>1fqtVPg}m;L=GXmokS@ZBYNoLIj==Ve|eMJRWoMR*D3*De7uTj=}o0tY$&{{5TCE(%^&VPQq?Sz2Bm zv#sxv82)a-q`=sh-Z z=VI2rHh+hmf!><-1eRtGETD&roD=pX$?v&*`LcG|v)#v>oD8h2tYEIJOKQYgDkCi| z{rb2oGNU%j$;p`>I<#eadfN2B0oiX^E1*I?@mup%fq_;(jNsXa+E=&S^U{VMuXg$Z z-O-2kUS5`2CY~DL`ltBWBqb$1C@nqG+uM5#5d|>s%P23mA-AWT+0}KAW?R~~Nchd? zlzeoXn4G);H=IT!AsvHF$?HF1Nl$6;mh9Un9%p85zUSo0M@R`UOiNGCc=%8+%jAkt z#o%^qvhgswlk;?bCoW=YX100fi8lvis1D+I@s!QMYfM7HsxiyLm8A0pmyfITqs)n? zhvE2?X?=ps#KX%=Q<-?g#N>MT^5x5Z$B$WC;BUBF;sGBQR$ zGLQ8<6OqpeJPPCNtXMluY_1Ud=dI=pc|VcKcS`rLusgSKmsnVN zd6nP=Xu`@G9nE~)=50^jF97`p&d!ff#Te|}8$m=mX#H@0w^X;$mY$??ezHALrRXF( z>n1+Fc)UA`^odwZAoyY7H$>bQ+xO zCUR$4Cm4DE7MpkNsW2uYKiKK|w+1~I%JjFeYAN_zn!ewp=udv&f@s_l;ECF0j3Q}k z9RJfld^rP9zWTE}kvWdK4{Nr>>*i)+jUe2G5k-kF_fqH%;hy+6y3N&f8<8vsp8 ze{PN0*zLK97;a%XIaZ_#Rv{3BHs#&MBpk3`phWw6A(7w3zkdCKHzO$@k^;M0%m^+c#V&Mev)uGKqvDZ{~6t zNUnQ*_U~r`9*$gj=AbGPDKjsxo=K^nRNNN@+cT1Xk83><7mA|pP0DUTA{!SMg9tIN zbQ+=H5=kmqWY#4o-MV>`_Es}ZSWN(@l!aa2^V20>vqR^nl9Q8Xfjisd@h-63hA-lr z8XM`6-1sN8Au2lB;@in%$3CG}L-g?1ilngcFW-j*G)3OODkzYZmzV#9F#F8>{DvcE z%0kXhb$&(+(r{-?bTn*M_+iM&vr`EV&)|wrKv^i6%+86M_?`y&fGCh)H9Z9XKmU(z z4cNi#6q7$M{0Va22qmPO`2p1&J^xLqs;5P5l(sv$hS^=~R** zT+)vbs6$NP4CV&u#l;)XpFh7i@hC->pl^B&M-f{OvKHMxSJR*n;FLO)M@DmTbGvOc z+_KV2OL+xi=umUujtIs#{95w%8o93Tj&;ZzdC<@fkWZbGs94h{vkNSc*shQ5zrBNI z|Ni}iu_NxkeevI{G$No1dy!i)FMhnW-B!=hQS{rZA4c*D3is2}C=pA@&BId-N4ifQ zF9PT3Us}r`q>Jv_#_ge9Q@Ti%fR}1rD-E85AVv|9`!>|9*LyiOZgjM>Bf;Q|>6dAE z_^Q0XBO&a_f8PnbGO9H=^S?i<|Nnf!h;Q;^xA#d`GyA8i8%g+YkFF6pPuni={{oC` BBVGUi literal 0 HcmV?d00001