lldois/v04_balanced_core_lr2e5
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: lldois/v04_balanced_core_lr2e5
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-21 02:43:17 +08:00
commit ca1346ed8e
16 changed files with 3626 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

58
README.md Normal file
View File

@@ -0,0 +1,58 @@
---
library_name: transformers
license: other
base_model: OpenOneRec/OneReason-0.8B-pretrain-competition
tags:
- llama-factory
- full
- generated_from_trainer
model-index:
- name: v04_balanced_core_lr2e5
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# v04_balanced_core_lr2e5
This model is a fine-tuned version of [OpenOneRec/OneReason-0.8B-pretrain-competition](https://huggingface.co/OpenOneRec/OneReason-0.8B-pretrain-competition) on the overnight_balanced_core dataset.
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 1
- eval_batch_size: 8
- seed: 202607064
- gradient_accumulation_steps: 4
- total_train_batch_size: 4
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 0.03
- num_epochs: 1
### Training results
### Framework versions
- Transformers 5.6.0
- Pytorch 2.7.1+cu126
- Datasets 4.0.0
- Tokenizers 0.22.2

8
all_results.json Normal file
View File

@@ -0,0 +1,8 @@
{
"epoch": 1.0,
"total_flos": 2.961647362958254e+17,
"train_loss": 1.4641528666311143,
"train_runtime": 5382.2079,
"train_samples_per_second": 0.455,
"train_steps_per_second": 0.114
}

89
chat_template.jinja Normal file
View File

@@ -0,0 +1,89 @@
{%- if tools %}
{{- '<|im_start|>system\n' }}
{%- if messages[0].role == 'system' %}
{{- messages[0].content + '\n\n' }}
{%- endif %}
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
{%- if messages[0].role == 'system' %}
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
{%- set index = (messages|length - 1) - loop.index0 %}
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
{%- set ns.multi_step_tool = false %}
{%- set ns.last_query_index = index %}
{%- endif %}
{%- endfor %}
{%- for message in messages %}
{%- if message.content is string %}
{%- set content = message.content %}
{%- else %}
{%- set content = '' %}
{%- endif %}
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
{%- elif message.role == "assistant" %}
{%- set reasoning_content = '' %}
{%- if message.reasoning_content is string %}
{%- set reasoning_content = message.reasoning_content %}
{%- else %}
{%- if '</think>' in content %}
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
{%- endif %}
{%- endif %}
{%- if loop.index0 > ns.last_query_index %}
{%- if loop.last or (not loop.last and reasoning_content) %}
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- if message.tool_calls %}
{%- for tool_call in message.tool_calls %}
{%- if (loop.first and content) or (not loop.first) %}
{{- '\n' }}
{%- endif %}
{%- if tool_call.function %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{{- '<tool_call>\n{"name": "' }}
{{- tool_call.name }}
{{- '", "arguments": ' }}
{%- if tool_call.arguments is string %}
{{- tool_call.arguments }}
{%- else %}
{{- tool_call.arguments | tojson }}
{%- endif %}
{{- '}\n</tool_call>' }}
{%- endfor %}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- elif message.role == "tool" %}
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' }}
{{- content }}
{{- '\n</tool_response>' }}
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- if enable_thinking is defined and enable_thinking is false %}
{{- '<think>\n\n</think>\n\n' }}
{%- endif %}
{%- endif %}

63
config.json Normal file
View File

@@ -0,0 +1,63 @@
{
"architectures": [
"Qwen3ForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": null,
"dtype": "bfloat16",
"eos_token_id": 151645,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 1024,
"initializer_range": 0.02,
"intermediate_size": 3072,
"layer_types": [
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention"
],
"max_position_embeddings": 40960,
"max_window_layers": 28,
"model_type": "qwen3",
"num_attention_heads": 16,
"num_hidden_layers": 28,
"num_key_value_heads": 8,
"pad_token_id": 151643,
"rms_norm_eps": 1e-06,
"rope_parameters": {
"rope_theta": 1000000,
"rope_type": "default"
},
"sliding_window": null,
"tie_word_embeddings": false,
"transformers_version": "5.6.0",
"use_cache": false,
"use_sliding_window": false,
"vocab_size": 176253
}

12
generation_config.json Normal file
View File

@@ -0,0 +1,12 @@
{
"do_sample": true,
"eos_token_id": [
151645,
151643
],
"pad_token_id": 151643,
"temperature": 0.6,
"top_k": 20,
"top_p": 0.95,
"transformers_version": "5.6.0"
}

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d8543615a93427e241f6dd0981ad13234cf088ea185e310b95bab5a9de081044
size 1602902832

3
runs/Jul06_01-01-23_ubuntu-System-Product-Name/events.out.tfevents.1783270883.ubuntu-System-Product-Name.1469155.0 Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f9b91773cecbf0e6ebfd7fcd266520ccd4340654a059519ac4179623a3247385
size 31964

3
tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cd4d15f596979aecbc11ba12668cb21c9fb8452ce1235cd9d7878cc950170421
size 16016554

16
tokenizer_config.json Normal file
View File

@@ -0,0 +1,16 @@
{
"add_prefix_space": false,
"backend": "tokenizers",
"bos_token": null,
"clean_up_tokenization_spaces": false,
"eos_token": "<|im_end|>",
"errors": "replace",
"is_local": false,
"local_files_only": false,
"model_max_length": 131072,
"pad_token": "<|endoftext|>",
"padding_side": "right",
"split_special_tokens": false,
"tokenizer_class": "Qwen2Tokenizer",
"unk_token": null
}

2304
train.log Normal file
View File

File diff suppressed because one or more lines are too long

8
train_results.json Normal file
View File

@@ -0,0 +1,8 @@
{
"epoch": 1.0,
"total_flos": 2.961647362958254e+17,
"train_loss": 1.4641528666311143,
"train_runtime": 5382.2079,
"train_samples_per_second": 0.455,
"train_steps_per_second": 0.114
}

123
trainer_log.jsonl Normal file
View File

@@ -0,0 +1,123 @@
{"current_steps": 5, "total_steps": 613, "loss": 2.764590835571289, "lr": 4.210526315789474e-06, "epoch": 0.008159934720522236, "percentage": 0.82, "elapsed_time": "0:00:43", "remaining_time": "1:27:20"}
{"current_steps": 10, "total_steps": 613, "loss": 2.489868927001953, "lr": 9.473684210526315e-06, "epoch": 0.016319869441044473, "percentage": 1.63, "elapsed_time": "0:01:27", "remaining_time": "1:28:07"}
{"current_steps": 15, "total_steps": 613, "loss": 2.2080495834350584, "lr": 1.4736842105263159e-05, "epoch": 0.02447980416156671, "percentage": 2.45, "elapsed_time": "0:02:11", "remaining_time": "1:27:27"}
{"current_steps": 20, "total_steps": 613, "loss": 2.0085981369018553, "lr": 2e-05, "epoch": 0.032639738882088945, "percentage": 3.26, "elapsed_time": "0:02:56", "remaining_time": "1:27:09"}
{"current_steps": 25, "total_steps": 613, "loss": 1.855879020690918, "lr": 1.9996503676899863e-05, "epoch": 0.04079967360261118, "percentage": 4.08, "elapsed_time": "0:03:40", "remaining_time": "1:26:33"}
{"current_steps": 30, "total_steps": 613, "loss": 1.8655593872070313, "lr": 1.9986017152454497e-05, "epoch": 0.04895960832313342, "percentage": 4.89, "elapsed_time": "0:04:24", "remaining_time": "1:25:36"}
{"current_steps": 35, "total_steps": 613, "loss": 1.764651107788086, "lr": 1.9968547759519426e-05, "epoch": 0.05711954304365565, "percentage": 5.71, "elapsed_time": "0:05:07", "remaining_time": "1:24:38"}
{"current_steps": 40, "total_steps": 613, "loss": 1.8568502426147462, "lr": 1.9944107713823068e-05, "epoch": 0.06527947776417789, "percentage": 6.53, "elapsed_time": "0:05:52", "remaining_time": "1:24:03"}
{"current_steps": 45, "total_steps": 613, "loss": 1.7390193939208984, "lr": 1.9912714105424694e-05, "epoch": 0.07343941248470012, "percentage": 7.34, "elapsed_time": "0:06:35", "remaining_time": "1:23:12"}
{"current_steps": 50, "total_steps": 613, "loss": 1.5365073204040527, "lr": 1.9874388886763944e-05, "epoch": 0.08159934720522236, "percentage": 8.16, "elapsed_time": "0:07:20", "remaining_time": "1:22:41"}
{"current_steps": 55, "total_steps": 613, "loss": 1.7221796035766601, "lr": 1.9829158857310288e-05, "epoch": 0.08975928192574459, "percentage": 8.97, "elapsed_time": "0:08:03", "remaining_time": "1:21:48"}
{"current_steps": 60, "total_steps": 613, "loss": 1.6801658630371095, "lr": 1.9777055644823087e-05, "epoch": 0.09791921664626684, "percentage": 9.79, "elapsed_time": "0:08:46", "remaining_time": "1:20:52"}
{"current_steps": 65, "total_steps": 613, "loss": 1.6599308013916017, "lr": 1.9718115683235418e-05, "epoch": 0.10607915136678907, "percentage": 10.6, "elapsed_time": "0:09:28", "remaining_time": "1:19:49"}
{"current_steps": 70, "total_steps": 613, "loss": 1.5088156700134276, "lr": 1.9652380187177128e-05, "epoch": 0.1142390860873113, "percentage": 11.42, "elapsed_time": "0:10:13", "remaining_time": "1:19:22"}
{"current_steps": 75, "total_steps": 613, "loss": 1.541905117034912, "lr": 1.957989512315489e-05, "epoch": 0.12239902080783353, "percentage": 12.23, "elapsed_time": "0:10:58", "remaining_time": "1:18:46"}
{"current_steps": 80, "total_steps": 613, "loss": 1.635506820678711, "lr": 1.9500711177409456e-05, "epoch": 0.13055895552835578, "percentage": 13.05, "elapsed_time": "0:11:42", "remaining_time": "1:18:03"}
{"current_steps": 85, "total_steps": 613, "loss": 1.4972270011901856, "lr": 1.9414883720472557e-05, "epoch": 0.138718890248878, "percentage": 13.87, "elapsed_time": "0:12:26", "remaining_time": "1:17:15"}
{"current_steps": 90, "total_steps": 613, "loss": 1.5120899200439453, "lr": 1.932247276844826e-05, "epoch": 0.14687882496940025, "percentage": 14.68, "elapsed_time": "0:13:10", "remaining_time": "1:16:33"}
{"current_steps": 95, "total_steps": 613, "loss": 1.516195011138916, "lr": 1.9223542941045817e-05, "epoch": 0.15503875968992248, "percentage": 15.5, "elapsed_time": "0:13:54", "remaining_time": "1:15:51"}
{"current_steps": 100, "total_steps": 613, "loss": 1.5576241493225098, "lr": 1.9118163416393392e-05, "epoch": 0.1631986944104447, "percentage": 16.31, "elapsed_time": "0:14:36", "remaining_time": "1:14:55"}
{"current_steps": 105, "total_steps": 613, "loss": 1.4911109924316406, "lr": 1.9006407882664256e-05, "epoch": 0.17135862913096694, "percentage": 17.13, "elapsed_time": "0:15:20", "remaining_time": "1:14:12"}
{"current_steps": 110, "total_steps": 613, "loss": 1.4046802520751953, "lr": 1.8888354486549238e-05, "epoch": 0.17951856385148918, "percentage": 17.94, "elapsed_time": "0:16:04", "remaining_time": "1:13:32"}
{"current_steps": 115, "total_steps": 613, "loss": 1.4821024894714356, "lr": 1.8764085778611507e-05, "epoch": 0.18767849857201144, "percentage": 18.76, "elapsed_time": "0:16:48", "remaining_time": "1:12:48"}
{"current_steps": 120, "total_steps": 613, "loss": 1.4233411788940429, "lr": 1.863368865556191e-05, "epoch": 0.19583843329253367, "percentage": 19.58, "elapsed_time": "0:17:33", "remaining_time": "1:12:06"}
{"current_steps": 125, "total_steps": 613, "loss": 1.4180265426635743, "lr": 1.8497254299495147e-05, "epoch": 0.2039983680130559, "percentage": 20.39, "elapsed_time": "0:18:18", "remaining_time": "1:11:29"}
{"current_steps": 130, "total_steps": 613, "loss": 1.4138151168823243, "lr": 1.8354878114129368e-05, "epoch": 0.21215830273357814, "percentage": 21.21, "elapsed_time": "0:19:03", "remaining_time": "1:10:47"}
{"current_steps": 135, "total_steps": 613, "loss": 1.5245219230651856, "lr": 1.820665965809373e-05, "epoch": 0.22031823745410037, "percentage": 22.02, "elapsed_time": "0:19:46", "remaining_time": "1:10:01"}
{"current_steps": 140, "total_steps": 613, "loss": 1.4437243461608886, "lr": 1.8052702575310588e-05, "epoch": 0.2284781721746226, "percentage": 22.84, "elapsed_time": "0:20:30", "remaining_time": "1:09:17"}
{"current_steps": 145, "total_steps": 613, "loss": 1.3076929092407226, "lr": 1.789311452252092e-05, "epoch": 0.23663810689514483, "percentage": 23.65, "elapsed_time": "0:21:15", "remaining_time": "1:08:38"}
{"current_steps": 150, "total_steps": 613, "loss": 1.4008633613586425, "lr": 1.772800709400383e-05, "epoch": 0.24479804161566707, "percentage": 24.47, "elapsed_time": "0:22:00", "remaining_time": "1:07:56"}
{"current_steps": 155, "total_steps": 613, "loss": 1.5305479049682618, "lr": 1.7557495743542586e-05, "epoch": 0.2529579763361893, "percentage": 25.29, "elapsed_time": "0:22:44", "remaining_time": "1:07:12"}
{"current_steps": 160, "total_steps": 613, "loss": 1.3546552658081055, "lr": 1.7381699703691866e-05, "epoch": 0.26111791105671156, "percentage": 26.1, "elapsed_time": "0:23:30", "remaining_time": "1:06:34"}
{"current_steps": 165, "total_steps": 613, "loss": 1.374640655517578, "lr": 1.720074190240269e-05, "epoch": 0.2692778457772338, "percentage": 26.92, "elapsed_time": "0:24:15", "remaining_time": "1:05:50"}
{"current_steps": 170, "total_steps": 613, "loss": 1.4571109771728517, "lr": 1.7014748877063212e-05, "epoch": 0.277437780497756, "percentage": 27.73, "elapsed_time": "0:24:59", "remaining_time": "1:05:06"}
{"current_steps": 175, "total_steps": 613, "loss": 1.3947142601013183, "lr": 1.682385068601563e-05, "epoch": 0.28559771521827826, "percentage": 28.55, "elapsed_time": "0:25:44", "remaining_time": "1:04:26"}
{"current_steps": 180, "total_steps": 613, "loss": 1.425752830505371, "lr": 1.6628180817610963e-05, "epoch": 0.2937576499388005, "percentage": 29.36, "elapsed_time": "0:26:29", "remaining_time": "1:03:44"}
{"current_steps": 185, "total_steps": 613, "loss": 1.4002631187438965, "lr": 1.6427876096865394e-05, "epoch": 0.3019175846593227, "percentage": 30.18, "elapsed_time": "0:27:14", "remaining_time": "1:03:00"}
{"current_steps": 190, "total_steps": 613, "loss": 1.3912601470947266, "lr": 1.6223076589783368e-05, "epoch": 0.31007751937984496, "percentage": 31.0, "elapsed_time": "0:27:58", "remaining_time": "1:02:16"}
{"current_steps": 195, "total_steps": 613, "loss": 1.2885801315307617, "lr": 1.6013925505414386e-05, "epoch": 0.3182374541003672, "percentage": 31.81, "elapsed_time": "0:28:43", "remaining_time": "1:01:33"}
{"current_steps": 200, "total_steps": 613, "loss": 1.3117466926574708, "lr": 1.5800569095711983e-05, "epoch": 0.3263973888208894, "percentage": 32.63, "elapsed_time": "0:29:28", "remaining_time": "1:00:51"}
{"current_steps": 205, "total_steps": 613, "loss": 1.4435410499572754, "lr": 1.5583156553264923e-05, "epoch": 0.33455732354141166, "percentage": 33.44, "elapsed_time": "0:30:11", "remaining_time": "1:00:05"}
{"current_steps": 210, "total_steps": 613, "loss": 1.3575885772705079, "lr": 1.5361839906972095e-05, "epoch": 0.3427172582619339, "percentage": 34.26, "elapsed_time": "0:30:56", "remaining_time": "0:59:22"}
{"current_steps": 215, "total_steps": 613, "loss": 1.3277423858642579, "lr": 1.5136773915734067e-05, "epoch": 0.3508771929824561, "percentage": 35.07, "elapsed_time": "0:31:41", "remaining_time": "0:58:40"}
{"current_steps": 220, "total_steps": 613, "loss": 1.4906869888305665, "lr": 1.4908115960235683e-05, "epoch": 0.35903712770297835, "percentage": 35.89, "elapsed_time": "0:32:25", "remaining_time": "0:57:55"}
{"current_steps": 225, "total_steps": 613, "loss": 1.472239875793457, "lr": 1.4676025932895315e-05, "epoch": 0.3671970624235006, "percentage": 36.7, "elapsed_time": "0:33:09", "remaining_time": "0:57:10"}
{"current_steps": 230, "total_steps": 613, "loss": 1.3827289581298827, "lr": 1.4440666126057743e-05, "epoch": 0.3753569971440229, "percentage": 37.52, "elapsed_time": "0:33:54", "remaining_time": "0:56:27"}
{"current_steps": 235, "total_steps": 613, "loss": 1.4698705673217773, "lr": 1.4202201118508863e-05, "epoch": 0.3835169318645451, "percentage": 38.34, "elapsed_time": "0:34:38", "remaining_time": "0:55:43"}
{"current_steps": 240, "total_steps": 613, "loss": 1.390995407104492, "lr": 1.396079766039157e-05, "epoch": 0.39167686658506734, "percentage": 39.15, "elapsed_time": "0:35:20", "remaining_time": "0:54:55"}
{"current_steps": 245, "total_steps": 613, "loss": 1.4784428596496582, "lr": 1.3716624556603275e-05, "epoch": 0.3998368013055896, "percentage": 39.97, "elapsed_time": "0:36:04", "remaining_time": "0:54:10"}
{"current_steps": 250, "total_steps": 613, "loss": 1.4209571838378907, "lr": 1.3469852548756626e-05, "epoch": 0.4079967360261118, "percentage": 40.78, "elapsed_time": "0:36:46", "remaining_time": "0:53:23"}
{"current_steps": 255, "total_steps": 613, "loss": 1.3656881332397461, "lr": 1.3220654195785917e-05, "epoch": 0.41615667074663404, "percentage": 41.6, "elapsed_time": "0:37:30", "remaining_time": "0:52:40"}
{"current_steps": 260, "total_steps": 613, "loss": 1.5011524200439452, "lr": 1.296920375328275e-05, "epoch": 0.42431660546715627, "percentage": 42.41, "elapsed_time": "0:38:13", "remaining_time": "0:51:54"}
{"current_steps": 265, "total_steps": 613, "loss": 1.4276366233825684, "lr": 1.2715677051645259e-05, "epoch": 0.4324765401876785, "percentage": 43.23, "elapsed_time": "0:38:58", "remaining_time": "0:51:11"}
{"current_steps": 270, "total_steps": 613, "loss": 1.390436553955078, "lr": 1.2460251373126136e-05, "epoch": 0.44063647490820074, "percentage": 44.05, "elapsed_time": "0:39:41", "remaining_time": "0:50:25"}
{"current_steps": 275, "total_steps": 613, "loss": 1.3565399169921875, "lr": 1.2203105327865407e-05, "epoch": 0.44879640962872297, "percentage": 44.86, "elapsed_time": "0:40:25", "remaining_time": "0:49:41"}
{"current_steps": 280, "total_steps": 613, "loss": 1.472177791595459, "lr": 1.194441872899471e-05, "epoch": 0.4569563443492452, "percentage": 45.68, "elapsed_time": "0:41:10", "remaining_time": "0:48:57"}
{"current_steps": 285, "total_steps": 613, "loss": 1.4198420524597168, "lr": 1.1684372466900306e-05, "epoch": 0.46511627906976744, "percentage": 46.49, "elapsed_time": "0:41:53", "remaining_time": "0:48:12"}
{"current_steps": 290, "total_steps": 613, "loss": 1.322580337524414, "lr": 1.1423148382732854e-05, "epoch": 0.47327621379028967, "percentage": 47.31, "elapsed_time": "0:42:39", "remaining_time": "0:47:30"}
{"current_steps": 295, "total_steps": 613, "loss": 1.4380602836608887, "lr": 1.1160929141252303e-05, "epoch": 0.4814361485108119, "percentage": 48.12, "elapsed_time": "0:43:23", "remaining_time": "0:46:46"}
{"current_steps": 300, "total_steps": 613, "loss": 1.285576343536377, "lr": 1.0897898103096917e-05, "epoch": 0.48959608323133413, "percentage": 48.94, "elapsed_time": "0:44:08", "remaining_time": "0:46:02"}
{"current_steps": 305, "total_steps": 613, "loss": 1.4786340713500976, "lr": 1.0634239196565646e-05, "epoch": 0.49775601795185637, "percentage": 49.76, "elapsed_time": "0:44:50", "remaining_time": "0:45:16"}
{"current_steps": 310, "total_steps": 613, "loss": 1.374602699279785, "lr": 1.0370136789003582e-05, "epoch": 0.5059159526723787, "percentage": 50.57, "elapsed_time": "0:45:34", "remaining_time": "0:44:32"}
{"current_steps": 315, "total_steps": 613, "loss": 1.4532788276672364, "lr": 1.0105775557880398e-05, "epoch": 0.5140758873929009, "percentage": 51.39, "elapsed_time": "0:46:18", "remaining_time": "0:43:48"}
{"current_steps": 320, "total_steps": 613, "loss": 1.3536710739135742, "lr": 9.841340361651921e-06, "epoch": 0.5222358221134231, "percentage": 52.2, "elapsed_time": "0:47:03", "remaining_time": "0:43:04"}
{"current_steps": 325, "total_steps": 613, "loss": 1.3883792877197265, "lr": 9.57701611049517e-06, "epoch": 0.5303957568339454, "percentage": 53.02, "elapsed_time": "0:47:46", "remaining_time": "0:42:20"}
{"current_steps": 330, "total_steps": 613, "loss": 1.3699709892272949, "lr": 9.312987637007191e-06, "epoch": 0.5385556915544676, "percentage": 53.83, "elapsed_time": "0:48:30", "remaining_time": "0:41:36"}
{"current_steps": 335, "total_steps": 613, "loss": 1.4548574447631837, "lr": 9.049439566958176e-06, "epoch": 0.5467156262749898, "percentage": 54.65, "elapsed_time": "0:49:14", "remaining_time": "0:40:51"}
{"current_steps": 340, "total_steps": 613, "loss": 1.3517833709716798, "lr": 8.786556190189183e-06, "epoch": 0.554875560995512, "percentage": 55.46, "elapsed_time": "0:49:59", "remaining_time": "0:40:08"}
{"current_steps": 345, "total_steps": 613, "loss": 1.426644992828369, "lr": 8.52452133174478e-06, "epoch": 0.5630354957160343, "percentage": 56.28, "elapsed_time": "0:50:43", "remaining_time": "0:39:24"}
{"current_steps": 350, "total_steps": 613, "loss": 1.383060359954834, "lr": 8.263518223330698e-06, "epoch": 0.5711954304365565, "percentage": 57.1, "elapsed_time": "0:51:28", "remaining_time": "0:38:41"}
{"current_steps": 355, "total_steps": 613, "loss": 1.331951904296875, "lr": 8.00372937518636e-06, "epoch": 0.5793553651570787, "percentage": 57.91, "elapsed_time": "0:52:13", "remaining_time": "0:37:56"}
{"current_steps": 360, "total_steps": 613, "loss": 1.445413303375244, "lr": 7.745336448461958e-06, "epoch": 0.587515299877601, "percentage": 58.73, "elapsed_time": "0:52:55", "remaining_time": "0:37:11"}
{"current_steps": 365, "total_steps": 613, "loss": 1.4074223518371582, "lr": 7.488520128189209e-06, "epoch": 0.5956752345981232, "percentage": 59.54, "elapsed_time": "0:53:39", "remaining_time": "0:36:27"}
{"current_steps": 370, "total_steps": 613, "loss": 1.3592453956604005, "lr": 7.233459996934731e-06, "epoch": 0.6038351693186454, "percentage": 60.36, "elapsed_time": "0:54:23", "remaining_time": "0:35:43"}
{"current_steps": 375, "total_steps": 613, "loss": 1.3151020050048827, "lr": 6.9803344092242855e-06, "epoch": 0.6119951040391677, "percentage": 61.17, "elapsed_time": "0:55:08", "remaining_time": "0:34:59"}
{"current_steps": 380, "total_steps": 613, "loss": 1.3170394897460938, "lr": 6.729320366825785e-06, "epoch": 0.6201550387596899, "percentage": 61.99, "elapsed_time": "0:55:53", "remaining_time": "0:34:16"}
{"current_steps": 385, "total_steps": 613, "loss": 1.4254400253295898, "lr": 6.480593394978208e-06, "epoch": 0.6283149734802121, "percentage": 62.81, "elapsed_time": "0:56:36", "remaining_time": "0:33:31"}
{"current_steps": 390, "total_steps": 613, "loss": 1.3528937339782714, "lr": 6.234327419653013e-06, "epoch": 0.6364749082007344, "percentage": 63.62, "elapsed_time": "0:57:21", "remaining_time": "0:32:48"}
{"current_steps": 395, "total_steps": 613, "loss": 1.3909344673156738, "lr": 5.990694645933866e-06, "epoch": 0.6446348429212566, "percentage": 64.44, "elapsed_time": "0:58:05", "remaining_time": "0:32:03"}
{"current_steps": 400, "total_steps": 613, "loss": 1.4332430839538575, "lr": 5.749865437599703e-06, "epoch": 0.6527947776417788, "percentage": 65.25, "elapsed_time": "0:58:48", "remaining_time": "0:31:18"}
{"current_steps": 405, "total_steps": 613, "loss": 1.4415637016296388, "lr": 5.512008197995379e-06, "epoch": 0.6609547123623011, "percentage": 66.07, "elapsed_time": "0:59:31", "remaining_time": "0:30:34"}
{"current_steps": 410, "total_steps": 613, "loss": 1.3970324516296386, "lr": 5.277289252273175e-06, "epoch": 0.6691146470828233, "percentage": 66.88, "elapsed_time": "1:00:15", "remaining_time": "0:29:49"}
{"current_steps": 415, "total_steps": 613, "loss": 1.3478738784790039, "lr": 5.045872731087479e-06, "epoch": 0.6772745818033455, "percentage": 67.7, "elapsed_time": "1:01:00", "remaining_time": "0:29:06"}
{"current_steps": 420, "total_steps": 613, "loss": 1.3553836822509766, "lr": 4.817920455824045e-06, "epoch": 0.6854345165238678, "percentage": 68.52, "elapsed_time": "1:01:44", "remaining_time": "0:28:22"}
{"current_steps": 425, "total_steps": 613, "loss": 1.4708582878112793, "lr": 4.593591825444028e-06, "epoch": 0.69359445124439, "percentage": 69.33, "elapsed_time": "1:02:24", "remaining_time": "0:27:36"}
{"current_steps": 430, "total_steps": 613, "loss": 1.4501720428466798, "lr": 4.373043705021899e-06, "epoch": 0.7017543859649122, "percentage": 70.15, "elapsed_time": "1:03:06", "remaining_time": "0:26:51"}
{"current_steps": 435, "total_steps": 613, "loss": 1.3744462966918944, "lr": 4.1564303160552935e-06, "epoch": 0.7099143206854345, "percentage": 70.96, "elapsed_time": "1:03:50", "remaining_time": "0:26:07"}
{"current_steps": 440, "total_steps": 613, "loss": 1.3946028709411622, "lr": 3.943903128623336e-06, "epoch": 0.7180742554059567, "percentage": 71.78, "elapsed_time": "1:04:31", "remaining_time": "0:25:22"}
{"current_steps": 445, "total_steps": 613, "loss": 1.2606407165527345, "lr": 3.735610755468988e-06, "epoch": 0.7262341901264789, "percentage": 72.59, "elapsed_time": "1:05:17", "remaining_time": "0:24:38"}
{"current_steps": 450, "total_steps": 613, "loss": 1.3681606292724608, "lr": 3.5316988480794255e-06, "epoch": 0.7343941248470012, "percentage": 73.41, "elapsed_time": "1:06:01", "remaining_time": "0:23:55"}
{"current_steps": 455, "total_steps": 613, "loss": 1.3252838134765625, "lr": 3.3323099948370853e-06, "epoch": 0.7425540595675234, "percentage": 74.23, "elapsed_time": "1:06:46", "remaining_time": "0:23:11"}
{"current_steps": 460, "total_steps": 613, "loss": 1.414201831817627, "lr": 3.1375836213126653e-06, "epoch": 0.7507139942880457, "percentage": 75.04, "elapsed_time": "1:07:30", "remaining_time": "0:22:27"}
{"current_steps": 465, "total_steps": 613, "loss": 1.318328857421875, "lr": 2.9476558927697605e-06, "epoch": 0.758873929008568, "percentage": 75.86, "elapsed_time": "1:08:13", "remaining_time": "0:21:42"}
{"current_steps": 470, "total_steps": 613, "loss": 1.3034313201904297, "lr": 2.7626596189492983e-06, "epoch": 0.7670338637290902, "percentage": 76.67, "elapsed_time": "1:08:58", "remaining_time": "0:20:59"}
{"current_steps": 475, "total_steps": 613, "loss": 1.3811626434326172, "lr": 2.582724161200405e-06, "epoch": 0.7751937984496124, "percentage": 77.49, "elapsed_time": "1:09:41", "remaining_time": "0:20:14"}
{"current_steps": 480, "total_steps": 613, "loss": 1.3948455810546876, "lr": 2.4079753420225694e-06, "epoch": 0.7833537331701347, "percentage": 78.3, "elapsed_time": "1:10:23", "remaining_time": "0:19:30"}
{"current_steps": 485, "total_steps": 613, "loss": 1.3906720161437989, "lr": 2.2385353570824308e-06, "epoch": 0.7915136678906569, "percentage": 79.12, "elapsed_time": "1:11:06", "remaining_time": "0:18:46"}
{"current_steps": 490, "total_steps": 613, "loss": 1.4307458877563477, "lr": 2.0745226897666858e-06, "epoch": 0.7996736026111791, "percentage": 79.93, "elapsed_time": "1:11:51", "remaining_time": "0:18:02"}
{"current_steps": 495, "total_steps": 613, "loss": 1.3465310096740724, "lr": 1.9160520283308115e-06, "epoch": 0.8078335373317014, "percentage": 80.75, "elapsed_time": "1:12:34", "remaining_time": "0:17:18"}
{"current_steps": 500, "total_steps": 613, "loss": 1.4635851860046387, "lr": 1.7632341857016733e-06, "epoch": 0.8159934720522236, "percentage": 81.57, "elapsed_time": "1:13:18", "remaining_time": "0:16:33"}
{"current_steps": 505, "total_steps": 613, "loss": 1.4342742919921876, "lr": 1.616176021989926e-06, "epoch": 0.8241534067727458, "percentage": 82.38, "elapsed_time": "1:14:03", "remaining_time": "0:15:50"}
{"current_steps": 510, "total_steps": 613, "loss": 1.4339244842529297, "lr": 1.4749803697665366e-06, "epoch": 0.8323133414932681, "percentage": 83.2, "elapsed_time": "1:14:47", "remaining_time": "0:15:06"}
{"current_steps": 515, "total_steps": 613, "loss": 1.3769190788269043, "lr": 1.339745962155613e-06, "epoch": 0.8404732762137903, "percentage": 84.01, "elapsed_time": "1:15:31", "remaining_time": "0:14:22"}
{"current_steps": 520, "total_steps": 613, "loss": 1.3848706245422364, "lr": 1.2105673637938054e-06, "epoch": 0.8486332109343125, "percentage": 84.83, "elapsed_time": "1:16:15", "remaining_time": "0:13:38"}
{"current_steps": 525, "total_steps": 613, "loss": 1.460084819793701, "lr": 1.0875349047046113e-06, "epoch": 0.8567931456548348, "percentage": 85.64, "elapsed_time": "1:16:57", "remaining_time": "0:12:53"}
{"current_steps": 530, "total_steps": 613, "loss": 1.4155464172363281, "lr": 9.707346171337895e-07, "epoch": 0.864953080375357, "percentage": 86.46, "elapsed_time": "1:17:38", "remaining_time": "0:12:09"}
{"current_steps": 535, "total_steps": 613, "loss": 1.370467472076416, "lr": 8.602481753900427e-07, "epoch": 0.8731130150958792, "percentage": 87.28, "elapsed_time": "1:18:22", "remaining_time": "0:11:25"}
{"current_steps": 540, "total_steps": 613, "loss": 1.4545866012573243, "lr": 7.561528387330797e-07, "epoch": 0.8812729498164015, "percentage": 88.09, "elapsed_time": "1:19:04", "remaining_time": "0:10:41"}
{"current_steps": 545, "total_steps": 613, "loss": 1.3990278244018555, "lr": 6.585213973489335e-07, "epoch": 0.8894328845369237, "percentage": 88.91, "elapsed_time": "1:19:49", "remaining_time": "0:09:57"}
{"current_steps": 550, "total_steps": 613, "loss": 1.3850707054138183, "lr": 5.674221214503639e-07, "epoch": 0.8975928192574459, "percentage": 89.72, "elapsed_time": "1:20:32", "remaining_time": "0:09:13"}
{"current_steps": 555, "total_steps": 613, "loss": 1.4300135612487792, "lr": 4.829187135379221e-07, "epoch": 0.9057527539779682, "percentage": 90.54, "elapsed_time": "1:21:15", "remaining_time": "0:08:29"}
{"current_steps": 560, "total_steps": 613, "loss": 1.3187259674072265, "lr": 4.0507026385502747e-07, "epoch": 0.9139126886984904, "percentage": 91.35, "elapsed_time": "1:21:58", "remaining_time": "0:07:45"}
{"current_steps": 565, "total_steps": 613, "loss": 1.4685959815979004, "lr": 3.339312090682689e-07, "epoch": 0.9220726234190126, "percentage": 92.17, "elapsed_time": "1:22:41", "remaining_time": "0:07:01"}
{"current_steps": 570, "total_steps": 613, "loss": 1.4852921485900878, "lr": 2.6955129420176193e-07, "epoch": 0.9302325581395349, "percentage": 92.99, "elapsed_time": "1:23:25", "remaining_time": "0:06:17"}
{"current_steps": 575, "total_steps": 613, "loss": 1.3241841316223144, "lr": 2.119755378522137e-07, "epoch": 0.9383924928600571, "percentage": 93.8, "elapsed_time": "1:24:09", "remaining_time": "0:05:33"}
{"current_steps": 580, "total_steps": 613, "loss": 1.3388869285583496, "lr": 1.612442007090076e-07, "epoch": 0.9465524275805793, "percentage": 94.62, "elapsed_time": "1:24:53", "remaining_time": "0:04:49"}
{"current_steps": 585, "total_steps": 613, "loss": 1.2855671882629394, "lr": 1.1739275740134004e-07, "epoch": 0.9547123623011016, "percentage": 95.43, "elapsed_time": "1:25:39", "remaining_time": "0:04:05"}
{"current_steps": 590, "total_steps": 613, "loss": 1.3641891479492188, "lr": 8.04518716920466e-08, "epoch": 0.9628722970216238, "percentage": 96.25, "elapsed_time": "1:26:23", "remaining_time": "0:03:22"}
{"current_steps": 595, "total_steps": 613, "loss": 1.5328051567077636, "lr": 5.044737503554165e-08, "epoch": 0.971032231742146, "percentage": 97.06, "elapsed_time": "1:27:03", "remaining_time": "0:02:38"}
{"current_steps": 600, "total_steps": 613, "loss": 1.3465785026550292, "lr": 2.7400248514776184e-08, "epoch": 0.9791921664626683, "percentage": 97.88, "elapsed_time": "1:27:48", "remaining_time": "0:01:54"}
{"current_steps": 605, "total_steps": 613, "loss": 1.39981107711792, "lr": 1.1326608169920373e-08, "epoch": 0.9873521011831905, "percentage": 98.69, "elapsed_time": "1:28:32", "remaining_time": "0:01:10"}
{"current_steps": 610, "total_steps": 613, "loss": 1.4022762298583984, "lr": 2.237693728981416e-09, "epoch": 0.9955120359037127, "percentage": 99.51, "elapsed_time": "1:29:16", "remaining_time": "0:00:26"}
{"current_steps": 613, "total_steps": 613, "epoch": 1.0, "percentage": 100.0, "elapsed_time": "1:29:42", "remaining_time": "0:00:00"}

897
trainer_state.json Normal file
View File

@@ -0,0 +1,897 @@
{
"best_global_step": null,
"best_metric": null,
"best_model_checkpoint": null,
"epoch": 1.0,
"eval_steps": 500,
"global_step": 613,
"is_hyper_param_search": false,
"is_local_process_zero": true,
"is_world_process_zero": true,
"log_history": [
{
"epoch": 0.008159934720522236,
"grad_norm": 42.25,
"learning_rate": 4.210526315789474e-06,
"loss": 2.764590835571289,
"step": 5
},
{
"epoch": 0.016319869441044473,
"grad_norm": 9.75,
"learning_rate": 9.473684210526315e-06,
"loss": 2.489868927001953,
"step": 10
},
{
"epoch": 0.02447980416156671,
"grad_norm": 8.1875,
"learning_rate": 1.4736842105263159e-05,
"loss": 2.2080495834350584,
"step": 15
},
{
"epoch": 0.032639738882088945,
"grad_norm": 4.4375,
"learning_rate": 2e-05,
"loss": 2.0085981369018553,
"step": 20
},
{
"epoch": 0.04079967360261118,
"grad_norm": 3.515625,
"learning_rate": 1.9996503676899863e-05,
"loss": 1.855879020690918,
"step": 25
},
{
"epoch": 0.04895960832313342,
"grad_norm": 3.359375,
"learning_rate": 1.9986017152454497e-05,
"loss": 1.8655593872070313,
"step": 30
},
{
"epoch": 0.05711954304365565,
"grad_norm": 3.140625,
"learning_rate": 1.9968547759519426e-05,
"loss": 1.764651107788086,
"step": 35
},
{
"epoch": 0.06527947776417789,
"grad_norm": 3.453125,
"learning_rate": 1.9944107713823068e-05,
"loss": 1.8568502426147462,
"step": 40
},
{
"epoch": 0.07343941248470012,
"grad_norm": 2.734375,
"learning_rate": 1.9912714105424694e-05,
"loss": 1.7390193939208984,
"step": 45
},
{
"epoch": 0.08159934720522236,
"grad_norm": 4.03125,
"learning_rate": 1.9874388886763944e-05,
"loss": 1.5365073204040527,
"step": 50
},
{
"epoch": 0.08975928192574459,
"grad_norm": 2.296875,
"learning_rate": 1.9829158857310288e-05,
"loss": 1.7221796035766601,
"step": 55
},
{
"epoch": 0.09791921664626684,
"grad_norm": 3.25,
"learning_rate": 1.9777055644823087e-05,
"loss": 1.6801658630371095,
"step": 60
},
{
"epoch": 0.10607915136678907,
"grad_norm": 3.75,
"learning_rate": 1.9718115683235418e-05,
"loss": 1.6599308013916017,
"step": 65
},
{
"epoch": 0.1142390860873113,
"grad_norm": 2.96875,
"learning_rate": 1.9652380187177128e-05,
"loss": 1.5088156700134276,
"step": 70
},
{
"epoch": 0.12239902080783353,
"grad_norm": 3.078125,
"learning_rate": 1.957989512315489e-05,
"loss": 1.541905117034912,
"step": 75
},
{
"epoch": 0.13055895552835578,
"grad_norm": 2.640625,
"learning_rate": 1.9500711177409456e-05,
"loss": 1.635506820678711,
"step": 80
},
{
"epoch": 0.138718890248878,
"grad_norm": 3.421875,
"learning_rate": 1.9414883720472557e-05,
"loss": 1.4972270011901856,
"step": 85
},
{
"epoch": 0.14687882496940025,
"grad_norm": 2.921875,
"learning_rate": 1.932247276844826e-05,
"loss": 1.5120899200439453,
"step": 90
},
{
"epoch": 0.15503875968992248,
"grad_norm": 2.421875,
"learning_rate": 1.9223542941045817e-05,
"loss": 1.516195011138916,
"step": 95
},
{
"epoch": 0.1631986944104447,
"grad_norm": 3.578125,
"learning_rate": 1.9118163416393392e-05,
"loss": 1.5576241493225098,
"step": 100
},
{
"epoch": 0.17135862913096694,
"grad_norm": 3.875,
"learning_rate": 1.9006407882664256e-05,
"loss": 1.4911109924316406,
"step": 105
},
{
"epoch": 0.17951856385148918,
"grad_norm": 2.875,
"learning_rate": 1.8888354486549238e-05,
"loss": 1.4046802520751953,
"step": 110
},
{
"epoch": 0.18767849857201144,
"grad_norm": 2.890625,
"learning_rate": 1.8764085778611507e-05,
"loss": 1.4821024894714356,
"step": 115
},
{
"epoch": 0.19583843329253367,
"grad_norm": 2.625,
"learning_rate": 1.863368865556191e-05,
"loss": 1.4233411788940429,
"step": 120
},
{
"epoch": 0.2039983680130559,
"grad_norm": 4.0,
"learning_rate": 1.8497254299495147e-05,
"loss": 1.4180265426635743,
"step": 125
},
{
"epoch": 0.21215830273357814,
"grad_norm": 3.796875,
"learning_rate": 1.8354878114129368e-05,
"loss": 1.4138151168823243,
"step": 130
},
{
"epoch": 0.22031823745410037,
"grad_norm": 2.4375,
"learning_rate": 1.820665965809373e-05,
"loss": 1.5245219230651856,
"step": 135
},
{
"epoch": 0.2284781721746226,
"grad_norm": 3.75,
"learning_rate": 1.8052702575310588e-05,
"loss": 1.4437243461608886,
"step": 140
},
{
"epoch": 0.23663810689514483,
"grad_norm": 3.078125,
"learning_rate": 1.789311452252092e-05,
"loss": 1.3076929092407226,
"step": 145
},
{
"epoch": 0.24479804161566707,
"grad_norm": 3.890625,
"learning_rate": 1.772800709400383e-05,
"loss": 1.4008633613586425,
"step": 150
},
{
"epoch": 0.2529579763361893,
"grad_norm": 2.375,
"learning_rate": 1.7557495743542586e-05,
"loss": 1.5305479049682618,
"step": 155
},
{
"epoch": 0.26111791105671156,
"grad_norm": 3.34375,
"learning_rate": 1.7381699703691866e-05,
"loss": 1.3546552658081055,
"step": 160
},
{
"epoch": 0.2692778457772338,
"grad_norm": 3.953125,
"learning_rate": 1.720074190240269e-05,
"loss": 1.374640655517578,
"step": 165
},
{
"epoch": 0.277437780497756,
"grad_norm": 2.9375,
"learning_rate": 1.7014748877063212e-05,
"loss": 1.4571109771728517,
"step": 170
},
{
"epoch": 0.28559771521827826,
"grad_norm": 3.9375,
"learning_rate": 1.682385068601563e-05,
"loss": 1.3947142601013183,
"step": 175
},
{
"epoch": 0.2937576499388005,
"grad_norm": 2.421875,
"learning_rate": 1.6628180817610963e-05,
"loss": 1.425752830505371,
"step": 180
},
{
"epoch": 0.3019175846593227,
"grad_norm": 3.4375,
"learning_rate": 1.6427876096865394e-05,
"loss": 1.4002631187438965,
"step": 185
},
{
"epoch": 0.31007751937984496,
"grad_norm": 3.8125,
"learning_rate": 1.6223076589783368e-05,
"loss": 1.3912601470947266,
"step": 190
},
{
"epoch": 0.3182374541003672,
"grad_norm": 4.46875,
"learning_rate": 1.6013925505414386e-05,
"loss": 1.2885801315307617,
"step": 195
},
{
"epoch": 0.3263973888208894,
"grad_norm": 3.96875,
"learning_rate": 1.5800569095711983e-05,
"loss": 1.3117466926574708,
"step": 200
},
{
"epoch": 0.33455732354141166,
"grad_norm": 3.6875,
"learning_rate": 1.5583156553264923e-05,
"loss": 1.4435410499572754,
"step": 205
},
{
"epoch": 0.3427172582619339,
"grad_norm": 2.71875,
"learning_rate": 1.5361839906972095e-05,
"loss": 1.3575885772705079,
"step": 210
},
{
"epoch": 0.3508771929824561,
"grad_norm": 4.125,
"learning_rate": 1.5136773915734067e-05,
"loss": 1.3277423858642579,
"step": 215
},
{
"epoch": 0.35903712770297835,
"grad_norm": 3.25,
"learning_rate": 1.4908115960235683e-05,
"loss": 1.4906869888305665,
"step": 220
},
{
"epoch": 0.3671970624235006,
"grad_norm": 2.875,
"learning_rate": 1.4676025932895315e-05,
"loss": 1.472239875793457,
"step": 225
},
{
"epoch": 0.3753569971440229,
"grad_norm": 3.0625,
"learning_rate": 1.4440666126057743e-05,
"loss": 1.3827289581298827,
"step": 230
},
{
"epoch": 0.3835169318645451,
"grad_norm": 2.90625,
"learning_rate": 1.4202201118508863e-05,
"loss": 1.4698705673217773,
"step": 235
},
{
"epoch": 0.39167686658506734,
"grad_norm": 2.21875,
"learning_rate": 1.396079766039157e-05,
"loss": 1.390995407104492,
"step": 240
},
{
"epoch": 0.3998368013055896,
"grad_norm": 2.734375,
"learning_rate": 1.3716624556603275e-05,
"loss": 1.4784428596496582,
"step": 245
},
{
"epoch": 0.4079967360261118,
"grad_norm": 2.59375,
"learning_rate": 1.3469852548756626e-05,
"loss": 1.4209571838378907,
"step": 250
},
{
"epoch": 0.41615667074663404,
"grad_norm": 3.6875,
"learning_rate": 1.3220654195785917e-05,
"loss": 1.3656881332397461,
"step": 255
},
{
"epoch": 0.42431660546715627,
"grad_norm": 2.53125,
"learning_rate": 1.296920375328275e-05,
"loss": 1.5011524200439452,
"step": 260
},
{
"epoch": 0.4324765401876785,
"grad_norm": 3.421875,
"learning_rate": 1.2715677051645259e-05,
"loss": 1.4276366233825684,
"step": 265
},
{
"epoch": 0.44063647490820074,
"grad_norm": 2.828125,
"learning_rate": 1.2460251373126136e-05,
"loss": 1.390436553955078,
"step": 270
},
{
"epoch": 0.44879640962872297,
"grad_norm": 3.265625,
"learning_rate": 1.2203105327865407e-05,
"loss": 1.3565399169921875,
"step": 275
},
{
"epoch": 0.4569563443492452,
"grad_norm": 2.84375,
"learning_rate": 1.194441872899471e-05,
"loss": 1.472177791595459,
"step": 280
},
{
"epoch": 0.46511627906976744,
"grad_norm": 2.296875,
"learning_rate": 1.1684372466900306e-05,
"loss": 1.4198420524597168,
"step": 285
},
{
"epoch": 0.47327621379028967,
"grad_norm": 4.03125,
"learning_rate": 1.1423148382732854e-05,
"loss": 1.322580337524414,
"step": 290
},
{
"epoch": 0.4814361485108119,
"grad_norm": 2.296875,
"learning_rate": 1.1160929141252303e-05,
"loss": 1.4380602836608887,
"step": 295
},
{
"epoch": 0.48959608323133413,
"grad_norm": 2.796875,
"learning_rate": 1.0897898103096917e-05,
"loss": 1.285576343536377,
"step": 300
},
{
"epoch": 0.49775601795185637,
"grad_norm": 2.84375,
"learning_rate": 1.0634239196565646e-05,
"loss": 1.4786340713500976,
"step": 305
},
{
"epoch": 0.5059159526723787,
"grad_norm": 2.953125,
"learning_rate": 1.0370136789003582e-05,
"loss": 1.374602699279785,
"step": 310
},
{
"epoch": 0.5140758873929009,
"grad_norm": 3.84375,
"learning_rate": 1.0105775557880398e-05,
"loss": 1.4532788276672364,
"step": 315
},
{
"epoch": 0.5222358221134231,
"grad_norm": 2.90625,
"learning_rate": 9.841340361651921e-06,
"loss": 1.3536710739135742,
"step": 320
},
{
"epoch": 0.5303957568339454,
"grad_norm": 2.765625,
"learning_rate": 9.57701611049517e-06,
"loss": 1.3883792877197265,
"step": 325
},
{
"epoch": 0.5385556915544676,
"grad_norm": 3.78125,
"learning_rate": 9.312987637007191e-06,
"loss": 1.3699709892272949,
"step": 330
},
{
"epoch": 0.5467156262749898,
"grad_norm": 3.140625,
"learning_rate": 9.049439566958176e-06,
"loss": 1.4548574447631837,
"step": 335
},
{
"epoch": 0.554875560995512,
"grad_norm": 3.0,
"learning_rate": 8.786556190189183e-06,
"loss": 1.3517833709716798,
"step": 340
},
{
"epoch": 0.5630354957160343,
"grad_norm": 3.25,
"learning_rate": 8.52452133174478e-06,
"loss": 1.426644992828369,
"step": 345
},
{
"epoch": 0.5711954304365565,
"grad_norm": 2.53125,
"learning_rate": 8.263518223330698e-06,
"loss": 1.383060359954834,
"step": 350
},
{
"epoch": 0.5793553651570787,
"grad_norm": 2.796875,
"learning_rate": 8.00372937518636e-06,
"loss": 1.331951904296875,
"step": 355
},
{
"epoch": 0.587515299877601,
"grad_norm": 2.609375,
"learning_rate": 7.745336448461958e-06,
"loss": 1.445413303375244,
"step": 360
},
{
"epoch": 0.5956752345981232,
"grad_norm": 2.4375,
"learning_rate": 7.488520128189209e-06,
"loss": 1.4074223518371582,
"step": 365
},
{
"epoch": 0.6038351693186454,
"grad_norm": 3.609375,
"learning_rate": 7.233459996934731e-06,
"loss": 1.3592453956604005,
"step": 370
},
{
"epoch": 0.6119951040391677,
"grad_norm": 4.28125,
"learning_rate": 6.9803344092242855e-06,
"loss": 1.3151020050048827,
"step": 375
},
{
"epoch": 0.6201550387596899,
"grad_norm": 3.078125,
"learning_rate": 6.729320366825785e-06,
"loss": 1.3170394897460938,
"step": 380
},
{
"epoch": 0.6283149734802121,
"grad_norm": 2.46875,
"learning_rate": 6.480593394978208e-06,
"loss": 1.4254400253295898,
"step": 385
},
{
"epoch": 0.6364749082007344,
"grad_norm": 4.34375,
"learning_rate": 6.234327419653013e-06,
"loss": 1.3528937339782714,
"step": 390
},
{
"epoch": 0.6446348429212566,
"grad_norm": 2.171875,
"learning_rate": 5.990694645933866e-06,
"loss": 1.3909344673156738,
"step": 395
},
{
"epoch": 0.6527947776417788,
"grad_norm": 2.359375,
"learning_rate": 5.749865437599703e-06,
"loss": 1.4332430839538575,
"step": 400
},
{
"epoch": 0.6609547123623011,
"grad_norm": 2.78125,
"learning_rate": 5.512008197995379e-06,
"loss": 1.4415637016296388,
"step": 405
},
{
"epoch": 0.6691146470828233,
"grad_norm": 2.109375,
"learning_rate": 5.277289252273175e-06,
"loss": 1.3970324516296386,
"step": 410
},
{
"epoch": 0.6772745818033455,
"grad_norm": 3.9375,
"learning_rate": 5.045872731087479e-06,
"loss": 1.3478738784790039,
"step": 415
},
{
"epoch": 0.6854345165238678,
"grad_norm": 2.734375,
"learning_rate": 4.817920455824045e-06,
"loss": 1.3553836822509766,
"step": 420
},
{
"epoch": 0.69359445124439,
"grad_norm": 2.828125,
"learning_rate": 4.593591825444028e-06,
"loss": 1.4708582878112793,
"step": 425
},
{
"epoch": 0.7017543859649122,
"grad_norm": 2.328125,
"learning_rate": 4.373043705021899e-06,
"loss": 1.4501720428466798,
"step": 430
},
{
"epoch": 0.7099143206854345,
"grad_norm": 4.3125,
"learning_rate": 4.1564303160552935e-06,
"loss": 1.3744462966918944,
"step": 435
},
{
"epoch": 0.7180742554059567,
"grad_norm": 1.859375,
"learning_rate": 3.943903128623336e-06,
"loss": 1.3946028709411622,
"step": 440
},
{
"epoch": 0.7262341901264789,
"grad_norm": 3.4375,
"learning_rate": 3.735610755468988e-06,
"loss": 1.2606407165527345,
"step": 445
},
{
"epoch": 0.7343941248470012,
"grad_norm": 2.96875,
"learning_rate": 3.5316988480794255e-06,
"loss": 1.3681606292724608,
"step": 450
},
{
"epoch": 0.7425540595675234,
"grad_norm": 2.46875,
"learning_rate": 3.3323099948370853e-06,
"loss": 1.3252838134765625,
"step": 455
},
{
"epoch": 0.7507139942880457,
"grad_norm": 3.90625,
"learning_rate": 3.1375836213126653e-06,
"loss": 1.414201831817627,
"step": 460
},
{
"epoch": 0.758873929008568,
"grad_norm": 4.4375,
"learning_rate": 2.9476558927697605e-06,
"loss": 1.318328857421875,
"step": 465
},
{
"epoch": 0.7670338637290902,
"grad_norm": 3.78125,
"learning_rate": 2.7626596189492983e-06,
"loss": 1.3034313201904297,
"step": 470
},
{
"epoch": 0.7751937984496124,
"grad_norm": 2.25,
"learning_rate": 2.582724161200405e-06,
"loss": 1.3811626434326172,
"step": 475
},
{
"epoch": 0.7833537331701347,
"grad_norm": 3.125,
"learning_rate": 2.4079753420225694e-06,
"loss": 1.3948455810546876,
"step": 480
},
{
"epoch": 0.7915136678906569,
"grad_norm": 2.125,
"learning_rate": 2.2385353570824308e-06,
"loss": 1.3906720161437989,
"step": 485
},
{
"epoch": 0.7996736026111791,
"grad_norm": 2.859375,
"learning_rate": 2.0745226897666858e-06,
"loss": 1.4307458877563477,
"step": 490
},
{
"epoch": 0.8078335373317014,
"grad_norm": 2.65625,
"learning_rate": 1.9160520283308115e-06,
"loss": 1.3465310096740724,
"step": 495
},
{
"epoch": 0.8159934720522236,
"grad_norm": 2.96875,
"learning_rate": 1.7632341857016733e-06,
"loss": 1.4635851860046387,
"step": 500
},
{
"epoch": 0.8241534067727458,
"grad_norm": 3.578125,
"learning_rate": 1.616176021989926e-06,
"loss": 1.4342742919921876,
"step": 505
},
{
"epoch": 0.8323133414932681,
"grad_norm": 2.390625,
"learning_rate": 1.4749803697665366e-06,
"loss": 1.4339244842529297,
"step": 510
},
{
"epoch": 0.8404732762137903,
"grad_norm": 2.65625,
"learning_rate": 1.339745962155613e-06,
"loss": 1.3769190788269043,
"step": 515
},
{
"epoch": 0.8486332109343125,
"grad_norm": 3.4375,
"learning_rate": 1.2105673637938054e-06,
"loss": 1.3848706245422364,
"step": 520
},
{
"epoch": 0.8567931456548348,
"grad_norm": 2.75,
"learning_rate": 1.0875349047046113e-06,
"loss": 1.460084819793701,
"step": 525
},
{
"epoch": 0.864953080375357,
"grad_norm": 4.3125,
"learning_rate": 9.707346171337895e-07,
"loss": 1.4155464172363281,
"step": 530
},
{
"epoch": 0.8731130150958792,
"grad_norm": 3.78125,
"learning_rate": 8.602481753900427e-07,
"loss": 1.370467472076416,
"step": 535
},
{
"epoch": 0.8812729498164015,
"grad_norm": 2.765625,
"learning_rate": 7.561528387330797e-07,
"loss": 1.4545866012573243,
"step": 540
},
{
"epoch": 0.8894328845369237,
"grad_norm": 3.203125,
"learning_rate": 6.585213973489335e-07,
"loss": 1.3990278244018555,
"step": 545
},
{
"epoch": 0.8975928192574459,
"grad_norm": 2.375,
"learning_rate": 5.674221214503639e-07,
"loss": 1.3850707054138183,
"step": 550
},
{
"epoch": 0.9057527539779682,
"grad_norm": 3.125,
"learning_rate": 4.829187135379221e-07,
"loss": 1.4300135612487792,
"step": 555
},
{
"epoch": 0.9139126886984904,
"grad_norm": 2.140625,
"learning_rate": 4.0507026385502747e-07,
"loss": 1.3187259674072265,
"step": 560
},
{
"epoch": 0.9220726234190126,
"grad_norm": 2.53125,
"learning_rate": 3.339312090682689e-07,
"loss": 1.4685959815979004,
"step": 565
},
{
"epoch": 0.9302325581395349,
"grad_norm": 2.59375,
"learning_rate": 2.6955129420176193e-07,
"loss": 1.4852921485900878,
"step": 570
},
{
"epoch": 0.9383924928600571,
"grad_norm": 3.4375,
"learning_rate": 2.119755378522137e-07,
"loss": 1.3241841316223144,
"step": 575
},
{
"epoch": 0.9465524275805793,
"grad_norm": 2.578125,
"learning_rate": 1.612442007090076e-07,
"loss": 1.3388869285583496,
"step": 580
},
{
"epoch": 0.9547123623011016,
"grad_norm": 4.625,
"learning_rate": 1.1739275740134004e-07,
"loss": 1.2855671882629394,
"step": 585
},
{
"epoch": 0.9628722970216238,
"grad_norm": 2.53125,
"learning_rate": 8.04518716920466e-08,
"loss": 1.3641891479492188,
"step": 590
},
{
"epoch": 0.971032231742146,
"grad_norm": 2.78125,
"learning_rate": 5.044737503554165e-08,
"loss": 1.5328051567077636,
"step": 595
},
{
"epoch": 0.9791921664626683,
"grad_norm": 2.578125,
"learning_rate": 2.7400248514776184e-08,
"loss": 1.3465785026550292,
"step": 600
},
{
"epoch": 0.9873521011831905,
"grad_norm": 2.71875,
"learning_rate": 1.1326608169920373e-08,
"loss": 1.39981107711792,
"step": 605
},
{
"epoch": 0.9955120359037127,
"grad_norm": 3.125,
"learning_rate": 2.237693728981416e-09,
"loss": 1.4022762298583984,
"step": 610
},
{
"epoch": 1.0,
"step": 613,
"total_flos": 2.961647362958254e+17,
"train_loss": 1.4641528666311143,
"train_runtime": 5382.2079,
"train_samples_per_second": 0.455,
"train_steps_per_second": 0.114
}
],
"logging_steps": 5,
"max_steps": 613,
"num_input_tokens_seen": 0,
"num_train_epochs": 1,
"save_steps": 500,
"stateful_callbacks": {
"TrainerControl": {
"args": {
"should_epoch_stop": false,
"should_evaluate": false,
"should_log": false,
"should_save": true,
"should_training_stop": true
},
"attributes": {}
}
},
"total_flos": 2.961647362958254e+17,
"train_batch_size": 1,
"trial_name": null,
"trial_params": null
}

3
training_args.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6516224cadc9cbb209709ae24495c06e85025ef848ced11f26ab06f1fe651ae8
size 6033

BIN
training_loss.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 47 KiB