lldois/v13_rec_cot_focus_lr2e5
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: lldois/v13_rec_cot_focus_lr2e5
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-30 16:11:19 +08:00
commit 6950d2ba7b
16 changed files with 3770 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

58
README.md Normal file
View File

@@ -0,0 +1,58 @@
---
library_name: transformers
license: other
base_model: OpenOneRec/OneReason-0.8B-pretrain-competition
tags:
- llama-factory
- full
- generated_from_trainer
model-index:
- name: v13_rec_cot_focus_lr2e5
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# v13_rec_cot_focus_lr2e5
This model is a fine-tuned version of [OpenOneRec/OneReason-0.8B-pretrain-competition](https://huggingface.co/OpenOneRec/OneReason-0.8B-pretrain-competition) on the exp_rec_cot_focus dataset.
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 1
- eval_batch_size: 8
- seed: 202607113
- gradient_accumulation_steps: 4
- total_train_batch_size: 4
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 0.03
- num_epochs: 1
### Training results
### Framework versions
- Transformers 5.6.0
- Pytorch 2.7.1+cu126
- Datasets 4.0.0
- Tokenizers 0.22.2

8
all_results.json Normal file
View File

@@ -0,0 +1,8 @@
{
"epoch": 1.0,
"total_flos": 3.2515782676161946e+17,
"train_loss": 1.470360949480975,
"train_runtime": 5409.9531,
"train_samples_per_second": 0.499,
"train_steps_per_second": 0.125
}

89
chat_template.jinja Normal file
View File

@@ -0,0 +1,89 @@
{%- if tools %}
{{- '<|im_start|>system\n' }}
{%- if messages[0].role == 'system' %}
{{- messages[0].content + '\n\n' }}
{%- endif %}
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
{%- if messages[0].role == 'system' %}
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
{%- set index = (messages|length - 1) - loop.index0 %}
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
{%- set ns.multi_step_tool = false %}
{%- set ns.last_query_index = index %}
{%- endif %}
{%- endfor %}
{%- for message in messages %}
{%- if message.content is string %}
{%- set content = message.content %}
{%- else %}
{%- set content = '' %}
{%- endif %}
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
{%- elif message.role == "assistant" %}
{%- set reasoning_content = '' %}
{%- if message.reasoning_content is string %}
{%- set reasoning_content = message.reasoning_content %}
{%- else %}
{%- if '</think>' in content %}
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
{%- endif %}
{%- endif %}
{%- if loop.index0 > ns.last_query_index %}
{%- if loop.last or (not loop.last and reasoning_content) %}
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- if message.tool_calls %}
{%- for tool_call in message.tool_calls %}
{%- if (loop.first and content) or (not loop.first) %}
{{- '\n' }}
{%- endif %}
{%- if tool_call.function %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{{- '<tool_call>\n{"name": "' }}
{{- tool_call.name }}
{{- '", "arguments": ' }}
{%- if tool_call.arguments is string %}
{{- tool_call.arguments }}
{%- else %}
{{- tool_call.arguments | tojson }}
{%- endif %}
{{- '}\n</tool_call>' }}
{%- endfor %}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- elif message.role == "tool" %}
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' }}
{{- content }}
{{- '\n</tool_response>' }}
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- if enable_thinking is defined and enable_thinking is false %}
{{- '<think>\n\n</think>\n\n' }}
{%- endif %}
{%- endif %}

63
config.json Normal file
View File

@@ -0,0 +1,63 @@
{
"architectures": [
"Qwen3ForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": null,
"dtype": "bfloat16",
"eos_token_id": 151645,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 1024,
"initializer_range": 0.02,
"intermediate_size": 3072,
"layer_types": [
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention"
],
"max_position_embeddings": 40960,
"max_window_layers": 28,
"model_type": "qwen3",
"num_attention_heads": 16,
"num_hidden_layers": 28,
"num_key_value_heads": 8,
"pad_token_id": 151643,
"rms_norm_eps": 1e-06,
"rope_parameters": {
"rope_theta": 1000000,
"rope_type": "default"
},
"sliding_window": null,
"tie_word_embeddings": false,
"transformers_version": "5.6.0",
"use_cache": false,
"use_sliding_window": false,
"vocab_size": 176253
}

12
generation_config.json Normal file
View File

@@ -0,0 +1,12 @@
{
"do_sample": true,
"eos_token_id": [
151645,
151643
],
"pad_token_id": 151643,
"temperature": 0.6,
"top_k": 20,
"top_p": 0.95,
"transformers_version": "5.6.0"
}

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0a972149f95e07f4694fe706bafbadd147d316f6490dad287e1ce0d2a49e32b5
size 1602902832

3
runs/Jul07_03-50-00_ubuntu-System-Product-Name/events.out.tfevents.1783367400.ubuntu-System-Product-Name.2811805.0 Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:45b04c578953d2e752ce41f94fa6c3c035e681c323fd0ffde26c60a72d14b30c
size 34666

3
tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cd4d15f596979aecbc11ba12668cb21c9fb8452ce1235cd9d7878cc950170421
size 16016554

16
tokenizer_config.json Normal file
View File

@@ -0,0 +1,16 @@
{
"add_prefix_space": false,
"backend": "tokenizers",
"bos_token": null,
"clean_up_tokenization_spaces": false,
"eos_token": "<|im_end|>",
"errors": "replace",
"is_local": false,
"local_files_only": false,
"model_max_length": 131072,
"pad_token": "<|endoftext|>",
"padding_side": "right",
"split_special_tokens": false,
"tokenizer_class": "Qwen2Tokenizer",
"unk_token": null
}

2344
train.log Normal file
View File

File diff suppressed because one or more lines are too long

8
train_results.json Normal file
View File

@@ -0,0 +1,8 @@
{
"epoch": 1.0,
"total_flos": 3.2515782676161946e+17,
"train_loss": 1.470360949480975,
"train_runtime": 5409.9531,
"train_samples_per_second": 0.499,
"train_steps_per_second": 0.125
}

136
trainer_log.jsonl Normal file
View File

@@ -0,0 +1,136 @@
{"current_steps": 5, "total_steps": 675, "loss": 2.981045722961426, "lr": 3.80952380952381e-06, "epoch": 0.007412898443291327, "percentage": 0.74, "elapsed_time": "0:00:39", "remaining_time": "1:28:55"}
{"current_steps": 10, "total_steps": 675, "loss": 2.7763238906860352, "lr": 8.571428571428571e-06, "epoch": 0.014825796886582653, "percentage": 1.48, "elapsed_time": "0:01:19", "remaining_time": "1:27:55"}
{"current_steps": 15, "total_steps": 675, "loss": 2.5071876525878904, "lr": 1.3333333333333333e-05, "epoch": 0.02223869532987398, "percentage": 2.22, "elapsed_time": "0:01:59", "remaining_time": "1:27:33"}
{"current_steps": 20, "total_steps": 675, "loss": 2.216544532775879, "lr": 1.8095238095238097e-05, "epoch": 0.029651593773165306, "percentage": 2.96, "elapsed_time": "0:02:40", "remaining_time": "1:27:36"}
{"current_steps": 25, "total_steps": 675, "loss": 2.02364444732666, "lr": 1.9998961636899736e-05, "epoch": 0.037064492216456635, "percentage": 3.7, "elapsed_time": "0:03:20", "remaining_time": "1:27:01"}
{"current_steps": 30, "total_steps": 675, "loss": 1.8605846405029296, "lr": 1.9992616865520515e-05, "epoch": 0.04447739065974796, "percentage": 4.44, "elapsed_time": "0:04:01", "remaining_time": "1:26:29"}
{"current_steps": 35, "total_steps": 675, "loss": 1.8314769744873047, "lr": 1.998050784669941e-05, "epoch": 0.05189028910303929, "percentage": 5.19, "elapsed_time": "0:04:41", "remaining_time": "1:25:47"}
{"current_steps": 40, "total_steps": 675, "loss": 1.7219036102294922, "lr": 1.9962641565531694e-05, "epoch": 0.05930318754633061, "percentage": 5.93, "elapsed_time": "0:05:22", "remaining_time": "1:25:25"}
{"current_steps": 45, "total_steps": 675, "loss": 1.6877906799316407, "lr": 1.9939028328193097e-05, "epoch": 0.06671608598962195, "percentage": 6.67, "elapsed_time": "0:06:01", "remaining_time": "1:24:20"}
{"current_steps": 50, "total_steps": 675, "loss": 1.6538002014160156, "lr": 1.990968175599471e-05, "epoch": 0.07412898443291327, "percentage": 7.41, "elapsed_time": "0:06:43", "remaining_time": "1:24:02"}
{"current_steps": 55, "total_steps": 675, "loss": 1.6635187149047852, "lr": 1.987461877752552e-05, "epoch": 0.08154188287620459, "percentage": 8.15, "elapsed_time": "0:07:22", "remaining_time": "1:23:07"}
{"current_steps": 60, "total_steps": 675, "loss": 1.5999790191650392, "lr": 1.983385961888711e-05, "epoch": 0.08895478131949593, "percentage": 8.89, "elapsed_time": "0:08:04", "remaining_time": "1:22:49"}
{"current_steps": 65, "total_steps": 675, "loss": 1.572991943359375, "lr": 1.978742779202627e-05, "epoch": 0.09636767976278725, "percentage": 9.63, "elapsed_time": "0:08:45", "remaining_time": "1:22:09"}
{"current_steps": 70, "total_steps": 675, "loss": 1.6114467620849608, "lr": 1.973535008117207e-05, "epoch": 0.10378057820607858, "percentage": 10.37, "elapsed_time": "0:09:25", "remaining_time": "1:21:31"}
{"current_steps": 75, "total_steps": 675, "loss": 1.5769593238830566, "lr": 1.9677656527385386e-05, "epoch": 0.1111934766493699, "percentage": 11.11, "elapsed_time": "0:10:06", "remaining_time": "1:20:55"}
{"current_steps": 80, "total_steps": 675, "loss": 1.5900885581970214, "lr": 1.9614380411229693e-05, "epoch": 0.11860637509266123, "percentage": 11.85, "elapsed_time": "0:10:46", "remaining_time": "1:20:10"}
{"current_steps": 85, "total_steps": 675, "loss": 1.5888105392456056, "lr": 1.9545558233573136e-05, "epoch": 0.12601927353595255, "percentage": 12.59, "elapsed_time": "0:11:26", "remaining_time": "1:19:25"}
{"current_steps": 90, "total_steps": 675, "loss": 1.563032054901123, "lr": 1.9471229694533003e-05, "epoch": 0.1334321719792439, "percentage": 13.33, "elapsed_time": "0:12:05", "remaining_time": "1:18:34"}
{"current_steps": 95, "total_steps": 675, "loss": 1.5782450675964355, "lr": 1.9391437670574656e-05, "epoch": 0.14084507042253522, "percentage": 14.07, "elapsed_time": "0:12:44", "remaining_time": "1:17:50"}
{"current_steps": 100, "total_steps": 675, "loss": 1.5569299697875976, "lr": 1.9306228189778255e-05, "epoch": 0.14825796886582654, "percentage": 14.81, "elapsed_time": "0:13:24", "remaining_time": "1:17:07"}
{"current_steps": 105, "total_steps": 675, "loss": 1.5072972297668457, "lr": 1.9215650405287392e-05, "epoch": 0.15567086730911786, "percentage": 15.56, "elapsed_time": "0:14:05", "remaining_time": "1:16:28"}
{"current_steps": 110, "total_steps": 675, "loss": 1.4942115783691405, "lr": 1.9119756566955092e-05, "epoch": 0.16308376575240918, "percentage": 16.3, "elapsed_time": "0:14:45", "remaining_time": "1:15:50"}
{"current_steps": 115, "total_steps": 675, "loss": 1.5515302658081054, "lr": 1.901860199120344e-05, "epoch": 0.17049666419570053, "percentage": 17.04, "elapsed_time": "0:15:27", "remaining_time": "1:15:16"}
{"current_steps": 120, "total_steps": 675, "loss": 1.4991122245788575, "lr": 1.891224502911428e-05, "epoch": 0.17790956263899185, "percentage": 17.78, "elapsed_time": "0:16:08", "remaining_time": "1:14:37"}
{"current_steps": 125, "total_steps": 675, "loss": 1.4753767013549806, "lr": 1.8800747032769343e-05, "epoch": 0.18532246108228317, "percentage": 18.52, "elapsed_time": "0:16:48", "remaining_time": "1:13:58"}
{"current_steps": 130, "total_steps": 675, "loss": 1.4607695579528808, "lr": 1.8684172319859258e-05, "epoch": 0.1927353595255745, "percentage": 19.26, "elapsed_time": "0:17:28", "remaining_time": "1:13:15"}
{"current_steps": 135, "total_steps": 675, "loss": 1.461857509613037, "lr": 1.856258813658188e-05, "epoch": 0.20014825796886582, "percentage": 20.0, "elapsed_time": "0:18:09", "remaining_time": "1:12:38"}
{"current_steps": 140, "total_steps": 675, "loss": 1.530456256866455, "lr": 1.8436064618851225e-05, "epoch": 0.20756115641215717, "percentage": 20.74, "elapsed_time": "0:18:47", "remaining_time": "1:11:48"}
{"current_steps": 145, "total_steps": 675, "loss": 1.4979378700256347, "lr": 1.8304674751839583e-05, "epoch": 0.2149740548554485, "percentage": 21.48, "elapsed_time": "0:19:25", "remaining_time": "1:10:58"}
{"current_steps": 150, "total_steps": 675, "loss": 1.4886467933654786, "lr": 1.8168494327875918e-05, "epoch": 0.2223869532987398, "percentage": 22.22, "elapsed_time": "0:20:05", "remaining_time": "1:10:17"}
{"current_steps": 155, "total_steps": 675, "loss": 1.4672227859497071, "lr": 1.8027601902725065e-05, "epoch": 0.22979985174203113, "percentage": 22.96, "elapsed_time": "0:20:45", "remaining_time": "1:09:38"}
{"current_steps": 160, "total_steps": 675, "loss": 1.4558910369873046, "lr": 1.788207875027274e-05, "epoch": 0.23721275018532245, "percentage": 23.7, "elapsed_time": "0:21:23", "remaining_time": "1:08:51"}
{"current_steps": 165, "total_steps": 675, "loss": 1.4654417037963867, "lr": 1.77320088156427e-05, "epoch": 0.2446256486286138, "percentage": 24.44, "elapsed_time": "0:22:04", "remaining_time": "1:08:12"}
{"current_steps": 170, "total_steps": 675, "loss": 1.4390137672424317, "lr": 1.7577478666772886e-05, "epoch": 0.2520385470719051, "percentage": 25.19, "elapsed_time": "0:22:44", "remaining_time": "1:07:32"}
{"current_steps": 175, "total_steps": 675, "loss": 1.4661806106567383, "lr": 1.741857744447869e-05, "epoch": 0.25945144551519644, "percentage": 25.93, "elapsed_time": "0:23:22", "remaining_time": "1:06:47"}
{"current_steps": 180, "total_steps": 675, "loss": 1.472686767578125, "lr": 1.7255396811032014e-05, "epoch": 0.2668643439584878, "percentage": 26.67, "elapsed_time": "0:24:01", "remaining_time": "1:06:03"}
{"current_steps": 185, "total_steps": 675, "loss": 1.4672610282897949, "lr": 1.708803089728583e-05, "epoch": 0.2742772424017791, "percentage": 27.41, "elapsed_time": "0:24:42", "remaining_time": "1:05:26"}
{"current_steps": 190, "total_steps": 675, "loss": 1.4069836616516114, "lr": 1.691657624837472e-05, "epoch": 0.28169014084507044, "percentage": 28.15, "elapsed_time": "0:25:23", "remaining_time": "1:04:49"}
{"current_steps": 195, "total_steps": 675, "loss": 1.471778392791748, "lr": 1.6741131768022777e-05, "epoch": 0.28910303928836173, "percentage": 28.89, "elapsed_time": "0:26:03", "remaining_time": "1:04:08"}
{"current_steps": 200, "total_steps": 675, "loss": 1.4518500328063966, "lr": 1.6561798661490904e-05, "epoch": 0.2965159377316531, "percentage": 29.63, "elapsed_time": "0:26:42", "remaining_time": "1:03:25"}
{"current_steps": 205, "total_steps": 675, "loss": 1.4200240135192872, "lr": 1.6378680377196526e-05, "epoch": 0.30392883617494443, "percentage": 30.37, "elapsed_time": "0:27:22", "remaining_time": "1:02:46"}
{"current_steps": 210, "total_steps": 675, "loss": 1.4366942405700684, "lr": 1.619188254703927e-05, "epoch": 0.3113417346182357, "percentage": 31.11, "elapsed_time": "0:28:02", "remaining_time": "1:02:06"}
{"current_steps": 215, "total_steps": 675, "loss": 1.4304126739501952, "lr": 1.6001512925467153e-05, "epoch": 0.31875463306152707, "percentage": 31.85, "elapsed_time": "0:28:42", "remaining_time": "1:01:25"}
{"current_steps": 220, "total_steps": 675, "loss": 1.4636366844177247, "lr": 1.5807681327318372e-05, "epoch": 0.32616753150481836, "percentage": 32.59, "elapsed_time": "0:29:21", "remaining_time": "1:00:43"}
{"current_steps": 225, "total_steps": 675, "loss": 1.440121841430664, "lr": 1.5610499564474512e-05, "epoch": 0.3335804299481097, "percentage": 33.33, "elapsed_time": "0:30:00", "remaining_time": "1:00:01"}
{"current_steps": 230, "total_steps": 675, "loss": 1.4860041618347168, "lr": 1.541008138136183e-05, "epoch": 0.34099332839140106, "percentage": 34.07, "elapsed_time": "0:30:40", "remaining_time": "0:59:20"}
{"current_steps": 235, "total_steps": 675, "loss": 1.4189085006713866, "lr": 1.520654238933767e-05, "epoch": 0.34840622683469236, "percentage": 34.81, "elapsed_time": "0:31:20", "remaining_time": "0:58:40"}
{"current_steps": 240, "total_steps": 675, "loss": 1.407963466644287, "lr": 1.5000000000000002e-05, "epoch": 0.3558191252779837, "percentage": 35.56, "elapsed_time": "0:31:59", "remaining_time": "0:57:58"}
{"current_steps": 245, "total_steps": 675, "loss": 1.4134508132934571, "lr": 1.4790573357458464e-05, "epoch": 0.363232023721275, "percentage": 36.3, "elapsed_time": "0:32:38", "remaining_time": "0:57:17"}
{"current_steps": 250, "total_steps": 675, "loss": 1.425547218322754, "lr": 1.4578383269606004e-05, "epoch": 0.37064492216456635, "percentage": 37.04, "elapsed_time": "0:33:18", "remaining_time": "0:56:37"}
{"current_steps": 255, "total_steps": 675, "loss": 1.402323055267334, "lr": 1.4363552138430764e-05, "epoch": 0.3780578206078577, "percentage": 37.78, "elapsed_time": "0:33:58", "remaining_time": "0:55:56"}
{"current_steps": 260, "total_steps": 675, "loss": 1.4572528839111327, "lr": 1.4146203889408418e-05, "epoch": 0.385470719051149, "percentage": 38.52, "elapsed_time": "0:34:36", "remaining_time": "0:55:13"}
{"current_steps": 265, "total_steps": 675, "loss": 1.3995424270629884, "lr": 1.392646390001569e-05, "epoch": 0.39288361749444034, "percentage": 39.26, "elapsed_time": "0:35:15", "remaining_time": "0:54:32"}
{"current_steps": 270, "total_steps": 675, "loss": 1.4161681175231933, "lr": 1.3704458927406261e-05, "epoch": 0.40029651593773163, "percentage": 40.0, "elapsed_time": "0:35:55", "remaining_time": "0:53:52"}
{"current_steps": 275, "total_steps": 675, "loss": 1.4083374977111816, "lr": 1.3480317035290837e-05, "epoch": 0.407709414381023, "percentage": 40.74, "elapsed_time": "0:36:36", "remaining_time": "0:53:15"}
{"current_steps": 280, "total_steps": 675, "loss": 1.3747909545898438, "lr": 1.325416752006351e-05, "epoch": 0.41512231282431433, "percentage": 41.48, "elapsed_time": "0:37:15", "remaining_time": "0:52:33"}
{"current_steps": 285, "total_steps": 675, "loss": 1.4275038719177247, "lr": 1.302614083621705e-05, "epoch": 0.4225352112676056, "percentage": 42.22, "elapsed_time": "0:37:56", "remaining_time": "0:51:54"}
{"current_steps": 290, "total_steps": 675, "loss": 1.3964252471923828, "lr": 1.2796368521090143e-05, "epoch": 0.429948109710897, "percentage": 42.96, "elapsed_time": "0:38:36", "remaining_time": "0:51:15"}
{"current_steps": 295, "total_steps": 675, "loss": 1.401359462738037, "lr": 1.256498311899001e-05, "epoch": 0.43736100815418827, "percentage": 43.7, "elapsed_time": "0:39:17", "remaining_time": "0:50:36"}
{"current_steps": 300, "total_steps": 675, "loss": 1.4066211700439453, "lr": 1.233211810473411e-05, "epoch": 0.4447739065974796, "percentage": 44.44, "elapsed_time": "0:39:58", "remaining_time": "0:49:57"}
{"current_steps": 305, "total_steps": 675, "loss": 1.3704119682312013, "lr": 1.2097907806655136e-05, "epoch": 0.45218680504077097, "percentage": 45.19, "elapsed_time": "0:40:38", "remaining_time": "0:49:18"}
{"current_steps": 310, "total_steps": 675, "loss": 1.3963244438171387, "lr": 1.1862487329113606e-05, "epoch": 0.45959970348406226, "percentage": 45.93, "elapsed_time": "0:41:18", "remaining_time": "0:48:38"}
{"current_steps": 315, "total_steps": 675, "loss": 1.402764892578125, "lr": 1.1625992474562846e-05, "epoch": 0.4670126019273536, "percentage": 46.67, "elapsed_time": "0:41:59", "remaining_time": "0:47:59"}
{"current_steps": 320, "total_steps": 675, "loss": 1.417808437347412, "lr": 1.1388559665211241e-05, "epoch": 0.4744255003706449, "percentage": 47.41, "elapsed_time": "0:42:38", "remaining_time": "0:47:18"}
{"current_steps": 325, "total_steps": 675, "loss": 1.4581987380981445, "lr": 1.1150325864327003e-05, "epoch": 0.48183839881393625, "percentage": 48.15, "elapsed_time": "0:43:19", "remaining_time": "0:46:39"}
{"current_steps": 330, "total_steps": 675, "loss": 1.4179252624511718, "lr": 1.0911428497230834e-05, "epoch": 0.4892512972572276, "percentage": 48.89, "elapsed_time": "0:43:58", "remaining_time": "0:45:58"}
{"current_steps": 335, "total_steps": 675, "loss": 1.4111376762390138, "lr": 1.0672005372022014e-05, "epoch": 0.4966641957005189, "percentage": 49.63, "elapsed_time": "0:44:39", "remaining_time": "0:45:19"}
{"current_steps": 340, "total_steps": 675, "loss": 1.3600208282470703, "lr": 1.043219460008374e-05, "epoch": 0.5040770941438102, "percentage": 50.37, "elapsed_time": "0:45:20", "remaining_time": "0:44:40"}
{"current_steps": 345, "total_steps": 675, "loss": 1.3771323204040526, "lr": 1.019213451641347e-05, "epoch": 0.5114899925871016, "percentage": 51.11, "elapsed_time": "0:46:00", "remaining_time": "0:44:00"}
{"current_steps": 350, "total_steps": 675, "loss": 1.3850160598754884, "lr": 9.951963599824294e-06, "epoch": 0.5189028910303929, "percentage": 51.85, "elapsed_time": "0:46:40", "remaining_time": "0:43:20"}
{"current_steps": 355, "total_steps": 675, "loss": 1.4168268203735352, "lr": 9.711820393063337e-06, "epoch": 0.5263157894736842, "percentage": 52.59, "elapsed_time": "0:47:21", "remaining_time": "0:42:41"}
{"current_steps": 360, "total_steps": 675, "loss": 1.3890551567077636, "lr": 9.471843422893299e-06, "epoch": 0.5337286879169756, "percentage": 53.33, "elapsed_time": "0:48:01", "remaining_time": "0:42:01"}
{"current_steps": 365, "total_steps": 675, "loss": 1.4085680961608886, "lr": 9.232171120183193e-06, "epoch": 0.5411415863602669, "percentage": 54.07, "elapsed_time": "0:48:41", "remaining_time": "0:41:21"}
{"current_steps": 370, "total_steps": 675, "loss": 1.3939090728759767, "lr": 8.992941740054418e-06, "epoch": 0.5485544848035582, "percentage": 54.81, "elapsed_time": "0:49:20", "remaining_time": "0:40:40"}
{"current_steps": 375, "total_steps": 675, "loss": 1.4120423316955566, "lr": 8.754293282128187e-06, "epoch": 0.5559673832468495, "percentage": 55.56, "elapsed_time": "0:50:01", "remaining_time": "0:40:01"}
{"current_steps": 380, "total_steps": 675, "loss": 1.342286968231201, "lr": 8.516363410920376e-06, "epoch": 0.5633802816901409, "percentage": 56.3, "elapsed_time": "0:50:41", "remaining_time": "0:39:21"}
{"current_steps": 385, "total_steps": 675, "loss": 1.3686384201049804, "lr": 8.279289376429653e-06, "epoch": 0.5707931801334322, "percentage": 57.04, "elapsed_time": "0:51:22", "remaining_time": "0:38:41"}
{"current_steps": 390, "total_steps": 675, "loss": 1.4198535919189452, "lr": 8.043207934964722e-06, "epoch": 0.5782060785767235, "percentage": 57.78, "elapsed_time": "0:52:02", "remaining_time": "0:38:01"}
{"current_steps": 395, "total_steps": 675, "loss": 1.4073737144470215, "lr": 7.808255270256368e-06, "epoch": 0.5856189770200149, "percentage": 58.52, "elapsed_time": "0:52:41", "remaining_time": "0:37:20"}
{"current_steps": 400, "total_steps": 675, "loss": 1.3731096267700196, "lr": 7.574566914899779e-06, "epoch": 0.5930318754633062, "percentage": 59.26, "elapsed_time": "0:53:21", "remaining_time": "0:36:41"}
{"current_steps": 405, "total_steps": 675, "loss": 1.3820195198059082, "lr": 7.342277672172483e-06, "epoch": 0.6004447739065975, "percentage": 60.0, "elapsed_time": "0:54:01", "remaining_time": "0:36:01"}
{"current_steps": 410, "total_steps": 675, "loss": 1.3672608375549316, "lr": 7.111521538272997e-06, "epoch": 0.6078576723498889, "percentage": 60.74, "elapsed_time": "0:54:42", "remaining_time": "0:35:21"}
{"current_steps": 415, "total_steps": 675, "loss": 1.352785873413086, "lr": 6.882431625025016e-06, "epoch": 0.6152705707931801, "percentage": 61.48, "elapsed_time": "0:55:23", "remaining_time": "0:34:42"}
{"current_steps": 420, "total_steps": 675, "loss": 1.3680495262145995, "lr": 6.655140083091794e-06, "epoch": 0.6226834692364714, "percentage": 62.22, "elapsed_time": "0:56:02", "remaining_time": "0:34:01"}
{"current_steps": 425, "total_steps": 675, "loss": 1.4069809913635254, "lr": 6.42977802574494e-06, "epoch": 0.6300963676797627, "percentage": 62.96, "elapsed_time": "0:56:40", "remaining_time": "0:33:20"}
{"current_steps": 430, "total_steps": 675, "loss": 1.3522788047790528, "lr": 6.206475453231644e-06, "epoch": 0.6375092661230541, "percentage": 63.7, "elapsed_time": "0:57:21", "remaining_time": "0:32:40"}
{"current_steps": 435, "total_steps": 675, "loss": 1.3570075035095215, "lr": 5.98536117778394e-06, "epoch": 0.6449221645663454, "percentage": 64.44, "elapsed_time": "0:58:01", "remaining_time": "0:32:00"}
{"current_steps": 440, "total_steps": 675, "loss": 1.3594365119934082, "lr": 5.766562749313309e-06, "epoch": 0.6523350630096367, "percentage": 65.19, "elapsed_time": "0:58:41", "remaining_time": "0:31:20"}
{"current_steps": 445, "total_steps": 675, "loss": 1.4155049324035645, "lr": 5.550206381833409e-06, "epoch": 0.6597479614529281, "percentage": 65.93, "elapsed_time": "0:59:19", "remaining_time": "0:30:39"}
{"current_steps": 450, "total_steps": 675, "loss": 1.3647008895874024, "lr": 5.336416880653461e-06, "epoch": 0.6671608598962194, "percentage": 66.67, "elapsed_time": "0:59:59", "remaining_time": "0:29:59"}
{"current_steps": 455, "total_steps": 675, "loss": 1.3679018020629883, "lr": 5.125317570384211e-06, "epoch": 0.6745737583395107, "percentage": 67.41, "elapsed_time": "1:00:40", "remaining_time": "0:29:20"}
{"current_steps": 460, "total_steps": 675, "loss": 1.4130691528320312, "lr": 4.917030223798057e-06, "epoch": 0.6819866567828021, "percentage": 68.15, "elapsed_time": "1:01:19", "remaining_time": "0:28:39"}
{"current_steps": 465, "total_steps": 675, "loss": 1.3584203720092773, "lr": 4.7116749915843356e-06, "epoch": 0.6893995552260934, "percentage": 68.89, "elapsed_time": "1:01:59", "remaining_time": "0:27:59"}
{"current_steps": 470, "total_steps": 675, "loss": 1.348178005218506, "lr": 4.5093703330403385e-06, "epoch": 0.6968124536693847, "percentage": 69.63, "elapsed_time": "1:02:38", "remaining_time": "0:27:19"}
{"current_steps": 475, "total_steps": 675, "loss": 1.3591523170471191, "lr": 4.310232947737979e-06, "epoch": 0.704225352112676, "percentage": 70.37, "elapsed_time": "1:03:20", "remaining_time": "0:26:40"}
{"current_steps": 480, "total_steps": 675, "loss": 1.3993351936340332, "lr": 4.1143777082055715e-06, "epoch": 0.7116382505559674, "percentage": 71.11, "elapsed_time": "1:04:01", "remaining_time": "0:26:00"}
{"current_steps": 485, "total_steps": 675, "loss": 1.3535401344299316, "lr": 3.921917593663538e-06, "epoch": 0.7190511489992587, "percentage": 71.85, "elapsed_time": "1:04:42", "remaining_time": "0:25:20"}
{"current_steps": 490, "total_steps": 675, "loss": 1.392369556427002, "lr": 3.732963624852275e-06, "epoch": 0.72646404744255, "percentage": 72.59, "elapsed_time": "1:05:22", "remaining_time": "0:24:40"}
{"current_steps": 495, "total_steps": 675, "loss": 1.4015738487243652, "lr": 3.5476247999897707e-06, "epoch": 0.7338769458858414, "percentage": 73.33, "elapsed_time": "1:06:01", "remaining_time": "0:24:00"}
{"current_steps": 500, "total_steps": 675, "loss": 1.3607392311096191, "lr": 3.3660080318959043e-06, "epoch": 0.7412898443291327, "percentage": 74.07, "elapsed_time": "1:06:41", "remaining_time": "0:23:20"}
{"current_steps": 505, "total_steps": 675, "loss": 1.3873835563659669, "lr": 3.1882180863197308e-06, "epoch": 0.748702742772424, "percentage": 74.81, "elapsed_time": "1:07:21", "remaining_time": "0:22:40"}
{"current_steps": 510, "total_steps": 675, "loss": 1.3514761924743652, "lr": 3.0143575215052732e-06, "epoch": 0.7561156412157154, "percentage": 75.56, "elapsed_time": "1:08:02", "remaining_time": "0:22:00"}
{"current_steps": 515, "total_steps": 675, "loss": 1.3852715492248535, "lr": 2.8445266290307515e-06, "epoch": 0.7635285396590067, "percentage": 76.3, "elapsed_time": "1:08:42", "remaining_time": "0:21:20"}
{"current_steps": 520, "total_steps": 675, "loss": 1.3499971389770509, "lr": 2.678823375955314e-06, "epoch": 0.770941438102298, "percentage": 77.04, "elapsed_time": "1:09:23", "remaining_time": "0:20:40"}
{"current_steps": 525, "total_steps": 675, "loss": 1.3455326080322265, "lr": 2.5173433483066777e-06, "epoch": 0.7783543365455893, "percentage": 77.78, "elapsed_time": "1:10:03", "remaining_time": "0:20:00"}
{"current_steps": 530, "total_steps": 675, "loss": 1.3331590652465821, "lr": 2.3601796959422585e-06, "epoch": 0.7857672349888807, "percentage": 78.52, "elapsed_time": "1:10:44", "remaining_time": "0:19:21"}
{"current_steps": 535, "total_steps": 675, "loss": 1.3671817779541016, "lr": 2.2074230788156325e-06, "epoch": 0.793180133432172, "percentage": 79.26, "elapsed_time": "1:11:24", "remaining_time": "0:18:41"}
{"current_steps": 540, "total_steps": 675, "loss": 1.3592339515686036, "lr": 2.0591616146792705e-06, "epoch": 0.8005930318754633, "percentage": 80.0, "elapsed_time": "1:12:04", "remaining_time": "0:18:01"}
{"current_steps": 545, "total_steps": 675, "loss": 1.3896028518676757, "lr": 1.915480828253763e-06, "epoch": 0.8080059303187547, "percentage": 80.74, "elapsed_time": "1:12:44", "remaining_time": "0:17:21"}
{"current_steps": 550, "total_steps": 675, "loss": 1.3742392539978028, "lr": 1.7764636018928249e-06, "epoch": 0.815418828762046, "percentage": 81.48, "elapsed_time": "1:13:24", "remaining_time": "0:16:40"}
{"current_steps": 555, "total_steps": 675, "loss": 1.391466236114502, "lr": 1.6421901277725627e-06, "epoch": 0.8228317272053373, "percentage": 82.22, "elapsed_time": "1:14:04", "remaining_time": "0:16:00"}
{"current_steps": 560, "total_steps": 675, "loss": 1.3652151107788086, "lr": 1.5127378616325606e-06, "epoch": 0.8302446256486287, "percentage": 82.96, "elapsed_time": "1:14:45", "remaining_time": "0:15:21"}
{"current_steps": 565, "total_steps": 675, "loss": 1.3537821769714355, "lr": 1.3881814780954916e-06, "epoch": 0.83765752409192, "percentage": 83.7, "elapsed_time": "1:15:24", "remaining_time": "0:14:40"}
{"current_steps": 570, "total_steps": 675, "loss": 1.347853660583496, "lr": 1.2685928275910142e-06, "epoch": 0.8450704225352113, "percentage": 84.44, "elapsed_time": "1:16:06", "remaining_time": "0:14:01"}
{"current_steps": 575, "total_steps": 675, "loss": 1.3285847663879395, "lr": 1.1540408949087956e-06, "epoch": 0.8524833209785025, "percentage": 85.19, "elapsed_time": "1:16:46", "remaining_time": "0:13:21"}
{"current_steps": 580, "total_steps": 675, "loss": 1.4402085304260255, "lr": 1.0445917594046073e-06, "epoch": 0.859896219421794, "percentage": 85.93, "elapsed_time": "1:17:26", "remaining_time": "0:12:41"}
{"current_steps": 585, "total_steps": 675, "loss": 1.4003355979919434, "lr": 9.403085568823988e-07, "epoch": 0.8673091178650852, "percentage": 86.67, "elapsed_time": "1:18:07", "remaining_time": "0:12:01"}
{"current_steps": 590, "total_steps": 675, "loss": 1.3969636917114259, "lr": 8.412514431743657e-07, "epoch": 0.8747220163083765, "percentage": 87.41, "elapsed_time": "1:18:47", "remaining_time": "0:11:21"}
{"current_steps": 595, "total_steps": 675, "loss": 1.3984580993652345, "lr": 7.474775594400252e-07, "epoch": 0.8821349147516679, "percentage": 88.15, "elapsed_time": "1:19:28", "remaining_time": "0:10:41"}
{"current_steps": 600, "total_steps": 675, "loss": 1.3271192550659179, "lr": 6.590409992042957e-07, "epoch": 0.8895478131949592, "percentage": 88.89, "elapsed_time": "1:20:09", "remaining_time": "0:10:01"}
{"current_steps": 605, "total_steps": 675, "loss": 1.3371945381164552, "lr": 5.759927771536167e-07, "epoch": 0.8969607116382505, "percentage": 89.63, "elapsed_time": "1:20:49", "remaining_time": "0:09:21"}
{"current_steps": 610, "total_steps": 675, "loss": 1.3641469955444336, "lr": 4.983807997080925e-07, "epoch": 0.9043736100815419, "percentage": 90.37, "elapsed_time": "1:21:30", "remaining_time": "0:08:41"}
{"current_steps": 615, "total_steps": 675, "loss": 1.3556190490722657, "lr": 4.262498373866486e-07, "epoch": 0.9117865085248332, "percentage": 91.11, "elapsed_time": "1:22:09", "remaining_time": "0:08:00"}
{"current_steps": 620, "total_steps": 675, "loss": 1.3831118583679198, "lr": 3.5964149898111587e-07, "epoch": 0.9191994069681245, "percentage": 91.85, "elapsed_time": "1:22:48", "remaining_time": "0:07:20"}
{"current_steps": 625, "total_steps": 675, "loss": 1.3703168869018554, "lr": 2.985942075541848e-07, "epoch": 0.9266123054114158, "percentage": 92.59, "elapsed_time": "1:23:29", "remaining_time": "0:06:40"}
{"current_steps": 630, "total_steps": 675, "loss": 1.3450684547424316, "lr": 2.4314317827503375e-07, "epoch": 0.9340252038547072, "percentage": 93.33, "elapsed_time": "1:24:09", "remaining_time": "0:06:00"}
{"current_steps": 635, "total_steps": 675, "loss": 1.3397465705871583, "lr": 1.9332039810543324e-07, "epoch": 0.9414381022979985, "percentage": 94.07, "elapsed_time": "1:24:52", "remaining_time": "0:05:20"}
{"current_steps": 640, "total_steps": 675, "loss": 1.3559181213378906, "lr": 1.49154607348051e-07, "epoch": 0.9488510007412898, "percentage": 94.81, "elapsed_time": "1:25:33", "remaining_time": "0:04:40"}
{"current_steps": 645, "total_steps": 675, "loss": 1.4002655029296875, "lr": 1.1067128306758267e-07, "epoch": 0.9562638991845812, "percentage": 95.56, "elapsed_time": "1:26:13", "remaining_time": "0:04:00"}
{"current_steps": 650, "total_steps": 675, "loss": 1.3875154495239257, "lr": 7.789262439430012e-08, "epoch": 0.9636767976278725, "percentage": 96.3, "elapsed_time": "1:26:53", "remaining_time": "0:03:20"}
{"current_steps": 655, "total_steps": 675, "loss": 1.3760144233703613, "lr": 5.083753971846239e-08, "epoch": 0.9710896960711638, "percentage": 97.04, "elapsed_time": "1:27:33", "remaining_time": "0:02:40"}
{"current_steps": 660, "total_steps": 675, "loss": 1.3631325721740724, "lr": 2.9521635783001932e-08, "epoch": 0.9785025945144552, "percentage": 97.78, "elapsed_time": "1:28:13", "remaining_time": "0:02:00"}
{"current_steps": 665, "total_steps": 675, "loss": 1.3682563781738282, "lr": 1.3957208680761558e-08, "epoch": 0.9859154929577465, "percentage": 98.52, "elapsed_time": "1:28:54", "remaining_time": "0:01:20"}
{"current_steps": 670, "total_steps": 675, "loss": 1.3901843070983886, "lr": 4.153236761488266e-09, "epoch": 0.9933283914010378, "percentage": 99.26, "elapsed_time": "1:29:35", "remaining_time": "0:00:40"}
{"current_steps": 675, "total_steps": 675, "loss": 1.4335641860961914, "lr": 1.1537545266726568e-10, "epoch": 1.0, "percentage": 100.0, "elapsed_time": "1:30:09", "remaining_time": "0:00:00"}
{"current_steps": 675, "total_steps": 675, "epoch": 1.0, "percentage": 100.0, "elapsed_time": "1:30:09", "remaining_time": "0:00:00"}

988
trainer_state.json Normal file
View File

@@ -0,0 +1,988 @@
{
"best_global_step": null,
"best_metric": null,
"best_model_checkpoint": null,
"epoch": 1.0,
"eval_steps": 500,
"global_step": 675,
"is_hyper_param_search": false,
"is_local_process_zero": true,
"is_world_process_zero": true,
"log_history": [
{
"epoch": 0.007412898443291327,
"grad_norm": 49.5,
"learning_rate": 3.80952380952381e-06,
"loss": 2.981045722961426,
"step": 5
},
{
"epoch": 0.014825796886582653,
"grad_norm": 13.0,
"learning_rate": 8.571428571428571e-06,
"loss": 2.7763238906860352,
"step": 10
},
{
"epoch": 0.02223869532987398,
"grad_norm": 7.59375,
"learning_rate": 1.3333333333333333e-05,
"loss": 2.5071876525878904,
"step": 15
},
{
"epoch": 0.029651593773165306,
"grad_norm": 4.0625,
"learning_rate": 1.8095238095238097e-05,
"loss": 2.216544532775879,
"step": 20
},
{
"epoch": 0.037064492216456635,
"grad_norm": 3.453125,
"learning_rate": 1.9998961636899736e-05,
"loss": 2.02364444732666,
"step": 25
},
{
"epoch": 0.04447739065974796,
"grad_norm": 2.640625,
"learning_rate": 1.9992616865520515e-05,
"loss": 1.8605846405029296,
"step": 30
},
{
"epoch": 0.05189028910303929,
"grad_norm": 2.5,
"learning_rate": 1.998050784669941e-05,
"loss": 1.8314769744873047,
"step": 35
},
{
"epoch": 0.05930318754633061,
"grad_norm": 2.546875,
"learning_rate": 1.9962641565531694e-05,
"loss": 1.7219036102294922,
"step": 40
},
{
"epoch": 0.06671608598962195,
"grad_norm": 2.640625,
"learning_rate": 1.9939028328193097e-05,
"loss": 1.6877906799316407,
"step": 45
},
{
"epoch": 0.07412898443291327,
"grad_norm": 2.625,
"learning_rate": 1.990968175599471e-05,
"loss": 1.6538002014160156,
"step": 50
},
{
"epoch": 0.08154188287620459,
"grad_norm": 2.328125,
"learning_rate": 1.987461877752552e-05,
"loss": 1.6635187149047852,
"step": 55
},
{
"epoch": 0.08895478131949593,
"grad_norm": 2.40625,
"learning_rate": 1.983385961888711e-05,
"loss": 1.5999790191650392,
"step": 60
},
{
"epoch": 0.09636767976278725,
"grad_norm": 2.59375,
"learning_rate": 1.978742779202627e-05,
"loss": 1.572991943359375,
"step": 65
},
{
"epoch": 0.10378057820607858,
"grad_norm": 2.640625,
"learning_rate": 1.973535008117207e-05,
"loss": 1.6114467620849608,
"step": 70
},
{
"epoch": 0.1111934766493699,
"grad_norm": 2.21875,
"learning_rate": 1.9677656527385386e-05,
"loss": 1.5769593238830566,
"step": 75
},
{
"epoch": 0.11860637509266123,
"grad_norm": 2.109375,
"learning_rate": 1.9614380411229693e-05,
"loss": 1.5900885581970214,
"step": 80
},
{
"epoch": 0.12601927353595255,
"grad_norm": 2.125,
"learning_rate": 1.9545558233573136e-05,
"loss": 1.5888105392456056,
"step": 85
},
{
"epoch": 0.1334321719792439,
"grad_norm": 2.578125,
"learning_rate": 1.9471229694533003e-05,
"loss": 1.563032054901123,
"step": 90
},
{
"epoch": 0.14084507042253522,
"grad_norm": 2.296875,
"learning_rate": 1.9391437670574656e-05,
"loss": 1.5782450675964355,
"step": 95
},
{
"epoch": 0.14825796886582654,
"grad_norm": 2.28125,
"learning_rate": 1.9306228189778255e-05,
"loss": 1.5569299697875976,
"step": 100
},
{
"epoch": 0.15567086730911786,
"grad_norm": 2.015625,
"learning_rate": 1.9215650405287392e-05,
"loss": 1.5072972297668457,
"step": 105
},
{
"epoch": 0.16308376575240918,
"grad_norm": 2.375,
"learning_rate": 1.9119756566955092e-05,
"loss": 1.4942115783691405,
"step": 110
},
{
"epoch": 0.17049666419570053,
"grad_norm": 2.390625,
"learning_rate": 1.901860199120344e-05,
"loss": 1.5515302658081054,
"step": 115
},
{
"epoch": 0.17790956263899185,
"grad_norm": 2.890625,
"learning_rate": 1.891224502911428e-05,
"loss": 1.4991122245788575,
"step": 120
},
{
"epoch": 0.18532246108228317,
"grad_norm": 2.28125,
"learning_rate": 1.8800747032769343e-05,
"loss": 1.4753767013549806,
"step": 125
},
{
"epoch": 0.1927353595255745,
"grad_norm": 2.234375,
"learning_rate": 1.8684172319859258e-05,
"loss": 1.4607695579528808,
"step": 130
},
{
"epoch": 0.20014825796886582,
"grad_norm": 2.125,
"learning_rate": 1.856258813658188e-05,
"loss": 1.461857509613037,
"step": 135
},
{
"epoch": 0.20756115641215717,
"grad_norm": 2.265625,
"learning_rate": 1.8436064618851225e-05,
"loss": 1.530456256866455,
"step": 140
},
{
"epoch": 0.2149740548554485,
"grad_norm": 2.1875,
"learning_rate": 1.8304674751839583e-05,
"loss": 1.4979378700256347,
"step": 145
},
{
"epoch": 0.2223869532987398,
"grad_norm": 2.203125,
"learning_rate": 1.8168494327875918e-05,
"loss": 1.4886467933654786,
"step": 150
},
{
"epoch": 0.22979985174203113,
"grad_norm": 1.9453125,
"learning_rate": 1.8027601902725065e-05,
"loss": 1.4672227859497071,
"step": 155
},
{
"epoch": 0.23721275018532245,
"grad_norm": 2.0,
"learning_rate": 1.788207875027274e-05,
"loss": 1.4558910369873046,
"step": 160
},
{
"epoch": 0.2446256486286138,
"grad_norm": 2.1875,
"learning_rate": 1.77320088156427e-05,
"loss": 1.4654417037963867,
"step": 165
},
{
"epoch": 0.2520385470719051,
"grad_norm": 2.21875,
"learning_rate": 1.7577478666772886e-05,
"loss": 1.4390137672424317,
"step": 170
},
{
"epoch": 0.25945144551519644,
"grad_norm": 2.3125,
"learning_rate": 1.741857744447869e-05,
"loss": 1.4661806106567383,
"step": 175
},
{
"epoch": 0.2668643439584878,
"grad_norm": 2.28125,
"learning_rate": 1.7255396811032014e-05,
"loss": 1.472686767578125,
"step": 180
},
{
"epoch": 0.2742772424017791,
"grad_norm": 3.328125,
"learning_rate": 1.708803089728583e-05,
"loss": 1.4672610282897949,
"step": 185
},
{
"epoch": 0.28169014084507044,
"grad_norm": 2.4375,
"learning_rate": 1.691657624837472e-05,
"loss": 1.4069836616516114,
"step": 190
},
{
"epoch": 0.28910303928836173,
"grad_norm": 2.453125,
"learning_rate": 1.6741131768022777e-05,
"loss": 1.471778392791748,
"step": 195
},
{
"epoch": 0.2965159377316531,
"grad_norm": 2.703125,
"learning_rate": 1.6561798661490904e-05,
"loss": 1.4518500328063966,
"step": 200
},
{
"epoch": 0.30392883617494443,
"grad_norm": 2.015625,
"learning_rate": 1.6378680377196526e-05,
"loss": 1.4200240135192872,
"step": 205
},
{
"epoch": 0.3113417346182357,
"grad_norm": 2.484375,
"learning_rate": 1.619188254703927e-05,
"loss": 1.4366942405700684,
"step": 210
},
{
"epoch": 0.31875463306152707,
"grad_norm": 2.140625,
"learning_rate": 1.6001512925467153e-05,
"loss": 1.4304126739501952,
"step": 215
},
{
"epoch": 0.32616753150481836,
"grad_norm": 2.234375,
"learning_rate": 1.5807681327318372e-05,
"loss": 1.4636366844177247,
"step": 220
},
{
"epoch": 0.3335804299481097,
"grad_norm": 2.046875,
"learning_rate": 1.5610499564474512e-05,
"loss": 1.440121841430664,
"step": 225
},
{
"epoch": 0.34099332839140106,
"grad_norm": 2.203125,
"learning_rate": 1.541008138136183e-05,
"loss": 1.4860041618347168,
"step": 230
},
{
"epoch": 0.34840622683469236,
"grad_norm": 1.984375,
"learning_rate": 1.520654238933767e-05,
"loss": 1.4189085006713866,
"step": 235
},
{
"epoch": 0.3558191252779837,
"grad_norm": 2.328125,
"learning_rate": 1.5000000000000002e-05,
"loss": 1.407963466644287,
"step": 240
},
{
"epoch": 0.363232023721275,
"grad_norm": 2.1875,
"learning_rate": 1.4790573357458464e-05,
"loss": 1.4134508132934571,
"step": 245
},
{
"epoch": 0.37064492216456635,
"grad_norm": 2.875,
"learning_rate": 1.4578383269606004e-05,
"loss": 1.425547218322754,
"step": 250
},
{
"epoch": 0.3780578206078577,
"grad_norm": 2.5625,
"learning_rate": 1.4363552138430764e-05,
"loss": 1.402323055267334,
"step": 255
},
{
"epoch": 0.385470719051149,
"grad_norm": 2.703125,
"learning_rate": 1.4146203889408418e-05,
"loss": 1.4572528839111327,
"step": 260
},
{
"epoch": 0.39288361749444034,
"grad_norm": 1.9921875,
"learning_rate": 1.392646390001569e-05,
"loss": 1.3995424270629884,
"step": 265
},
{
"epoch": 0.40029651593773163,
"grad_norm": 2.3125,
"learning_rate": 1.3704458927406261e-05,
"loss": 1.4161681175231933,
"step": 270
},
{
"epoch": 0.407709414381023,
"grad_norm": 2.6875,
"learning_rate": 1.3480317035290837e-05,
"loss": 1.4083374977111816,
"step": 275
},
{
"epoch": 0.41512231282431433,
"grad_norm": 2.390625,
"learning_rate": 1.325416752006351e-05,
"loss": 1.3747909545898438,
"step": 280
},
{
"epoch": 0.4225352112676056,
"grad_norm": 2.6875,
"learning_rate": 1.302614083621705e-05,
"loss": 1.4275038719177247,
"step": 285
},
{
"epoch": 0.429948109710897,
"grad_norm": 2.171875,
"learning_rate": 1.2796368521090143e-05,
"loss": 1.3964252471923828,
"step": 290
},
{
"epoch": 0.43736100815418827,
"grad_norm": 2.328125,
"learning_rate": 1.256498311899001e-05,
"loss": 1.401359462738037,
"step": 295
},
{
"epoch": 0.4447739065974796,
"grad_norm": 2.046875,
"learning_rate": 1.233211810473411e-05,
"loss": 1.4066211700439453,
"step": 300
},
{
"epoch": 0.45218680504077097,
"grad_norm": 2.265625,
"learning_rate": 1.2097907806655136e-05,
"loss": 1.3704119682312013,
"step": 305
},
{
"epoch": 0.45959970348406226,
"grad_norm": 2.125,
"learning_rate": 1.1862487329113606e-05,
"loss": 1.3963244438171387,
"step": 310
},
{
"epoch": 0.4670126019273536,
"grad_norm": 2.40625,
"learning_rate": 1.1625992474562846e-05,
"loss": 1.402764892578125,
"step": 315
},
{
"epoch": 0.4744255003706449,
"grad_norm": 2.34375,
"learning_rate": 1.1388559665211241e-05,
"loss": 1.417808437347412,
"step": 320
},
{
"epoch": 0.48183839881393625,
"grad_norm": 2.625,
"learning_rate": 1.1150325864327003e-05,
"loss": 1.4581987380981445,
"step": 325
},
{
"epoch": 0.4892512972572276,
"grad_norm": 2.546875,
"learning_rate": 1.0911428497230834e-05,
"loss": 1.4179252624511718,
"step": 330
},
{
"epoch": 0.4966641957005189,
"grad_norm": 2.65625,
"learning_rate": 1.0672005372022014e-05,
"loss": 1.4111376762390138,
"step": 335
},
{
"epoch": 0.5040770941438102,
"grad_norm": 2.234375,
"learning_rate": 1.043219460008374e-05,
"loss": 1.3600208282470703,
"step": 340
},
{
"epoch": 0.5114899925871016,
"grad_norm": 2.390625,
"learning_rate": 1.019213451641347e-05,
"loss": 1.3771323204040526,
"step": 345
},
{
"epoch": 0.5189028910303929,
"grad_norm": 2.28125,
"learning_rate": 9.951963599824294e-06,
"loss": 1.3850160598754884,
"step": 350
},
{
"epoch": 0.5263157894736842,
"grad_norm": 2.03125,
"learning_rate": 9.711820393063337e-06,
"loss": 1.4168268203735352,
"step": 355
},
{
"epoch": 0.5337286879169756,
"grad_norm": 2.125,
"learning_rate": 9.471843422893299e-06,
"loss": 1.3890551567077636,
"step": 360
},
{
"epoch": 0.5411415863602669,
"grad_norm": 1.9375,
"learning_rate": 9.232171120183193e-06,
"loss": 1.4085680961608886,
"step": 365
},
{
"epoch": 0.5485544848035582,
"grad_norm": 2.0625,
"learning_rate": 8.992941740054418e-06,
"loss": 1.3939090728759767,
"step": 370
},
{
"epoch": 0.5559673832468495,
"grad_norm": 2.109375,
"learning_rate": 8.754293282128187e-06,
"loss": 1.4120423316955566,
"step": 375
},
{
"epoch": 0.5633802816901409,
"grad_norm": 2.40625,
"learning_rate": 8.516363410920376e-06,
"loss": 1.342286968231201,
"step": 380
},
{
"epoch": 0.5707931801334322,
"grad_norm": 2.640625,
"learning_rate": 8.279289376429653e-06,
"loss": 1.3686384201049804,
"step": 385
},
{
"epoch": 0.5782060785767235,
"grad_norm": 2.25,
"learning_rate": 8.043207934964722e-06,
"loss": 1.4198535919189452,
"step": 390
},
{
"epoch": 0.5856189770200149,
"grad_norm": 1.8515625,
"learning_rate": 7.808255270256368e-06,
"loss": 1.4073737144470215,
"step": 395
},
{
"epoch": 0.5930318754633062,
"grad_norm": 3.125,
"learning_rate": 7.574566914899779e-06,
"loss": 1.3731096267700196,
"step": 400
},
{
"epoch": 0.6004447739065975,
"grad_norm": 2.109375,
"learning_rate": 7.342277672172483e-06,
"loss": 1.3820195198059082,
"step": 405
},
{
"epoch": 0.6078576723498889,
"grad_norm": 2.421875,
"learning_rate": 7.111521538272997e-06,
"loss": 1.3672608375549316,
"step": 410
},
{
"epoch": 0.6152705707931801,
"grad_norm": 2.859375,
"learning_rate": 6.882431625025016e-06,
"loss": 1.352785873413086,
"step": 415
},
{
"epoch": 0.6226834692364714,
"grad_norm": 2.265625,
"learning_rate": 6.655140083091794e-06,
"loss": 1.3680495262145995,
"step": 420
},
{
"epoch": 0.6300963676797627,
"grad_norm": 2.375,
"learning_rate": 6.42977802574494e-06,
"loss": 1.4069809913635254,
"step": 425
},
{
"epoch": 0.6375092661230541,
"grad_norm": 1.953125,
"learning_rate": 6.206475453231644e-06,
"loss": 1.3522788047790528,
"step": 430
},
{
"epoch": 0.6449221645663454,
"grad_norm": 2.859375,
"learning_rate": 5.98536117778394e-06,
"loss": 1.3570075035095215,
"step": 435
},
{
"epoch": 0.6523350630096367,
"grad_norm": 2.453125,
"learning_rate": 5.766562749313309e-06,
"loss": 1.3594365119934082,
"step": 440
},
{
"epoch": 0.6597479614529281,
"grad_norm": 2.015625,
"learning_rate": 5.550206381833409e-06,
"loss": 1.4155049324035645,
"step": 445
},
{
"epoch": 0.6671608598962194,
"grad_norm": 2.46875,
"learning_rate": 5.336416880653461e-06,
"loss": 1.3647008895874024,
"step": 450
},
{
"epoch": 0.6745737583395107,
"grad_norm": 2.4375,
"learning_rate": 5.125317570384211e-06,
"loss": 1.3679018020629883,
"step": 455
},
{
"epoch": 0.6819866567828021,
"grad_norm": 2.1875,
"learning_rate": 4.917030223798057e-06,
"loss": 1.4130691528320312,
"step": 460
},
{
"epoch": 0.6893995552260934,
"grad_norm": 2.25,
"learning_rate": 4.7116749915843356e-06,
"loss": 1.3584203720092773,
"step": 465
},
{
"epoch": 0.6968124536693847,
"grad_norm": 2.90625,
"learning_rate": 4.5093703330403385e-06,
"loss": 1.348178005218506,
"step": 470
},
{
"epoch": 0.704225352112676,
"grad_norm": 3.09375,
"learning_rate": 4.310232947737979e-06,
"loss": 1.3591523170471191,
"step": 475
},
{
"epoch": 0.7116382505559674,
"grad_norm": 2.703125,
"learning_rate": 4.1143777082055715e-06,
"loss": 1.3993351936340332,
"step": 480
},
{
"epoch": 0.7190511489992587,
"grad_norm": 2.515625,
"learning_rate": 3.921917593663538e-06,
"loss": 1.3535401344299316,
"step": 485
},
{
"epoch": 0.72646404744255,
"grad_norm": 2.3125,
"learning_rate": 3.732963624852275e-06,
"loss": 1.392369556427002,
"step": 490
},
{
"epoch": 0.7338769458858414,
"grad_norm": 2.171875,
"learning_rate": 3.5476247999897707e-06,
"loss": 1.4015738487243652,
"step": 495
},
{
"epoch": 0.7412898443291327,
"grad_norm": 2.609375,
"learning_rate": 3.3660080318959043e-06,
"loss": 1.3607392311096191,
"step": 500
},
{
"epoch": 0.748702742772424,
"grad_norm": 1.9296875,
"learning_rate": 3.1882180863197308e-06,
"loss": 1.3873835563659669,
"step": 505
},
{
"epoch": 0.7561156412157154,
"grad_norm": 2.046875,
"learning_rate": 3.0143575215052732e-06,
"loss": 1.3514761924743652,
"step": 510
},
{
"epoch": 0.7635285396590067,
"grad_norm": 2.03125,
"learning_rate": 2.8445266290307515e-06,
"loss": 1.3852715492248535,
"step": 515
},
{
"epoch": 0.770941438102298,
"grad_norm": 2.390625,
"learning_rate": 2.678823375955314e-06,
"loss": 1.3499971389770509,
"step": 520
},
{
"epoch": 0.7783543365455893,
"grad_norm": 2.234375,
"learning_rate": 2.5173433483066777e-06,
"loss": 1.3455326080322265,
"step": 525
},
{
"epoch": 0.7857672349888807,
"grad_norm": 1.984375,
"learning_rate": 2.3601796959422585e-06,
"loss": 1.3331590652465821,
"step": 530
},
{
"epoch": 0.793180133432172,
"grad_norm": 1.9375,
"learning_rate": 2.2074230788156325e-06,
"loss": 1.3671817779541016,
"step": 535
},
{
"epoch": 0.8005930318754633,
"grad_norm": 2.390625,
"learning_rate": 2.0591616146792705e-06,
"loss": 1.3592339515686036,
"step": 540
},
{
"epoch": 0.8080059303187547,
"grad_norm": 2.296875,
"learning_rate": 1.915480828253763e-06,
"loss": 1.3896028518676757,
"step": 545
},
{
"epoch": 0.815418828762046,
"grad_norm": 2.09375,
"learning_rate": 1.7764636018928249e-06,
"loss": 1.3742392539978028,
"step": 550
},
{
"epoch": 0.8228317272053373,
"grad_norm": 2.09375,
"learning_rate": 1.6421901277725627e-06,
"loss": 1.391466236114502,
"step": 555
},
{
"epoch": 0.8302446256486287,
"grad_norm": 2.1875,
"learning_rate": 1.5127378616325606e-06,
"loss": 1.3652151107788086,
"step": 560
},
{
"epoch": 0.83765752409192,
"grad_norm": 2.421875,
"learning_rate": 1.3881814780954916e-06,
"loss": 1.3537821769714355,
"step": 565
},
{
"epoch": 0.8450704225352113,
"grad_norm": 2.171875,
"learning_rate": 1.2685928275910142e-06,
"loss": 1.347853660583496,
"step": 570
},
{
"epoch": 0.8524833209785025,
"grad_norm": 2.03125,
"learning_rate": 1.1540408949087956e-06,
"loss": 1.3285847663879395,
"step": 575
},
{
"epoch": 0.859896219421794,
"grad_norm": 2.109375,
"learning_rate": 1.0445917594046073e-06,
"loss": 1.4402085304260255,
"step": 580
},
{
"epoch": 0.8673091178650852,
"grad_norm": 2.671875,
"learning_rate": 9.403085568823988e-07,
"loss": 1.4003355979919434,
"step": 585
},
{
"epoch": 0.8747220163083765,
"grad_norm": 2.25,
"learning_rate": 8.412514431743657e-07,
"loss": 1.3969636917114259,
"step": 590
},
{
"epoch": 0.8821349147516679,
"grad_norm": 2.234375,
"learning_rate": 7.474775594400252e-07,
"loss": 1.3984580993652345,
"step": 595
},
{
"epoch": 0.8895478131949592,
"grad_norm": 3.46875,
"learning_rate": 6.590409992042957e-07,
"loss": 1.3271192550659179,
"step": 600
},
{
"epoch": 0.8969607116382505,
"grad_norm": 2.34375,
"learning_rate": 5.759927771536167e-07,
"loss": 1.3371945381164552,
"step": 605
},
{
"epoch": 0.9043736100815419,
"grad_norm": 1.8515625,
"learning_rate": 4.983807997080925e-07,
"loss": 1.3641469955444336,
"step": 610
},
{
"epoch": 0.9117865085248332,
"grad_norm": 1.875,
"learning_rate": 4.262498373866486e-07,
"loss": 1.3556190490722657,
"step": 615
},
{
"epoch": 0.9191994069681245,
"grad_norm": 2.3125,
"learning_rate": 3.5964149898111587e-07,
"loss": 1.3831118583679198,
"step": 620
},
{
"epoch": 0.9266123054114158,
"grad_norm": 2.203125,
"learning_rate": 2.985942075541848e-07,
"loss": 1.3703168869018554,
"step": 625
},
{
"epoch": 0.9340252038547072,
"grad_norm": 2.15625,
"learning_rate": 2.4314317827503375e-07,
"loss": 1.3450684547424316,
"step": 630
},
{
"epoch": 0.9414381022979985,
"grad_norm": 2.328125,
"learning_rate": 1.9332039810543324e-07,
"loss": 1.3397465705871583,
"step": 635
},
{
"epoch": 0.9488510007412898,
"grad_norm": 1.8125,
"learning_rate": 1.49154607348051e-07,
"loss": 1.3559181213378906,
"step": 640
},
{
"epoch": 0.9562638991845812,
"grad_norm": 2.171875,
"learning_rate": 1.1067128306758267e-07,
"loss": 1.4002655029296875,
"step": 645
},
{
"epoch": 0.9636767976278725,
"grad_norm": 2.125,
"learning_rate": 7.789262439430012e-08,
"loss": 1.3875154495239257,
"step": 650
},
{
"epoch": 0.9710896960711638,
"grad_norm": 1.96875,
"learning_rate": 5.083753971846239e-08,
"loss": 1.3760144233703613,
"step": 655
},
{
"epoch": 0.9785025945144552,
"grad_norm": 1.828125,
"learning_rate": 2.9521635783001932e-08,
"loss": 1.3631325721740724,
"step": 660
},
{
"epoch": 0.9859154929577465,
"grad_norm": 2.21875,
"learning_rate": 1.3957208680761558e-08,
"loss": 1.3682563781738282,
"step": 665
},
{
"epoch": 0.9933283914010378,
"grad_norm": 1.953125,
"learning_rate": 4.153236761488266e-09,
"loss": 1.3901843070983886,
"step": 670
},
{
"epoch": 1.0,
"grad_norm": 3.625,
"learning_rate": 1.1537545266726568e-10,
"loss": 1.4335641860961914,
"step": 675
},
{
"epoch": 1.0,
"step": 675,
"total_flos": 3.2515782676161946e+17,
"train_loss": 1.470360949480975,
"train_runtime": 5409.9531,
"train_samples_per_second": 0.499,
"train_steps_per_second": 0.125
}
],
"logging_steps": 5,
"max_steps": 675,
"num_input_tokens_seen": 0,
"num_train_epochs": 1,
"save_steps": 500,
"stateful_callbacks": {
"TrainerControl": {
"args": {
"should_epoch_stop": false,
"should_evaluate": false,
"should_log": false,
"should_save": false,
"should_training_stop": false
},
"attributes": {}
}
},
"total_flos": 3.2515782676161946e+17,
"train_batch_size": 1,
"trial_name": null,
"trial_params": null
}

3
training_args.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1a63ade1d570f2861164953d0bb398492bcb2c134cb47c446ca7ae88a600535b
size 6033

BIN
training_loss.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 41 KiB