shuoxing/llama3-8b-full-sft-c4-1m-en
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: shuoxing/llama3-8b-full-sft-c4-1m-en
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-17 06:25:09 +08:00
commit 70a995b910
10 changed files with 294 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

58
README.md Normal file
View File

@@ -0,0 +1,58 @@
---
library_name: transformers
tags:
- llama-factory
- generated_from_trainer
model-index:
- name: llama3-8b-full-sft-c4-1m-en
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# llama3-8b-full-sft-c4-1m-en
This model was trained from scratch on an unknown dataset.
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 8
- gradient_accumulation_steps: 2
- total_train_batch_size: 128
- total_eval_batch_size: 64
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 0.1
- num_epochs: 3.0
### Training results
### Framework versions
- Transformers 5.6.0
- Pytorch 2.12.0+cu130
- Datasets 4.0.0
- Tokenizers 0.22.2

5
chat_template.jinja Normal file
View File

@@ -0,0 +1,5 @@
{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>
'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>
' }}{% endif %}

32
config.json Normal file
View File

@@ -0,0 +1,32 @@
{
"architectures": [
"LlamaForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 128000,
"dtype": "bfloat16",
"eos_token_id": 128009,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 14336,
"max_position_embeddings": 8192,
"mlp_bias": false,
"model_type": "llama",
"num_attention_heads": 32,
"num_hidden_layers": 32,
"num_key_value_heads": 8,
"pad_token_id": 128009,
"pretraining_tp": 1,
"rms_norm_eps": 1e-05,
"rope_parameters": {
"rope_theta": 500000.0,
"rope_type": "default"
},
"tie_word_embeddings": false,
"transformers_version": "5.6.0",
"use_cache": false,
"vocab_size": 128256
}

13
generation_config.json Normal file
View File

@@ -0,0 +1,13 @@
{
"bos_token_id": 128000,
"do_sample": true,
"eos_token_id": [
128001,
128009
],
"max_length": 4096,
"pad_token_id": 128009,
"temperature": 0.6,
"top_p": 0.9,
"transformers_version": "5.6.0"
}

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ce59746a8261007cd8d4cb9a510d577d998a5f25d27bca13b87fc5edc18cd51b
size 16060556616

3
tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8c1dcab308e7cf5970ea38815e0a62887d705c5b436f869ca27a5dcdd40c36a6
size 17210148

20
tokenizer_config.json Normal file
View File

@@ -0,0 +1,20 @@
{
"backend": "tokenizers",
"bos_token": "<|begin_of_text|>",
"clean_up_tokenization_spaces": true,
"eos_token": "<|eot_id|>",
"extra_special_tokens": [
"<|eom_id|>"
],
"is_local": true,
"local_files_only": false,
"model_input_names": [
"input_ids",
"attention_mask"
],
"model_max_length": 1000000000000000019884624838656,
"pad_token": "<|eot_id|>",
"padding_side": "right",
"split_special_tokens": false,
"tokenizer_class": "TokenizersBackend"
}

121
trainer_log.jsonl Normal file
View File

@@ -0,0 +1,121 @@
{"current_steps": 1, "total_steps": 120, "loss": 1.818462610244751, "lr": 0.0, "epoch": 0.02531645569620253, "percentage": 0.83, "elapsed_time": "0:00:11", "remaining_time": "0:22:43"}
{"current_steps": 2, "total_steps": 120, "loss": 1.7896065711975098, "lr": 8.333333333333333e-07, "epoch": 0.05063291139240506, "percentage": 1.67, "elapsed_time": "0:00:15", "remaining_time": "0:15:36"}
{"current_steps": 3, "total_steps": 120, "loss": 1.8898143768310547, "lr": 1.6666666666666667e-06, "epoch": 0.0759493670886076, "percentage": 2.5, "elapsed_time": "0:00:20", "remaining_time": "0:13:06"}
{"current_steps": 4, "total_steps": 120, "loss": 1.5612382888793945, "lr": 2.5e-06, "epoch": 0.10126582278481013, "percentage": 3.33, "elapsed_time": "0:00:24", "remaining_time": "0:12:02"}
{"current_steps": 5, "total_steps": 120, "loss": 1.6267149448394775, "lr": 3.3333333333333333e-06, "epoch": 0.12658227848101267, "percentage": 4.17, "elapsed_time": "0:00:29", "remaining_time": "0:11:10"}
{"current_steps": 6, "total_steps": 120, "loss": 1.5447708368301392, "lr": 4.166666666666667e-06, "epoch": 0.1518987341772152, "percentage": 5.0, "elapsed_time": "0:00:33", "remaining_time": "0:10:34"}
{"current_steps": 7, "total_steps": 120, "loss": 1.5526912212371826, "lr": 5e-06, "epoch": 0.17721518987341772, "percentage": 5.83, "elapsed_time": "0:00:37", "remaining_time": "0:10:07"}
{"current_steps": 8, "total_steps": 120, "loss": 1.3726903200149536, "lr": 5.833333333333334e-06, "epoch": 0.20253164556962025, "percentage": 6.67, "elapsed_time": "0:00:41", "remaining_time": "0:09:46"}
{"current_steps": 9, "total_steps": 120, "loss": 1.3341686725616455, "lr": 6.666666666666667e-06, "epoch": 0.22784810126582278, "percentage": 7.5, "elapsed_time": "0:00:46", "remaining_time": "0:09:28"}
{"current_steps": 10, "total_steps": 120, "loss": 1.4077244997024536, "lr": 7.500000000000001e-06, "epoch": 0.25316455696202533, "percentage": 8.33, "elapsed_time": "0:00:50", "remaining_time": "0:09:13"}
{"current_steps": 11, "total_steps": 120, "loss": 1.3136383295059204, "lr": 8.333333333333334e-06, "epoch": 0.27848101265822783, "percentage": 9.17, "elapsed_time": "0:00:54", "remaining_time": "0:09:01"}
{"current_steps": 12, "total_steps": 120, "loss": 1.3540279865264893, "lr": 9.166666666666666e-06, "epoch": 0.3037974683544304, "percentage": 10.0, "elapsed_time": "0:00:58", "remaining_time": "0:08:48"}
{"current_steps": 13, "total_steps": 120, "loss": 1.2260899543762207, "lr": 1e-05, "epoch": 0.3291139240506329, "percentage": 10.83, "elapsed_time": "0:01:03", "remaining_time": "0:08:39"}
{"current_steps": 14, "total_steps": 120, "loss": 1.3063011169433594, "lr": 9.997884750411004e-06, "epoch": 0.35443037974683544, "percentage": 11.67, "elapsed_time": "0:01:07", "remaining_time": "0:08:30"}
{"current_steps": 15, "total_steps": 120, "loss": 1.2736425399780273, "lr": 9.991540791356342e-06, "epoch": 0.379746835443038, "percentage": 12.5, "elapsed_time": "0:01:11", "remaining_time": "0:08:22"}
{"current_steps": 16, "total_steps": 120, "loss": 1.3190453052520752, "lr": 9.980973490458728e-06, "epoch": 0.4050632911392405, "percentage": 13.33, "elapsed_time": "0:01:16", "remaining_time": "0:08:14"}
{"current_steps": 17, "total_steps": 120, "loss": 1.310528039932251, "lr": 9.966191788709716e-06, "epoch": 0.43037974683544306, "percentage": 14.17, "elapsed_time": "0:01:20", "remaining_time": "0:08:05"}
{"current_steps": 18, "total_steps": 120, "loss": 1.3213602304458618, "lr": 9.947208192904722e-06, "epoch": 0.45569620253164556, "percentage": 15.0, "elapsed_time": "0:01:24", "remaining_time": "0:07:58"}
{"current_steps": 19, "total_steps": 120, "loss": 1.2818647623062134, "lr": 9.924038765061042e-06, "epoch": 0.4810126582278481, "percentage": 15.83, "elapsed_time": "0:01:28", "remaining_time": "0:07:51"}
{"current_steps": 20, "total_steps": 120, "loss": 1.2110847234725952, "lr": 9.896703108827758e-06, "epoch": 0.5063291139240507, "percentage": 16.67, "elapsed_time": "0:01:32", "remaining_time": "0:07:44"}
{"current_steps": 21, "total_steps": 120, "loss": 1.2585198879241943, "lr": 9.86522435289912e-06, "epoch": 0.5316455696202531, "percentage": 17.5, "elapsed_time": "0:01:37", "remaining_time": "0:07:37"}
{"current_steps": 22, "total_steps": 120, "loss": 1.362351655960083, "lr": 9.829629131445342e-06, "epoch": 0.5569620253164557, "percentage": 18.33, "elapsed_time": "0:01:41", "remaining_time": "0:07:31"}
{"current_steps": 23, "total_steps": 120, "loss": 1.3219285011291504, "lr": 9.789947561577445e-06, "epoch": 0.5822784810126582, "percentage": 19.17, "elapsed_time": "0:01:45", "remaining_time": "0:07:24"}
{"current_steps": 24, "total_steps": 120, "loss": 1.3466389179229736, "lr": 9.74621321786517e-06, "epoch": 0.6075949367088608, "percentage": 20.0, "elapsed_time": "0:01:49", "remaining_time": "0:07:18"}
{"current_steps": 25, "total_steps": 120, "loss": 1.2454345226287842, "lr": 9.698463103929542e-06, "epoch": 0.6329113924050633, "percentage": 20.83, "elapsed_time": "0:01:53", "remaining_time": "0:07:12"}
{"current_steps": 26, "total_steps": 120, "loss": 1.3177976608276367, "lr": 9.646737621134112e-06, "epoch": 0.6582278481012658, "percentage": 21.67, "elapsed_time": "0:01:57", "remaining_time": "0:07:06"}
{"current_steps": 27, "total_steps": 120, "loss": 1.1473119258880615, "lr": 9.591080534401371e-06, "epoch": 0.6835443037974683, "percentage": 22.5, "elapsed_time": "0:02:02", "remaining_time": "0:07:00"}
{"current_steps": 28, "total_steps": 120, "loss": 1.1627123355865479, "lr": 9.531538935183252e-06, "epoch": 0.7088607594936709, "percentage": 23.33, "elapsed_time": "0:02:06", "remaining_time": "0:06:55"}
{"current_steps": 29, "total_steps": 120, "loss": 1.2631337642669678, "lr": 9.468163201617063e-06, "epoch": 0.7341772151898734, "percentage": 24.17, "elapsed_time": "0:02:10", "remaining_time": "0:06:49"}
{"current_steps": 30, "total_steps": 120, "loss": 1.23032546043396, "lr": 9.401006955900555e-06, "epoch": 0.759493670886076, "percentage": 25.0, "elapsed_time": "0:02:14", "remaining_time": "0:06:44"}
{"current_steps": 31, "total_steps": 120, "loss": 1.2267570495605469, "lr": 9.330127018922195e-06, "epoch": 0.7848101265822784, "percentage": 25.83, "elapsed_time": "0:02:18", "remaining_time": "0:06:38"}
{"current_steps": 32, "total_steps": 120, "loss": 1.180215835571289, "lr": 9.255583362184998e-06, "epoch": 0.810126582278481, "percentage": 26.67, "elapsed_time": "0:02:23", "remaining_time": "0:06:33"}
{"current_steps": 33, "total_steps": 120, "loss": 1.2534652948379517, "lr": 9.177439057064684e-06, "epoch": 0.8354430379746836, "percentage": 27.5, "elapsed_time": "0:02:27", "remaining_time": "0:06:27"}
{"current_steps": 34, "total_steps": 120, "loss": 1.260343074798584, "lr": 9.09576022144496e-06, "epoch": 0.8607594936708861, "percentage": 28.33, "elapsed_time": "0:02:31", "remaining_time": "0:06:22"}
{"current_steps": 35, "total_steps": 120, "loss": 1.2050299644470215, "lr": 9.01061596377522e-06, "epoch": 0.8860759493670886, "percentage": 29.17, "elapsed_time": "0:02:35", "remaining_time": "0:06:17"}
{"current_steps": 36, "total_steps": 120, "loss": 1.2866182327270508, "lr": 8.92207832459788e-06, "epoch": 0.9113924050632911, "percentage": 30.0, "elapsed_time": "0:02:39", "remaining_time": "0:06:12"}
{"current_steps": 37, "total_steps": 120, "loss": 1.22445809841156, "lr": 8.83022221559489e-06, "epoch": 0.9367088607594937, "percentage": 30.83, "elapsed_time": "0:02:43", "remaining_time": "0:06:06"}
{"current_steps": 38, "total_steps": 120, "loss": 1.259000539779663, "lr": 8.735125356204982e-06, "epoch": 0.9620253164556962, "percentage": 31.67, "elapsed_time": "0:02:47", "remaining_time": "0:06:02"}
{"current_steps": 39, "total_steps": 120, "loss": 1.2420016527175903, "lr": 8.636868207865244e-06, "epoch": 0.9873417721518988, "percentage": 32.5, "elapsed_time": "0:02:51", "remaining_time": "0:05:57"}
{"current_steps": 40, "total_steps": 120, "loss": 1.1530793905258179, "lr": 8.535533905932739e-06, "epoch": 1.0, "percentage": 33.33, "elapsed_time": "0:02:53", "remaining_time": "0:05:47"}
{"current_steps": 41, "total_steps": 120, "loss": 0.7639079093933105, "lr": 8.43120818934367e-06, "epoch": 1.0253164556962024, "percentage": 34.17, "elapsed_time": "0:02:58", "remaining_time": "0:05:43"}
{"current_steps": 42, "total_steps": 120, "loss": 0.6992146968841553, "lr": 8.323979328069689e-06, "epoch": 1.0506329113924051, "percentage": 35.0, "elapsed_time": "0:03:02", "remaining_time": "0:05:39"}
{"current_steps": 43, "total_steps": 120, "loss": 0.7556097507476807, "lr": 8.213938048432697e-06, "epoch": 1.0759493670886076, "percentage": 35.83, "elapsed_time": "0:03:06", "remaining_time": "0:05:34"}
{"current_steps": 44, "total_steps": 120, "loss": 0.6626423597335815, "lr": 8.101177456341301e-06, "epoch": 1.1012658227848102, "percentage": 36.67, "elapsed_time": "0:03:10", "remaining_time": "0:05:29"}
{"current_steps": 45, "total_steps": 120, "loss": 0.6483436226844788, "lr": 7.985792958513932e-06, "epoch": 1.1265822784810127, "percentage": 37.5, "elapsed_time": "0:03:14", "remaining_time": "0:05:24"}
{"current_steps": 46, "total_steps": 120, "loss": 0.6214762926101685, "lr": 7.86788218175523e-06, "epoch": 1.1518987341772151, "percentage": 38.33, "elapsed_time": "0:03:18", "remaining_time": "0:05:19"}
{"current_steps": 47, "total_steps": 120, "loss": 0.6857700347900391, "lr": 7.747544890354031e-06, "epoch": 1.1772151898734178, "percentage": 39.17, "elapsed_time": "0:03:23", "remaining_time": "0:05:15"}
{"current_steps": 48, "total_steps": 120, "loss": 0.7192323803901672, "lr": 7.624882901672801e-06, "epoch": 1.2025316455696202, "percentage": 40.0, "elapsed_time": "0:03:27", "remaining_time": "0:05:10"}
{"current_steps": 49, "total_steps": 120, "loss": 0.6561193466186523, "lr": 7.500000000000001e-06, "epoch": 1.2278481012658227, "percentage": 40.83, "elapsed_time": "0:03:31", "remaining_time": "0:05:06"}
{"current_steps": 50, "total_steps": 120, "loss": 0.7041677236557007, "lr": 7.373001848738203e-06, "epoch": 1.2531645569620253, "percentage": 41.67, "elapsed_time": "0:03:35", "remaining_time": "0:05:01"}
{"current_steps": 51, "total_steps": 120, "loss": 0.6008931994438171, "lr": 7.243995901002312e-06, "epoch": 1.2784810126582278, "percentage": 42.5, "elapsed_time": "0:03:39", "remaining_time": "0:04:56"}
{"current_steps": 52, "total_steps": 120, "loss": 0.557420551776886, "lr": 7.113091308703498e-06, "epoch": 1.3037974683544304, "percentage": 43.33, "elapsed_time": "0:03:43", "remaining_time": "0:04:51"}
{"current_steps": 53, "total_steps": 120, "loss": 0.6586855053901672, "lr": 6.980398830195785e-06, "epoch": 1.3291139240506329, "percentage": 44.17, "elapsed_time": "0:03:47", "remaining_time": "0:04:47"}
{"current_steps": 54, "total_steps": 120, "loss": 0.646077036857605, "lr": 6.8460307365634225e-06, "epoch": 1.3544303797468356, "percentage": 45.0, "elapsed_time": "0:03:51", "remaining_time": "0:04:42"}
{"current_steps": 55, "total_steps": 120, "loss": 0.6395582556724548, "lr": 6.710100716628345e-06, "epoch": 1.379746835443038, "percentage": 45.83, "elapsed_time": "0:03:55", "remaining_time": "0:04:38"}
{"current_steps": 56, "total_steps": 120, "loss": 0.6356006860733032, "lr": 6.572723780758069e-06, "epoch": 1.4050632911392404, "percentage": 46.67, "elapsed_time": "0:03:59", "remaining_time": "0:04:33"}
{"current_steps": 57, "total_steps": 120, "loss": 0.5863984227180481, "lr": 6.434016163555452e-06, "epoch": 1.4303797468354431, "percentage": 47.5, "elapsed_time": "0:04:03", "remaining_time": "0:04:29"}
{"current_steps": 58, "total_steps": 120, "loss": 0.6374719142913818, "lr": 6.294095225512604e-06, "epoch": 1.4556962025316456, "percentage": 48.33, "elapsed_time": "0:04:07", "remaining_time": "0:04:24"}
{"current_steps": 59, "total_steps": 120, "loss": 0.5927392244338989, "lr": 6.153079353712201e-06, "epoch": 1.481012658227848, "percentage": 49.17, "elapsed_time": "0:04:11", "remaining_time": "0:04:20"}
{"current_steps": 60, "total_steps": 120, "loss": 0.6336349248886108, "lr": 6.011087861660191e-06, "epoch": 1.5063291139240507, "percentage": 50.0, "elapsed_time": "0:04:15", "remaining_time": "0:04:15"}
{"current_steps": 61, "total_steps": 120, "loss": 0.6073538064956665, "lr": 5.8682408883346535e-06, "epoch": 1.5316455696202531, "percentage": 50.83, "elapsed_time": "0:04:20", "remaining_time": "0:04:11"}
{"current_steps": 62, "total_steps": 120, "loss": 0.6669062972068787, "lr": 5.724659296536234e-06, "epoch": 1.5569620253164556, "percentage": 51.67, "elapsed_time": "0:04:24", "remaining_time": "0:04:07"}
{"current_steps": 63, "total_steps": 120, "loss": 0.6189376711845398, "lr": 5.5804645706261515e-06, "epoch": 1.5822784810126582, "percentage": 52.5, "elapsed_time": "0:04:28", "remaining_time": "0:04:02"}
{"current_steps": 64, "total_steps": 120, "loss": 0.6027208566665649, "lr": 5.435778713738292e-06, "epoch": 1.6075949367088609, "percentage": 53.33, "elapsed_time": "0:04:32", "remaining_time": "0:03:58"}
{"current_steps": 65, "total_steps": 120, "loss": 0.5961810350418091, "lr": 5.290724144552379e-06, "epoch": 1.6329113924050633, "percentage": 54.17, "elapsed_time": "0:04:36", "remaining_time": "0:03:54"}
{"current_steps": 66, "total_steps": 120, "loss": 0.6135658621788025, "lr": 5.145423593715558e-06, "epoch": 1.6582278481012658, "percentage": 55.0, "elapsed_time": "0:04:40", "remaining_time": "0:03:49"}
{"current_steps": 67, "total_steps": 120, "loss": 0.5762044787406921, "lr": 5e-06, "epoch": 1.6835443037974684, "percentage": 55.83, "elapsed_time": "0:04:44", "remaining_time": "0:03:45"}
{"current_steps": 68, "total_steps": 120, "loss": 0.6422979235649109, "lr": 4.854576406284443e-06, "epoch": 1.7088607594936709, "percentage": 56.67, "elapsed_time": "0:04:48", "remaining_time": "0:03:40"}
{"current_steps": 69, "total_steps": 120, "loss": 0.6250797510147095, "lr": 4.7092758554476215e-06, "epoch": 1.7341772151898733, "percentage": 57.5, "elapsed_time": "0:04:53", "remaining_time": "0:03:36"}
{"current_steps": 70, "total_steps": 120, "loss": 0.5917418599128723, "lr": 4.564221286261709e-06, "epoch": 1.759493670886076, "percentage": 58.33, "elapsed_time": "0:04:57", "remaining_time": "0:03:32"}
{"current_steps": 71, "total_steps": 120, "loss": 0.6207754611968994, "lr": 4.4195354293738484e-06, "epoch": 1.7848101265822784, "percentage": 59.17, "elapsed_time": "0:05:01", "remaining_time": "0:03:27"}
{"current_steps": 72, "total_steps": 120, "loss": 0.5877741575241089, "lr": 4.275340703463767e-06, "epoch": 1.810126582278481, "percentage": 60.0, "elapsed_time": "0:05:05", "remaining_time": "0:03:23"}
{"current_steps": 73, "total_steps": 120, "loss": 0.6282405853271484, "lr": 4.131759111665349e-06, "epoch": 1.8354430379746836, "percentage": 60.83, "elapsed_time": "0:05:09", "remaining_time": "0:03:19"}
{"current_steps": 74, "total_steps": 120, "loss": 0.6667063236236572, "lr": 3.988912138339812e-06, "epoch": 1.8607594936708862, "percentage": 61.67, "elapsed_time": "0:05:13", "remaining_time": "0:03:14"}
{"current_steps": 75, "total_steps": 120, "loss": 0.6348490715026855, "lr": 3.8469206462878e-06, "epoch": 1.8860759493670884, "percentage": 62.5, "elapsed_time": "0:05:17", "remaining_time": "0:03:10"}
{"current_steps": 76, "total_steps": 120, "loss": 0.6106809377670288, "lr": 3.705904774487396e-06, "epoch": 1.9113924050632911, "percentage": 63.33, "elapsed_time": "0:05:21", "remaining_time": "0:03:06"}
{"current_steps": 77, "total_steps": 120, "loss": 0.5391103029251099, "lr": 3.5659838364445505e-06, "epoch": 1.9367088607594938, "percentage": 64.17, "elapsed_time": "0:05:25", "remaining_time": "0:03:01"}
{"current_steps": 78, "total_steps": 120, "loss": 0.6897211074829102, "lr": 3.427276219241933e-06, "epoch": 1.9620253164556962, "percentage": 65.0, "elapsed_time": "0:05:29", "remaining_time": "0:02:57"}
{"current_steps": 79, "total_steps": 120, "loss": 0.5842834711074829, "lr": 3.289899283371657e-06, "epoch": 1.9873417721518987, "percentage": 65.83, "elapsed_time": "0:05:33", "remaining_time": "0:02:53"}
{"current_steps": 80, "total_steps": 120, "loss": 0.4493752419948578, "lr": 3.1539692634365788e-06, "epoch": 2.0, "percentage": 66.67, "elapsed_time": "0:05:35", "remaining_time": "0:02:47"}
{"current_steps": 81, "total_steps": 120, "loss": 0.24466681480407715, "lr": 3.019601169804216e-06, "epoch": 2.0253164556962027, "percentage": 67.5, "elapsed_time": "0:05:39", "remaining_time": "0:02:43"}
{"current_steps": 82, "total_steps": 120, "loss": 0.285966694355011, "lr": 2.886908691296504e-06, "epoch": 2.050632911392405, "percentage": 68.33, "elapsed_time": "0:05:43", "remaining_time": "0:02:39"}
{"current_steps": 83, "total_steps": 120, "loss": 0.31480997800827026, "lr": 2.7560040989976894e-06, "epoch": 2.0759493670886076, "percentage": 69.17, "elapsed_time": "0:05:47", "remaining_time": "0:02:35"}
{"current_steps": 84, "total_steps": 120, "loss": 0.25318849086761475, "lr": 2.626998151261798e-06, "epoch": 2.1012658227848102, "percentage": 70.0, "elapsed_time": "0:05:51", "remaining_time": "0:02:30"}
{"current_steps": 85, "total_steps": 120, "loss": 0.24628899991512299, "lr": 2.5000000000000015e-06, "epoch": 2.1265822784810124, "percentage": 70.83, "elapsed_time": "0:05:55", "remaining_time": "0:02:26"}
{"current_steps": 86, "total_steps": 120, "loss": 0.30567923188209534, "lr": 2.3751170983272e-06, "epoch": 2.151898734177215, "percentage": 71.67, "elapsed_time": "0:06:00", "remaining_time": "0:02:22"}
{"current_steps": 87, "total_steps": 120, "loss": 0.28589311242103577, "lr": 2.2524551096459703e-06, "epoch": 2.1772151898734178, "percentage": 72.5, "elapsed_time": "0:06:04", "remaining_time": "0:02:18"}
{"current_steps": 88, "total_steps": 120, "loss": 0.25083789229393005, "lr": 2.132117818244771e-06, "epoch": 2.2025316455696204, "percentage": 73.33, "elapsed_time": "0:06:08", "remaining_time": "0:02:13"}
{"current_steps": 89, "total_steps": 120, "loss": 0.25631198287010193, "lr": 2.0142070414860704e-06, "epoch": 2.2278481012658227, "percentage": 74.17, "elapsed_time": "0:06:12", "remaining_time": "0:02:09"}
{"current_steps": 90, "total_steps": 120, "loss": 0.25462988018989563, "lr": 1.8988225436587005e-06, "epoch": 2.2531645569620253, "percentage": 75.0, "elapsed_time": "0:06:16", "remaining_time": "0:02:05"}
{"current_steps": 91, "total_steps": 120, "loss": 0.23561763763427734, "lr": 1.7860619515673034e-06, "epoch": 2.278481012658228, "percentage": 75.83, "elapsed_time": "0:06:20", "remaining_time": "0:02:01"}
{"current_steps": 92, "total_steps": 120, "loss": 0.20739705860614777, "lr": 1.6760206719303107e-06, "epoch": 2.3037974683544302, "percentage": 76.67, "elapsed_time": "0:06:24", "remaining_time": "0:01:57"}
{"current_steps": 93, "total_steps": 120, "loss": 0.24953943490982056, "lr": 1.5687918106563326e-06, "epoch": 2.329113924050633, "percentage": 77.5, "elapsed_time": "0:06:28", "remaining_time": "0:01:52"}
{"current_steps": 94, "total_steps": 120, "loss": 0.1974938064813614, "lr": 1.4644660940672628e-06, "epoch": 2.3544303797468356, "percentage": 78.33, "elapsed_time": "0:06:33", "remaining_time": "0:01:48"}
{"current_steps": 95, "total_steps": 120, "loss": 0.23599892854690552, "lr": 1.3631317921347564e-06, "epoch": 2.379746835443038, "percentage": 79.17, "elapsed_time": "0:06:37", "remaining_time": "0:01:44"}
{"current_steps": 96, "total_steps": 120, "loss": 0.20944546163082123, "lr": 1.264874643795021e-06, "epoch": 2.4050632911392404, "percentage": 80.0, "elapsed_time": "0:06:40", "remaining_time": "0:01:40"}
{"current_steps": 97, "total_steps": 120, "loss": 0.20870286226272583, "lr": 1.1697777844051105e-06, "epoch": 2.430379746835443, "percentage": 80.83, "elapsed_time": "0:06:44", "remaining_time": "0:01:36"}
{"current_steps": 98, "total_steps": 120, "loss": 0.2570308744907379, "lr": 1.0779216754021215e-06, "epoch": 2.4556962025316453, "percentage": 81.67, "elapsed_time": "0:06:48", "remaining_time": "0:01:31"}
{"current_steps": 99, "total_steps": 120, "loss": 0.20367807149887085, "lr": 9.893840362247809e-07, "epoch": 2.481012658227848, "percentage": 82.5, "elapsed_time": "0:06:52", "remaining_time": "0:01:27"}
{"current_steps": 100, "total_steps": 120, "loss": 0.23436129093170166, "lr": 9.042397785550405e-07, "epoch": 2.5063291139240507, "percentage": 83.33, "elapsed_time": "0:06:57", "remaining_time": "0:01:23"}
{"current_steps": 101, "total_steps": 120, "loss": 0.22350996732711792, "lr": 8.225609429353187e-07, "epoch": 2.5316455696202533, "percentage": 84.17, "elapsed_time": "0:07:01", "remaining_time": "0:01:19"}
{"current_steps": 102, "total_steps": 120, "loss": 0.22677019238471985, "lr": 7.444166378150014e-07, "epoch": 2.5569620253164556, "percentage": 85.0, "elapsed_time": "0:07:05", "remaining_time": "0:01:15"}
{"current_steps": 103, "total_steps": 120, "loss": 0.23749305307865143, "lr": 6.698729810778065e-07, "epoch": 2.5822784810126582, "percentage": 85.83, "elapsed_time": "0:07:09", "remaining_time": "0:01:10"}
{"current_steps": 104, "total_steps": 120, "loss": 0.20964664220809937, "lr": 5.989930440994451e-07, "epoch": 2.607594936708861, "percentage": 86.67, "elapsed_time": "0:07:13", "remaining_time": "0:01:06"}
{"current_steps": 105, "total_steps": 120, "loss": 0.2642308473587036, "lr": 5.318367983829393e-07, "epoch": 2.632911392405063, "percentage": 87.5, "elapsed_time": "0:07:17", "remaining_time": "0:01:02"}
{"current_steps": 106, "total_steps": 120, "loss": 0.1657794564962387, "lr": 4.6846106481675035e-07, "epoch": 2.6582278481012658, "percentage": 88.33, "elapsed_time": "0:07:21", "remaining_time": "0:00:58"}
{"current_steps": 107, "total_steps": 120, "loss": 0.24359145760536194, "lr": 4.089194655986306e-07, "epoch": 2.6835443037974684, "percentage": 89.17, "elapsed_time": "0:07:25", "remaining_time": "0:00:54"}
{"current_steps": 108, "total_steps": 120, "loss": 0.23948456346988678, "lr": 3.5326237886588734e-07, "epoch": 2.708860759493671, "percentage": 90.0, "elapsed_time": "0:07:29", "remaining_time": "0:00:49"}
{"current_steps": 109, "total_steps": 120, "loss": 0.25552040338516235, "lr": 3.015368960704584e-07, "epoch": 2.7341772151898733, "percentage": 90.83, "elapsed_time": "0:07:33", "remaining_time": "0:00:45"}
{"current_steps": 110, "total_steps": 120, "loss": 0.25873929262161255, "lr": 2.5378678213483057e-07, "epoch": 2.759493670886076, "percentage": 91.67, "elapsed_time": "0:07:37", "remaining_time": "0:00:41"}
{"current_steps": 111, "total_steps": 120, "loss": 0.21623635292053223, "lr": 2.1005243842255552e-07, "epoch": 2.7848101265822782, "percentage": 92.5, "elapsed_time": "0:07:41", "remaining_time": "0:00:37"}
{"current_steps": 112, "total_steps": 120, "loss": 0.21631255745887756, "lr": 1.7037086855465902e-07, "epoch": 2.810126582278481, "percentage": 93.33, "elapsed_time": "0:07:45", "remaining_time": "0:00:33"}
{"current_steps": 113, "total_steps": 120, "loss": 0.21876609325408936, "lr": 1.3477564710088097e-07, "epoch": 2.8354430379746836, "percentage": 94.17, "elapsed_time": "0:07:49", "remaining_time": "0:00:29"}
{"current_steps": 114, "total_steps": 120, "loss": 0.1940886378288269, "lr": 1.0329689117224262e-07, "epoch": 2.8607594936708862, "percentage": 95.0, "elapsed_time": "0:07:53", "remaining_time": "0:00:24"}
{"current_steps": 115, "total_steps": 120, "loss": 0.23064503073692322, "lr": 7.59612349389599e-08, "epoch": 2.8860759493670884, "percentage": 95.83, "elapsed_time": "0:07:57", "remaining_time": "0:00:20"}
{"current_steps": 116, "total_steps": 120, "loss": 0.21075034141540527, "lr": 5.279180709527765e-08, "epoch": 2.911392405063291, "percentage": 96.67, "elapsed_time": "0:08:01", "remaining_time": "0:00:16"}
{"current_steps": 117, "total_steps": 120, "loss": 0.2221754789352417, "lr": 3.3808211290284886e-08, "epoch": 2.9367088607594938, "percentage": 97.5, "elapsed_time": "0:08:06", "remaining_time": "0:00:12"}
{"current_steps": 118, "total_steps": 120, "loss": 0.2319803237915039, "lr": 1.9026509541272276e-08, "epoch": 2.962025316455696, "percentage": 98.33, "elapsed_time": "0:08:10", "remaining_time": "0:00:08"}
{"current_steps": 119, "total_steps": 120, "loss": 0.25988614559173584, "lr": 8.459208643659122e-09, "epoch": 2.9873417721518987, "percentage": 99.17, "elapsed_time": "0:08:14", "remaining_time": "0:00:04"}
{"current_steps": 120, "total_steps": 120, "loss": 0.09604214131832123, "lr": 2.1152495889970035e-09, "epoch": 3.0, "percentage": 100.0, "elapsed_time": "0:08:16", "remaining_time": "0:00:00"}
{"current_steps": 120, "total_steps": 120, "epoch": 3.0, "percentage": 100.0, "elapsed_time": "0:09:02", "remaining_time": "0:00:00"}

3
training_args.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:218319acf1d357e9d1e1f92d1a4ede7b4cc70a0e059f333ab954213e948b86ae
size 7761