初始化项目，由ModelHub XC社区提供模型
Model: jackf857/llama-3-8b-base-slic-hf-ultrafeedback-4xh200 Source: Original Platform
This commit is contained in:
ModelHub XC
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
78
README.md
Normal file
78
README.md
Normal file
@@ -0,0 +1,78 @@
|
||||
---
|
||||
library_name: transformers
|
||||
base_model: W-61/llama-3-8b-base-sft-ultrachat-8xh200
|
||||
tags:
|
||||
- alignment-handbook
|
||||
- slic-hf
|
||||
- generated_from_trainer
|
||||
datasets:
|
||||
- HuggingFaceH4/ultrafeedback_binarized
|
||||
model-index:
|
||||
- name: llama-3-8b-base-slic-hf-ultrafeedback-8xh200
|
||||
results: []
|
||||
---
|
||||
|
||||
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
||||
should probably proofread and complete it, then remove this comment. -->
|
||||
|
||||
# llama-3-8b-base-slic-hf-ultrafeedback-8xh200
|
||||
|
||||
This model is a fine-tuned version of [W-61/llama-3-8b-base-sft-ultrachat-8xh200](https://huggingface.co/W-61/llama-3-8b-base-sft-ultrachat-8xh200) on the HuggingFaceH4/ultrafeedback_binarized dataset.
|
||||
It achieves the following results on the evaluation set:
|
||||
- Loss: 341.9101
|
||||
- Rewards/chosen: -260.8732
|
||||
- Rewards/rejected: -246.9854
|
||||
- Rewards/accuracies: 0.4919
|
||||
- Rewards/margins: -13.8878
|
||||
- Logps/chosen: -260.8732
|
||||
- Logps/rejected: -246.9854
|
||||
- Slic/rank Loss: 81.0369
|
||||
- Slic/ce Loss: 260.8732
|
||||
- Logits/chosen: -0.5998
|
||||
- Logits/rejected: -0.6056
|
||||
|
||||
## Model description
|
||||
|
||||
More information needed
|
||||
|
||||
## Intended uses & limitations
|
||||
|
||||
More information needed
|
||||
|
||||
## Training and evaluation data
|
||||
|
||||
More information needed
|
||||
|
||||
## Training procedure
|
||||
|
||||
### Training hyperparameters
|
||||
|
||||
The following hyperparameters were used during training:
|
||||
- learning_rate: 5e-07
|
||||
- train_batch_size: 4
|
||||
- eval_batch_size: 4
|
||||
- seed: 42
|
||||
- distributed_type: multi-GPU
|
||||
- num_devices: 8
|
||||
- gradient_accumulation_steps: 4
|
||||
- total_train_batch_size: 128
|
||||
- total_eval_batch_size: 32
|
||||
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
|
||||
- lr_scheduler_type: cosine
|
||||
- lr_scheduler_warmup_ratio: 0.1
|
||||
- num_epochs: 1
|
||||
|
||||
### Training results
|
||||
|
||||
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/chosen | Logps/rejected | Slic/rank Loss | Slic/ce Loss | Logits/chosen | Logits/rejected |
|
||||
|:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:------------:|:--------------:|:--------------:|:------------:|:-------------:|:---------------:|
|
||||
| 1368.2239 | 0.4188 | 200 | 345.5856 | -262.1702 | -246.1536 | 0.4874 | -16.0166 | -262.1702 | -246.1536 | 83.4155 | 262.1702 | -0.6125 | -0.6205 |
|
||||
| 1395.7896 | 0.8377 | 400 | 341.9101 | -260.8732 | -246.9854 | 0.4919 | -13.8878 | -260.8732 | -246.9854 | 81.0369 | 260.8732 | -0.5998 | -0.6056 |
|
||||
|
||||
|
||||
### Framework versions
|
||||
|
||||
- Transformers 4.51.0
|
||||
- Pytorch 2.3.1+cu121
|
||||
- Datasets 2.21.0
|
||||
- Tokenizers 0.21.4
|
||||
24
all_results.json
Normal file
24
all_results.json
Normal file
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"epoch": 0.9989528795811519,
|
||||
"eval_logits/chosen": -0.6057044267654419,
|
||||
"eval_logits/rejected": -0.6120408177375793,
|
||||
"eval_logps/chosen": -260.8712158203125,
|
||||
"eval_logps/rejected": -247.0758819580078,
|
||||
"eval_loss": 341.7819519042969,
|
||||
"eval_rewards/accuracies": 0.49294355511665344,
|
||||
"eval_rewards/chosen": -260.8712158203125,
|
||||
"eval_rewards/margins": -13.795334815979004,
|
||||
"eval_rewards/rejected": -247.0758819580078,
|
||||
"eval_runtime": 23.6719,
|
||||
"eval_samples": 2000,
|
||||
"eval_samples_per_second": 84.488,
|
||||
"eval_slic/ce_loss": 260.8712158203125,
|
||||
"eval_slic/rank_loss": 80.91075134277344,
|
||||
"eval_steps_per_second": 2.661,
|
||||
"total_flos": 0.0,
|
||||
"train_loss": 1398.719245958628,
|
||||
"train_runtime": 2677.8284,
|
||||
"train_samples": 61135,
|
||||
"train_samples_per_second": 22.83,
|
||||
"train_steps_per_second": 0.178
|
||||
}
|
||||
29
config.json
Normal file
29
config.json
Normal file
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 128000,
|
||||
"eos_token_id": 128001,
|
||||
"head_dim": 128,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 4096,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 14336,
|
||||
"max_position_embeddings": 8192,
|
||||
"mlp_bias": false,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 32,
|
||||
"num_key_value_heads": 8,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 500000.0,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "float32",
|
||||
"transformers_version": "4.51.0",
|
||||
"use_cache": true,
|
||||
"vocab_size": 128256
|
||||
}
|
||||
18
eval_results.json
Normal file
18
eval_results.json
Normal file
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"epoch": 0.9989528795811519,
|
||||
"eval_logits/chosen": -0.6057044267654419,
|
||||
"eval_logits/rejected": -0.6120408177375793,
|
||||
"eval_logps/chosen": -260.8712158203125,
|
||||
"eval_logps/rejected": -247.0758819580078,
|
||||
"eval_loss": 341.7819519042969,
|
||||
"eval_rewards/accuracies": 0.49294355511665344,
|
||||
"eval_rewards/chosen": -260.8712158203125,
|
||||
"eval_rewards/margins": -13.795334815979004,
|
||||
"eval_rewards/rejected": -247.0758819580078,
|
||||
"eval_runtime": 23.6719,
|
||||
"eval_samples": 2000,
|
||||
"eval_samples_per_second": 84.488,
|
||||
"eval_slic/ce_loss": 260.8712158203125,
|
||||
"eval_slic/rank_loss": 80.91075134277344,
|
||||
"eval_steps_per_second": 2.661
|
||||
}
|
||||
9
generation_config.json
Normal file
9
generation_config.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"bos_token_id": 128000,
|
||||
"do_sample": true,
|
||||
"eos_token_id": 128001,
|
||||
"max_length": 4096,
|
||||
"temperature": 0.6,
|
||||
"top_p": 0.9,
|
||||
"transformers_version": "4.51.0"
|
||||
}
|
||||
3
model-00001-of-00007.safetensors
Normal file
3
model-00001-of-00007.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e996bd3f5beb6ccc6e9ddf3636d52610453e27f2a26cc3cff53ed080077dace8
|
||||
size 4886466168
|
||||
3
model-00002-of-00007.safetensors
Normal file
3
model-00002-of-00007.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8b9f3876cbc52a56af8fa6224915f80d6d804f28430f5a6856c5236803a58aa2
|
||||
size 4832007448
|
||||
3
model-00003-of-00007.safetensors
Normal file
3
model-00003-of-00007.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:14f420a77d452208572be96f4d3a5cde9b0dc2445ff688cc3590d28387b46b11
|
||||
size 4999813112
|
||||
3
model-00004-of-00007.safetensors
Normal file
3
model-00004-of-00007.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:835b9641ef2ac4dc7a060f280fdd7375054b3d603597d0075bd3ff3e69561c54
|
||||
size 4999813128
|
||||
3
model-00005-of-00007.safetensors
Normal file
3
model-00005-of-00007.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:705cbb49136b83d897e2b6bfa6f69f328db96634cb36930f20378349022e94a6
|
||||
size 4832007496
|
||||
3
model-00006-of-00007.safetensors
Normal file
3
model-00006-of-00007.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4bc6ebdb2a6bd51ac32be6a60beccb4fa517591b4c020bb227ccf9618b5e50da
|
||||
size 4999813120
|
||||
3
model-00007-of-00007.safetensors
Normal file
3
model-00007-of-00007.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:ea059fc87278c784e8dd4cac93f6b0611970d29616af548cd3529cdeacd3eac3
|
||||
size 2571158184
|
||||
298
model.safetensors.index.json
Normal file
298
model.safetensors.index.json
Normal file
@@ -0,0 +1,298 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 32121044992
|
||||
},
|
||||
"weight_map": {
|
||||
"lm_head.weight": "model-00007-of-00007.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.mlp.down_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.mlp.gate_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.mlp.up_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.mlp.down_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.mlp.gate_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.mlp.up_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.mlp.down_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.mlp.gate_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.mlp.up_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00007-of-00007.safetensors",
|
||||
"model.layers.31.mlp.down_proj.weight": "model-00007-of-00007.safetensors",
|
||||
"model.layers.31.mlp.gate_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.31.mlp.up_proj.weight": "model-00007-of-00007.safetensors",
|
||||
"model.layers.31.post_attention_layernorm.weight": "model-00007-of-00007.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.31.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.norm.weight": "model-00007-of-00007.safetensors"
|
||||
}
|
||||
}
|
||||
23
special_tokens_map.json
Normal file
23
special_tokens_map.json
Normal file
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<|begin_of_text|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<|end_of_text|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<|end_of_text|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3c5cf44023714fb39b05e71e425f8d7b92805ff73f7988b083b8c87f0bf87393
|
||||
size 17209961
|
||||
2064
tokenizer_config.json
Normal file
2064
tokenizer_config.json
Normal file
File diff suppressed because it is too large
Load Diff
9
train_results.json
Normal file
9
train_results.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"epoch": 0.9989528795811519,
|
||||
"total_flos": 0.0,
|
||||
"train_loss": 1398.719245958628,
|
||||
"train_runtime": 2677.8284,
|
||||
"train_samples": 61135,
|
||||
"train_samples_per_second": 22.83,
|
||||
"train_steps_per_second": 0.178
|
||||
}
|
||||
895
trainer_state.json
Normal file
895
trainer_state.json
Normal file
@@ -0,0 +1,895 @@
|
||||
{
|
||||
"best_global_step": null,
|
||||
"best_metric": null,
|
||||
"best_model_checkpoint": null,
|
||||
"epoch": 0.9989528795811519,
|
||||
"eval_steps": 200,
|
||||
"global_step": 477,
|
||||
"is_hyper_param_search": false,
|
||||
"is_local_process_zero": true,
|
||||
"is_world_process_zero": true,
|
||||
"log_history": [
|
||||
{
|
||||
"epoch": 0.0020942408376963353,
|
||||
"grad_norm": 6127.23876953125,
|
||||
"learning_rate": 0.0,
|
||||
"logits/chosen": -0.6042112112045288,
|
||||
"logits/rejected": -0.6177991628646851,
|
||||
"logps/chosen": -275.48590087890625,
|
||||
"logps/rejected": -223.16470336914062,
|
||||
"loss": 1522.3201,
|
||||
"rewards/accuracies": 0.46875,
|
||||
"rewards/chosen": -275.48590087890625,
|
||||
"rewards/margins": -52.32117462158203,
|
||||
"rewards/rejected": -223.16470336914062,
|
||||
"slic/ce_loss": 275.48590087890625,
|
||||
"slic/rank_loss": 105.09413146972656,
|
||||
"step": 1
|
||||
},
|
||||
{
|
||||
"epoch": 0.020942408376963352,
|
||||
"grad_norm": 5911.2392578125,
|
||||
"learning_rate": 9.375e-08,
|
||||
"logits/chosen": -0.6444513201713562,
|
||||
"logits/rejected": -0.6520731449127197,
|
||||
"logps/chosen": -290.2828674316406,
|
||||
"logps/rejected": -264.8289489746094,
|
||||
"loss": 1545.4015,
|
||||
"rewards/accuracies": 0.4618055522441864,
|
||||
"rewards/chosen": -290.2828674316406,
|
||||
"rewards/margins": -25.45393943786621,
|
||||
"rewards/rejected": -264.8289489746094,
|
||||
"slic/ce_loss": 290.2828674316406,
|
||||
"slic/rank_loss": 96.06748962402344,
|
||||
"step": 10
|
||||
},
|
||||
{
|
||||
"epoch": 0.041884816753926704,
|
||||
"grad_norm": 6397.529296875,
|
||||
"learning_rate": 1.9791666666666664e-07,
|
||||
"logits/chosen": -0.6179991364479065,
|
||||
"logits/rejected": -0.6330140829086304,
|
||||
"logps/chosen": -286.71014404296875,
|
||||
"logps/rejected": -259.08758544921875,
|
||||
"loss": 1526.0234,
|
||||
"rewards/accuracies": 0.47265625,
|
||||
"rewards/chosen": -286.71014404296875,
|
||||
"rewards/margins": -27.622533798217773,
|
||||
"rewards/rejected": -259.08758544921875,
|
||||
"slic/ce_loss": 286.71014404296875,
|
||||
"slic/rank_loss": 94.79573059082031,
|
||||
"step": 20
|
||||
},
|
||||
{
|
||||
"epoch": 0.06282722513089005,
|
||||
"grad_norm": 4871.51513671875,
|
||||
"learning_rate": 3.020833333333333e-07,
|
||||
"logits/chosen": -0.6329924464225769,
|
||||
"logits/rejected": -0.6390506029129028,
|
||||
"logps/chosen": -277.3640441894531,
|
||||
"logps/rejected": -255.4713134765625,
|
||||
"loss": 1477.4605,
|
||||
"rewards/accuracies": 0.48124998807907104,
|
||||
"rewards/chosen": -277.3640441894531,
|
||||
"rewards/margins": -21.892715454101562,
|
||||
"rewards/rejected": -255.4713134765625,
|
||||
"slic/ce_loss": 277.3640441894531,
|
||||
"slic/rank_loss": 92.00110626220703,
|
||||
"step": 30
|
||||
},
|
||||
{
|
||||
"epoch": 0.08376963350785341,
|
||||
"grad_norm": 4104.8447265625,
|
||||
"learning_rate": 4.0625e-07,
|
||||
"logits/chosen": -0.6297017931938171,
|
||||
"logits/rejected": -0.6496596932411194,
|
||||
"logps/chosen": -279.53680419921875,
|
||||
"logps/rejected": -251.68435668945312,
|
||||
"loss": 1506.1701,
|
||||
"rewards/accuracies": 0.47578126192092896,
|
||||
"rewards/chosen": -279.53680419921875,
|
||||
"rewards/margins": -27.85247230529785,
|
||||
"rewards/rejected": -251.68435668945312,
|
||||
"slic/ce_loss": 279.53680419921875,
|
||||
"slic/rank_loss": 97.00569915771484,
|
||||
"step": 40
|
||||
},
|
||||
{
|
||||
"epoch": 0.10471204188481675,
|
||||
"grad_norm": 3677.18603515625,
|
||||
"learning_rate": 4.999932966293553e-07,
|
||||
"logits/chosen": -0.6434795260429382,
|
||||
"logits/rejected": -0.6600109338760376,
|
||||
"logps/chosen": -273.2533874511719,
|
||||
"logps/rejected": -256.40338134765625,
|
||||
"loss": 1450.0907,
|
||||
"rewards/accuracies": 0.48359376192092896,
|
||||
"rewards/chosen": -273.2533874511719,
|
||||
"rewards/margins": -16.849985122680664,
|
||||
"rewards/rejected": -256.40338134765625,
|
||||
"slic/ce_loss": 273.2533874511719,
|
||||
"slic/rank_loss": 89.26930236816406,
|
||||
"step": 50
|
||||
},
|
||||
{
|
||||
"epoch": 0.1256544502617801,
|
||||
"grad_norm": 3486.959716796875,
|
||||
"learning_rate": 4.991893270335525e-07,
|
||||
"logits/chosen": -0.6477519869804382,
|
||||
"logits/rejected": -0.6574610471725464,
|
||||
"logps/chosen": -261.81854248046875,
|
||||
"logps/rejected": -248.384521484375,
|
||||
"loss": 1407.8849,
|
||||
"rewards/accuracies": 0.503125011920929,
|
||||
"rewards/chosen": -261.81854248046875,
|
||||
"rewards/margins": -13.434022903442383,
|
||||
"rewards/rejected": -248.384521484375,
|
||||
"slic/ce_loss": 261.81854248046875,
|
||||
"slic/rank_loss": 90.15267944335938,
|
||||
"step": 60
|
||||
},
|
||||
{
|
||||
"epoch": 0.14659685863874344,
|
||||
"grad_norm": 3554.351806640625,
|
||||
"learning_rate": 4.970496218214204e-07,
|
||||
"logits/chosen": -0.6433520913124084,
|
||||
"logits/rejected": -0.6552119851112366,
|
||||
"logps/chosen": -261.0797424316406,
|
||||
"logps/rejected": -245.8175506591797,
|
||||
"loss": 1383.9182,
|
||||
"rewards/accuracies": 0.48828125,
|
||||
"rewards/chosen": -261.0797424316406,
|
||||
"rewards/margins": -15.262176513671875,
|
||||
"rewards/rejected": -245.8175506591797,
|
||||
"slic/ce_loss": 261.0797424316406,
|
||||
"slic/rank_loss": 84.89978790283203,
|
||||
"step": 70
|
||||
},
|
||||
{
|
||||
"epoch": 0.16753926701570682,
|
||||
"grad_norm": 3468.174072265625,
|
||||
"learning_rate": 4.935856505068998e-07,
|
||||
"logits/chosen": -0.6140779256820679,
|
||||
"logits/rejected": -0.6229384541511536,
|
||||
"logps/chosen": -262.1397705078125,
|
||||
"logps/rejected": -246.156494140625,
|
||||
"loss": 1382.4392,
|
||||
"rewards/accuracies": 0.47578126192092896,
|
||||
"rewards/chosen": -262.1397705078125,
|
||||
"rewards/margins": -15.983256340026855,
|
||||
"rewards/rejected": -246.156494140625,
|
||||
"slic/ce_loss": 262.1397705078125,
|
||||
"slic/rank_loss": 83.47003173828125,
|
||||
"step": 80
|
||||
},
|
||||
{
|
||||
"epoch": 0.18848167539267016,
|
||||
"grad_norm": 3276.579833984375,
|
||||
"learning_rate": 4.8881598109976e-07,
|
||||
"logits/chosen": -0.637958288192749,
|
||||
"logits/rejected": -0.6412946581840515,
|
||||
"logps/chosen": -259.00439453125,
|
||||
"logps/rejected": -244.78060913085938,
|
||||
"loss": 1378.6654,
|
||||
"rewards/accuracies": 0.5,
|
||||
"rewards/chosen": -259.00439453125,
|
||||
"rewards/margins": -14.223783493041992,
|
||||
"rewards/rejected": -244.78060913085938,
|
||||
"slic/ce_loss": 259.00439453125,
|
||||
"slic/rank_loss": 85.66192626953125,
|
||||
"step": 90
|
||||
},
|
||||
{
|
||||
"epoch": 0.2094240837696335,
|
||||
"grad_norm": 3682.858642578125,
|
||||
"learning_rate": 4.827661805750437e-07,
|
||||
"logits/chosen": -0.6100295782089233,
|
||||
"logits/rejected": -0.6224995255470276,
|
||||
"logps/chosen": -260.15911865234375,
|
||||
"logps/rejected": -240.53005981445312,
|
||||
"loss": 1390.1286,
|
||||
"rewards/accuracies": 0.49921876192092896,
|
||||
"rewards/chosen": -260.15911865234375,
|
||||
"rewards/margins": -19.629114151000977,
|
||||
"rewards/rejected": -240.53005981445312,
|
||||
"slic/ce_loss": 260.15911865234375,
|
||||
"slic/rank_loss": 87.37300109863281,
|
||||
"step": 100
|
||||
},
|
||||
{
|
||||
"epoch": 0.23036649214659685,
|
||||
"grad_norm": 3362.56884765625,
|
||||
"learning_rate": 4.75468677825789e-07,
|
||||
"logits/chosen": -0.6101893186569214,
|
||||
"logits/rejected": -0.6191698312759399,
|
||||
"logps/chosen": -259.8664245605469,
|
||||
"logps/rejected": -244.67868041992188,
|
||||
"loss": 1384.9318,
|
||||
"rewards/accuracies": 0.4765625,
|
||||
"rewards/chosen": -259.8664245605469,
|
||||
"rewards/margins": -15.187728881835938,
|
||||
"rewards/rejected": -244.67868041992188,
|
||||
"slic/ce_loss": 259.8664245605469,
|
||||
"slic/rank_loss": 86.36650848388672,
|
||||
"step": 110
|
||||
},
|
||||
{
|
||||
"epoch": 0.2513089005235602,
|
||||
"grad_norm": 3407.73095703125,
|
||||
"learning_rate": 4.669625898336438e-07,
|
||||
"logits/chosen": -0.6205486059188843,
|
||||
"logits/rejected": -0.62354576587677,
|
||||
"logps/chosen": -264.4170227050781,
|
||||
"logps/rejected": -248.2465362548828,
|
||||
"loss": 1411.6574,
|
||||
"rewards/accuracies": 0.4749999940395355,
|
||||
"rewards/chosen": -264.4170227050781,
|
||||
"rewards/margins": -16.170438766479492,
|
||||
"rewards/rejected": -248.2465362548828,
|
||||
"slic/ce_loss": 264.4170227050781,
|
||||
"slic/rank_loss": 88.49736022949219,
|
||||
"step": 120
|
||||
},
|
||||
{
|
||||
"epoch": 0.27225130890052357,
|
||||
"grad_norm": 3431.46142578125,
|
||||
"learning_rate": 4.5729351198915705e-07,
|
||||
"logits/chosen": -0.6135658025741577,
|
||||
"logits/rejected": -0.6137000322341919,
|
||||
"logps/chosen": -263.35980224609375,
|
||||
"logps/rejected": -245.09103393554688,
|
||||
"loss": 1415.0682,
|
||||
"rewards/accuracies": 0.4921875,
|
||||
"rewards/chosen": -263.35980224609375,
|
||||
"rewards/margins": -18.268768310546875,
|
||||
"rewards/rejected": -245.09103393554688,
|
||||
"slic/ce_loss": 263.35980224609375,
|
||||
"slic/rank_loss": 90.4072494506836,
|
||||
"step": 130
|
||||
},
|
||||
{
|
||||
"epoch": 0.2931937172774869,
|
||||
"grad_norm": 3545.783935546875,
|
||||
"learning_rate": 4.4651327368569684e-07,
|
||||
"logits/chosen": -0.6140355467796326,
|
||||
"logits/rejected": -0.6084020733833313,
|
||||
"logps/chosen": -265.9866027832031,
|
||||
"logps/rejected": -250.787109375,
|
||||
"loss": 1405.8035,
|
||||
"rewards/accuracies": 0.4984374940395355,
|
||||
"rewards/chosen": -265.9866027832031,
|
||||
"rewards/margins": -15.199483871459961,
|
||||
"rewards/rejected": -250.787109375,
|
||||
"slic/ce_loss": 265.9866027832031,
|
||||
"slic/rank_loss": 85.46427917480469,
|
||||
"step": 140
|
||||
},
|
||||
{
|
||||
"epoch": 0.31413612565445026,
|
||||
"grad_norm": 3279.9306640625,
|
||||
"learning_rate": 4.346796604970912e-07,
|
||||
"logits/chosen": -0.610595703125,
|
||||
"logits/rejected": -0.6149523258209229,
|
||||
"logps/chosen": -262.4765930175781,
|
||||
"logps/rejected": -238.65792846679688,
|
||||
"loss": 1396.4196,
|
||||
"rewards/accuracies": 0.4585937559604645,
|
||||
"rewards/chosen": -262.4765930175781,
|
||||
"rewards/margins": -23.818683624267578,
|
||||
"rewards/rejected": -238.65792846679688,
|
||||
"slic/ce_loss": 262.4765930175781,
|
||||
"slic/rank_loss": 86.62831115722656,
|
||||
"step": 150
|
||||
},
|
||||
{
|
||||
"epoch": 0.33507853403141363,
|
||||
"grad_norm": 3371.940673828125,
|
||||
"learning_rate": 4.218561044282098e-07,
|
||||
"logits/chosen": -0.6132291555404663,
|
||||
"logits/rejected": -0.615606963634491,
|
||||
"logps/chosen": -260.2372131347656,
|
||||
"logps/rejected": -246.6212158203125,
|
||||
"loss": 1395.4537,
|
||||
"rewards/accuracies": 0.4976562559604645,
|
||||
"rewards/chosen": -260.2372131347656,
|
||||
"rewards/margins": -13.615992546081543,
|
||||
"rewards/rejected": -246.6212158203125,
|
||||
"slic/ce_loss": 260.2372131347656,
|
||||
"slic/rank_loss": 88.62622833251953,
|
||||
"step": 160
|
||||
},
|
||||
{
|
||||
"epoch": 0.35602094240837695,
|
||||
"grad_norm": 3451.259521484375,
|
||||
"learning_rate": 4.081113438988443e-07,
|
||||
"logits/chosen": -0.6058305501937866,
|
||||
"logits/rejected": -0.6137093305587769,
|
||||
"logps/chosen": -264.1838684082031,
|
||||
"logps/rejected": -232.7599639892578,
|
||||
"loss": 1435.0535,
|
||||
"rewards/accuracies": 0.46406251192092896,
|
||||
"rewards/chosen": -264.1838684082031,
|
||||
"rewards/margins": -31.423877716064453,
|
||||
"rewards/rejected": -232.7599639892578,
|
||||
"slic/ce_loss": 264.1838684082031,
|
||||
"slic/rank_loss": 94.57951354980469,
|
||||
"step": 170
|
||||
},
|
||||
{
|
||||
"epoch": 0.3769633507853403,
|
||||
"grad_norm": 3250.255126953125,
|
||||
"learning_rate": 3.935190552834828e-07,
|
||||
"logits/chosen": -0.604538083076477,
|
||||
"logits/rejected": -0.6160414814949036,
|
||||
"logps/chosen": -263.60711669921875,
|
||||
"logps/rejected": -244.922607421875,
|
||||
"loss": 1397.7748,
|
||||
"rewards/accuracies": 0.48828125,
|
||||
"rewards/chosen": -263.60711669921875,
|
||||
"rewards/margins": -18.68451499938965,
|
||||
"rewards/rejected": -244.922607421875,
|
||||
"slic/ce_loss": 263.60711669921875,
|
||||
"slic/rank_loss": 85.83655548095703,
|
||||
"step": 180
|
||||
},
|
||||
{
|
||||
"epoch": 0.39790575916230364,
|
||||
"grad_norm": 3169.67041015625,
|
||||
"learning_rate": 3.781574579820464e-07,
|
||||
"logits/chosen": -0.604019820690155,
|
||||
"logits/rejected": -0.6174649596214294,
|
||||
"logps/chosen": -261.44183349609375,
|
||||
"logps/rejected": -239.5850830078125,
|
||||
"loss": 1405.417,
|
||||
"rewards/accuracies": 0.4765625,
|
||||
"rewards/chosen": -261.44183349609375,
|
||||
"rewards/margins": -21.85675621032715,
|
||||
"rewards/rejected": -239.5850830078125,
|
||||
"slic/ce_loss": 261.44183349609375,
|
||||
"slic/rank_loss": 89.91242980957031,
|
||||
"step": 190
|
||||
},
|
||||
{
|
||||
"epoch": 0.418848167539267,
|
||||
"grad_norm": 3375.69482421875,
|
||||
"learning_rate": 3.621088951385353e-07,
|
||||
"logits/chosen": -0.6043376922607422,
|
||||
"logits/rejected": -0.6085049510002136,
|
||||
"logps/chosen": -257.392333984375,
|
||||
"logps/rejected": -241.6600799560547,
|
||||
"loss": 1368.2239,
|
||||
"rewards/accuracies": 0.49609375,
|
||||
"rewards/chosen": -257.392333984375,
|
||||
"rewards/margins": -15.732267379760742,
|
||||
"rewards/rejected": -241.6600799560547,
|
||||
"slic/ce_loss": 257.392333984375,
|
||||
"slic/rank_loss": 84.66365814208984,
|
||||
"step": 200
|
||||
},
|
||||
{
|
||||
"epoch": 0.418848167539267,
|
||||
"eval_logits/chosen": -0.6125423312187195,
|
||||
"eval_logits/rejected": -0.6204564571380615,
|
||||
"eval_logps/chosen": -262.1701965332031,
|
||||
"eval_logps/rejected": -246.15359497070312,
|
||||
"eval_loss": 345.58563232421875,
|
||||
"eval_rewards/accuracies": 0.4873991906642914,
|
||||
"eval_rewards/chosen": -262.1701965332031,
|
||||
"eval_rewards/margins": -16.016645431518555,
|
||||
"eval_rewards/rejected": -246.15359497070312,
|
||||
"eval_runtime": 23.996,
|
||||
"eval_samples_per_second": 83.347,
|
||||
"eval_slic/ce_loss": 262.1701965332031,
|
||||
"eval_slic/rank_loss": 83.41545104980469,
|
||||
"eval_steps_per_second": 2.625,
|
||||
"step": 200
|
||||
},
|
||||
{
|
||||
"epoch": 0.4397905759162304,
|
||||
"grad_norm": 3124.86767578125,
|
||||
"learning_rate": 3.454593922550693e-07,
|
||||
"logits/chosen": -0.6011446714401245,
|
||||
"logits/rejected": -0.6014553308486938,
|
||||
"logps/chosen": -262.05340576171875,
|
||||
"logps/rejected": -250.4723358154297,
|
||||
"loss": 1388.7663,
|
||||
"rewards/accuracies": 0.4781250059604645,
|
||||
"rewards/chosen": -262.05340576171875,
|
||||
"rewards/margins": -11.581093788146973,
|
||||
"rewards/rejected": -250.4723358154297,
|
||||
"slic/ce_loss": 262.05340576171875,
|
||||
"slic/rank_loss": 85.13812255859375,
|
||||
"step": 210
|
||||
},
|
||||
{
|
||||
"epoch": 0.4607329842931937,
|
||||
"grad_norm": 3453.45849609375,
|
||||
"learning_rate": 3.2829819606729477e-07,
|
||||
"logits/chosen": -0.595199704170227,
|
||||
"logits/rejected": -0.6031100153923035,
|
||||
"logps/chosen": -268.83941650390625,
|
||||
"logps/rejected": -252.3756866455078,
|
||||
"loss": 1436.1566,
|
||||
"rewards/accuracies": 0.48124998807907104,
|
||||
"rewards/chosen": -268.83941650390625,
|
||||
"rewards/margins": -16.46368980407715,
|
||||
"rewards/rejected": -252.3756866455078,
|
||||
"slic/ce_loss": 268.83941650390625,
|
||||
"slic/rank_loss": 90.19975280761719,
|
||||
"step": 220
|
||||
},
|
||||
{
|
||||
"epoch": 0.4816753926701571,
|
||||
"grad_norm": 3144.963623046875,
|
||||
"learning_rate": 3.1071729615293424e-07,
|
||||
"logits/chosen": -0.5905068516731262,
|
||||
"logits/rejected": -0.6006661653518677,
|
||||
"logps/chosen": -256.08294677734375,
|
||||
"logps/rejected": -239.11331176757812,
|
||||
"loss": 1356.9873,
|
||||
"rewards/accuracies": 0.48359376192092896,
|
||||
"rewards/chosen": -256.08294677734375,
|
||||
"rewards/margins": -16.969640731811523,
|
||||
"rewards/rejected": -239.11331176757812,
|
||||
"slic/ce_loss": 256.08294677734375,
|
||||
"slic/rank_loss": 83.16390228271484,
|
||||
"step": 230
|
||||
},
|
||||
{
|
||||
"epoch": 0.5026178010471204,
|
||||
"grad_norm": 3449.269287109375,
|
||||
"learning_rate": 2.9281093183781403e-07,
|
||||
"logits/chosen": -0.5943664312362671,
|
||||
"logits/rejected": -0.6033399701118469,
|
||||
"logps/chosen": -262.95050048828125,
|
||||
"logps/rejected": -244.4707794189453,
|
||||
"loss": 1402.5626,
|
||||
"rewards/accuracies": 0.4742187559604645,
|
||||
"rewards/chosen": -262.95050048828125,
|
||||
"rewards/margins": -18.479671478271484,
|
||||
"rewards/rejected": -244.4707794189453,
|
||||
"slic/ce_loss": 262.95050048828125,
|
||||
"slic/rank_loss": 87.6901626586914,
|
||||
"step": 240
|
||||
},
|
||||
{
|
||||
"epoch": 0.5235602094240838,
|
||||
"grad_norm": 3244.26611328125,
|
||||
"learning_rate": 2.7467508704251135e-07,
|
||||
"logits/chosen": -0.5862177610397339,
|
||||
"logits/rejected": -0.5929856300354004,
|
||||
"logps/chosen": -261.1794738769531,
|
||||
"logps/rejected": -238.1936492919922,
|
||||
"loss": 1404.9589,
|
||||
"rewards/accuracies": 0.4742187559604645,
|
||||
"rewards/chosen": -261.1794738769531,
|
||||
"rewards/margins": -22.985858917236328,
|
||||
"rewards/rejected": -238.1936492919922,
|
||||
"slic/ce_loss": 261.1794738769531,
|
||||
"slic/rank_loss": 90.06024169921875,
|
||||
"step": 250
|
||||
},
|
||||
{
|
||||
"epoch": 0.5445026178010471,
|
||||
"grad_norm": 3298.73388671875,
|
||||
"learning_rate": 2.5640697577740815e-07,
|
||||
"logits/chosen": -0.6019514203071594,
|
||||
"logits/rejected": -0.5996894836425781,
|
||||
"logps/chosen": -261.6252746582031,
|
||||
"logps/rejected": -237.8870391845703,
|
||||
"loss": 1411.4449,
|
||||
"rewards/accuracies": 0.46484375,
|
||||
"rewards/chosen": -261.6252746582031,
|
||||
"rewards/margins": -23.738243103027344,
|
||||
"rewards/rejected": -237.8870391845703,
|
||||
"slic/ce_loss": 261.6252746582031,
|
||||
"slic/rank_loss": 91.23591613769531,
|
||||
"step": 260
|
||||
},
|
||||
{
|
||||
"epoch": 0.5654450261780105,
|
||||
"grad_norm": 3326.556884765625,
|
||||
"learning_rate": 2.381045210440644e-07,
|
||||
"logits/chosen": -0.5840014219284058,
|
||||
"logits/rejected": -0.5949230790138245,
|
||||
"logps/chosen": -254.5701446533203,
|
||||
"logps/rejected": -237.69515991210938,
|
||||
"loss": 1350.7662,
|
||||
"rewards/accuracies": 0.48046875,
|
||||
"rewards/chosen": -254.5701446533203,
|
||||
"rewards/margins": -16.87495231628418,
|
||||
"rewards/rejected": -237.69515991210938,
|
||||
"slic/ce_loss": 254.5701446533203,
|
||||
"slic/rank_loss": 83.12141418457031,
|
||||
"step": 270
|
||||
},
|
||||
{
|
||||
"epoch": 0.5863874345549738,
|
||||
"grad_norm": 3088.087646484375,
|
||||
"learning_rate": 2.1986582993616925e-07,
|
||||
"logits/chosen": -0.5930246114730835,
|
||||
"logits/rejected": -0.5939750075340271,
|
||||
"logps/chosen": -253.7728271484375,
|
||||
"logps/rejected": -241.112548828125,
|
||||
"loss": 1342.3729,
|
||||
"rewards/accuracies": 0.48906248807907104,
|
||||
"rewards/chosen": -253.7728271484375,
|
||||
"rewards/margins": -12.66028118133545,
|
||||
"rewards/rejected": -241.112548828125,
|
||||
"slic/ce_loss": 253.7728271484375,
|
||||
"slic/rank_loss": 81.8204116821289,
|
||||
"step": 280
|
||||
},
|
||||
{
|
||||
"epoch": 0.6073298429319371,
|
||||
"grad_norm": 3437.228759765625,
|
||||
"learning_rate": 2.0178866775369774e-07,
|
||||
"logits/chosen": -0.5804582238197327,
|
||||
"logits/rejected": -0.5853986740112305,
|
||||
"logps/chosen": -268.73333740234375,
|
||||
"logps/rejected": -250.84423828125,
|
||||
"loss": 1440.4078,
|
||||
"rewards/accuracies": 0.48046875,
|
||||
"rewards/chosen": -268.73333740234375,
|
||||
"rewards/margins": -17.889144897460938,
|
||||
"rewards/rejected": -250.84423828125,
|
||||
"slic/ce_loss": 268.73333740234375,
|
||||
"slic/rank_loss": 91.36856842041016,
|
||||
"step": 290
|
||||
},
|
||||
{
|
||||
"epoch": 0.6282722513089005,
|
||||
"grad_norm": 3455.057861328125,
|
||||
"learning_rate": 1.839699339491937e-07,
|
||||
"logits/chosen": -0.590076744556427,
|
||||
"logits/rejected": -0.5911703109741211,
|
||||
"logps/chosen": -255.82763671875,
|
||||
"logps/rejected": -247.8886260986328,
|
||||
"loss": 1343.457,
|
||||
"rewards/accuracies": 0.507031261920929,
|
||||
"rewards/chosen": -255.82763671875,
|
||||
"rewards/margins": -7.939053535461426,
|
||||
"rewards/rejected": -247.8886260986328,
|
||||
"slic/ce_loss": 255.82763671875,
|
||||
"slic/rank_loss": 80.03659057617188,
|
||||
"step": 300
|
||||
},
|
||||
{
|
||||
"epoch": 0.6492146596858639,
|
||||
"grad_norm": 3349.302734375,
|
||||
"learning_rate": 1.6650514271527465e-07,
|
||||
"logits/chosen": -0.5793516635894775,
|
||||
"logits/rejected": -0.5949603319168091,
|
||||
"logps/chosen": -258.7000427246094,
|
||||
"logps/rejected": -238.9949188232422,
|
||||
"loss": 1385.5395,
|
||||
"rewards/accuracies": 0.4749999940395355,
|
||||
"rewards/chosen": -258.7000427246094,
|
||||
"rewards/margins": -19.705102920532227,
|
||||
"rewards/rejected": -238.9949188232422,
|
||||
"slic/ce_loss": 258.7000427246094,
|
||||
"slic/rank_loss": 87.68482971191406,
|
||||
"step": 310
|
||||
},
|
||||
{
|
||||
"epoch": 0.6701570680628273,
|
||||
"grad_norm": 3584.829345703125,
|
||||
"learning_rate": 1.4948791099758052e-07,
|
||||
"logits/chosen": -0.6042912006378174,
|
||||
"logits/rejected": -0.602425217628479,
|
||||
"logps/chosen": -263.06732177734375,
|
||||
"logps/rejected": -240.22341918945312,
|
||||
"loss": 1407.0932,
|
||||
"rewards/accuracies": 0.4867187440395355,
|
||||
"rewards/chosen": -263.06732177734375,
|
||||
"rewards/margins": -22.843936920166016,
|
||||
"rewards/rejected": -240.22341918945312,
|
||||
"slic/ce_loss": 263.06732177734375,
|
||||
"slic/rank_loss": 88.70594787597656,
|
||||
"step": 320
|
||||
},
|
||||
{
|
||||
"epoch": 0.6910994764397905,
|
||||
"grad_norm": 3191.279296875,
|
||||
"learning_rate": 1.3300945667758012e-07,
|
||||
"logits/chosen": -0.5990831255912781,
|
||||
"logits/rejected": -0.5979740023612976,
|
||||
"logps/chosen": -257.674072265625,
|
||||
"logps/rejected": -244.77487182617188,
|
||||
"loss": 1362.2186,
|
||||
"rewards/accuracies": 0.504687488079071,
|
||||
"rewards/chosen": -257.674072265625,
|
||||
"rewards/margins": -12.899203300476074,
|
||||
"rewards/rejected": -244.77487182617188,
|
||||
"slic/ce_loss": 257.674072265625,
|
||||
"slic/rank_loss": 82.88054656982422,
|
||||
"step": 330
|
||||
},
|
||||
{
|
||||
"epoch": 0.7120418848167539,
|
||||
"grad_norm": 2967.82421875,
|
||||
"learning_rate": 1.1715810961514072e-07,
|
||||
"logits/chosen": -0.6025289297103882,
|
||||
"logits/rejected": -0.5973036885261536,
|
||||
"logps/chosen": -251.06820678710938,
|
||||
"logps/rejected": -234.9936065673828,
|
||||
"loss": 1341.9119,
|
||||
"rewards/accuracies": 0.4859375059604645,
|
||||
"rewards/chosen": -251.06820678710938,
|
||||
"rewards/margins": -16.074626922607422,
|
||||
"rewards/rejected": -234.9936065673828,
|
||||
"slic/ce_loss": 251.06820678710938,
|
||||
"slic/rank_loss": 84.40972900390625,
|
||||
"step": 340
|
||||
},
|
||||
{
|
||||
"epoch": 0.7329842931937173,
|
||||
"grad_norm": 4420.080078125,
|
||||
"learning_rate": 1.0201883817182949e-07,
|
||||
"logits/chosen": -0.5965454578399658,
|
||||
"logits/rejected": -0.6024629473686218,
|
||||
"logps/chosen": -265.9053955078125,
|
||||
"logps/rejected": -244.15786743164062,
|
||||
"loss": 1410.9908,
|
||||
"rewards/accuracies": 0.4546875059604645,
|
||||
"rewards/chosen": -265.9053955078125,
|
||||
"rewards/margins": -21.74753189086914,
|
||||
"rewards/rejected": -244.15786743164062,
|
||||
"slic/ce_loss": 265.9053955078125,
|
||||
"slic/rank_loss": 86.84226989746094,
|
||||
"step": 350
|
||||
},
|
||||
{
|
||||
"epoch": 0.7539267015706806,
|
||||
"grad_norm": 3473.728515625,
|
||||
"learning_rate": 8.76727937529367e-08,
|
||||
"logits/chosen": -0.5978257060050964,
|
||||
"logits/rejected": -0.6121166944503784,
|
||||
"logps/chosen": -250.9609832763672,
|
||||
"logps/rejected": -233.37783813476562,
|
||||
"loss": 1331.2162,
|
||||
"rewards/accuracies": 0.500781238079071,
|
||||
"rewards/chosen": -250.9609832763672,
|
||||
"rewards/margins": -17.583179473876953,
|
||||
"rewards/rejected": -233.37783813476562,
|
||||
"slic/ce_loss": 250.9609832763672,
|
||||
"slic/rank_loss": 81.8430404663086,
|
||||
"step": 360
|
||||
},
|
||||
{
|
||||
"epoch": 0.774869109947644,
|
||||
"grad_norm": 3074.599853515625,
|
||||
"learning_rate": 7.419687580962222e-08,
|
||||
"logits/chosen": -0.5856522917747498,
|
||||
"logits/rejected": -0.5918489098548889,
|
||||
"logps/chosen": -257.7669677734375,
|
||||
"logps/rejected": -240.96908569335938,
|
||||
"loss": 1375.6232,
|
||||
"rewards/accuracies": 0.49609375,
|
||||
"rewards/chosen": -257.7669677734375,
|
||||
"rewards/margins": -16.797880172729492,
|
||||
"rewards/rejected": -240.96908569335938,
|
||||
"slic/ce_loss": 257.7669677734375,
|
||||
"slic/rank_loss": 86.13883972167969,
|
||||
"step": 370
|
||||
},
|
||||
{
|
||||
"epoch": 0.7958115183246073,
|
||||
"grad_norm": 3393.130859375,
|
||||
"learning_rate": 6.166331963291519e-08,
|
||||
"logits/chosen": -0.5945131182670593,
|
||||
"logits/rejected": -0.5999140739440918,
|
||||
"logps/chosen": -275.88726806640625,
|
||||
"logps/rejected": -248.7072296142578,
|
||||
"loss": 1463.2406,
|
||||
"rewards/accuracies": 0.47187501192092896,
|
||||
"rewards/chosen": -275.88726806640625,
|
||||
"rewards/margins": -27.180028915405273,
|
||||
"rewards/rejected": -248.7072296142578,
|
||||
"slic/ce_loss": 275.88726806640625,
|
||||
"slic/rank_loss": 89.92294311523438,
|
||||
"step": 380
|
||||
},
|
||||
{
|
||||
"epoch": 0.8167539267015707,
|
||||
"grad_norm": 3126.19189453125,
|
||||
"learning_rate": 5.013930914912476e-08,
|
||||
"logits/chosen": -0.5959919095039368,
|
||||
"logits/rejected": -0.596234917640686,
|
||||
"logps/chosen": -252.68051147460938,
|
||||
"logps/rejected": -245.86611938476562,
|
||||
"loss": 1317.6791,
|
||||
"rewards/accuracies": 0.514843761920929,
|
||||
"rewards/chosen": -252.68051147460938,
|
||||
"rewards/margins": -6.814435005187988,
|
||||
"rewards/rejected": -245.86611938476562,
|
||||
"slic/ce_loss": 252.68051147460938,
|
||||
"slic/rank_loss": 76.7392807006836,
|
||||
"step": 390
|
||||
},
|
||||
{
|
||||
"epoch": 0.837696335078534,
|
||||
"grad_norm": 3121.567626953125,
|
||||
"learning_rate": 3.968661679220467e-08,
|
||||
"logits/chosen": -0.5884934663772583,
|
||||
"logits/rejected": -0.5969708561897278,
|
||||
"logps/chosen": -265.9261779785156,
|
||||
"logps/rejected": -247.37106323242188,
|
||||
"loss": 1395.7896,
|
||||
"rewards/accuracies": 0.46406251192092896,
|
||||
"rewards/chosen": -265.9261779785156,
|
||||
"rewards/margins": -18.555116653442383,
|
||||
"rewards/rejected": -247.37106323242188,
|
||||
"slic/ce_loss": 265.9261779785156,
|
||||
"slic/rank_loss": 83.02122497558594,
|
||||
"step": 400
|
||||
},
|
||||
{
|
||||
"epoch": 0.837696335078534,
|
||||
"eval_logits/chosen": -0.5998061299324036,
|
||||
"eval_logits/rejected": -0.6055657267570496,
|
||||
"eval_logps/chosen": -260.8731994628906,
|
||||
"eval_logps/rejected": -246.98541259765625,
|
||||
"eval_loss": 341.91009521484375,
|
||||
"eval_rewards/accuracies": 0.49193549156188965,
|
||||
"eval_rewards/chosen": -260.8731994628906,
|
||||
"eval_rewards/margins": -13.887796401977539,
|
||||
"eval_rewards/rejected": -246.98541259765625,
|
||||
"eval_runtime": 23.8106,
|
||||
"eval_samples_per_second": 83.996,
|
||||
"eval_slic/ce_loss": 260.8731994628906,
|
||||
"eval_slic/rank_loss": 81.03688049316406,
|
||||
"eval_steps_per_second": 2.646,
|
||||
"step": 400
|
||||
},
|
||||
{
|
||||
"epoch": 0.8586387434554974,
|
||||
"grad_norm": 2985.495849609375,
|
||||
"learning_rate": 3.036127238347164e-08,
|
||||
"logits/chosen": -0.6078251600265503,
|
||||
"logits/rejected": -0.6096236109733582,
|
||||
"logps/chosen": -263.81060791015625,
|
||||
"logps/rejected": -248.4971923828125,
|
||||
"loss": 1406.5264,
|
||||
"rewards/accuracies": 0.48906248807907104,
|
||||
"rewards/chosen": -263.81060791015625,
|
||||
"rewards/margins": -15.313403129577637,
|
||||
"rewards/rejected": -248.4971923828125,
|
||||
"slic/ce_loss": 263.81060791015625,
|
||||
"slic/rank_loss": 87.82101440429688,
|
||||
"step": 410
|
||||
},
|
||||
{
|
||||
"epoch": 0.8795811518324608,
|
||||
"grad_norm": 3290.060791015625,
|
||||
"learning_rate": 2.2213262793589482e-08,
|
||||
"logits/chosen": -0.6001953482627869,
|
||||
"logits/rejected": -0.6040524244308472,
|
||||
"logps/chosen": -262.39385986328125,
|
||||
"logps/rejected": -246.28182983398438,
|
||||
"loss": 1379.8988,
|
||||
"rewards/accuracies": 0.484375,
|
||||
"rewards/chosen": -262.39385986328125,
|
||||
"rewards/margins": -16.112010955810547,
|
||||
"rewards/rejected": -246.28182983398438,
|
||||
"slic/ce_loss": 262.39385986328125,
|
||||
"slic/rank_loss": 82.58085632324219,
|
||||
"step": 420
|
||||
},
|
||||
{
|
||||
"epoch": 0.900523560209424,
|
||||
"grad_norm": 3281.253662109375,
|
||||
"learning_rate": 1.5286263996730026e-08,
|
||||
"logits/chosen": -0.5900901556015015,
|
||||
"logits/rejected": -0.6069929003715515,
|
||||
"logps/chosen": -264.75018310546875,
|
||||
"logps/rejected": -240.9290313720703,
|
||||
"loss": 1414.56,
|
||||
"rewards/accuracies": 0.484375,
|
||||
"rewards/chosen": -264.75018310546875,
|
||||
"rewards/margins": -23.821134567260742,
|
||||
"rewards/rejected": -240.9290313720703,
|
||||
"slic/ce_loss": 264.75018310546875,
|
||||
"slic/rank_loss": 88.88983154296875,
|
||||
"step": 430
|
||||
},
|
||||
{
|
||||
"epoch": 0.9214659685863874,
|
||||
"grad_norm": 3399.375,
|
||||
"learning_rate": 9.617406953185136e-09,
|
||||
"logits/chosen": -0.5953913927078247,
|
||||
"logits/rejected": -0.6039350628852844,
|
||||
"logps/chosen": -252.87686157226562,
|
||||
"logps/rejected": -241.09890747070312,
|
||||
"loss": 1344.8779,
|
||||
"rewards/accuracies": 0.503125011920929,
|
||||
"rewards/chosen": -252.87686157226562,
|
||||
"rewards/margins": -11.77791976928711,
|
||||
"rewards/rejected": -241.09890747070312,
|
||||
"slic/ce_loss": 252.87686157226562,
|
||||
"slic/rank_loss": 83.34260559082031,
|
||||
"step": 440
|
||||
},
|
||||
{
|
||||
"epoch": 0.9424083769633508,
|
||||
"grad_norm": 3320.21875,
|
||||
"learning_rate": 5.2370785753763356e-09,
|
||||
"logits/chosen": -0.5980680584907532,
|
||||
"logits/rejected": -0.6059701442718506,
|
||||
"logps/chosen": -259.66754150390625,
|
||||
"logps/rejected": -238.99984741210938,
|
||||
"loss": 1389.4178,
|
||||
"rewards/accuracies": 0.4859375059604645,
|
||||
"rewards/chosen": -259.66754150390625,
|
||||
"rewards/margins": -20.667722702026367,
|
||||
"rewards/rejected": -238.99984741210938,
|
||||
"slic/ce_loss": 259.66754150390625,
|
||||
"slic/rank_loss": 87.68685150146484,
|
||||
"step": 450
|
||||
},
|
||||
{
|
||||
"epoch": 0.9633507853403142,
|
||||
"grad_norm": 3521.455322265625,
|
||||
"learning_rate": 2.168758844148272e-09,
|
||||
"logits/chosen": -0.5775882005691528,
|
||||
"logits/rejected": -0.5878804922103882,
|
||||
"logps/chosen": -275.74432373046875,
|
||||
"logps/rejected": -259.4674072265625,
|
||||
"loss": 1447.9953,
|
||||
"rewards/accuracies": 0.4867187440395355,
|
||||
"rewards/chosen": -275.74432373046875,
|
||||
"rewards/margins": -16.27688217163086,
|
||||
"rewards/rejected": -259.4674072265625,
|
||||
"slic/ce_loss": 275.74432373046875,
|
||||
"slic/rank_loss": 86.25456237792969,
|
||||
"step": 460
|
||||
},
|
||||
{
|
||||
"epoch": 0.9842931937172775,
|
||||
"grad_norm": 3612.229736328125,
|
||||
"learning_rate": 4.288949484559934e-10,
|
||||
"logits/chosen": -0.5883210897445679,
|
||||
"logits/rejected": -0.5901591181755066,
|
||||
"logps/chosen": -265.39093017578125,
|
||||
"logps/rejected": -239.3176727294922,
|
||||
"loss": 1424.0908,
|
||||
"rewards/accuracies": 0.4867187440395355,
|
||||
"rewards/chosen": -265.39093017578125,
|
||||
"rewards/margins": -26.073261260986328,
|
||||
"rewards/rejected": -239.3176727294922,
|
||||
"slic/ce_loss": 265.39093017578125,
|
||||
"slic/rank_loss": 90.63172912597656,
|
||||
"step": 470
|
||||
},
|
||||
{
|
||||
"epoch": 0.9989528795811519,
|
||||
"step": 477,
|
||||
"total_flos": 0.0,
|
||||
"train_loss": 1398.719245958628,
|
||||
"train_runtime": 2677.8284,
|
||||
"train_samples_per_second": 22.83,
|
||||
"train_steps_per_second": 0.178
|
||||
}
|
||||
],
|
||||
"logging_steps": 10,
|
||||
"max_steps": 477,
|
||||
"num_input_tokens_seen": 0,
|
||||
"num_train_epochs": 1,
|
||||
"save_steps": 500,
|
||||
"stateful_callbacks": {
|
||||
"TrainerControl": {
|
||||
"args": {
|
||||
"should_epoch_stop": false,
|
||||
"should_evaluate": false,
|
||||
"should_log": false,
|
||||
"should_save": false,
|
||||
"should_training_stop": false
|
||||
},
|
||||
"attributes": {}
|
||||
}
|
||||
},
|
||||
"total_flos": 0.0,
|
||||
"train_batch_size": 4,
|
||||
"trial_name": null,
|
||||
"trial_params": null
|
||||
}
|
||||
Reference in New Issue
Block a user