HoangHa/Pensez-v0.1-e5
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: HoangHa/Pensez-v0.1-e5
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-04-24 16:55:06 +08:00
commit 1f4723fe30
17 changed files with 152530 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

170
README.md Normal file
View File

@@ -0,0 +1,170 @@
---
base_model:
- Qwen/Qwen2.5-7B-Instruct
datasets:
- HoangHa/Pensez-v0.1
language:
- en
- fr
library_name: transformers
license: apache-2.0
pipeline_tag: text-generation
---
<div align="center">
# Pensez: Less Data, Better Reasoning – Rethinking French LLM
[**About**](#about) | [**How to Run Locally**](#run-locally) | [**Models and Datasets**](#models-and-datasets) | [**Benchmarks**](#benchmarks) | [**Training Details**](#training-details)
![image/png](https://cdn-uploads.huggingface.co/production/uploads/630a5ef0e81e1dea2cedcec0/-QnXjQ3SRkGgYpYK9wvff.png)
</div>
## About
Paper: [Pensez: Less Data, Better Reasoning - Rethinking French LLM](https://huggingface.co/papers/2503.13661)
Pensez is a bilingual (French-English) reasoning model designed to maximize efficiency with significantly reduced training data. The model leverages a curated dataset focusing on daily reasoning tasks and scientific questions to enhance performance.
Key strategies for improved reasoning:
- **Concise reasoning** for simple tasks to prevent overthinking.
- **Extended reasoning** for complex domains like mathematics, coding, and science.
- **Special tokens (`<think>...</think>`)** to explicitly guide the model’s reasoning process.
These optimizations result in superior reasoning capabilities while maintaining robust general understanding compared to models like [DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B).
## Models and Datasets
### Model Versions
Pensez is built upon [Qwen 2.5 Instruct 7B](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) and trained over five epochs.
| Model | Backbone | Size | Download Link |
|---------------|----------------------------------------|------|---------------|
| Pensez-v0.1-e1 | Qwen2.5-7B-Instruct | 7B | [🤗 Pensez-v0.1-e1](https://huggingface.co/HoangHa/Pensez-v0.1-e1) |
| Pensez-v0.1-e2 | Qwen2.5-7B-Instruct | 7B | [🤗 Pensez-v0.1-e2](https://huggingface.co/HoangHa/Pensez-v0.1-e2) |
| Pensez-v0.1-e3 | Qwen2.5-7B-Instruct | 7B | [🤗 Pensez-v0.1-e3](https://huggingface.co/HoangHa/Pensez-v0.1-e3) |
| Pensez-v0.1-e4 | Qwen2.5-7B-Instruct | 7B | [🤗 Pensez-v0.1-e4](https://huggingface.co/HoangHa/Pensez-v0.1-e4) |
| Pensez-v0.1-e5 | Qwen2.5-7B-Instruct | 7B | [🤗 Pensez-v0.1-e5](https://huggingface.co/HoangHa/Pensez-v0.1-e5) |
### Dataset
Pensez was trained on the hand-curated [Pensez v0.1](https://huggingface.co/datasets/HoangHa/Pensez-v0.1) dataset containing 2,000 samples (1,000 French, 1,000 English).
| Dataset | Description | Size | Link |
|--------------|----------------------|-------|-------|
| Pensez v0.1 | SFT Training Dataset | 2K samples | [🤗 Pensez v0.1](https://huggingface.co/datasets/HoangHa/Pensez-v0.1) |
## Benchmarks
Pensez was evaluated on French-specific benchmarks, demonstrating strong reasoning ability and improved task-specific performance:
| Benchmark | Pensez-v0.1-e5 | DeepSeek-R1-Distill-Qwen-7B | Qwen2.5-7B-Instruct |
|-----------|---------------|-----------------------------|----------------------|
| Math-hard (fr) | 0.3458 | 0.3403 | 0.2253 |
| MMLU (fr) | 0.5766 | 0.4961 | 0.6612 |
| BoolQA (fr) | 0.9157 | 0.7079 | 0.9382 |
| Trivia (en) | 0.4421 | 0.2711 | 0.5316 |
| HellaSwag (en) | 0.5050 | 0.3540 | 0.5258 |
**Key Observations:**
- Pensez outperforms Qwen2.5-7B-Instruct in reasoning tasks.
- Comparable to DeepSeek-R1-Distill-Qwen-7B in reasoning while maintaining strong understanding.
- Reduced degradation in knowledge-based tasks.
<details>
<summary>Click for detailed benchmark results</summary>
| Tasks | Pensez v0.1 e1 | Pensez v0.1 e2 | Pensez v0.1 e3 | Pensez v0.1 e4 | Pensez v0.1 e5 | Qwen 7B instruct | R1 distil |
|------------------------------------------------|---------------|---------------|---------------|---------------|---------------|-----------------|-----------|
| leaderboard_math_hard_fr | 0.0918 | 0.2547 | 0.2783 | 0.3035 | 0.3458 | 0.2253 | 0.3403 |
| leaderboard_math_algebra_hard_fr | 0.1029 | 0.3914 | 0.3971 | 0.5114 | 0.5000 | 0.4229 | 0.4771 |
| leaderboard_math_counting_and_prob_hard_fr | 0.0765 | 0.1378 | 0.1939 | 0.2041 | 0.2398 | 0.1224 | 0.2347 |
| leaderboard_math_geometry_hard_fr | 0.0388 | 0.1019 | 0.1408 | 0.1359 | 0.1748 | 0.1019 | 0.2330 |
| leaderboard_math_num_theory_hard_fr | 0.1198 | 0.2581 | 0.3502 | 0.3548 | 0.4332 | 0.3180 | 0.3963 |
| leaderboard_math_prealgebra_hard_fr | 0.1681 | 0.4425 | 0.4690 | 0.4956 | 0.5841 | 0.3274 | 0.4867 |
| leaderboard_math_precalculus_hard_fr | 0.0357 | 0.0714 | 0.1190 | 0.1190 | 0.1429 | 0.0595 | 0.2143 |
| leaderboard_mmlu_fr | 0.3806 | 0.3329 | - | - | 0.5766 | 0.6612 | 0.4961 |
| french_bench_arc_challenge | 0.5047 | 0.5021 | 0.4919 | 0.4859 | 0.4842 | 0.5518 | 0.3447 |
| french_bench_boolqa | 0.9326 | 0.9326 | 0.9326 | 0.9270 | 0.9157 | 0.9382 | 0.7079 |
| french_bench_fquadv2 | 0.4325 | 0.4400 | 0.4412 | 0.4375 | 0.4387 | 0.4800 | 0.2988 |
| french_bench_hellaswag | 0.4970 | 0.5055 | 0.5092 | 0.5058 | 0.5050 | 0.5258 | 0.3540 |
| french_bench_trivia | 0.4763 | 0.4763 | 0.4553 | 0.4395 | 0.4421 | 0.5316 | 0.2711 |
</details>
## Run Locally
You can run Pensez using Hugging Face’s `transformers` library:
```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_path = "HoangHa/Pensez-v0.1-e5"
# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(
model_path, torch_dtype=torch.float16, device_map="auto"
)
# Example input
messages = [{"role": "user", "content": "Bonjour!"}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors='pt').to("cuda")
generated_ids = model.generate(input_ids, max_new_tokens=2500, temperature=0.8, repetition_penalty=1.1, do_sample=True, eos_token_id=tokenizer.eos_token_id)
response = tokenizer.decode(generated_ids[0], skip_special_tokens=True, clean_up_tokenization_space=True)
print(f"Réponse: {response}")
```
## Training Details
Pensez was trained with:
- **Packing Inputs Without Cross-Contamination Attention** ([Reference](https://github.com/MeetKai/functionary/tree/main/functionary/train/packing))
- **Liger Kernel** ([Reference](https://github.com/linkedin/Liger-Kernel))
- **DeepSpeed 3** ([Reference](https://github.com/deepspeedai/DeepSpeed))
- **NEFTune Noise** ([Reference](https://arxiv.org/abs/2310.05914)) for robustness.
| **Parameter** | **Value** |
|--------------|----------|
| Epochs | 5 |
| Global Batch Size | 200 |
| Learning Rate | 1e-5 |
| Scheduler | Cosine |
| Optimizer | AdamW |
| Warmup Ratio | 0.05 |
| Weight Decay | 0.01 |
| Max Sequence Length | 16,384 |
More details: [Training Config](https://huggingface.co/HoangHa/Pensez-v0.1-e5/blob/main/fr_full_sft.yaml) | Loss curves: [Wandb](https://wandb.ai/hahuyhoanghhh41/llamafactory?nw=nwuserhahuyhoanghhh41)
## Citation
```bibtex
@misc{ha2025pensezreasoningfrenchllm,
title={Pensez: Less Data, Better Reasoning – Rethinking French LLM},
author={Ha Huy Hoang},
year={2025},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2503.13661},
}
```
## Acknowledgement
- [llama-factory](https://github.com/hiyouga/LLaMA-Factory)
- [Deepseek R1](https://github.com/deepseek-ai/DeepSeek-R1)
- [Qwen 2.5](https://github.com/QwenLM/Qwen2.5)
- [NEFTune Noise](https://arxiv.org/abs/2310.05914)
- [Packing Inputs Without Cross-Contamination Attention](https://github.com/MeetKai/functionary/tree/main/functionary/train/packing)
- [Liger Kernel](https://github.com/linkedin/Liger-Kernel)
- [Deepspeed](https://github.com/deepspeedai/DeepSpeed)
- [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness)
- [Hyperbolic](https://hyperbolic.xyz/)
- [Modal](https://modal.com/)
```

26
added_tokens.json Normal file
View File

@@ -0,0 +1,26 @@
{
"</think>": 151666,
"</tool_call>": 151658,
"<think>": 151665,
"<tool_call>": 151657,
"<|box_end|>": 151649,
"<|box_start|>": 151648,
"<|endoftext|>": 151643,
"<|file_sep|>": 151664,
"<|fim_middle|>": 151660,
"<|fim_pad|>": 151662,
"<|fim_prefix|>": 151659,
"<|fim_suffix|>": 151661,
"<|im_end|>": 151645,
"<|im_start|>": 151644,
"<|image_pad|>": 151655,
"<|object_ref_end|>": 151647,
"<|object_ref_start|>": 151646,
"<|quad_end|>": 151651,
"<|quad_start|>": 151650,
"<|repo_name|>": 151663,
"<|video_pad|>": 151656,
"<|vision_end|>": 151653,
"<|vision_pad|>": 151654,
"<|vision_start|>": 151652
}

29
config.json Normal file
View File

@@ -0,0 +1,29 @@
{
"_name_or_path": "./checkpoint-200",
"architectures": [
"Qwen2ForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 151643,
"eos_token_id": 151645,
"hidden_act": "silu",
"hidden_size": 3584,
"initializer_range": 0.02,
"intermediate_size": 18944,
"max_position_embeddings": 32768,
"max_window_layers": 28,
"model_type": "qwen2",
"num_attention_heads": 28,
"num_hidden_layers": 28,
"num_key_value_heads": 4,
"rms_norm_eps": 1e-06,
"rope_scaling": null,
"rope_theta": 1000000.0,
"sliding_window": null,
"tie_word_embeddings": false,
"torch_dtype": "bfloat16",
"transformers_version": "4.49.0",
"use_cache": false,
"use_sliding_window": false,
"vocab_size": 151667
}

48
fr_full_sft.yaml Normal file
View File

@@ -0,0 +1,48 @@
### model
model_name_or_path: HoangHa/Pensez-v0.1-init
trust_remote_code: true
enable_liger_kernel: true
use_unsloth_gc: true
### method
stage: sft
do_train: true
finetuning_type: full
deepspeed: examples/deepspeed/ds_z3_offload_config.json # choices: [ds_z0_config.json, ds_z2_config.json, ds_z3_config.json]
### dataset
dataset: pensez
template: qwen
cutoff_len: 16384
# max_samples: 1000
overwrite_cache: true
preprocessing_num_workers: 16
neat_packing: true
### output
output_dir: saves/fr-8b/full/sft
logging_steps: 1
# save_steps: 500
save_strategy: "epoch"
plot_loss: true
overwrite_output_dir: true
### train
per_device_train_batch_size: 2
gradient_accumulation_steps: 1
learning_rate: 1.0e-5
num_train_epochs: 5.0
lr_scheduler_type: cosine
warmup_ratio: 0.05
bf16: true
ddp_timeout: 180000000
neftune_noise_alpha: 5
### eval
# val_size: 0.1
# per_device_eval_batch_size: 1
# eval_strategy: steps
# eval_steps: 500
report_to: wandb
run_name: fr-pensez # optional

14
generation_config.json Normal file
View File

@@ -0,0 +1,14 @@
{
"bos_token_id": 151643,
"do_sample": true,
"eos_token_id": [
151645,
151643
],
"pad_token_id": 151643,
"repetition_penalty": 1.05,
"temperature": 0.7,
"top_k": 20,
"top_p": 0.8,
"transformers_version": "4.49.0"
}

151388
merges.txt Normal file
View File

File diff suppressed because it is too large Load Diff

3
model-00001-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:433552f273d5e9e5ea7d55cef3f8fd61bd3c28bbbdd8cb319df782aa91ef1614
size 4874815080

3
model-00002-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b764aad3f86f487823f38ef5ff0e6521f34ad07b53306f8811f4618e49ca4c7f
size 4932751008

3
model-00003-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7ee2c0a59057450306798f94e90e590696d63db9c765799cfa454a9cb1a02224
size 4330865200

3
model-00004-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4263467a93396e2b26032e82776276f3cf7ebfc14b9c3302c2ee0dbd1ba99376
size 1087149184

346
model.safetensors.index.json Normal file
View File

@@ -0,0 +1,346 @@
{
"metadata": {
"total_size": 15225541632
},
"weight_map": {
"lm_head.weight": "model-00004-of-00004.safetensors",
"model.embed_tokens.weight": "model-00001-of-00004.safetensors",
"model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.16.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.17.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.17.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.17.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.17.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.17.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.18.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.18.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.18.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.18.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.18.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.18.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.19.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.19.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.19.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.19.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.19.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.20.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.20.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.20.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.20.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.20.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.21.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.25.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.26.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.26.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.26.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.26.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.26.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.27.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.27.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.27.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.27.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.27.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.27.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.27.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.27.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.27.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.27.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.27.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.27.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.6.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.6.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.6.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.6.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.6.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.7.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.7.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.7.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.7.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.7.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.8.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.8.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.8.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.8.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.8.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.8.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.8.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.8.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.8.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.8.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.8.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.8.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.9.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.norm.weight": "model-00003-of-00004.safetensors"
}
}

31
special_tokens_map.json Normal file
View File

@@ -0,0 +1,31 @@
{
"additional_special_tokens": [
"<|im_start|>",
"<|im_end|>",
"<|object_ref_start|>",
"<|object_ref_end|>",
"<|box_start|>",
"<|box_end|>",
"<|quad_start|>",
"<|quad_end|>",
"<|vision_start|>",
"<|vision_end|>",
"<|vision_pad|>",
"<|image_pad|>",
"<|video_pad|>"
],
"eos_token": {
"content": "<|im_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

3
tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6fb3aae6d45fa4a15c9546a0bb628f161cea617ed2bbc3ffbdf61f9f372c3deb
size 11422265

225
tokenizer_config.json Normal file
View File

@@ -0,0 +1,225 @@
{
"add_bos_token": false,
"add_prefix_space": false,
"added_tokens_decoder": {
"151643": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151644": {
"content": "<|im_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151645": {
"content": "<|im_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151646": {
"content": "<|object_ref_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151647": {
"content": "<|object_ref_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151648": {
"content": "<|box_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151649": {
"content": "<|box_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151650": {
"content": "<|quad_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151651": {
"content": "<|quad_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151652": {
"content": "<|vision_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151653": {
"content": "<|vision_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151654": {
"content": "<|vision_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151655": {
"content": "<|image_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151656": {
"content": "<|video_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151657": {
"content": "<tool_call>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151658": {
"content": "</tool_call>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151659": {
"content": "<|fim_prefix|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151660": {
"content": "<|fim_middle|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151661": {
"content": "<|fim_suffix|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151662": {
"content": "<|fim_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151663": {
"content": "<|repo_name|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151664": {
"content": "<|file_sep|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151665": {
"content": "<think>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false,
"special": false
},
"151666": {
"content": "</think>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false,
"special": false
}
},
"additional_special_tokens": [
"<|im_start|>",
"<|im_end|>",
"<|object_ref_start|>",
"<|object_ref_end|>",
"<|box_start|>",
"<|box_end|>",
"<|quad_start|>",
"<|quad_end|>",
"<|vision_start|>",
"<|vision_end|>",
"<|vision_pad|>",
"<|image_pad|>",
"<|video_pad|>"
],
"bos_token": null,
"chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0]['role'] == 'system' %}\n {{- messages[0]['content'] }}\n {%- else %}\n {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n {%- endif %}\n {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0]['role'] == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n {%- else %}\n {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {{- '<|im_start|>' + message.role }}\n {%- if message.content %}\n {{- '\\n' + message.content }}\n {%- endif %}\n {%- for tool_call in message.tool_calls %}\n {%- if tool_call.function is defined %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '\\n<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {{- tool_call.arguments | tojson }}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- message.content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
"clean_up_tokenization_spaces": false,
"eos_token": "<|im_end|>",
"errors": "replace",
"extra_special_tokens": {},
"model_max_length": 16384,
"pad_token": "<|endoftext|>",
"padding_side": "right",
"split_special_tokens": false,
"tokenizer_class": "Qwen2Tokenizer",
"unk_token": null
}

201
trainer_log.jsonl Normal file
View File

@@ -0,0 +1,201 @@
{"current_steps": 1, "total_steps": 200, "loss": 0.9829, "lr": 1.0000000000000002e-06, "epoch": 0.025, "percentage": 0.5, "elapsed_time": "0:00:50", "remaining_time": "2:46:59"}
{"current_steps": 2, "total_steps": 200, "loss": 0.96, "lr": 2.0000000000000003e-06, "epoch": 0.05, "percentage": 1.0, "elapsed_time": "0:01:18", "remaining_time": "2:10:06"}
{"current_steps": 3, "total_steps": 200, "loss": 1.0704, "lr": 3e-06, "epoch": 0.075, "percentage": 1.5, "elapsed_time": "0:01:43", "remaining_time": "1:52:55"}
{"current_steps": 4, "total_steps": 200, "loss": 1.0042, "lr": 4.000000000000001e-06, "epoch": 0.1, "percentage": 2.0, "elapsed_time": "0:02:07", "remaining_time": "1:44:04"}
{"current_steps": 5, "total_steps": 200, "loss": 0.968, "lr": 5e-06, "epoch": 0.125, "percentage": 2.5, "elapsed_time": "0:02:31", "remaining_time": "1:38:44"}
{"current_steps": 6, "total_steps": 200, "loss": 0.8274, "lr": 6e-06, "epoch": 0.15, "percentage": 3.0, "elapsed_time": "0:02:58", "remaining_time": "1:36:00"}
{"current_steps": 7, "total_steps": 200, "loss": 0.859, "lr": 7e-06, "epoch": 0.175, "percentage": 3.5, "elapsed_time": "0:03:22", "remaining_time": "1:33:01"}
{"current_steps": 8, "total_steps": 200, "loss": 0.8704, "lr": 8.000000000000001e-06, "epoch": 0.2, "percentage": 4.0, "elapsed_time": "0:03:46", "remaining_time": "1:30:41"}
{"current_steps": 9, "total_steps": 200, "loss": 0.915, "lr": 9e-06, "epoch": 0.225, "percentage": 4.5, "elapsed_time": "0:04:11", "remaining_time": "1:28:48"}
{"current_steps": 10, "total_steps": 200, "loss": 0.8445, "lr": 1e-05, "epoch": 0.25, "percentage": 5.0, "elapsed_time": "0:04:35", "remaining_time": "1:27:11"}
{"current_steps": 11, "total_steps": 200, "loss": 0.9181, "lr": 9.999316524962347e-06, "epoch": 0.275, "percentage": 5.5, "elapsed_time": "0:04:59", "remaining_time": "1:25:48"}
{"current_steps": 12, "total_steps": 200, "loss": 0.7154, "lr": 9.99726628670463e-06, "epoch": 0.3, "percentage": 6.0, "elapsed_time": "0:05:23", "remaining_time": "1:24:34"}
{"current_steps": 13, "total_steps": 200, "loss": 0.8721, "lr": 9.993849845741525e-06, "epoch": 0.325, "percentage": 6.5, "elapsed_time": "0:05:48", "remaining_time": "1:23:28"}
{"current_steps": 14, "total_steps": 200, "loss": 0.784, "lr": 9.989068136093873e-06, "epoch": 0.35, "percentage": 7.0, "elapsed_time": "0:06:13", "remaining_time": "1:22:44"}
{"current_steps": 15, "total_steps": 200, "loss": 0.8239, "lr": 9.98292246503335e-06, "epoch": 0.375, "percentage": 7.5, "elapsed_time": "0:06:38", "remaining_time": "1:21:49"}
{"current_steps": 16, "total_steps": 200, "loss": 0.7897, "lr": 9.975414512725058e-06, "epoch": 0.4, "percentage": 8.0, "elapsed_time": "0:07:02", "remaining_time": "1:20:56"}
{"current_steps": 17, "total_steps": 200, "loss": 0.7054, "lr": 9.966546331768192e-06, "epoch": 0.425, "percentage": 8.5, "elapsed_time": "0:07:26", "remaining_time": "1:20:07"}
{"current_steps": 18, "total_steps": 200, "loss": 0.6976, "lr": 9.956320346634877e-06, "epoch": 0.45, "percentage": 9.0, "elapsed_time": "0:07:52", "remaining_time": "1:19:41"}
{"current_steps": 19, "total_steps": 200, "loss": 0.7125, "lr": 9.944739353007344e-06, "epoch": 0.475, "percentage": 9.5, "elapsed_time": "0:08:17", "remaining_time": "1:18:57"}
{"current_steps": 20, "total_steps": 200, "loss": 0.7321, "lr": 9.931806517013612e-06, "epoch": 0.5, "percentage": 10.0, "elapsed_time": "0:08:41", "remaining_time": "1:18:13"}
{"current_steps": 21, "total_steps": 200, "loss": 0.6899, "lr": 9.917525374361913e-06, "epoch": 0.525, "percentage": 10.5, "elapsed_time": "0:09:05", "remaining_time": "1:17:31"}
{"current_steps": 22, "total_steps": 200, "loss": 0.6679, "lr": 9.901899829374048e-06, "epoch": 0.55, "percentage": 11.0, "elapsed_time": "0:09:30", "remaining_time": "1:16:51"}
{"current_steps": 23, "total_steps": 200, "loss": 0.6094, "lr": 9.884934153917998e-06, "epoch": 0.575, "percentage": 11.5, "elapsed_time": "0:09:54", "remaining_time": "1:16:13"}
{"current_steps": 24, "total_steps": 200, "loss": 0.827, "lr": 9.86663298624003e-06, "epoch": 0.6, "percentage": 12.0, "elapsed_time": "0:10:18", "remaining_time": "1:15:34"}
{"current_steps": 25, "total_steps": 200, "loss": 0.7592, "lr": 9.847001329696653e-06, "epoch": 0.625, "percentage": 12.5, "elapsed_time": "0:10:42", "remaining_time": "1:14:58"}
{"current_steps": 26, "total_steps": 200, "loss": 0.7081, "lr": 9.826044551386743e-06, "epoch": 0.65, "percentage": 13.0, "elapsed_time": "0:11:07", "remaining_time": "1:14:23"}
{"current_steps": 27, "total_steps": 200, "loss": 0.7622, "lr": 9.803768380684242e-06, "epoch": 0.675, "percentage": 13.5, "elapsed_time": "0:11:31", "remaining_time": "1:13:49"}
{"current_steps": 28, "total_steps": 200, "loss": 0.6869, "lr": 9.780178907671788e-06, "epoch": 0.7, "percentage": 14.0, "elapsed_time": "0:11:55", "remaining_time": "1:13:16"}
{"current_steps": 29, "total_steps": 200, "loss": 0.6229, "lr": 9.755282581475769e-06, "epoch": 0.725, "percentage": 14.5, "elapsed_time": "0:12:20", "remaining_time": "1:12:44"}
{"current_steps": 30, "total_steps": 200, "loss": 0.7262, "lr": 9.729086208503174e-06, "epoch": 0.75, "percentage": 15.0, "elapsed_time": "0:12:44", "remaining_time": "1:12:11"}
{"current_steps": 31, "total_steps": 200, "loss": 0.7781, "lr": 9.701596950580807e-06, "epoch": 0.775, "percentage": 15.5, "elapsed_time": "0:13:08", "remaining_time": "1:11:39"}
{"current_steps": 32, "total_steps": 200, "loss": 0.7153, "lr": 9.672822322997305e-06, "epoch": 0.8, "percentage": 16.0, "elapsed_time": "0:13:32", "remaining_time": "1:11:06"}
{"current_steps": 33, "total_steps": 200, "loss": 0.627, "lr": 9.642770192448537e-06, "epoch": 0.825, "percentage": 16.5, "elapsed_time": "0:13:57", "remaining_time": "1:10:35"}
{"current_steps": 34, "total_steps": 200, "loss": 0.6303, "lr": 9.611448774886925e-06, "epoch": 0.85, "percentage": 17.0, "elapsed_time": "0:14:21", "remaining_time": "1:10:05"}
{"current_steps": 35, "total_steps": 200, "loss": 0.6458, "lr": 9.578866633275289e-06, "epoch": 0.875, "percentage": 17.5, "elapsed_time": "0:14:45", "remaining_time": "1:09:34"}
{"current_steps": 36, "total_steps": 200, "loss": 0.5871, "lr": 9.545032675245814e-06, "epoch": 0.9, "percentage": 18.0, "elapsed_time": "0:15:09", "remaining_time": "1:09:04"}
{"current_steps": 37, "total_steps": 200, "loss": 0.6505, "lr": 9.509956150664796e-06, "epoch": 0.925, "percentage": 18.5, "elapsed_time": "0:15:34", "remaining_time": "1:08:35"}
{"current_steps": 38, "total_steps": 200, "loss": 0.6133, "lr": 9.473646649103819e-06, "epoch": 0.95, "percentage": 19.0, "elapsed_time": "0:15:58", "remaining_time": "1:08:05"}
{"current_steps": 39, "total_steps": 200, "loss": 0.7444, "lr": 9.43611409721806e-06, "epoch": 0.975, "percentage": 19.5, "elapsed_time": "0:16:22", "remaining_time": "1:07:37"}
{"current_steps": 40, "total_steps": 200, "loss": 0.5992, "lr": 9.397368756032445e-06, "epoch": 1.0, "percentage": 20.0, "elapsed_time": "0:16:36", "remaining_time": "1:06:26"}
{"current_steps": 41, "total_steps": 200, "loss": 0.5628, "lr": 9.357421218136387e-06, "epoch": 1.025, "percentage": 20.5, "elapsed_time": "0:17:50", "remaining_time": "1:09:09"}
{"current_steps": 42, "total_steps": 200, "loss": 0.5189, "lr": 9.31628240478787e-06, "epoch": 1.05, "percentage": 21.0, "elapsed_time": "0:18:14", "remaining_time": "1:08:38"}
{"current_steps": 43, "total_steps": 200, "loss": 0.6368, "lr": 9.273963562927695e-06, "epoch": 1.075, "percentage": 21.5, "elapsed_time": "0:18:40", "remaining_time": "1:08:12"}
{"current_steps": 44, "total_steps": 200, "loss": 0.6484, "lr": 9.230476262104678e-06, "epoch": 1.1, "percentage": 22.0, "elapsed_time": "0:19:09", "remaining_time": "1:07:55"}
{"current_steps": 45, "total_steps": 200, "loss": 0.548, "lr": 9.185832391312644e-06, "epoch": 1.125, "percentage": 22.5, "elapsed_time": "0:19:34", "remaining_time": "1:07:26"}
{"current_steps": 46, "total_steps": 200, "loss": 0.5877, "lr": 9.140044155740102e-06, "epoch": 1.15, "percentage": 23.0, "elapsed_time": "0:19:59", "remaining_time": "1:06:54"}
{"current_steps": 47, "total_steps": 200, "loss": 0.5464, "lr": 9.093124073433464e-06, "epoch": 1.175, "percentage": 23.5, "elapsed_time": "0:20:23", "remaining_time": "1:06:22"}
{"current_steps": 48, "total_steps": 200, "loss": 0.5666, "lr": 9.045084971874738e-06, "epoch": 1.2, "percentage": 24.0, "elapsed_time": "0:20:47", "remaining_time": "1:05:51"}
{"current_steps": 49, "total_steps": 200, "loss": 0.5671, "lr": 8.995939984474624e-06, "epoch": 1.225, "percentage": 24.5, "elapsed_time": "0:21:11", "remaining_time": "1:05:19"}
{"current_steps": 50, "total_steps": 200, "loss": 0.5979, "lr": 8.94570254698197e-06, "epoch": 1.25, "percentage": 25.0, "elapsed_time": "0:21:36", "remaining_time": "1:04:50"}
{"current_steps": 51, "total_steps": 200, "loss": 0.578, "lr": 8.894386393810563e-06, "epoch": 1.275, "percentage": 25.5, "elapsed_time": "0:22:01", "remaining_time": "1:04:20"}
{"current_steps": 52, "total_steps": 200, "loss": 0.6398, "lr": 8.842005554284296e-06, "epoch": 1.3, "percentage": 26.0, "elapsed_time": "0:22:25", "remaining_time": "1:03:49"}
{"current_steps": 53, "total_steps": 200, "loss": 0.5285, "lr": 8.788574348801676e-06, "epoch": 1.325, "percentage": 26.5, "elapsed_time": "0:22:49", "remaining_time": "1:03:18"}
{"current_steps": 54, "total_steps": 200, "loss": 0.5435, "lr": 8.734107384920771e-06, "epoch": 1.35, "percentage": 27.0, "elapsed_time": "0:23:13", "remaining_time": "1:02:48"}
{"current_steps": 55, "total_steps": 200, "loss": 0.5677, "lr": 8.67861955336566e-06, "epoch": 1.375, "percentage": 27.5, "elapsed_time": "0:23:37", "remaining_time": "1:02:18"}
{"current_steps": 56, "total_steps": 200, "loss": 0.6359, "lr": 8.622126023955446e-06, "epoch": 1.4, "percentage": 28.0, "elapsed_time": "0:24:02", "remaining_time": "1:01:48"}
{"current_steps": 57, "total_steps": 200, "loss": 0.5924, "lr": 8.564642241456986e-06, "epoch": 1.425, "percentage": 28.5, "elapsed_time": "0:24:26", "remaining_time": "1:01:19"}
{"current_steps": 58, "total_steps": 200, "loss": 0.5036, "lr": 8.506183921362443e-06, "epoch": 1.45, "percentage": 29.0, "elapsed_time": "0:24:50", "remaining_time": "1:00:49"}
{"current_steps": 59, "total_steps": 200, "loss": 0.511, "lr": 8.446767045592829e-06, "epoch": 1.475, "percentage": 29.5, "elapsed_time": "0:25:14", "remaining_time": "1:00:20"}
{"current_steps": 60, "total_steps": 200, "loss": 0.6183, "lr": 8.386407858128707e-06, "epoch": 1.5, "percentage": 30.0, "elapsed_time": "0:25:39", "remaining_time": "0:59:51"}
{"current_steps": 61, "total_steps": 200, "loss": 0.668, "lr": 8.325122860569241e-06, "epoch": 1.525, "percentage": 30.5, "elapsed_time": "0:26:03", "remaining_time": "0:59:22"}
{"current_steps": 62, "total_steps": 200, "loss": 0.6144, "lr": 8.262928807620843e-06, "epoch": 1.55, "percentage": 31.0, "elapsed_time": "0:26:27", "remaining_time": "0:58:53"}
{"current_steps": 63, "total_steps": 200, "loss": 0.4912, "lr": 8.199842702516584e-06, "epoch": 1.575, "percentage": 31.5, "elapsed_time": "0:26:51", "remaining_time": "0:58:24"}
{"current_steps": 64, "total_steps": 200, "loss": 0.58, "lr": 8.135881792367686e-06, "epoch": 1.6, "percentage": 32.0, "elapsed_time": "0:27:15", "remaining_time": "0:57:55"}
{"current_steps": 65, "total_steps": 200, "loss": 0.6149, "lr": 8.071063563448341e-06, "epoch": 1.625, "percentage": 32.5, "elapsed_time": "0:27:39", "remaining_time": "0:57:27"}
{"current_steps": 66, "total_steps": 200, "loss": 0.6245, "lr": 8.005405736415127e-06, "epoch": 1.65, "percentage": 33.0, "elapsed_time": "0:28:04", "remaining_time": "0:56:59"}
{"current_steps": 67, "total_steps": 200, "loss": 0.5635, "lr": 7.938926261462366e-06, "epoch": 1.675, "percentage": 33.5, "elapsed_time": "0:28:28", "remaining_time": "0:56:31"}
{"current_steps": 68, "total_steps": 200, "loss": 0.6097, "lr": 7.871643313414718e-06, "epoch": 1.7, "percentage": 34.0, "elapsed_time": "0:28:52", "remaining_time": "0:56:03"}
{"current_steps": 69, "total_steps": 200, "loss": 0.6498, "lr": 7.803575286758365e-06, "epoch": 1.725, "percentage": 34.5, "elapsed_time": "0:29:16", "remaining_time": "0:55:35"}
{"current_steps": 70, "total_steps": 200, "loss": 0.5906, "lr": 7.734740790612137e-06, "epoch": 1.75, "percentage": 35.0, "elapsed_time": "0:29:40", "remaining_time": "0:55:07"}
{"current_steps": 71, "total_steps": 200, "loss": 0.6059, "lr": 7.66515864363997e-06, "epoch": 1.775, "percentage": 35.5, "elapsed_time": "0:30:05", "remaining_time": "0:54:39"}
{"current_steps": 72, "total_steps": 200, "loss": 0.5358, "lr": 7.594847868906076e-06, "epoch": 1.8, "percentage": 36.0, "elapsed_time": "0:30:29", "remaining_time": "0:54:12"}
{"current_steps": 73, "total_steps": 200, "loss": 0.6218, "lr": 7.52382768867422e-06, "epoch": 1.825, "percentage": 36.5, "elapsed_time": "0:30:53", "remaining_time": "0:53:44"}
{"current_steps": 74, "total_steps": 200, "loss": 0.5731, "lr": 7.452117519152542e-06, "epoch": 1.85, "percentage": 37.0, "elapsed_time": "0:31:17", "remaining_time": "0:53:17"}
{"current_steps": 75, "total_steps": 200, "loss": 0.5855, "lr": 7.379736965185369e-06, "epoch": 1.875, "percentage": 37.5, "elapsed_time": "0:31:42", "remaining_time": "0:52:50"}
{"current_steps": 76, "total_steps": 200, "loss": 0.5999, "lr": 7.30670581489344e-06, "epoch": 1.9, "percentage": 38.0, "elapsed_time": "0:32:06", "remaining_time": "0:52:22"}
{"current_steps": 77, "total_steps": 200, "loss": 0.599, "lr": 7.233044034264034e-06, "epoch": 1.925, "percentage": 38.5, "elapsed_time": "0:32:30", "remaining_time": "0:51:55"}
{"current_steps": 78, "total_steps": 200, "loss": 0.5371, "lr": 7.158771761692464e-06, "epoch": 1.95, "percentage": 39.0, "elapsed_time": "0:32:54", "remaining_time": "0:51:28"}
{"current_steps": 79, "total_steps": 200, "loss": 0.5353, "lr": 7.083909302476453e-06, "epoch": 1.975, "percentage": 39.5, "elapsed_time": "0:33:18", "remaining_time": "0:51:01"}
{"current_steps": 80, "total_steps": 200, "loss": 0.5258, "lr": 7.008477123264849e-06, "epoch": 2.0, "percentage": 40.0, "elapsed_time": "0:33:32", "remaining_time": "0:50:18"}
{"current_steps": 81, "total_steps": 200, "loss": 0.4706, "lr": 6.932495846462262e-06, "epoch": 2.025, "percentage": 40.5, "elapsed_time": "0:34:45", "remaining_time": "0:51:03"}
{"current_steps": 82, "total_steps": 200, "loss": 0.5239, "lr": 6.855986244591104e-06, "epoch": 2.05, "percentage": 41.0, "elapsed_time": "0:35:09", "remaining_time": "0:50:35"}
{"current_steps": 83, "total_steps": 200, "loss": 0.559, "lr": 6.778969234612583e-06, "epoch": 2.075, "percentage": 41.5, "elapsed_time": "0:35:34", "remaining_time": "0:50:08"}
{"current_steps": 84, "total_steps": 200, "loss": 0.477, "lr": 6.701465872208216e-06, "epoch": 2.1, "percentage": 42.0, "elapsed_time": "0:36:02", "remaining_time": "0:49:46"}
{"current_steps": 85, "total_steps": 200, "loss": 0.4634, "lr": 6.6234973460234184e-06, "epoch": 2.125, "percentage": 42.5, "elapsed_time": "0:36:30", "remaining_time": "0:49:23"}
{"current_steps": 86, "total_steps": 200, "loss": 0.4589, "lr": 6.545084971874738e-06, "epoch": 2.15, "percentage": 43.0, "elapsed_time": "0:36:57", "remaining_time": "0:48:59"}
{"current_steps": 87, "total_steps": 200, "loss": 0.506, "lr": 6.466250186922325e-06, "epoch": 2.175, "percentage": 43.5, "elapsed_time": "0:37:21", "remaining_time": "0:48:31"}
{"current_steps": 88, "total_steps": 200, "loss": 0.5267, "lr": 6.387014543809224e-06, "epoch": 2.2, "percentage": 44.0, "elapsed_time": "0:37:46", "remaining_time": "0:48:04"}
{"current_steps": 89, "total_steps": 200, "loss": 0.4994, "lr": 6.3073997047691e-06, "epoch": 2.225, "percentage": 44.5, "elapsed_time": "0:38:11", "remaining_time": "0:47:37"}
{"current_steps": 90, "total_steps": 200, "loss": 0.5213, "lr": 6.227427435703997e-06, "epoch": 2.25, "percentage": 45.0, "elapsed_time": "0:38:35", "remaining_time": "0:47:09"}
{"current_steps": 91, "total_steps": 200, "loss": 0.4933, "lr": 6.147119600233758e-06, "epoch": 2.275, "percentage": 45.5, "elapsed_time": "0:38:59", "remaining_time": "0:46:42"}
{"current_steps": 92, "total_steps": 200, "loss": 0.4585, "lr": 6.066498153718735e-06, "epoch": 2.3, "percentage": 46.0, "elapsed_time": "0:39:23", "remaining_time": "0:46:15"}
{"current_steps": 93, "total_steps": 200, "loss": 0.4509, "lr": 5.985585137257401e-06, "epoch": 2.325, "percentage": 46.5, "elapsed_time": "0:39:48", "remaining_time": "0:45:47"}
{"current_steps": 94, "total_steps": 200, "loss": 0.4399, "lr": 5.904402671660551e-06, "epoch": 2.35, "percentage": 47.0, "elapsed_time": "0:40:11", "remaining_time": "0:45:19"}
{"current_steps": 95, "total_steps": 200, "loss": 0.5698, "lr": 5.82297295140367e-06, "epoch": 2.375, "percentage": 47.5, "elapsed_time": "0:40:36", "remaining_time": "0:44:52"}
{"current_steps": 96, "total_steps": 200, "loss": 0.49, "lr": 5.74131823855921e-06, "epoch": 2.4, "percentage": 48.0, "elapsed_time": "0:41:00", "remaining_time": "0:44:25"}
{"current_steps": 97, "total_steps": 200, "loss": 0.5098, "lr": 5.659460856710346e-06, "epoch": 2.425, "percentage": 48.5, "elapsed_time": "0:41:24", "remaining_time": "0:43:58"}
{"current_steps": 98, "total_steps": 200, "loss": 0.5692, "lr": 5.577423184847932e-06, "epoch": 2.45, "percentage": 49.0, "elapsed_time": "0:41:48", "remaining_time": "0:43:30"}
{"current_steps": 99, "total_steps": 200, "loss": 0.5643, "lr": 5.495227651252315e-06, "epoch": 2.475, "percentage": 49.5, "elapsed_time": "0:42:12", "remaining_time": "0:43:03"}
{"current_steps": 100, "total_steps": 200, "loss": 0.4395, "lr": 5.412896727361663e-06, "epoch": 2.5, "percentage": 50.0, "elapsed_time": "0:42:36", "remaining_time": "0:42:36"}
{"current_steps": 101, "total_steps": 200, "loss": 0.4573, "lr": 5.3304529216284974e-06, "epoch": 2.525, "percentage": 50.5, "elapsed_time": "0:43:01", "remaining_time": "0:42:10"}
{"current_steps": 102, "total_steps": 200, "loss": 0.445, "lr": 5.247918773366112e-06, "epoch": 2.55, "percentage": 51.0, "elapsed_time": "0:43:25", "remaining_time": "0:41:43"}
{"current_steps": 103, "total_steps": 200, "loss": 0.4861, "lr": 5.165316846586541e-06, "epoch": 2.575, "percentage": 51.5, "elapsed_time": "0:43:49", "remaining_time": "0:41:16"}
{"current_steps": 104, "total_steps": 200, "loss": 0.5349, "lr": 5.082669723831793e-06, "epoch": 2.6, "percentage": 52.0, "elapsed_time": "0:44:15", "remaining_time": "0:40:51"}
{"current_steps": 105, "total_steps": 200, "loss": 0.5738, "lr": 5e-06, "epoch": 2.625, "percentage": 52.5, "elapsed_time": "0:44:39", "remaining_time": "0:40:24"}
{"current_steps": 106, "total_steps": 200, "loss": 0.4086, "lr": 4.917330276168208e-06, "epoch": 2.65, "percentage": 53.0, "elapsed_time": "0:45:04", "remaining_time": "0:39:58"}
{"current_steps": 107, "total_steps": 200, "loss": 0.4416, "lr": 4.8346831534134595e-06, "epoch": 2.675, "percentage": 53.5, "elapsed_time": "0:45:28", "remaining_time": "0:39:31"}
{"current_steps": 108, "total_steps": 200, "loss": 0.4277, "lr": 4.752081226633888e-06, "epoch": 2.7, "percentage": 54.0, "elapsed_time": "0:45:52", "remaining_time": "0:39:04"}
{"current_steps": 109, "total_steps": 200, "loss": 0.5048, "lr": 4.669547078371503e-06, "epoch": 2.725, "percentage": 54.5, "elapsed_time": "0:46:16", "remaining_time": "0:38:38"}
{"current_steps": 110, "total_steps": 200, "loss": 0.443, "lr": 4.587103272638339e-06, "epoch": 2.75, "percentage": 55.0, "elapsed_time": "0:46:41", "remaining_time": "0:38:11"}
{"current_steps": 111, "total_steps": 200, "loss": 0.4801, "lr": 4.504772348747687e-06, "epoch": 2.775, "percentage": 55.5, "elapsed_time": "0:47:05", "remaining_time": "0:37:45"}
{"current_steps": 112, "total_steps": 200, "loss": 0.4787, "lr": 4.42257681515207e-06, "epoch": 2.8, "percentage": 56.0, "elapsed_time": "0:47:29", "remaining_time": "0:37:19"}
{"current_steps": 113, "total_steps": 200, "loss": 0.4195, "lr": 4.340539143289655e-06, "epoch": 2.825, "percentage": 56.5, "elapsed_time": "0:47:53", "remaining_time": "0:36:52"}
{"current_steps": 114, "total_steps": 200, "loss": 0.482, "lr": 4.25868176144079e-06, "epoch": 2.85, "percentage": 57.0, "elapsed_time": "0:48:18", "remaining_time": "0:36:26"}
{"current_steps": 115, "total_steps": 200, "loss": 0.4667, "lr": 4.17702704859633e-06, "epoch": 2.875, "percentage": 57.5, "elapsed_time": "0:48:42", "remaining_time": "0:36:00"}
{"current_steps": 116, "total_steps": 200, "loss": 0.4429, "lr": 4.0955973283394525e-06, "epoch": 2.9, "percentage": 58.0, "elapsed_time": "0:49:07", "remaining_time": "0:35:34"}
{"current_steps": 117, "total_steps": 200, "loss": 0.4499, "lr": 4.0144148627426e-06, "epoch": 2.925, "percentage": 58.5, "elapsed_time": "0:49:31", "remaining_time": "0:35:07"}
{"current_steps": 118, "total_steps": 200, "loss": 0.4487, "lr": 3.9335018462812664e-06, "epoch": 2.95, "percentage": 59.0, "elapsed_time": "0:49:55", "remaining_time": "0:34:41"}
{"current_steps": 119, "total_steps": 200, "loss": 0.5287, "lr": 3.852880399766243e-06, "epoch": 2.975, "percentage": 59.5, "elapsed_time": "0:50:20", "remaining_time": "0:34:15"}
{"current_steps": 120, "total_steps": 200, "loss": 0.5056, "lr": 3.7725725642960047e-06, "epoch": 3.0, "percentage": 60.0, "elapsed_time": "0:50:33", "remaining_time": "0:33:42"}
{"current_steps": 121, "total_steps": 200, "loss": 0.4617, "lr": 3.6926002952309015e-06, "epoch": 3.025, "percentage": 60.5, "elapsed_time": "0:51:46", "remaining_time": "0:33:48"}
{"current_steps": 122, "total_steps": 200, "loss": 0.5074, "lr": 3.6129854561907786e-06, "epoch": 3.05, "percentage": 61.0, "elapsed_time": "0:52:13", "remaining_time": "0:33:23"}
{"current_steps": 123, "total_steps": 200, "loss": 0.4298, "lr": 3.533749813077677e-06, "epoch": 3.075, "percentage": 61.5, "elapsed_time": "0:52:37", "remaining_time": "0:32:56"}
{"current_steps": 124, "total_steps": 200, "loss": 0.4162, "lr": 3.4549150281252635e-06, "epoch": 3.1, "percentage": 62.0, "elapsed_time": "0:53:01", "remaining_time": "0:32:30"}
{"current_steps": 125, "total_steps": 200, "loss": 0.3577, "lr": 3.3765026539765832e-06, "epoch": 3.125, "percentage": 62.5, "elapsed_time": "0:53:26", "remaining_time": "0:32:03"}
{"current_steps": 126, "total_steps": 200, "loss": 0.426, "lr": 3.298534127791785e-06, "epoch": 3.15, "percentage": 63.0, "elapsed_time": "0:53:50", "remaining_time": "0:31:37"}
{"current_steps": 127, "total_steps": 200, "loss": 0.4097, "lr": 3.2210307653874175e-06, "epoch": 3.175, "percentage": 63.5, "elapsed_time": "0:54:14", "remaining_time": "0:31:10"}
{"current_steps": 128, "total_steps": 200, "loss": 0.3412, "lr": 3.1440137554088957e-06, "epoch": 3.2, "percentage": 64.0, "elapsed_time": "0:54:39", "remaining_time": "0:30:44"}
{"current_steps": 129, "total_steps": 200, "loss": 0.3622, "lr": 3.06750415353774e-06, "epoch": 3.225, "percentage": 64.5, "elapsed_time": "0:55:03", "remaining_time": "0:30:18"}
{"current_steps": 130, "total_steps": 200, "loss": 0.3622, "lr": 2.991522876735154e-06, "epoch": 3.25, "percentage": 65.0, "elapsed_time": "0:55:27", "remaining_time": "0:29:51"}
{"current_steps": 131, "total_steps": 200, "loss": 0.4541, "lr": 2.9160906975235493e-06, "epoch": 3.275, "percentage": 65.5, "elapsed_time": "0:55:52", "remaining_time": "0:29:25"}
{"current_steps": 132, "total_steps": 200, "loss": 0.4208, "lr": 2.8412282383075362e-06, "epoch": 3.3, "percentage": 66.0, "elapsed_time": "0:56:16", "remaining_time": "0:28:59"}
{"current_steps": 133, "total_steps": 200, "loss": 0.494, "lr": 2.766955965735968e-06, "epoch": 3.325, "percentage": 66.5, "elapsed_time": "0:56:40", "remaining_time": "0:28:33"}
{"current_steps": 134, "total_steps": 200, "loss": 0.4109, "lr": 2.693294185106562e-06, "epoch": 3.35, "percentage": 67.0, "elapsed_time": "0:57:04", "remaining_time": "0:28:06"}
{"current_steps": 135, "total_steps": 200, "loss": 0.4869, "lr": 2.6202630348146323e-06, "epoch": 3.375, "percentage": 67.5, "elapsed_time": "0:57:28", "remaining_time": "0:27:40"}
{"current_steps": 136, "total_steps": 200, "loss": 0.4479, "lr": 2.5478824808474613e-06, "epoch": 3.4, "percentage": 68.0, "elapsed_time": "0:57:53", "remaining_time": "0:27:14"}
{"current_steps": 137, "total_steps": 200, "loss": 0.4011, "lr": 2.476172311325783e-06, "epoch": 3.425, "percentage": 68.5, "elapsed_time": "0:58:17", "remaining_time": "0:26:48"}
{"current_steps": 138, "total_steps": 200, "loss": 0.4558, "lr": 2.4051521310939258e-06, "epoch": 3.45, "percentage": 69.0, "elapsed_time": "0:58:41", "remaining_time": "0:26:22"}
{"current_steps": 139, "total_steps": 200, "loss": 0.3495, "lr": 2.3348413563600324e-06, "epoch": 3.475, "percentage": 69.5, "elapsed_time": "0:59:05", "remaining_time": "0:25:56"}
{"current_steps": 140, "total_steps": 200, "loss": 0.3597, "lr": 2.265259209387867e-06, "epoch": 3.5, "percentage": 70.0, "elapsed_time": "0:59:31", "remaining_time": "0:25:30"}
{"current_steps": 141, "total_steps": 200, "loss": 0.5161, "lr": 2.1964247132416373e-06, "epoch": 3.525, "percentage": 70.5, "elapsed_time": "0:59:55", "remaining_time": "0:25:04"}
{"current_steps": 142, "total_steps": 200, "loss": 0.3994, "lr": 2.1283566865852824e-06, "epoch": 3.55, "percentage": 71.0, "elapsed_time": "1:00:19", "remaining_time": "0:24:38"}
{"current_steps": 143, "total_steps": 200, "loss": 0.4615, "lr": 2.061073738537635e-06, "epoch": 3.575, "percentage": 71.5, "elapsed_time": "1:00:43", "remaining_time": "0:24:12"}
{"current_steps": 144, "total_steps": 200, "loss": 0.3435, "lr": 1.9945942635848745e-06, "epoch": 3.6, "percentage": 72.0, "elapsed_time": "1:01:08", "remaining_time": "0:23:46"}
{"current_steps": 145, "total_steps": 200, "loss": 0.4297, "lr": 1.928936436551661e-06, "epoch": 3.625, "percentage": 72.5, "elapsed_time": "1:01:32", "remaining_time": "0:23:20"}
{"current_steps": 146, "total_steps": 200, "loss": 0.504, "lr": 1.864118207632315e-06, "epoch": 3.65, "percentage": 73.0, "elapsed_time": "1:01:56", "remaining_time": "0:22:54"}
{"current_steps": 147, "total_steps": 200, "loss": 0.4454, "lr": 1.8001572974834169e-06, "epoch": 3.675, "percentage": 73.5, "elapsed_time": "1:02:20", "remaining_time": "0:22:28"}
{"current_steps": 148, "total_steps": 200, "loss": 0.4563, "lr": 1.7370711923791567e-06, "epoch": 3.7, "percentage": 74.0, "elapsed_time": "1:02:45", "remaining_time": "0:22:02"}
{"current_steps": 149, "total_steps": 200, "loss": 0.3498, "lr": 1.6748771394307584e-06, "epoch": 3.725, "percentage": 74.5, "elapsed_time": "1:03:09", "remaining_time": "0:21:37"}
{"current_steps": 150, "total_steps": 200, "loss": 0.3093, "lr": 1.6135921418712959e-06, "epoch": 3.75, "percentage": 75.0, "elapsed_time": "1:03:33", "remaining_time": "0:21:11"}
{"current_steps": 151, "total_steps": 200, "loss": 0.4388, "lr": 1.5532329544071712e-06, "epoch": 3.775, "percentage": 75.5, "elapsed_time": "1:03:58", "remaining_time": "0:20:45"}
{"current_steps": 152, "total_steps": 200, "loss": 0.4505, "lr": 1.4938160786375571e-06, "epoch": 3.8, "percentage": 76.0, "elapsed_time": "1:04:22", "remaining_time": "0:20:19"}
{"current_steps": 153, "total_steps": 200, "loss": 0.3916, "lr": 1.4353577585430152e-06, "epoch": 3.825, "percentage": 76.5, "elapsed_time": "1:04:46", "remaining_time": "0:19:53"}
{"current_steps": 154, "total_steps": 200, "loss": 0.3634, "lr": 1.3778739760445552e-06, "epoch": 3.85, "percentage": 77.0, "elapsed_time": "1:05:10", "remaining_time": "0:19:28"}
{"current_steps": 155, "total_steps": 200, "loss": 0.4373, "lr": 1.321380446634342e-06, "epoch": 3.875, "percentage": 77.5, "elapsed_time": "1:05:34", "remaining_time": "0:19:02"}
{"current_steps": 156, "total_steps": 200, "loss": 0.3199, "lr": 1.2658926150792321e-06, "epoch": 3.9, "percentage": 78.0, "elapsed_time": "1:05:59", "remaining_time": "0:18:36"}
{"current_steps": 157, "total_steps": 200, "loss": 0.4332, "lr": 1.2114256511983274e-06, "epoch": 3.925, "percentage": 78.5, "elapsed_time": "1:06:23", "remaining_time": "0:18:10"}
{"current_steps": 158, "total_steps": 200, "loss": 0.3273, "lr": 1.157994445715706e-06, "epoch": 3.95, "percentage": 79.0, "elapsed_time": "1:06:47", "remaining_time": "0:17:45"}
{"current_steps": 159, "total_steps": 200, "loss": 0.465, "lr": 1.1056136061894386e-06, "epoch": 3.975, "percentage": 79.5, "elapsed_time": "1:07:11", "remaining_time": "0:17:19"}
{"current_steps": 160, "total_steps": 200, "loss": 0.4268, "lr": 1.0542974530180327e-06, "epoch": 4.0, "percentage": 80.0, "elapsed_time": "1:07:25", "remaining_time": "0:16:51"}
{"current_steps": 161, "total_steps": 200, "loss": 0.5297, "lr": 1.0040600155253766e-06, "epoch": 4.025, "percentage": 80.5, "elapsed_time": "1:08:38", "remaining_time": "0:16:37"}
{"current_steps": 162, "total_steps": 200, "loss": 0.4357, "lr": 9.549150281252633e-07, "epoch": 4.05, "percentage": 81.0, "elapsed_time": "1:09:06", "remaining_time": "0:16:12"}
{"current_steps": 163, "total_steps": 200, "loss": 0.4264, "lr": 9.068759265665384e-07, "epoch": 4.075, "percentage": 81.5, "elapsed_time": "1:09:35", "remaining_time": "0:15:47"}
{"current_steps": 164, "total_steps": 200, "loss": 0.3369, "lr": 8.599558442598998e-07, "epoch": 4.1, "percentage": 82.0, "elapsed_time": "1:09:59", "remaining_time": "0:15:21"}
{"current_steps": 165, "total_steps": 200, "loss": 0.4443, "lr": 8.141676086873574e-07, "epoch": 4.125, "percentage": 82.5, "elapsed_time": "1:10:23", "remaining_time": "0:14:55"}
{"current_steps": 166, "total_steps": 200, "loss": 0.3939, "lr": 7.695237378953224e-07, "epoch": 4.15, "percentage": 83.0, "elapsed_time": "1:10:47", "remaining_time": "0:14:30"}
{"current_steps": 167, "total_steps": 200, "loss": 0.4136, "lr": 7.260364370723044e-07, "epoch": 4.175, "percentage": 83.5, "elapsed_time": "1:11:12", "remaining_time": "0:14:04"}
{"current_steps": 168, "total_steps": 200, "loss": 0.3993, "lr": 6.837175952121305e-07, "epoch": 4.2, "percentage": 84.0, "elapsed_time": "1:11:36", "remaining_time": "0:13:38"}
{"current_steps": 169, "total_steps": 200, "loss": 0.3765, "lr": 6.425787818636131e-07, "epoch": 4.225, "percentage": 84.5, "elapsed_time": "1:12:00", "remaining_time": "0:13:12"}
{"current_steps": 170, "total_steps": 200, "loss": 0.4238, "lr": 6.026312439675553e-07, "epoch": 4.25, "percentage": 85.0, "elapsed_time": "1:12:24", "remaining_time": "0:12:46"}
{"current_steps": 171, "total_steps": 200, "loss": 0.3437, "lr": 5.63885902781941e-07, "epoch": 4.275, "percentage": 85.5, "elapsed_time": "1:12:49", "remaining_time": "0:12:20"}
{"current_steps": 172, "total_steps": 200, "loss": 0.3187, "lr": 5.263533508961827e-07, "epoch": 4.3, "percentage": 86.0, "elapsed_time": "1:13:13", "remaining_time": "0:11:55"}
{"current_steps": 173, "total_steps": 200, "loss": 0.3651, "lr": 4.900438493352056e-07, "epoch": 4.325, "percentage": 86.5, "elapsed_time": "1:13:37", "remaining_time": "0:11:29"}
{"current_steps": 174, "total_steps": 200, "loss": 0.3571, "lr": 4.549673247541875e-07, "epoch": 4.35, "percentage": 87.0, "elapsed_time": "1:14:01", "remaining_time": "0:11:03"}
{"current_steps": 175, "total_steps": 200, "loss": 0.4918, "lr": 4.211333667247125e-07, "epoch": 4.375, "percentage": 87.5, "elapsed_time": "1:14:25", "remaining_time": "0:10:37"}
{"current_steps": 176, "total_steps": 200, "loss": 0.3434, "lr": 3.885512251130763e-07, "epoch": 4.4, "percentage": 88.0, "elapsed_time": "1:14:49", "remaining_time": "0:10:12"}
{"current_steps": 177, "total_steps": 200, "loss": 0.3569, "lr": 3.572298075514652e-07, "epoch": 4.425, "percentage": 88.5, "elapsed_time": "1:15:13", "remaining_time": "0:09:46"}
{"current_steps": 178, "total_steps": 200, "loss": 0.3857, "lr": 3.271776770026963e-07, "epoch": 4.45, "percentage": 89.0, "elapsed_time": "1:15:37", "remaining_time": "0:09:20"}
{"current_steps": 179, "total_steps": 200, "loss": 0.3654, "lr": 2.984030494191942e-07, "epoch": 4.475, "percentage": 89.5, "elapsed_time": "1:16:02", "remaining_time": "0:08:55"}
{"current_steps": 180, "total_steps": 200, "loss": 0.4247, "lr": 2.7091379149682683e-07, "epoch": 4.5, "percentage": 90.0, "elapsed_time": "1:16:26", "remaining_time": "0:08:29"}
{"current_steps": 181, "total_steps": 200, "loss": 0.4008, "lr": 2.447174185242324e-07, "epoch": 4.525, "percentage": 90.5, "elapsed_time": "1:16:50", "remaining_time": "0:08:03"}
{"current_steps": 182, "total_steps": 200, "loss": 0.4076, "lr": 2.198210923282118e-07, "epoch": 4.55, "percentage": 91.0, "elapsed_time": "1:17:14", "remaining_time": "0:07:38"}
{"current_steps": 183, "total_steps": 200, "loss": 0.3259, "lr": 1.962316193157593e-07, "epoch": 4.575, "percentage": 91.5, "elapsed_time": "1:17:39", "remaining_time": "0:07:12"}
{"current_steps": 184, "total_steps": 200, "loss": 0.3999, "lr": 1.7395544861325718e-07, "epoch": 4.6, "percentage": 92.0, "elapsed_time": "1:18:03", "remaining_time": "0:06:47"}
{"current_steps": 185, "total_steps": 200, "loss": 0.4021, "lr": 1.5299867030334815e-07, "epoch": 4.625, "percentage": 92.5, "elapsed_time": "1:18:27", "remaining_time": "0:06:21"}
{"current_steps": 186, "total_steps": 200, "loss": 0.4134, "lr": 1.333670137599713e-07, "epoch": 4.65, "percentage": 93.0, "elapsed_time": "1:18:51", "remaining_time": "0:05:56"}
{"current_steps": 187, "total_steps": 200, "loss": 0.3711, "lr": 1.1506584608200366e-07, "epoch": 4.675, "percentage": 93.5, "elapsed_time": "1:19:15", "remaining_time": "0:05:30"}
{"current_steps": 188, "total_steps": 200, "loss": 0.4008, "lr": 9.810017062595322e-08, "epoch": 4.7, "percentage": 94.0, "elapsed_time": "1:19:39", "remaining_time": "0:05:05"}
{"current_steps": 189, "total_steps": 200, "loss": 0.3939, "lr": 8.247462563808816e-08, "epoch": 4.725, "percentage": 94.5, "elapsed_time": "1:20:03", "remaining_time": "0:04:39"}
{"current_steps": 190, "total_steps": 200, "loss": 0.3757, "lr": 6.819348298638839e-08, "epoch": 4.75, "percentage": 95.0, "elapsed_time": "1:20:27", "remaining_time": "0:04:14"}
{"current_steps": 191, "total_steps": 200, "loss": 0.3446, "lr": 5.526064699265754e-08, "epoch": 4.775, "percentage": 95.5, "elapsed_time": "1:20:52", "remaining_time": "0:03:48"}
{"current_steps": 192, "total_steps": 200, "loss": 0.4063, "lr": 4.367965336512403e-08, "epoch": 4.8, "percentage": 96.0, "elapsed_time": "1:21:16", "remaining_time": "0:03:23"}
{"current_steps": 193, "total_steps": 200, "loss": 0.4499, "lr": 3.345366823180929e-08, "epoch": 4.825, "percentage": 96.5, "elapsed_time": "1:21:40", "remaining_time": "0:02:57"}
{"current_steps": 194, "total_steps": 200, "loss": 0.3175, "lr": 2.4585487274942922e-08, "epoch": 4.85, "percentage": 97.0, "elapsed_time": "1:22:04", "remaining_time": "0:02:32"}
{"current_steps": 195, "total_steps": 200, "loss": 0.2726, "lr": 1.7077534966650767e-08, "epoch": 4.875, "percentage": 97.5, "elapsed_time": "1:22:28", "remaining_time": "0:02:06"}
{"current_steps": 196, "total_steps": 200, "loss": 0.2746, "lr": 1.0931863906127327e-08, "epoch": 4.9, "percentage": 98.0, "elapsed_time": "1:22:53", "remaining_time": "0:01:41"}
{"current_steps": 197, "total_steps": 200, "loss": 0.3432, "lr": 6.150154258476315e-09, "epoch": 4.925, "percentage": 98.5, "elapsed_time": "1:23:17", "remaining_time": "0:01:16"}
{"current_steps": 198, "total_steps": 200, "loss": 0.3215, "lr": 2.7337132953697555e-09, "epoch": 4.95, "percentage": 99.0, "elapsed_time": "1:23:41", "remaining_time": "0:00:50"}
{"current_steps": 199, "total_steps": 200, "loss": 0.426, "lr": 6.834750376549793e-10, "epoch": 4.975, "percentage": 99.5, "elapsed_time": "1:24:05", "remaining_time": "0:00:25"}
{"current_steps": 200, "total_steps": 200, "loss": 0.4279, "lr": 0.0, "epoch": 5.0, "percentage": 100.0, "elapsed_time": "1:24:19", "remaining_time": "0:00:00"}
{"current_steps": 200, "total_steps": 200, "epoch": 5.0, "percentage": 100.0, "elapsed_time": "1:27:05", "remaining_time": "0:00:00"}

1
vocab.json Normal file
View File

File diff suppressed because one or more lines are too long