reaperdoesntknow/DualMind-TKD-Agentic-1.7B
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: reaperdoesntknow/DualMind-TKD-Agentic-1.7B
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-27 15:46:10 +08:00
commit dc8d283c8f
15 changed files with 1885 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

134
README.md Normal file
View File

@@ -0,0 +1,134 @@
---
language:
- en
library_name: transformers
pipeline_tag: text-generation
license: other
base_model:
- Qwen/Qwen3-1.7B
datasets:
- 0xZee/dataset-CoT-Advanced-Calculus-268
- NousResearch/hermes-function-calling-v1
tags:
- transformers
- qwen3
- text-generation
- conversational
- agentic
- function-calling
- tool-use
- knowledge-distillation
- topology-guided-distillation
- lora-merged
- mathematics
- disc
- dualmind
- convergent-intelligence
---
# DualMind TKD Agentic 1.7B
DualMind TKD Agentic 1.7B is a two-stage derivative of
`Qwen/Qwen3-1.7B`.
It combines topology-guided mathematical knowledge distillation with
assistant-masked agentic and function-calling specialization.
## Training lineage
### Stage 1: topology-guided knowledge distillation
- Student: `Qwen/Qwen3-1.7B`
- Teacher: `Qwen/Qwen3-8B`
- Dataset: `0xZee/dataset-CoT-Advanced-Calculus-268`
- Training scope: full-model fine-tuning
- Objective: supervised cross-entropy plus sparse top-k-and-tail
teacher distillation
- Structural signals: teacher distribution discrepancy, transition
topology, gap-energy diagnostics, and phase-weighted
Explore/Examine/Response supervision
Stage 1 was designed to transfer mathematical reasoning behavior while
placing additional learning pressure on derivation, verification, and
high-discrepancy reasoning transitions.
### Stage 2: agentic specialization
- Dataset: `NousResearch/hermes-function-calling-v1`
- Training scope: LoRA specialization followed by weight merging
- Supervision: assistant and tool-call outputs only
- Tool schemas, user messages, and tool-result messages were visible as
context but excluded from direct loss
- Mathematical replay was mixed into Stage 2 to reduce catastrophic
forgetting
The files in this repository contain the merged standalone model.
A separate PEFT adapter is not required for inference.
## Intended uses
- Mathematical and technical reasoning
- Structured function calling
- Tool-selection experiments
- Agent-loop research
- Continued supervised or preference optimization
- Research on topology-aware distillation
## Tool execution
This model can generate tool calls, but it does not execute external
tools by itself.
A surrounding runtime must:
1. Parse the model's tool call.
2. Execute the selected tool.
3. Append the tool result to the conversation.
4. Invoke the model again for its next action or final response.
## Loading
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "reaperdoesntknow/DualMind-TKD-Agentic-1.7B"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
repo_id,
torch_dtype="auto",
device_map="auto",
)
Evaluation status
No formal benchmark results are claimed in this release.
The training pipeline includes held-out loss monitoring and qualitative
generation smoke tests, but external mathematics, function-calling,
retention, and safety benchmarks should be run before production use.
Limitations
The source mathematics dataset is small.
Synthetic or generated reasoning traces may contain incorrect
derivations or contradictory final answers.
Function-call formatting does not guarantee correct tool selection.
External tool outputs must be treated as untrusted input.
Mathematical replay reduces forgetting but does not prove retention.
The model has not been established as safe for autonomous,
high-impact, medical, financial, or legal action.
License note
The Qwen3-1.7B base model uses Apache-2.0 licensing. The Hermes
function-calling dataset also declares Apache-2.0.
The advanced-calculus dataset did not expose a clear license declaration
when this model card was prepared. Consequently, this repository is
temporarily marked license: other. Confirm the source dataset's reuse
terms before assigning a more permissive license to this derivative.
Developer
Convergent Intelligence LLC / Reaper
Hugging Face: reaperdoesntknow
<!-- cix-keeper-ts:2026-07-26T13:15:33Z -->

89
chat_template.jinja Normal file
View File

@@ -0,0 +1,89 @@
{%- if tools %}
{{- '<|im_start|>system\n' }}
{%- if messages[0].role == 'system' %}
{{- messages[0].content + '\n\n' }}
{%- endif %}
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
{%- if messages[0].role == 'system' %}
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
{%- set index = (messages|length - 1) - loop.index0 %}
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
{%- set ns.multi_step_tool = false %}
{%- set ns.last_query_index = index %}
{%- endif %}
{%- endfor %}
{%- for message in messages %}
{%- if message.content is string %}
{%- set content = message.content %}
{%- else %}
{%- set content = '' %}
{%- endif %}
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
{%- elif message.role == "assistant" %}
{%- set reasoning_content = '' %}
{%- if message.reasoning_content is string %}
{%- set reasoning_content = message.reasoning_content %}
{%- else %}
{%- if '</think>' in content %}
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
{%- endif %}
{%- endif %}
{%- if loop.index0 > ns.last_query_index %}
{%- if loop.last or (not loop.last and reasoning_content) %}
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- if message.tool_calls %}
{%- for tool_call in message.tool_calls %}
{%- if (loop.first and content) or (not loop.first) %}
{{- '\n' }}
{%- endif %}
{%- if tool_call.function %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{{- '<tool_call>\n{"name": "' }}
{{- tool_call.name }}
{{- '", "arguments": ' }}
{%- if tool_call.arguments is string %}
{{- tool_call.arguments }}
{%- else %}
{{- tool_call.arguments | tojson }}
{%- endif %}
{{- '}\n</tool_call>' }}
{%- endfor %}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- elif message.role == "tool" %}
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' }}
{{- content }}
{{- '\n</tool_response>' }}
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- if enable_thinking is defined and enable_thinking is false %}
{{- '<think>\n\n</think>\n\n' }}
{%- endif %}
{%- endif %}

63
config.json Normal file
View File

@@ -0,0 +1,63 @@
{
"architectures": [
"Qwen3ForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": null,
"dtype": "float32",
"eos_token_id": 151645,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 2048,
"initializer_range": 0.02,
"intermediate_size": 6144,
"layer_types": [
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention"
],
"max_position_embeddings": 40960,
"max_window_layers": 28,
"model_type": "qwen3",
"num_attention_heads": 16,
"num_hidden_layers": 28,
"num_key_value_heads": 8,
"pad_token_id": 151643,
"rms_norm_eps": 1e-06,
"rope_parameters": {
"rope_theta": 1000000,
"rope_type": "default"
},
"sliding_window": null,
"tie_word_embeddings": true,
"transformers_version": "5.13.1",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 151936
}

12
generation_config.json Normal file
View File

@@ -0,0 +1,12 @@
{
"do_sample": true,
"eos_token_id": [
151645,
151643
],
"pad_token_id": 151643,
"temperature": 0.6,
"top_k": 20,
"top_p": 0.95,
"transformers_version": "5.13.1"
}

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c3f6b13ea36d75eca61d4217c4319e8e5b9251cbc95fec07c41f427b34449c67
size 6882335328

26
release_manifest.json Normal file
View File

@@ -0,0 +1,26 @@
{
"repository": "reaperdoesntknow/DualMind-TKD-Agentic-1.7B",
"released_at_utc": "2026-07-12T09:29:33.617799+00:00",
"architecture": "Qwen3ForCausalLM",
"base_model": "Qwen/Qwen3-1.7B",
"distillation_teacher": "Qwen/Qwen3-8B",
"stage_1": {
"method": "Topology-guided knowledge distillation",
"training_scope": "Full-model",
"dataset": "0xZee/dataset-CoT-Advanced-Calculus-268",
"output": "/content/DualMind100/best"
},
"stage_2": {
"method": "Assistant-masked agentic LoRA SFT",
"training_scope": "LoRA followed by merge",
"dataset": "NousResearch/hermes-function-calling-v1",
"retention_replay": "0xZee/dataset-CoT-Advanced-Calculus-268",
"output": "/content/DualMind100_agentic/merged"
},
"formal_benchmarks_available": false,
"training_receipts": [
"training_artifacts/DualMind100_config.json",
"training_artifacts/DualMind100_train_log.jsonl",
"training_artifacts/DualMind100_agentic_trainer_state.json"
]
}

3
tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
size 11422650

30
tokenizer_config.json Normal file
View File

@@ -0,0 +1,30 @@
{
"add_prefix_space": false,
"backend": "tokenizers",
"bos_token": null,
"clean_up_tokenization_spaces": false,
"eos_token": "<|im_end|>",
"errors": "replace",
"extra_special_tokens": [
"<|im_start|>",
"<|im_end|>",
"<|object_ref_start|>",
"<|object_ref_end|>",
"<|box_start|>",
"<|box_end|>",
"<|quad_start|>",
"<|quad_end|>",
"<|vision_start|>",
"<|vision_end|>",
"<|vision_pad|>",
"<|image_pad|>",
"<|video_pad|>"
],
"is_local": true,
"local_files_only": false,
"model_max_length": 131072,
"pad_token": "<|endoftext|>",
"split_special_tokens": false,
"tokenizer_class": "Qwen2Tokenizer",
"unk_token": null
}

801
training_artifacts/DualMind100_agentic_trainer_state.json Normal file
View File

@@ -0,0 +1,801 @@
{
"best_global_step": 335,
"best_metric": 0.2155456691980362,
"best_model_checkpoint": "/content/DualMind100_agentic/checkpoint-335",
"epoch": 5.0,
"eval_steps": 50,
"global_step": 335,
"is_hyper_param_search": false,
"is_local_process_zero": true,
"is_world_process_zero": true,
"log_history": [
{
"entropy": 0.16182193332642783,
"epoch": 0.07092198581560284,
"grad_norm": 0.5555469989776611,
"learning_rate": 5e-05,
"loss": 0.43506731986999514,
"mean_token_accuracy": 0.8946891106665135,
"num_tokens": 123430.0,
"step": 5
},
{
"entropy": 0.26333686951547863,
"epoch": 0.14184397163120568,
"grad_norm": 0.2982931435108185,
"learning_rate": 9.99862592554908e-05,
"loss": 0.3709733247756958,
"mean_token_accuracy": 0.898155527561903,
"num_tokens": 240929.0,
"step": 10
},
{
"entropy": 0.34238293119706215,
"epoch": 0.2127659574468085,
"grad_norm": 0.27323758602142334,
"learning_rate": 9.950612572673255e-05,
"loss": 0.3364511489868164,
"mean_token_accuracy": 0.906800027191639,
"num_tokens": 373968.0,
"step": 15
},
{
"entropy": 0.2675066887401044,
"epoch": 0.28368794326241137,
"grad_norm": 0.21929873526096344,
"learning_rate": 9.834648865751254e-05,
"loss": 0.2727418184280396,
"mean_token_accuracy": 0.9184959597885609,
"num_tokens": 494846.0,
"step": 20
},
{
"entropy": 0.2389053585124202,
"epoch": 0.3546099290780142,
"grad_norm": 0.19836418330669403,
"learning_rate": 9.652326481535435e-05,
"loss": 0.3025625705718994,
"mean_token_accuracy": 0.9214970305562019,
"num_tokens": 630306.0,
"step": 25
},
{
"entropy": 0.25811560889706014,
"epoch": 0.425531914893617,
"grad_norm": 0.19982172548770905,
"learning_rate": 9.406147912470143e-05,
"loss": 0.330981969833374,
"mean_token_accuracy": 0.9110702231526375,
"num_tokens": 758739.0,
"step": 30
},
{
"entropy": 0.24841026657959447,
"epoch": 0.49645390070921985,
"grad_norm": 0.16917800903320312,
"learning_rate": 9.099492118367123e-05,
"loss": 0.26140174865722654,
"mean_token_accuracy": 0.9228238843381404,
"num_tokens": 886920.0,
"step": 35
},
{
"entropy": 0.2569453418778721,
"epoch": 0.5673758865248227,
"grad_norm": 0.2460022270679474,
"learning_rate": 8.736568148000386e-05,
"loss": 0.28326830863952634,
"mean_token_accuracy": 0.9195088751614093,
"num_tokens": 1010775.0,
"step": 40
},
{
"entropy": 0.29926245249807837,
"epoch": 0.6382978723404256,
"grad_norm": 0.17467764019966125,
"learning_rate": 8.322357367194109e-05,
"loss": 0.3357896566390991,
"mean_token_accuracy": 0.9085973531007767,
"num_tokens": 1134373.0,
"step": 45
},
{
"entropy": 0.2604390908149071,
"epoch": 0.7092198581560284,
"grad_norm": 0.16242940723896027,
"learning_rate": 7.86254508636036e-05,
"loss": 0.2815252304077148,
"mean_token_accuracy": 0.9152372993528843,
"num_tokens": 1256155.0,
"step": 50
},
{
"epoch": 0.7092198581560284,
"eval_entropy": 0.22815538255963475,
"eval_loss": 0.23423044383525848,
"eval_mean_token_accuracy": 0.9300008341670036,
"eval_num_tokens": 1256155.0,
"eval_runtime": 5.3417,
"eval_samples_per_second": 8.986,
"eval_steps_per_second": 8.986,
"step": 50
},
{
"entropy": 0.2784213587117847,
"epoch": 0.7801418439716312,
"grad_norm": 0.1878730058670044,
"learning_rate": 7.363442525942826e-05,
"loss": 0.2845616340637207,
"mean_token_accuracy": 0.9107947684824467,
"num_tokens": 1378868.0,
"step": 55
},
{
"entropy": 0.29253384250914677,
"epoch": 0.851063829787234,
"grad_norm": 0.17584209144115448,
"learning_rate": 6.831900190841232e-05,
"loss": 0.31951193809509276,
"mean_token_accuracy": 0.9116400554776192,
"num_tokens": 1509648.0,
"step": 60
},
{
"entropy": 0.27282086764462293,
"epoch": 0.9219858156028369,
"grad_norm": 0.1782023161649704,
"learning_rate": 6.275213842808383e-05,
"loss": 0.28685288429260253,
"mean_token_accuracy": 0.9180420659482479,
"num_tokens": 1640179.0,
"step": 65
},
{
"entropy": 0.26427618057350627,
"epoch": 0.9929078014184397,
"grad_norm": 0.2283456027507782,
"learning_rate": 5.70102436140943e-05,
"loss": 0.29142701625823975,
"mean_token_accuracy": 0.9174253843724728,
"num_tokens": 1764592.0,
"step": 70
},
{
"entropy": 0.2524041742629682,
"epoch": 1.0567375886524824,
"grad_norm": 0.16881126165390015,
"learning_rate": 5.117212868016303e-05,
"loss": 0.276317834854126,
"mean_token_accuracy": 0.9248060741358333,
"num_tokens": 1874267.0,
"step": 75
},
{
"entropy": 0.29945559421321377,
"epoch": 1.127659574468085,
"grad_norm": 0.2186647355556488,
"learning_rate": 4.5317925523282464e-05,
"loss": 0.29511370658874514,
"mean_token_accuracy": 0.907706456631422,
"num_tokens": 1991985.0,
"step": 80
},
{
"entropy": 0.24618784408958164,
"epoch": 1.198581560283688,
"grad_norm": 0.15828007459640503,
"learning_rate": 3.952798686169279e-05,
"loss": 0.2731397867202759,
"mean_token_accuracy": 0.9240930497646331,
"num_tokens": 2118168.0,
"step": 85
},
{
"entropy": 0.2373207072203513,
"epoch": 1.2695035460992907,
"grad_norm": 0.20958004891872406,
"learning_rate": 3.388178334194232e-05,
"loss": 0.27325007915496824,
"mean_token_accuracy": 0.9249208129942417,
"num_tokens": 2240951.0,
"step": 90
},
{
"entropy": 0.22738294367445633,
"epoch": 1.3404255319148937,
"grad_norm": 0.19565178453922272,
"learning_rate": 2.8456812752951485e-05,
"loss": 0.25463979244232177,
"mean_token_accuracy": 0.9295149140059948,
"num_tokens": 2364856.0,
"step": 95
},
{
"entropy": 0.247153190523386,
"epoch": 1.4113475177304964,
"grad_norm": 0.1792992502450943,
"learning_rate": 2.3327536318821495e-05,
"loss": 0.2785398244857788,
"mean_token_accuracy": 0.9210605584084988,
"num_tokens": 2496977.0,
"step": 100
},
{
"epoch": 1.4113475177304964,
"eval_entropy": 0.2170733372719648,
"eval_loss": 0.22858719527721405,
"eval_mean_token_accuracy": 0.9310164228081703,
"eval_num_tokens": 2496977.0,
"eval_runtime": 5.44,
"eval_samples_per_second": 8.824,
"eval_steps_per_second": 8.824,
"step": 100
},
{
"entropy": 0.2818379683652893,
"epoch": 1.4822695035460993,
"grad_norm": 0.17297224700450897,
"learning_rate": 1.8564356670455767e-05,
"loss": 0.28323366641998293,
"mean_token_accuracy": 0.9150983460247517,
"num_tokens": 2625956.0,
"step": 105
},
{
"entropy": 0.25849783700541595,
"epoch": 1.5531914893617023,
"grad_norm": 0.17231720685958862,
"learning_rate": 1.4232651523993634e-05,
"loss": 0.26291708946228026,
"mean_token_accuracy": 0.9214990958571434,
"num_tokens": 2762411.0,
"step": 110
},
{
"entropy": 0.23584751789458097,
"epoch": 1.624113475177305,
"grad_norm": 0.19976992905139923,
"learning_rate": 1.0391876329443533e-05,
"loss": 0.24894940853118896,
"mean_token_accuracy": 0.9259149916470051,
"num_tokens": 2887514.0,
"step": 115
},
{
"entropy": 0.2430717541486956,
"epoch": 1.6950354609929077,
"grad_norm": 0.21702121198177338,
"learning_rate": 7.094748206242796e-06,
"loss": 0.2538208246231079,
"mean_token_accuracy": 0.9232472732663155,
"num_tokens": 3013243.0,
"step": 120
},
{
"entropy": 0.23513522319262847,
"epoch": 1.7659574468085106,
"grad_norm": 0.16074544191360474,
"learning_rate": 4.386522366755169e-06,
"loss": 0.22128725051879883,
"mean_token_accuracy": 0.9245993517339229,
"num_tokens": 3134144.0,
"step": 125
},
{
"entropy": 0.24327249150956048,
"epoch": 1.8368794326241136,
"grad_norm": 0.16139043867588043,
"learning_rate": 2.3043709592610485e-06,
"loss": 0.26496360301971433,
"mean_token_accuracy": 0.9262310199439525,
"num_tokens": 3260737.0,
"step": 130
},
{
"entropy": 0.23524304252350703,
"epoch": 1.9078014184397163,
"grad_norm": 0.16922682523727417,
"learning_rate": 8.768728562211947e-07,
"loss": 0.2534662961959839,
"mean_token_accuracy": 0.924710838496685,
"num_tokens": 3383649.0,
"step": 135
},
{
"entropy": 0.2577046745980624,
"epoch": 1.978723404255319,
"grad_norm": 0.18219797313213348,
"learning_rate": 1.236213907994943e-07,
"loss": 0.2727572679519653,
"mean_token_accuracy": 0.9200983986258506,
"num_tokens": 3511363.0,
"step": 140
},
{
"epoch": 2.0,
"eval_entropy": 0.21362574538215995,
"eval_loss": 0.22699420154094696,
"eval_mean_token_accuracy": 0.9318861390153567,
"eval_num_tokens": 3550882.0,
"eval_runtime": 5.4445,
"eval_samples_per_second": 8.816,
"eval_steps_per_second": 8.816,
"step": 142
},
{
"entropy": 0.6595380927125613,
"epoch": 2.1641791044776117,
"grad_norm": 0.4891488254070282,
"learning_rate": 6.554477648417657e-05,
"loss": 0.3567054271697998,
"mean_token_accuracy": 0.8980764100948969,
"num_tokens": 74137.0,
"step": 145
},
{
"entropy": 0.3680407987907529,
"epoch": 2.2388059701492535,
"grad_norm": 0.2562958300113678,
"learning_rate": 6.317935830345338e-05,
"loss": 0.2808666467666626,
"mean_token_accuracy": 0.9229060940444469,
"num_tokens": 201191.0,
"step": 150
},
{
"epoch": 2.2388059701492535,
"eval_entropy": 0.29109591808584,
"eval_loss": 0.25212642550468445,
"eval_mean_token_accuracy": 0.9247896313667298,
"eval_num_tokens": 201191.0,
"eval_runtime": 5.3589,
"eval_samples_per_second": 8.397,
"eval_steps_per_second": 8.397,
"step": 150
},
{
"entropy": 0.28654855480417607,
"epoch": 2.3134328358208958,
"grad_norm": 0.22995419800281525,
"learning_rate": 6.078178934582885e-05,
"loss": 0.2800537347793579,
"mean_token_accuracy": 0.9219874046742916,
"num_tokens": 319457.0,
"step": 155
},
{
"entropy": 0.2231586970272474,
"epoch": 2.388059701492537,
"grad_norm": 0.17113032937049866,
"learning_rate": 5.835791843207916e-05,
"loss": 0.2297281265258789,
"mean_token_accuracy": 0.9307773187756538,
"num_tokens": 441969.0,
"step": 160
},
{
"entropy": 0.20086921484908088,
"epoch": 2.4626865671641793,
"grad_norm": 0.13817362487316132,
"learning_rate": 5.5913658546068295e-05,
"loss": 0.20193655490875245,
"mean_token_accuracy": 0.9379530139267445,
"num_tokens": 564392.0,
"step": 165
},
{
"entropy": 0.2754073503077962,
"epoch": 2.5373134328358207,
"grad_norm": 0.19452188909053802,
"learning_rate": 5.34549724101439e-05,
"loss": 0.27802190780639646,
"mean_token_accuracy": 0.9231111846864224,
"num_tokens": 696793.0,
"step": 170
},
{
"entropy": 0.24629125613719224,
"epoch": 2.611940298507463,
"grad_norm": 0.1801047921180725,
"learning_rate": 5.0987857939197324e-05,
"loss": 0.24364988803863524,
"mean_token_accuracy": 0.9304406151175499,
"num_tokens": 832284.0,
"step": 175
},
{
"entropy": 0.24637198023265228,
"epoch": 2.6865671641791042,
"grad_norm": 0.15811710059642792,
"learning_rate": 4.851833360887201e-05,
"loss": 0.2485351085662842,
"mean_token_accuracy": 0.9251487389206886,
"num_tokens": 953807.0,
"step": 180
},
{
"entropy": 0.19791743826935998,
"epoch": 2.7611940298507465,
"grad_norm": 0.17821364104747772,
"learning_rate": 4.6052423773614404e-05,
"loss": 0.21749985218048096,
"mean_token_accuracy": 0.9387934006750583,
"num_tokens": 1074631.0,
"step": 185
},
{
"entropy": 0.23313339608721434,
"epoch": 2.835820895522388,
"grad_norm": 0.22559864819049835,
"learning_rate": 4.3596143970383664e-05,
"loss": 0.23709943294525146,
"mean_token_accuracy": 0.9306622236967087,
"num_tokens": 1203325.0,
"step": 190
},
{
"entropy": 0.26784485091629906,
"epoch": 2.91044776119403,
"grad_norm": 0.20925050973892212,
"learning_rate": 4.115548624387137e-05,
"loss": 0.2629614114761353,
"mean_token_accuracy": 0.9191216990351677,
"num_tokens": 1332870.0,
"step": 195
},
{
"entropy": 0.2317873791558668,
"epoch": 2.9850746268656714,
"grad_norm": 0.15967224538326263,
"learning_rate": 3.873640452903026e-05,
"loss": 0.23617920875549317,
"mean_token_accuracy": 0.931964573264122,
"num_tokens": 1450793.0,
"step": 200
},
{
"epoch": 2.9850746268656714,
"eval_entropy": 0.2280733192132579,
"eval_loss": 0.21912212669849396,
"eval_mean_token_accuracy": 0.9303096612294515,
"eval_num_tokens": 1450793.0,
"eval_runtime": 5.2231,
"eval_samples_per_second": 8.615,
"eval_steps_per_second": 8.615,
"step": 200
},
{
"entropy": 0.24150694359559566,
"epoch": 3.0597014925373136,
"grad_norm": 0.17629101872444153,
"learning_rate": 3.6344800126570844e-05,
"loss": 0.24918053150177003,
"mean_token_accuracy": 0.929844381660223,
"num_tokens": 1577936.0,
"step": 205
},
{
"entropy": 0.20565980894316455,
"epoch": 3.1343283582089554,
"grad_norm": 0.15850670635700226,
"learning_rate": 3.3986507306858125e-05,
"loss": 0.2212688684463501,
"mean_token_accuracy": 0.9367655150592327,
"num_tokens": 1709033.0,
"step": 210
},
{
"entropy": 0.21145587725332007,
"epoch": 3.208955223880597,
"grad_norm": 0.2352381944656372,
"learning_rate": 3.16672790773276e-05,
"loss": 0.22223784923553466,
"mean_token_accuracy": 0.9365130670368671,
"num_tokens": 1831296.0,
"step": 215
},
{
"entropy": 0.22000169460952748,
"epoch": 3.283582089552239,
"grad_norm": 0.1637473851442337,
"learning_rate": 2.9392773148140408e-05,
"loss": 0.22411210536956788,
"mean_token_accuracy": 0.9312412738800049,
"num_tokens": 1950727.0,
"step": 220
},
{
"entropy": 0.21870606549200602,
"epoch": 3.3582089552238807,
"grad_norm": 0.18144957721233368,
"learning_rate": 2.716853813031435e-05,
"loss": 0.21267392635345458,
"mean_token_accuracy": 0.9330131947994232,
"num_tokens": 2084554.0,
"step": 225
},
{
"entropy": 0.18131871867808513,
"epoch": 3.4328358208955225,
"grad_norm": 0.14299768209457397,
"learning_rate": 2.500000000000001e-05,
"loss": 0.19915605783462526,
"mean_token_accuracy": 0.9421401508152485,
"num_tokens": 2207986.0,
"step": 230
},
{
"entropy": 0.18844851908506827,
"epoch": 3.5074626865671643,
"grad_norm": 0.21421347558498383,
"learning_rate": 2.289244886192207e-05,
"loss": 0.18854159116744995,
"mean_token_accuracy": 0.9412525840103626,
"num_tokens": 2330341.0,
"step": 235
},
{
"entropy": 0.2162812261347426,
"epoch": 3.582089552238806,
"grad_norm": 0.19194981455802917,
"learning_rate": 2.0851026044276406e-05,
"loss": 0.2259037971496582,
"mean_token_accuracy": 0.931151507049799,
"num_tokens": 2454431.0,
"step": 240
},
{
"entropy": 0.20769512332044543,
"epoch": 3.656716417910448,
"grad_norm": 0.1685398668050766,
"learning_rate": 1.888071155656421e-05,
"loss": 0.2129582643508911,
"mean_token_accuracy": 0.934239100664854,
"num_tokens": 2578412.0,
"step": 245
},
{
"entropy": 0.2097844194737263,
"epoch": 3.7313432835820897,
"grad_norm": 0.1834094077348709,
"learning_rate": 1.6986311940960147e-05,
"loss": 0.21508474349975587,
"mean_token_accuracy": 0.9365896977484226,
"num_tokens": 2704058.0,
"step": 250
},
{
"epoch": 3.7313432835820897,
"eval_entropy": 0.21005761296384864,
"eval_loss": 0.21609170734882355,
"eval_mean_token_accuracy": 0.9321770442856683,
"eval_num_tokens": 2704058.0,
"eval_runtime": 5.3012,
"eval_samples_per_second": 8.489,
"eval_steps_per_second": 8.489,
"step": 250
},
{
"entropy": 0.22172395456582308,
"epoch": 3.8059701492537314,
"grad_norm": 0.16993354260921478,
"learning_rate": 1.5172448546850165e-05,
"loss": 0.2224128484725952,
"mean_token_accuracy": 0.9313701257109642,
"num_tokens": 2822117.0,
"step": 255
},
{
"entropy": 0.20137031929916702,
"epoch": 3.8805970149253732,
"grad_norm": 0.18573690950870514,
"learning_rate": 1.3443546257143624e-05,
"loss": 0.20644056797027588,
"mean_token_accuracy": 0.935058569163084,
"num_tokens": 2954055.0,
"step": 260
},
{
"entropy": 0.21610731023247354,
"epoch": 3.955223880597015,
"grad_norm": 0.18307460844516754,
"learning_rate": 1.1803822693861378e-05,
"loss": 0.21918556690216065,
"mean_token_accuracy": 0.9331639289855957,
"num_tokens": 3076367.0,
"step": 265
},
{
"entropy": 0.18582435508724301,
"epoch": 4.029850746268656,
"grad_norm": 0.17089995741844177,
"learning_rate": 1.0257277929332332e-05,
"loss": 0.19541841745376587,
"mean_token_accuracy": 0.9410657376050949,
"num_tokens": 3200217.0,
"step": 270
},
{
"entropy": 0.1961809309606906,
"epoch": 4.104477611940299,
"grad_norm": 0.19942045211791992,
"learning_rate": 8.80768472809842e-06,
"loss": 0.18296244144439697,
"mean_token_accuracy": 0.9419995754957199,
"num_tokens": 3325910.0,
"step": 275
},
{
"entropy": 0.17922255574376322,
"epoch": 4.17910447761194,
"grad_norm": 0.17642585933208466,
"learning_rate": 7.458579343331995e-06,
"loss": 0.1955660581588745,
"mean_token_accuracy": 0.9431271247565747,
"num_tokens": 3449422.0,
"step": 280
},
{
"entropy": 0.20833609464461916,
"epoch": 4.253731343283582,
"grad_norm": 0.18559329211711884,
"learning_rate": 6.213252890218163e-06,
"loss": 0.20198218822479247,
"mean_token_accuracy": 0.934450775384903,
"num_tokens": 3569595.0,
"step": 285
},
{
"entropy": 0.20185105923446828,
"epoch": 4.3283582089552235,
"grad_norm": 0.14244937896728516,
"learning_rate": 5.074743317346009e-06,
"loss": 0.20316836833953858,
"mean_token_accuracy": 0.9381563618779183,
"num_tokens": 3688537.0,
"step": 290
},
{
"entropy": 0.21535796255921014,
"epoch": 4.402985074626866,
"grad_norm": 0.17846742272377014,
"learning_rate": 4.045827995694834e-06,
"loss": 0.2133563280105591,
"mean_token_accuracy": 0.9316843308508396,
"num_tokens": 3819875.0,
"step": 295
},
{
"entropy": 0.17974913779762575,
"epoch": 4.477611940298507,
"grad_norm": 0.20783886313438416,
"learning_rate": 3.1290169432939553e-06,
"loss": 0.2002777338027954,
"mean_token_accuracy": 0.9437027871608734,
"num_tokens": 3951096.0,
"step": 300
},
{
"epoch": 4.477611940298507,
"eval_entropy": 0.20027935401433045,
"eval_loss": 0.2157754898071289,
"eval_mean_token_accuracy": 0.9326928456624349,
"eval_num_tokens": 3951096.0,
"eval_runtime": 5.1907,
"eval_samples_per_second": 8.669,
"eval_steps_per_second": 8.669,
"step": 300
},
{
"entropy": 0.1885046276351204,
"epoch": 4.552238805970149,
"grad_norm": 0.20049376785755157,
"learning_rate": 2.3265467020847866e-06,
"loss": 0.1992756724357605,
"mean_token_accuracy": 0.9404157467186451,
"num_tokens": 4073686.0,
"step": 305
},
{
"entropy": 0.20385300312773325,
"epoch": 4.6268656716417915,
"grad_norm": 0.22206291556358337,
"learning_rate": 1.6403748819221466e-06,
"loss": 0.21113824844360352,
"mean_token_accuracy": 0.937010295689106,
"num_tokens": 4198560.0,
"step": 310
},
{
"entropy": 0.20165168155217544,
"epoch": 4.701492537313433,
"grad_norm": 0.22487279772758484,
"learning_rate": 1.0721753850247984e-06,
"loss": 0.2212970495223999,
"mean_token_accuracy": 0.9365487344563007,
"num_tokens": 4320893.0,
"step": 315
},
{
"entropy": 0.2170650392654352,
"epoch": 4.776119402985074,
"grad_norm": 0.1543811559677124,
"learning_rate": 6.233343225249933e-07,
"loss": 0.22227189540863038,
"mean_token_accuracy": 0.9313660271465778,
"num_tokens": 4453160.0,
"step": 320
},
{
"entropy": 0.18664323054545093,
"epoch": 4.850746268656716,
"grad_norm": 0.20742292702198029,
"learning_rate": 2.9494663307847447e-07,
"loss": 0.2171410322189331,
"mean_token_accuracy": 0.939918453246355,
"num_tokens": 4580581.0,
"step": 325
},
{
"entropy": 0.21867067418352235,
"epoch": 4.925373134328359,
"grad_norm": 0.17955397069454193,
"learning_rate": 8.781341178393244e-08,
"loss": 0.2199192762374878,
"mean_token_accuracy": 0.9308036096394062,
"num_tokens": 4705647.0,
"step": 330
},
{
"entropy": 0.19878723044821528,
"epoch": 5.0,
"grad_norm": 0.20007997751235962,
"learning_rate": 2.4399559277132888e-09,
"loss": 0.19873000383377076,
"mean_token_accuracy": 0.9358412712812424,
"num_tokens": 4828163.0,
"step": 335
},
{
"epoch": 5.0,
"eval_entropy": 0.20022574655918612,
"eval_loss": 0.2155456691980362,
"eval_mean_token_accuracy": 0.9328206459681193,
"eval_num_tokens": 4828163.0,
"eval_runtime": 5.2104,
"eval_samples_per_second": 8.637,
"eval_steps_per_second": 8.637,
"step": 335
},
{
"epoch": 5.0,
"step": 335,
"total_flos": 7.27718510456832e+16,
"train_loss": 0.12883905314687472,
"train_runtime": 1744.2725,
"train_samples_per_second": 3.073,
"train_steps_per_second": 0.192
}
],
"logging_steps": 5,
"max_steps": 335,
"num_input_tokens_seen": 0,
"num_train_epochs": 5,
"save_steps": 50,
"stateful_callbacks": {
"TrainerControl": {
"args": {
"should_epoch_stop": false,
"should_evaluate": false,
"should_log": false,
"should_save": true,
"should_training_stop": true
},
"attributes": {}
}
},
"total_flos": 7.27718510456832e+16,
"train_batch_size": 1,
"trial_name": null,
"trial_params": null
}

107
training_artifacts/DualMind100_config.json Normal file
View File

@@ -0,0 +1,107 @@
{
"dataset_ids": [
"0xZee/dataset-CoT-Advanced-Calculus-268"
],
"dataset_split": "train",
"eval_fraction": 0.1,
"seed": 42,
"max_examples": null,
"question_fields": [
"question",
"instruction",
"problem",
"prompt",
"input"
],
"reasoning_fields": [
"CoT",
"cot",
"chain_of_thought",
"reasoning",
"analysis",
"rationale",
"solution"
],
"answer_fields": [
"response",
"final_answer",
"answer",
"output",
"completion"
],
"save_data_audit": true,
"data_audit_path": "./DualMind100/data_audit.jsonl",
"student_id": "Qwen/Qwen3-1.7B",
"teacher_id": "Qwen/Qwen3-8B",
"student_revision": null,
"teacher_revision": null,
"save_dir": "./DualMind100",
"resume_from": null,
"use_bf16": true,
"keep_student_fp32": true,
"gradient_checkpointing": true,
"gradient_checkpointing_use_reentrant": false,
"attention_implementation": "sdpa",
"compile_student": false,
"teacher_load_in_4bit": false,
"teacher_device_map": "auto",
"teacher_offload_folder": "./teacher_offload",
"teacher_max_memory_gpu": null,
"teacher_max_memory_cpu": null,
"use_8bit_optimizer": false,
"use_paged_optimizer": true,
"dual_mind_format": true,
"use_chat_template": true,
"system_prompt": "Solve the problem rigorously. Preserve assumptions, derivation, verification, and final result.",
"preserve_full_answer": true,
"window_size": 2048,
"min_overlap": 128,
"max_overlap": 384,
"teacher_top_k": 64,
"teacher_cache_dir": "./tkd_cache_v4",
"teacher_context_tokens": 512,
"vocab_logsumexp_chunk": 16384,
"cache_log_every": 20,
"min_expected_topk_mass": 0.85,
"batch_size": 2,
"grad_accum_steps": 4,
"epochs": 3,
"lr": 3e-06,
"min_lr": 3e-07,
"warmup_ratio": 0.05,
"weight_decay": 0.01,
"adam_beta1": 0.9,
"adam_beta2": 0.95,
"adam_eps": 1e-08,
"grad_clip": 1.0,
"temperature": 2.0,
"kd_target_alpha": 0.35,
"kd_start_frac": 0.1,
"kd_end_frac": 0.4,
"eval_kd_alpha": null,
"jump_threshold_sigma": 3.5,
"jump_threshold_quantile": 0.995,
"support_turnover_weight": 0.15,
"jump_amplifier": 1.33,
"topology_energy_window": 32,
"topology_pair_chunk": 512,
"dg_window": 3,
"gap_energy_alpha": 0.0,
"proof_weight_start": 2.0,
"proof_weight_end": 1.1,
"examine_extra_multiplier": 1.15,
"response_weight": 1.0,
"curriculum_enabled": true,
"curriculum_phases": 4,
"easy_first_fraction": 0.3,
"num_workers": 2,
"prefetch_factor": 2,
"log_every": 10,
"eval_every": 100,
"checkpoint_every_opt_steps": 250,
"save_every_epoch": true,
"save_best": true,
"best_metric": "ce",
"run_internal_checks": true,
"inference_max_new_tokens": 768
}

59
training_artifacts/DualMind100_train_log.jsonl Normal file
View File

@@ -0,0 +1,59 @@
{"epoch": 1, "batch_idx": 0, "global_step": 1, "opt_step": 0, "combined": 2.19950724, "ce": 2.19950724, "kd": 0.0, "gap_energy": 0.0, "kd_alpha": 0.0, "proof_weight": 1.99846939, "ce_num": 60.49145889, "ce_den": 27.50227737, "kd_num": 0.0, "kd_den": 1.0, "gap_num": 0.0, "gap_den": 1.0, "kd_tokens": 0.0, "teacher_topk_mass_num": 0.0, "student_topk_mass_num": 0.0, "mass_den": 1.0, "lr": 4.2857142857142857e-07, "difficulty": 1.7609130144119263, "memory": "alloc=14.0GiB reserved=18.9GiB peak=16.4GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 9, "global_step": 10, "opt_step": 2, "combined": 1.21293616, "ce": 1.21293616, "kd": 0.0, "gap_energy": 0.0, "kd_alpha": 0.0, "proof_weight": 1.98469388, "ce_num": 586.51037598, "ce_den": 483.54595947, "kd_num": 0.0, "kd_den": 1.0, "gap_num": 0.0, "gap_den": 1.0, "kd_tokens": 0.0, "teacher_topk_mass_num": 0.0, "student_topk_mass_num": 0.0, "mass_den": 1.0, "lr": 1.2857142857142856e-06, "difficulty": 6.203751564025879, "memory": "alloc=26.8GiB reserved=38.6GiB peak=35.6GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 19, "global_step": 20, "opt_step": 5, "combined": 1.07730448, "ce": 1.07730448, "kd": 0.0, "gap_energy": 0.0, "kd_alpha": 0.0, "proof_weight": 1.96938776, "ce_num": 1827.9753418, "ce_den": 1696.80480957, "kd_num": 0.0, "kd_den": 1.0, "gap_num": 0.0, "gap_den": 1.0, "kd_tokens": 0.0, "teacher_topk_mass_num": 0.0, "student_topk_mass_num": 0.0, "mass_den": 1.0, "lr": 2.571428571428571e-06, "difficulty": 7.312134742736816, "memory": "alloc=20.4GiB reserved=38.6GiB peak=35.6GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 29, "global_step": 30, "opt_step": 7, "combined": 0.97815299, "ce": 0.97815299, "kd": 0.0, "gap_energy": 0.0, "kd_alpha": 0.0, "proof_weight": 1.95408163, "ce_num": 2458.76733398, "ce_den": 2513.68383789, "kd_num": 0.0, "kd_den": 1.0, "gap_num": 0.0, "gap_den": 1.0, "kd_tokens": 0.0, "teacher_topk_mass_num": 0.0, "student_topk_mass_num": 0.0, "mass_den": 1.0, "lr": 2.999660117172384e-06, "difficulty": 8.063447952270508, "memory": "alloc=26.8GiB reserved=38.6GiB peak=35.6GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 39, "global_step": 40, "opt_step": 10, "combined": 0.80651736, "ce": 0.80651736, "kd": 0.0, "gap_energy": 0.0, "kd_alpha": 0.0, "proof_weight": 1.93877551, "ce_num": 3162.8203125, "ce_den": 3921.57739258, "kd_num": 0.0, "kd_den": 1.0, "gap_num": 0.0, "gap_den": 1.0, "kd_tokens": 0.0, "teacher_topk_mass_num": 0.0, "student_topk_mass_num": 0.0, "mass_den": 1.0, "lr": 2.9945652968935727e-06, "difficulty": 8.520319938659668, "memory": "alloc=20.4GiB reserved=38.6GiB peak=35.6GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 49, "global_step": 50, "opt_step": 12, "combined": 0.60383594, "ce": 0.60383594, "kd": 0.0, "gap_energy": 0.0, "kd_alpha": 0.0, "proof_weight": 1.92346939, "ce_num": 2507.10180664, "ce_den": 4151.95849609, "kd_num": 0.0, "kd_den": 1.0, "gap_num": 0.0, "gap_den": 1.0, "kd_tokens": 0.0, "teacher_topk_mass_num": 0.0, "student_topk_mass_num": 0.0, "mass_den": 1.0, "lr": 2.987782178386712e-06, "difficulty": 8.635387420654297, "memory": "alloc=26.8GiB reserved=38.6GiB peak=35.6GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 59, "global_step": 60, "opt_step": 15, "combined": 0.48840919, "ce": 0.48823461, "kd": 0.53238404, "gap_energy": 0.13432807, "kd_alpha": 0.0039548, "proof_weight": 1.90816327, "ce_num": 2901.15576172, "ce_den": 5942.13476562, "kd_num": 3163.49755859, "kd_den": 5942.13476562, "gap_num": 797.59191895, "gap_den": 5937.64160156, "kd_tokens": 2650.0, "teacher_topk_mass_num": 2431.18603516, "student_topk_mass_num": 2339.93945312, "mass_den": 2650.0, "lr": 2.9725617878713683e-06, "difficulty": 9.01607894897461, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 69, "global_step": 70, "opt_step": 17, "combined": 0.64849538, "ce": 0.64515531, "kd": 0.78591424, "gap_energy": 0.12753046, "kd_alpha": 0.02372881, "proof_weight": 1.89285714, "ce_num": 4418.65429688, "ce_den": 6848.97753906, "kd_num": 5382.70898438, "kd_den": 6848.97753906, "gap_num": 872.86529541, "gap_den": 6844.3671875, "kd_tokens": 3039.0, "teacher_topk_mass_num": 2719.81274414, "student_topk_mass_num": 2384.62744141, "mass_den": 3039.0, "lr": 2.9590808525085274e-06, "difficulty": 9.196342468261719, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 79, "global_step": 80, "opt_step": 20, "combined": 0.45821229, "ce": 0.44078094, "kd": 0.84147513, "gap_energy": 0.19842488, "kd_alpha": 0.04350282, "proof_weight": 1.87755102, "ce_num": 3115.63183594, "ce_den": 7068.43603516, "kd_num": 5947.91308594, "kd_den": 7068.43603516, "gap_num": 1401.50024414, "gap_den": 7063.12792969, "kd_tokens": 2963.0, "teacher_topk_mass_num": 2739.51367188, "student_topk_mass_num": 2400.4050293, "mass_den": 2963.0, "lr": 2.9339262969984575e-06, "difficulty": 9.049243927001953, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 89, "global_step": 90, "opt_step": 22, "combined": 0.29784098, "ce": 0.2527481, "kd": 0.96537668, "gap_energy": 0.30821687, "kd_alpha": 0.06327684, "proof_weight": 1.8622449, "ce_num": 1589.84631348, "ce_den": 6290.24023438, "kd_num": 6072.45117188, "kd_den": 6290.24023438, "gap_num": 1937.32543945, "gap_den": 6285.59179688, "kd_tokens": 2697.0, "teacher_topk_mass_num": 2572.86108398, "student_topk_mass_num": 2246.55224609, "mass_den": 2697.0, "lr": 2.913917075363645e-06, "difficulty": 9.136136054992676, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 99, "global_step": 100, "opt_step": 25, "combined": 0.65063834, "ce": 0.62219143, "kd": 0.96471536, "gap_energy": 0.18682474, "kd_alpha": 0.08305085, "proof_weight": 1.84693878, "ce_num": 3035.79345703, "ce_den": 4879.1953125, "kd_num": 4707.03466797, "kd_den": 4879.1953125, "gap_num": 910.70910645, "gap_den": 4874.67089844, "kd_tokens": 2399.0, "teacher_topk_mass_num": 1882.3659668, "student_topk_mass_num": 1627.14233398, "mass_den": 2399.0, "lr": 2.8791446879894015e-06, "difficulty": 8.839614868164062, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 109, "global_step": 110, "opt_step": 27, "combined": 0.51954746, "ce": 0.49451286, "kd": 0.73798132, "gap_energy": 0.18867157, "kd_alpha": 0.10282486, "proof_weight": 1.83163265, "ce_num": 3955.71142578, "ce_den": 7999.20849609, "kd_num": 5903.26660156, "kd_den": 7999.20849609, "gap_num": 1508.29418945, "gap_den": 7994.28417969, "kd_tokens": 3590.0, "teacher_topk_mass_num": 3250.93408203, "student_topk_mass_num": 2921.56396484, "mass_den": 3590.0, "lr": 2.8528588076542965e-06, "difficulty": 9.443330764770508, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 119, "global_step": 120, "opt_step": 30, "combined": 0.35144728, "ce": 0.3200354, "kd": 0.57625228, "gap_energy": 0.0713951, "kd_alpha": 0.12259887, "proof_weight": 1.81632653, "ce_num": 2391.49194336, "ce_den": 7472.5859375, "kd_num": 4306.09472656, "kd_den": 7472.5859375, "gap_num": 533.13110352, "gap_den": 7467.33447266, "kd_tokens": 3282.0, "teacher_topk_mass_num": 2973.54248047, "student_topk_mass_num": 2701.47192383, "mass_den": 3282.0, "lr": 2.8089058713625192e-06, "difficulty": 9.33741569519043, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 129, "global_step": 130, "opt_step": 32, "combined": 0.61775106, "ce": 0.61014116, "kd": 0.6635918, "gap_energy": 0.10788401, "kd_alpha": 0.14237288, "proof_weight": 1.80102041, "ce_num": 5045.71289062, "ce_den": 8269.74707031, "kd_num": 5487.73632812, "kd_den": 8269.74707031, "gap_num": 891.60717773, "gap_den": 8264.49804688, "kd_tokens": 3530.0, "teacher_topk_mass_num": 2828.89599609, "student_topk_mass_num": 2750.65283203, "mass_den": 3530.0, "lr": 2.7766738925237584e-06, "difficulty": 9.378902435302734, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 139, "global_step": 140, "opt_step": 35, "combined": 0.46766806, "ce": 0.40705362, "kd": 0.78087783, "gap_energy": 0.33781546, "kd_alpha": 0.16214689, "proof_weight": 1.78571429, "ce_num": 3316.54199219, "ce_den": 8147.67871094, "kd_num": 6362.34179688, "kd_den": 8147.67871094, "gap_num": 2750.86181641, "gap_den": 8143.09033203, "kd_tokens": 3491.0, "teacher_topk_mass_num": 3209.97265625, "student_topk_mass_num": 3108.03955078, "mass_den": 3491.0, "lr": 2.7240931409904047e-06, "difficulty": 9.392586708068848, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 149, "global_step": 150, "opt_step": 37, "combined": 0.53738558, "ce": 0.45108336, "kd": 0.92547762, "gap_energy": 0.65703124, "kd_alpha": 0.1819209, "proof_weight": 1.77040816, "ce_num": 3765.02099609, "ce_den": 8346.61914062, "kd_num": 7724.609375, "kd_den": 8346.61914062, "gap_num": 5481.05664062, "gap_den": 8342.15527344, "kd_tokens": 3726.0, "teacher_topk_mass_num": 3358.96166992, "student_topk_mass_num": 3260.4050293, "mass_den": 3726.0, "lr": 2.6863203994833153e-06, "difficulty": 9.397011756896973, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 159, "global_step": 160, "opt_step": 40, "combined": 0.34333396, "ce": 0.26012513, "kd": 0.67267311, "gap_energy": 0.32599863, "kd_alpha": 0.20169492, "proof_weight": 1.75510204, "ce_num": 2292.73486328, "ce_den": 8813.96875, "kd_num": 5928.91992188, "kd_den": 8813.96875, "gap_num": 2871.65820312, "gap_den": 8808.8046875, "kd_tokens": 3818.0, "teacher_topk_mass_num": 3620.66650391, "student_topk_mass_num": 3510.72509766, "mass_den": 3818.0, "lr": 2.625773066166979e-06, "difficulty": 9.601572036743164, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 169, "global_step": 170, "opt_step": 42, "combined": 0.28035346, "ce": 0.27071178, "kd": 0.31424689, "gap_energy": 0.05172772, "kd_alpha": 0.22146893, "proof_weight": 1.73979592, "ce_num": 2396.01489258, "ce_den": 8850.796875, "kd_num": 2781.33544922, "kd_den": 8850.796875, "gap_num": 457.55905151, "gap_den": 8845.52832031, "kd_tokens": 3818.0, "teacher_topk_mass_num": 3658.04345703, "student_topk_mass_num": 3594.16357422, "mass_den": 3818.0, "lr": 2.5829345761322675e-06, "difficulty": 9.6195068359375, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 179, "global_step": 180, "opt_step": 45, "combined": 0.25831455, "ce": 0.2564809, "kd": 0.26408181, "gap_energy": 0.02563814, "kd_alpha": 0.24124294, "proof_weight": 1.7244898, "ce_num": 2221.45703125, "ce_den": 8661.29589844, "kd_num": 2287.29077148, "kd_den": 8661.29589844, "gap_num": 221.94287109, "gap_den": 8656.74804688, "kd_tokens": 3748.0, "teacher_topk_mass_num": 3553.6015625, "student_topk_mass_num": 3502.31494141, "mass_den": 3748.0, "lr": 2.5151820788797823e-06, "difficulty": 9.612282752990723, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 189, "global_step": 190, "opt_step": 47, "combined": 0.4030064, "ce": 0.38660949, "kd": 0.44942892, "gap_energy": 0.06206001, "kd_alpha": 0.26101695, "proof_weight": 1.70918367, "ce_num": 3204.91943359, "ce_den": 8289.81054688, "kd_num": 3725.68066406, "kd_den": 8289.81054688, "gap_num": 514.19836426, "gap_den": 8285.50292969, "kd_tokens": 3865.0, "teacher_topk_mass_num": 3518.7331543, "student_topk_mass_num": 3433.62817383, "mass_den": 3865.0, "lr": 2.4678165591851067e-06, "difficulty": 9.535543441772461, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 3, "global_step": 200, "opt_step": 50, "combined": 1.24225593, "ce": 1.39534032, "kd": 0.85015059, "gap_energy": 0.269521, "kd_alpha": 0.28079096, "proof_weight": 1.69387755, "ce_num": 446.31286621, "ce_den": 319.85949707, "kd_num": 271.92874146, "kd_den": 319.85949707, "gap_num": 85.47875977, "gap_den": 317.15066528, "kd_tokens": 246.0, "teacher_topk_mass_num": 142.12185669, "student_topk_mass_num": 135.27420044, "mass_den": 246.0, "lr": 2.3937109249603387e-06, "difficulty": 5.464296340942383, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 13, "global_step": 210, "opt_step": 52, "combined": 0.66685486, "ce": 0.63092232, "kd": 0.75047231, "gap_energy": 0.27310866, "kd_alpha": 0.30056497, "proof_weight": 1.67857143, "ce_num": 396.93060303, "ce_den": 629.12756348, "kd_num": 472.14282227, "kd_den": 629.12756348, "gap_num": 171.09176636, "gap_den": 626.46044922, "kd_tokens": 479.0, "teacher_topk_mass_num": 378.57348633, "student_topk_mass_num": 363.54272461, "mass_den": 479.0, "lr": 2.3424140244979735e-06, "difficulty": 6.788113594055176, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 23, "global_step": 220, "opt_step": 55, "combined": 0.59748399, "ce": 0.58014715, "kd": 0.63426751, "gap_energy": 0.12397655, "kd_alpha": 0.32033898, "proof_weight": 1.66326531, "ce_num": 1008.77160645, "ce_den": 1738.8203125, "kd_num": 1102.87719727, "kd_den": 1738.8203125, "gap_num": 215.08178711, "gap_den": 1734.85864258, "kd_tokens": 1021.0, "teacher_topk_mass_num": 801.39776611, "student_topk_mass_num": 810.95361328, "mass_den": 1021.0, "lr": 2.2628871746483883e-06, "difficulty": 7.726119041442871, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 33, "global_step": 230, "opt_step": 57, "combined": 0.35832763, "ce": 0.3339847, "kd": 0.40555772, "gap_energy": 0.0712208, "kd_alpha": 0.34011299, "proof_weight": 1.64795918, "ce_num": 980.17529297, "ce_den": 2934.79101562, "kd_num": 1190.22717285, "kd_den": 2934.79101562, "gap_num": 208.73866272, "gap_den": 2930.86669922, "kd_tokens": 1484.0, "teacher_topk_mass_num": 1332.2479248, "student_topk_mass_num": 1324.51086426, "mass_den": 1484.0, "lr": 2.208303981704803e-06, "difficulty": 8.34626579284668, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 43, "global_step": 240, "opt_step": 60, "combined": 0.35486567, "ce": 0.33603364, "kd": 0.38983947, "gap_energy": 0.05347834, "kd_alpha": 0.35, "proof_weight": 1.63265306, "ce_num": 1235.95581055, "ce_den": 3678.07177734, "kd_num": 1433.85754395, "kd_den": 3678.07177734, "gap_num": 196.49060059, "gap_den": 3674.20898438, "kd_tokens": 1778.0, "teacher_topk_mass_num": 1631.93139648, "student_topk_mass_num": 1639.41564941, "mass_den": 1778.0, "lr": 2.124356012509813e-06, "difficulty": 8.55252456665039, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 53, "global_step": 250, "opt_step": 62, "combined": 0.43773556, "ce": 0.46106946, "kd": 0.39440125, "gap_energy": 0.04402284, "kd_alpha": 0.35, "proof_weight": 1.61734694, "ce_num": 2207.42407227, "ce_den": 4787.6171875, "kd_num": 1888.2421875, "kd_den": 4787.6171875, "gap_num": 210.58125305, "gap_den": 4783.45410156, "kd_tokens": 2192.0, "teacher_topk_mass_num": 1961.63867188, "student_topk_mass_num": 1959.3458252, "mass_den": 2192.0, "lr": 2.067172942406179e-06, "difficulty": 8.677350997924805, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 63, "global_step": 260, "opt_step": 65, "combined": 0.41169688, "ce": 0.39724419, "kd": 0.4385376, "gap_energy": 0.05188184, "kd_alpha": 0.35, "proof_weight": 1.60204082, "ce_num": 2263.84179688, "ce_den": 5698.8671875, "kd_num": 2499.16748047, "kd_den": 5698.8671875, "gap_num": 295.4510498, "gap_den": 5694.69091797, "kd_tokens": 2943.0, "teacher_topk_mass_num": 2676.89599609, "student_topk_mass_num": 2602.33740234, "mass_den": 2943.0, "lr": 1.979859548286157e-06, "difficulty": 9.147865295410156, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 73, "global_step": 270, "opt_step": 67, "combined": 0.31871417, "ce": 0.29050359, "kd": 0.37110531, "gap_energy": 0.03902761, "kd_alpha": 0.35, "proof_weight": 1.58673469, "ce_num": 1429.89501953, "ce_den": 4922.125, "kd_num": 1826.62670898, "kd_den": 4922.125, "gap_num": 191.9515686, "gap_den": 4918.35351562, "kd_tokens": 2449.0, "teacher_topk_mass_num": 2294.19091797, "student_topk_mass_num": 2247.55615234, "mass_den": 2449.0, "lr": 1.9207957113074184e-06, "difficulty": 9.025297164916992, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 83, "global_step": 280, "opt_step": 70, "combined": 0.31959426, "ce": 0.30900902, "kd": 0.33925256, "gap_energy": 0.03257714, "kd_alpha": 0.35, "proof_weight": 1.57142857, "ce_num": 1426.18310547, "ce_den": 4615.34472656, "kd_num": 1565.76757812, "kd_den": 4615.34472656, "gap_num": 150.23193359, "gap_den": 4611.57519531, "kd_tokens": 2419.0, "teacher_topk_mass_num": 2247.19628906, "student_topk_mass_num": 2205.96362305, "mass_den": 2419.0, "lr": 1.831214908853835e-06, "difficulty": 8.914824485778809, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 93, "global_step": 290, "opt_step": 72, "combined": 0.40945679, "ce": 0.40299642, "kd": 0.4214547, "gap_energy": 0.05730927, "kd_alpha": 0.35, "proof_weight": 1.55612245, "ce_num": 2152.30273438, "ce_den": 5340.74902344, "kd_num": 2250.88378906, "kd_den": 5340.74902344, "gap_num": 305.85800171, "gap_den": 5336.97265625, "kd_tokens": 2809.0, "teacher_topk_mass_num": 2531.46459961, "student_topk_mass_num": 2505.27807617, "mass_den": 2809.0, "lr": 1.7710130670196354e-06, "difficulty": 9.15400505065918, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 103, "global_step": 300, "opt_step": 75, "combined": 0.39351124, "ce": 0.38491118, "kd": 0.40948281, "gap_energy": 0.05109351, "kd_alpha": 0.35, "proof_weight": 1.54081633, "ce_num": 1915.125, "ce_den": 4975.49853516, "kd_num": 2037.38110352, "kd_den": 4975.49853516, "gap_num": 254.01828003, "gap_den": 4971.63525391, "kd_tokens": 2583.0, "teacher_topk_mass_num": 2381.25341797, "student_topk_mass_num": 2361.17797852, "mass_den": 2583.0, "lr": 1.6802913867993369e-06, "difficulty": 9.028308868408203, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 113, "global_step": 310, "opt_step": 77, "combined": 0.36131692, "ce": 0.30587906, "kd": 0.46427298, "gap_energy": 0.0437492, "kd_alpha": 0.35, "proof_weight": 1.5255102, "ce_num": 2284.91210938, "ce_den": 7469.98535156, "kd_num": 3468.11230469, "kd_den": 7469.98535156, "gap_num": 326.60665894, "gap_den": 7465.43115234, "kd_tokens": 3472.0, "teacher_topk_mass_num": 3023.33691406, "student_topk_mass_num": 3130.13110352, "mass_den": 3472.0, "lr": 1.6197086132006635e-06, "difficulty": 9.511463165283203, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 123, "global_step": 320, "opt_step": 80, "combined": 0.31258738, "ce": 0.30897042, "kd": 0.31930465, "gap_energy": 0.03223233, "kd_alpha": 0.35, "proof_weight": 1.51020408, "ce_num": 2031.16308594, "ce_den": 6573.97265625, "kd_num": 2099.10009766, "kd_den": 6573.97265625, "gap_num": 211.77005005, "gap_den": 6570.11425781, "kd_tokens": 3547.0, "teacher_topk_mass_num": 3221.05810547, "student_topk_mass_num": 3193.78173828, "mass_den": 3547.0, "lr": 1.5289869329803647e-06, "difficulty": 9.425006866455078, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 133, "global_step": 330, "opt_step": 82, "combined": 0.37523079, "ce": 0.36893314, "kd": 0.38692647, "gap_energy": 0.04796501, "kd_alpha": 0.35, "proof_weight": 1.49489796, "ce_num": 2777.39501953, "ce_den": 7528.1796875, "kd_num": 2912.85205078, "kd_den": 7528.1796875, "gap_num": 360.87277222, "gap_den": 7523.66699219, "kd_tokens": 3835.0, "teacher_topk_mass_num": 3558.80957031, "student_topk_mass_num": 3521.45751953, "mass_den": 3835.0, "lr": 1.4687850911461655e-06, "difficulty": 9.515018463134766, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 143, "global_step": 340, "opt_step": 85, "combined": 0.3591302, "ce": 0.36028352, "kd": 0.3569884, "gap_energy": 0.03716599, "kd_alpha": 0.35, "proof_weight": 1.47959184, "ce_num": 2544.3293457, "ce_den": 7062.01953125, "kd_num": 2521.05908203, "kd_den": 7062.01953125, "gap_num": 262.29891968, "gap_den": 7057.49853516, "kd_tokens": 3668.0, "teacher_topk_mass_num": 3338.10302734, "student_topk_mass_num": 3297.52075195, "mass_den": 3668.0, "lr": 1.3792042886925817e-06, "difficulty": 9.386213302612305, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 153, "global_step": 350, "opt_step": 87, "combined": 0.31356943, "ce": 0.28112417, "kd": 0.37382498, "gap_energy": 0.03802348, "kd_alpha": 0.35, "proof_weight": 1.46428571, "ce_num": 2047.1159668, "ce_den": 7281.89257812, "kd_num": 2722.15332031, "kd_den": 7281.89257812, "gap_num": 276.73950195, "gap_den": 7278.12158203, "kd_tokens": 3810.0, "teacher_topk_mass_num": 3609.49633789, "student_topk_mass_num": 3521.68920898, "mass_den": 3810.0, "lr": 1.3201404517138427e-06, "difficulty": 9.56966781616211, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 163, "global_step": 360, "opt_step": 90, "combined": 0.3101933, "ce": 0.32049137, "kd": 0.29106832, "gap_energy": 0.02370189, "kd_alpha": 0.35, "proof_weight": 1.44897959, "ce_num": 2269.39501953, "ce_den": 7080.98632812, "kd_num": 2061.05078125, "kd_den": 7080.98632812, "gap_num": 167.73965454, "gap_den": 7077.05957031, "kd_tokens": 3767.0, "teacher_topk_mass_num": 3466.20703125, "student_topk_mass_num": 3470.0456543, "mass_den": 3767.0, "lr": 1.2328270575938212e-06, "difficulty": 9.567719459533691, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 173, "global_step": 370, "opt_step": 92, "combined": 0.25587186, "ce": 0.24827541, "kd": 0.2699796, "gap_energy": 0.03437245, "kd_alpha": 0.35, "proof_weight": 1.43367347, "ce_num": 1768.06774902, "ce_den": 7121.39697266, "kd_num": 1922.63183594, "kd_den": 7121.39697266, "gap_num": 244.64147949, "gap_den": 7117.37207031, "kd_tokens": 3774.0, "teacher_topk_mass_num": 3600.57617188, "student_topk_mass_num": 3582.96337891, "mass_den": 3774.0, "lr": 1.1756439874901873e-06, "difficulty": 9.657493591308594, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 183, "global_step": 380, "opt_step": 95, "combined": 0.2495057, "ce": 0.23697412, "kd": 0.27277863, "gap_energy": 0.03516343, "kd_alpha": 0.35, "proof_weight": 1.41836735, "ce_num": 1761.13122559, "ce_den": 7431.74511719, "kd_num": 2027.22119141, "kd_den": 7431.74511719, "gap_num": 261.18389893, "gap_den": 7427.71386719, "kd_tokens": 3983.0, "teacher_topk_mass_num": 3788.17041016, "student_topk_mass_num": 3781.07714844, "mass_den": 3983.0, "lr": 1.0916960182951974e-06, "difficulty": 9.683958053588867, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 193, "global_step": 390, "opt_step": 97, "combined": 0.30295968, "ce": 0.31477621, "kd": 0.28101465, "gap_energy": 0.02040295, "kd_alpha": 0.35, "proof_weight": 1.40306122, "ce_num": 2344.95214844, "ce_den": 7449.58496094, "kd_num": 2093.44262695, "kd_den": 7449.58496094, "gap_num": 151.9080658, "gap_den": 7445.39550781, "kd_tokens": 3860.0, "teacher_topk_mass_num": 3460.72460938, "student_topk_mass_num": 3475.14306641, "mass_den": 3860.0, "lr": 1.0371128253516119e-06, "difficulty": 9.59878158569336, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 7, "global_step": 400, "opt_step": 100, "combined": 0.92327392, "ce": 0.99677551, "kd": 0.78677112, "gap_energy": 0.13165115, "kd_alpha": 0.35, "proof_weight": 1.3877551, "ce_num": 421.11520386, "ce_den": 422.47747803, "kd_num": 332.39306641, "kd_den": 422.47747803, "gap_num": 55.2678833, "gap_den": 419.80554199, "kd_tokens": 324.0, "teacher_topk_mass_num": 214.39503479, "student_topk_mass_num": 201.57331848, "mass_den": 324.0, "lr": 9.57585975502027e-07, "difficulty": 5.915072441101074, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 17, "global_step": 410, "opt_step": 102, "combined": 0.50501037, "ce": 0.44381145, "kd": 0.61866558, "gap_energy": 0.0960589, "kd_alpha": 0.35, "proof_weight": 1.37244898, "ce_num": 392.18963623, "ce_den": 883.6852417, "kd_num": 546.70562744, "kd_den": 883.6852417, "gap_num": 84.62545776, "gap_den": 880.97467041, "kd_tokens": 674.0, "teacher_topk_mass_num": 562.82067871, "student_topk_mass_num": 528.24304199, "mass_den": 674.0, "lr": 9.062890750396618e-07, "difficulty": 7.160126686096191, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 27, "global_step": 420, "opt_step": 105, "combined": 0.56388533, "ce": 0.51320404, "kd": 0.65800786, "gap_energy": 0.12683682, "kd_alpha": 0.35, "proof_weight": 1.35714286, "ce_num": 1160.43701172, "ce_den": 2261.16113281, "kd_num": 1487.86181641, "kd_den": 2261.16113281, "gap_num": 286.32626343, "gap_den": 2257.43798828, "kd_tokens": 1385.0, "teacher_topk_mass_num": 1162.08874512, "student_topk_mass_num": 1125.13244629, "mass_den": 1385.0, "lr": 8.321834408148934e-07, "difficulty": 8.027276992797852, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 37, "global_step": 430, "opt_step": 107, "combined": 0.34577429, "ce": 0.33484906, "kd": 0.36606401, "gap_energy": 0.08746573, "kd_alpha": 0.35, "proof_weight": 1.34183673, "ce_num": 1006.72387695, "ce_den": 3006.50048828, "kd_num": 1100.57165527, "kd_den": 3006.50048828, "gap_num": 262.65948486, "gap_den": 3002.99902344, "kd_tokens": 1761.0, "teacher_topk_mass_num": 1567.68347168, "student_topk_mass_num": 1570.17810059, "mass_den": 1761.0, "lr": 7.848179211202175e-07, "difficulty": 8.473159790039062, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 47, "global_step": 440, "opt_step": 110, "combined": 0.39065716, "ce": 0.39188409, "kd": 0.38837859, "gap_energy": 0.03742781, "kd_alpha": 0.35, "proof_weight": 1.32653061, "ce_num": 1219.81176758, "ce_den": 3112.68505859, "kd_num": 1208.90026855, "kd_den": 3112.68505859, "gap_num": 116.3790741, "gap_den": 3109.42822266, "kd_tokens": 1870.0, "teacher_topk_mass_num": 1658.35217285, "student_topk_mass_num": 1659.3380127, "mass_den": 1870.0, "lr": 7.170654238677327e-07, "difficulty": 8.60601806640625, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 57, "global_step": 450, "opt_step": 112, "combined": 0.16394296, "ce": 0.14618537, "kd": 0.19692138, "gap_energy": 0.02102127, "kd_alpha": 0.35, "proof_weight": 1.31122449, "ce_num": 476.72827148, "ce_den": 3261.12158203, "kd_num": 642.18457031, "kd_den": 3261.12158203, "gap_num": 68.48384857, "gap_den": 3257.83666992, "kd_tokens": 1909.0, "teacher_topk_mass_num": 1841.36376953, "student_topk_mass_num": 1828.80688477, "mass_den": 1909.0, "lr": 6.742269338330216e-07, "difficulty": 8.738836288452148, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 67, "global_step": 460, "opt_step": 115, "combined": 0.34792715, "ce": 0.37267017, "kd": 0.30197588, "gap_energy": 0.01995143, "kd_alpha": 0.35, "proof_weight": 1.29591837, "ce_num": 1611.00402832, "ce_den": 4322.86816406, "kd_num": 1305.40185547, "kd_den": 4322.86816406, "gap_num": 86.17908478, "gap_den": 4319.44433594, "kd_tokens": 2567.0, "teacher_topk_mass_num": 2339.13134766, "student_topk_mass_num": 2327.57617188, "mass_den": 2567.0, "lr": 6.13679600516685e-07, "difficulty": 9.008158683776855, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 77, "global_step": 470, "opt_step": 117, "combined": 0.21805955, "ce": 0.20472163, "kd": 0.24283001, "gap_energy": 0.02222797, "kd_alpha": 0.35, "proof_weight": 1.28061224, "ce_num": 1057.07836914, "ce_den": 5163.49121094, "kd_num": 1253.85058594, "kd_den": 5163.49121094, "gap_num": 114.66514587, "gap_den": 5158.59716797, "kd_tokens": 3050.0, "teacher_topk_mass_num": 2907.29199219, "student_topk_mass_num": 2896.83374023, "mass_den": 3050.0, "lr": 5.759068590095958e-07, "difficulty": 8.891244888305664, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 87, "global_step": 480, "opt_step": 120, "combined": 0.4542684, "ce": 0.48165402, "kd": 0.40340948, "gap_energy": 0.06115909, "kd_alpha": 0.35, "proof_weight": 1.26530612, "ce_num": 2448.62890625, "ce_den": 5083.79199219, "kd_num": 2050.84985352, "kd_den": 5083.79199219, "gap_num": 310.69360352, "gap_den": 5080.08886719, "kd_tokens": 3270.0, "teacher_topk_mass_num": 2836.52124023, "student_topk_mass_num": 2793.31738281, "mass_den": 3270.0, "lr": 5.233261074762414e-07, "difficulty": 9.195501327514648, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 97, "global_step": 490, "opt_step": 122, "combined": 0.35276407, "ce": 0.34306455, "kd": 0.37077755, "gap_energy": 0.03944525, "kd_alpha": 0.35, "proof_weight": 1.25, "ce_num": 1354.73242188, "ce_den": 3948.9140625, "kd_num": 1464.16870117, "kd_den": 3948.9140625, "gap_num": 155.63720703, "gap_den": 3945.65087891, "kd_tokens": 2435.0, "teacher_topk_mass_num": 2298.7253418, "student_topk_mass_num": 2252.71826172, "mass_den": 2435.0, "lr": 4.910941286374807e-07, "difficulty": 8.969837188720703, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 107, "global_step": 500, "opt_step": 125, "combined": 0.30729848, "ce": 0.30684832, "kd": 0.30813453, "gap_energy": 0.02581502, "kd_alpha": 0.35, "proof_weight": 1.23469388, "ce_num": 1901.15356445, "ce_den": 6195.74365234, "kd_num": 1909.12255859, "kd_den": 6195.74365234, "gap_num": 159.84867859, "gap_den": 6192.08105469, "kd_tokens": 3523.0, "teacher_topk_mass_num": 3198.31567383, "student_topk_mass_num": 3213.59204102, "mass_den": 3523.0, "lr": 4.471411923457035e-07, "difficulty": 9.413188934326172, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 117, "global_step": 510, "opt_step": 127, "combined": 0.31296164, "ce": 0.30628365, "kd": 0.32536358, "gap_energy": 0.04044421, "kd_alpha": 0.35, "proof_weight": 1.21938776, "ce_num": 1758.57641602, "ce_den": 5741.65917969, "kd_num": 1868.12670898, "kd_den": 5741.65917969, "gap_num": 232.08267212, "gap_den": 5738.34179688, "kd_tokens": 3478.0, "teacher_topk_mass_num": 3133.03076172, "student_topk_mass_num": 3117.89526367, "mass_den": 3478.0, "lr": 4.208553120105986e-07, "difficulty": 9.388130187988281, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 127, "global_step": 520, "opt_step": 130, "combined": 0.54961079, "ce": 0.57953328, "kd": 0.49404049, "gap_energy": 0.02451204, "kd_alpha": 0.35, "proof_weight": 1.20408163, "ce_num": 3456.52905273, "ce_den": 5964.33203125, "kd_num": 2946.62158203, "kd_den": 5964.33203125, "gap_num": 146.1060791, "gap_den": 5960.58496094, "kd_tokens": 3780.0, "teacher_topk_mass_num": 2950.45654297, "student_topk_mass_num": 3000.10839844, "mass_den": 3780.0, "lr": 3.860829246363548e-07, "difficulty": 9.353657722473145, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 137, "global_step": 530, "opt_step": 132, "combined": 0.25616327, "ce": 0.2474604, "kd": 0.27232572, "gap_energy": 0.03138196, "kd_alpha": 0.35, "proof_weight": 1.18877551, "ce_num": 1285.36254883, "ce_den": 5194.21533203, "kd_num": 1414.51843262, "kd_den": 5194.21533203, "gap_num": 162.90638733, "gap_den": 5191.08300781, "kd_tokens": 3460.0, "teacher_topk_mass_num": 3235.41064453, "student_topk_mass_num": 3186.61279297, "mass_den": 3460.0, "lr": 3.6607370300154275e-07, "difficulty": 9.462833404541016, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 147, "global_step": 540, "opt_step": 135, "combined": 0.2797212, "ce": 0.27460113, "kd": 0.28922996, "gap_energy": 0.02170261, "kd_alpha": 0.35, "proof_weight": 1.17346939, "ce_num": 1391.22668457, "ce_den": 5066.35449219, "kd_num": 1465.34155273, "kd_den": 5066.35449219, "gap_num": 109.88298035, "gap_den": 5063.12304688, "kd_tokens": 3281.0, "teacher_topk_mass_num": 3077.32519531, "student_topk_mass_num": 3067.69018555, "mass_den": 3281.0, "lr": 3.409191474914727e-07, "difficulty": 9.389362335205078, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 157, "global_step": 550, "opt_step": 137, "combined": 0.191145, "ce": 0.17197637, "kd": 0.22674394, "gap_energy": 0.02278897, "kd_alpha": 0.35, "proof_weight": 1.15816327, "ce_num": 1032.36450195, "ce_den": 6002.94384766, "kd_num": 1361.13110352, "kd_den": 6002.94384766, "gap_num": 136.73205566, "gap_den": 5999.92285156, "kd_tokens": 3806.0, "teacher_topk_mass_num": 3634.18554688, "student_topk_mass_num": 3607.42773438, "mass_den": 3806.0, "lr": 3.274382121286321e-07, "difficulty": 9.663820266723633, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 167, "global_step": 560, "opt_step": 140, "combined": 0.27715239, "ce": 0.26765925, "kd": 0.29478255, "gap_energy": 0.02763841, "kd_alpha": 0.35, "proof_weight": 1.14285714, "ce_num": 1546.05029297, "ce_den": 5776.18847656, "kd_num": 1702.71960449, "kd_den": 5776.18847656, "gap_num": 159.55589294, "gap_den": 5772.97558594, "kd_tokens": 3862.0, "teacher_topk_mass_num": 3626.77905273, "student_topk_mass_num": 3650.0546875, "mass_den": 3862.0, "lr": 3.1221782161328813e-07, "difficulty": 9.599081039428711, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 177, "global_step": 570, "opt_step": 142, "combined": 0.36908805, "ce": 0.41059637, "kd": 0.29200116, "gap_energy": 0.01557151, "kd_alpha": 0.35, "proof_weight": 1.12755102, "ce_num": 2398.20751953, "ce_den": 5840.79101562, "kd_num": 1705.51782227, "kd_den": 5840.79101562, "gap_num": 90.89710236, "gap_den": 5837.39697266, "kd_tokens": 3945.0, "teacher_topk_mass_num": 3590.37695312, "student_topk_mass_num": 3634.23754883, "mass_den": 3945.0, "lr": 3.054347031064272e-07, "difficulty": 9.548123359680176, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 187, "global_step": 580, "opt_step": 145, "combined": 0.24322987, "ce": 0.23830642, "kd": 0.25237346, "gap_energy": 0.01909368, "kd_alpha": 0.35, "proof_weight": 1.1122449, "ce_num": 1339.30322266, "ce_den": 5620.08886719, "kd_num": 1418.36120605, "kd_den": 5620.08886719, "gap_num": 107.24766541, "gap_den": 5616.91796875, "kd_tokens": 3840.0, "teacher_topk_mass_num": 3661.22802734, "student_topk_mass_num": 3657.67651367, "mass_den": 3840.0, "lr": 3.0033988282761614e-07, "difficulty": 9.662643432617188, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}

107
training_artifacts/config.json Normal file
View File

@@ -0,0 +1,107 @@
{
"dataset_ids": [
"0xZee/dataset-CoT-Advanced-Calculus-268"
],
"dataset_split": "train",
"eval_fraction": 0.1,
"seed": 42,
"max_examples": null,
"question_fields": [
"question",
"instruction",
"problem",
"prompt",
"input"
],
"reasoning_fields": [
"CoT",
"cot",
"chain_of_thought",
"reasoning",
"analysis",
"rationale",
"solution"
],
"answer_fields": [
"response",
"final_answer",
"answer",
"output",
"completion"
],
"save_data_audit": true,
"data_audit_path": "./DualMind100/data_audit.jsonl",
"student_id": "Qwen/Qwen3-1.7B",
"teacher_id": "Qwen/Qwen3-8B",
"student_revision": null,
"teacher_revision": null,
"save_dir": "./DualMind100",
"resume_from": null,
"use_bf16": true,
"keep_student_fp32": true,
"gradient_checkpointing": true,
"gradient_checkpointing_use_reentrant": false,
"attention_implementation": "sdpa",
"compile_student": false,
"teacher_load_in_4bit": false,
"teacher_device_map": "auto",
"teacher_offload_folder": "./teacher_offload",
"teacher_max_memory_gpu": null,
"teacher_max_memory_cpu": null,
"use_8bit_optimizer": false,
"use_paged_optimizer": true,
"dual_mind_format": true,
"use_chat_template": true,
"system_prompt": "Solve the problem rigorously. Preserve assumptions, derivation, verification, and final result.",
"preserve_full_answer": true,
"window_size": 2048,
"min_overlap": 128,
"max_overlap": 384,
"teacher_top_k": 64,
"teacher_cache_dir": "./tkd_cache_v4",
"teacher_context_tokens": 512,
"vocab_logsumexp_chunk": 16384,
"cache_log_every": 20,
"min_expected_topk_mass": 0.85,
"batch_size": 2,
"grad_accum_steps": 4,
"epochs": 3,
"lr": 3e-06,
"min_lr": 3e-07,
"warmup_ratio": 0.05,
"weight_decay": 0.01,
"adam_beta1": 0.9,
"adam_beta2": 0.95,
"adam_eps": 1e-08,
"grad_clip": 1.0,
"temperature": 2.0,
"kd_target_alpha": 0.35,
"kd_start_frac": 0.1,
"kd_end_frac": 0.4,
"eval_kd_alpha": null,
"jump_threshold_sigma": 3.5,
"jump_threshold_quantile": 0.995,
"support_turnover_weight": 0.15,
"jump_amplifier": 1.33,
"topology_energy_window": 32,
"topology_pair_chunk": 512,
"dg_window": 3,
"gap_energy_alpha": 0.0,
"proof_weight_start": 2.0,
"proof_weight_end": 1.1,
"examine_extra_multiplier": 1.15,
"response_weight": 1.0,
"curriculum_enabled": true,
"curriculum_phases": 4,
"easy_first_fraction": 0.3,
"num_workers": 2,
"prefetch_factor": 2,
"log_every": 10,
"eval_every": 100,
"checkpoint_every_opt_steps": 250,
"save_every_epoch": true,
"save_best": true,
"best_metric": "ce",
"run_internal_checks": true,
"inference_max_new_tokens": 768
}

59
training_artifacts/train_log.jsonl Normal file
View File

@@ -0,0 +1,59 @@
{"epoch": 1, "batch_idx": 0, "global_step": 1, "opt_step": 0, "combined": 2.19950724, "ce": 2.19950724, "kd": 0.0, "gap_energy": 0.0, "kd_alpha": 0.0, "proof_weight": 1.99846939, "ce_num": 60.49145889, "ce_den": 27.50227737, "kd_num": 0.0, "kd_den": 1.0, "gap_num": 0.0, "gap_den": 1.0, "kd_tokens": 0.0, "teacher_topk_mass_num": 0.0, "student_topk_mass_num": 0.0, "mass_den": 1.0, "lr": 4.2857142857142857e-07, "difficulty": 1.7609130144119263, "memory": "alloc=14.0GiB reserved=18.9GiB peak=16.4GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 9, "global_step": 10, "opt_step": 2, "combined": 1.21293616, "ce": 1.21293616, "kd": 0.0, "gap_energy": 0.0, "kd_alpha": 0.0, "proof_weight": 1.98469388, "ce_num": 586.51037598, "ce_den": 483.54595947, "kd_num": 0.0, "kd_den": 1.0, "gap_num": 0.0, "gap_den": 1.0, "kd_tokens": 0.0, "teacher_topk_mass_num": 0.0, "student_topk_mass_num": 0.0, "mass_den": 1.0, "lr": 1.2857142857142856e-06, "difficulty": 6.203751564025879, "memory": "alloc=26.8GiB reserved=38.6GiB peak=35.6GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 19, "global_step": 20, "opt_step": 5, "combined": 1.07730448, "ce": 1.07730448, "kd": 0.0, "gap_energy": 0.0, "kd_alpha": 0.0, "proof_weight": 1.96938776, "ce_num": 1827.9753418, "ce_den": 1696.80480957, "kd_num": 0.0, "kd_den": 1.0, "gap_num": 0.0, "gap_den": 1.0, "kd_tokens": 0.0, "teacher_topk_mass_num": 0.0, "student_topk_mass_num": 0.0, "mass_den": 1.0, "lr": 2.571428571428571e-06, "difficulty": 7.312134742736816, "memory": "alloc=20.4GiB reserved=38.6GiB peak=35.6GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 29, "global_step": 30, "opt_step": 7, "combined": 0.97815299, "ce": 0.97815299, "kd": 0.0, "gap_energy": 0.0, "kd_alpha": 0.0, "proof_weight": 1.95408163, "ce_num": 2458.76733398, "ce_den": 2513.68383789, "kd_num": 0.0, "kd_den": 1.0, "gap_num": 0.0, "gap_den": 1.0, "kd_tokens": 0.0, "teacher_topk_mass_num": 0.0, "student_topk_mass_num": 0.0, "mass_den": 1.0, "lr": 2.999660117172384e-06, "difficulty": 8.063447952270508, "memory": "alloc=26.8GiB reserved=38.6GiB peak=35.6GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 39, "global_step": 40, "opt_step": 10, "combined": 0.80651736, "ce": 0.80651736, "kd": 0.0, "gap_energy": 0.0, "kd_alpha": 0.0, "proof_weight": 1.93877551, "ce_num": 3162.8203125, "ce_den": 3921.57739258, "kd_num": 0.0, "kd_den": 1.0, "gap_num": 0.0, "gap_den": 1.0, "kd_tokens": 0.0, "teacher_topk_mass_num": 0.0, "student_topk_mass_num": 0.0, "mass_den": 1.0, "lr": 2.9945652968935727e-06, "difficulty": 8.520319938659668, "memory": "alloc=20.4GiB reserved=38.6GiB peak=35.6GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 49, "global_step": 50, "opt_step": 12, "combined": 0.60383594, "ce": 0.60383594, "kd": 0.0, "gap_energy": 0.0, "kd_alpha": 0.0, "proof_weight": 1.92346939, "ce_num": 2507.10180664, "ce_den": 4151.95849609, "kd_num": 0.0, "kd_den": 1.0, "gap_num": 0.0, "gap_den": 1.0, "kd_tokens": 0.0, "teacher_topk_mass_num": 0.0, "student_topk_mass_num": 0.0, "mass_den": 1.0, "lr": 2.987782178386712e-06, "difficulty": 8.635387420654297, "memory": "alloc=26.8GiB reserved=38.6GiB peak=35.6GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 59, "global_step": 60, "opt_step": 15, "combined": 0.48840919, "ce": 0.48823461, "kd": 0.53238404, "gap_energy": 0.13432807, "kd_alpha": 0.0039548, "proof_weight": 1.90816327, "ce_num": 2901.15576172, "ce_den": 5942.13476562, "kd_num": 3163.49755859, "kd_den": 5942.13476562, "gap_num": 797.59191895, "gap_den": 5937.64160156, "kd_tokens": 2650.0, "teacher_topk_mass_num": 2431.18603516, "student_topk_mass_num": 2339.93945312, "mass_den": 2650.0, "lr": 2.9725617878713683e-06, "difficulty": 9.01607894897461, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 69, "global_step": 70, "opt_step": 17, "combined": 0.64849538, "ce": 0.64515531, "kd": 0.78591424, "gap_energy": 0.12753046, "kd_alpha": 0.02372881, "proof_weight": 1.89285714, "ce_num": 4418.65429688, "ce_den": 6848.97753906, "kd_num": 5382.70898438, "kd_den": 6848.97753906, "gap_num": 872.86529541, "gap_den": 6844.3671875, "kd_tokens": 3039.0, "teacher_topk_mass_num": 2719.81274414, "student_topk_mass_num": 2384.62744141, "mass_den": 3039.0, "lr": 2.9590808525085274e-06, "difficulty": 9.196342468261719, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 79, "global_step": 80, "opt_step": 20, "combined": 0.45821229, "ce": 0.44078094, "kd": 0.84147513, "gap_energy": 0.19842488, "kd_alpha": 0.04350282, "proof_weight": 1.87755102, "ce_num": 3115.63183594, "ce_den": 7068.43603516, "kd_num": 5947.91308594, "kd_den": 7068.43603516, "gap_num": 1401.50024414, "gap_den": 7063.12792969, "kd_tokens": 2963.0, "teacher_topk_mass_num": 2739.51367188, "student_topk_mass_num": 2400.4050293, "mass_den": 2963.0, "lr": 2.9339262969984575e-06, "difficulty": 9.049243927001953, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 89, "global_step": 90, "opt_step": 22, "combined": 0.29784098, "ce": 0.2527481, "kd": 0.96537668, "gap_energy": 0.30821687, "kd_alpha": 0.06327684, "proof_weight": 1.8622449, "ce_num": 1589.84631348, "ce_den": 6290.24023438, "kd_num": 6072.45117188, "kd_den": 6290.24023438, "gap_num": 1937.32543945, "gap_den": 6285.59179688, "kd_tokens": 2697.0, "teacher_topk_mass_num": 2572.86108398, "student_topk_mass_num": 2246.55224609, "mass_den": 2697.0, "lr": 2.913917075363645e-06, "difficulty": 9.136136054992676, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 99, "global_step": 100, "opt_step": 25, "combined": 0.65063834, "ce": 0.62219143, "kd": 0.96471536, "gap_energy": 0.18682474, "kd_alpha": 0.08305085, "proof_weight": 1.84693878, "ce_num": 3035.79345703, "ce_den": 4879.1953125, "kd_num": 4707.03466797, "kd_den": 4879.1953125, "gap_num": 910.70910645, "gap_den": 4874.67089844, "kd_tokens": 2399.0, "teacher_topk_mass_num": 1882.3659668, "student_topk_mass_num": 1627.14233398, "mass_den": 2399.0, "lr": 2.8791446879894015e-06, "difficulty": 8.839614868164062, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 109, "global_step": 110, "opt_step": 27, "combined": 0.51954746, "ce": 0.49451286, "kd": 0.73798132, "gap_energy": 0.18867157, "kd_alpha": 0.10282486, "proof_weight": 1.83163265, "ce_num": 3955.71142578, "ce_den": 7999.20849609, "kd_num": 5903.26660156, "kd_den": 7999.20849609, "gap_num": 1508.29418945, "gap_den": 7994.28417969, "kd_tokens": 3590.0, "teacher_topk_mass_num": 3250.93408203, "student_topk_mass_num": 2921.56396484, "mass_den": 3590.0, "lr": 2.8528588076542965e-06, "difficulty": 9.443330764770508, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 119, "global_step": 120, "opt_step": 30, "combined": 0.35144728, "ce": 0.3200354, "kd": 0.57625228, "gap_energy": 0.0713951, "kd_alpha": 0.12259887, "proof_weight": 1.81632653, "ce_num": 2391.49194336, "ce_den": 7472.5859375, "kd_num": 4306.09472656, "kd_den": 7472.5859375, "gap_num": 533.13110352, "gap_den": 7467.33447266, "kd_tokens": 3282.0, "teacher_topk_mass_num": 2973.54248047, "student_topk_mass_num": 2701.47192383, "mass_den": 3282.0, "lr": 2.8089058713625192e-06, "difficulty": 9.33741569519043, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 129, "global_step": 130, "opt_step": 32, "combined": 0.61775106, "ce": 0.61014116, "kd": 0.6635918, "gap_energy": 0.10788401, "kd_alpha": 0.14237288, "proof_weight": 1.80102041, "ce_num": 5045.71289062, "ce_den": 8269.74707031, "kd_num": 5487.73632812, "kd_den": 8269.74707031, "gap_num": 891.60717773, "gap_den": 8264.49804688, "kd_tokens": 3530.0, "teacher_topk_mass_num": 2828.89599609, "student_topk_mass_num": 2750.65283203, "mass_den": 3530.0, "lr": 2.7766738925237584e-06, "difficulty": 9.378902435302734, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 139, "global_step": 140, "opt_step": 35, "combined": 0.46766806, "ce": 0.40705362, "kd": 0.78087783, "gap_energy": 0.33781546, "kd_alpha": 0.16214689, "proof_weight": 1.78571429, "ce_num": 3316.54199219, "ce_den": 8147.67871094, "kd_num": 6362.34179688, "kd_den": 8147.67871094, "gap_num": 2750.86181641, "gap_den": 8143.09033203, "kd_tokens": 3491.0, "teacher_topk_mass_num": 3209.97265625, "student_topk_mass_num": 3108.03955078, "mass_den": 3491.0, "lr": 2.7240931409904047e-06, "difficulty": 9.392586708068848, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 149, "global_step": 150, "opt_step": 37, "combined": 0.53738558, "ce": 0.45108336, "kd": 0.92547762, "gap_energy": 0.65703124, "kd_alpha": 0.1819209, "proof_weight": 1.77040816, "ce_num": 3765.02099609, "ce_den": 8346.61914062, "kd_num": 7724.609375, "kd_den": 8346.61914062, "gap_num": 5481.05664062, "gap_den": 8342.15527344, "kd_tokens": 3726.0, "teacher_topk_mass_num": 3358.96166992, "student_topk_mass_num": 3260.4050293, "mass_den": 3726.0, "lr": 2.6863203994833153e-06, "difficulty": 9.397011756896973, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 159, "global_step": 160, "opt_step": 40, "combined": 0.34333396, "ce": 0.26012513, "kd": 0.67267311, "gap_energy": 0.32599863, "kd_alpha": 0.20169492, "proof_weight": 1.75510204, "ce_num": 2292.73486328, "ce_den": 8813.96875, "kd_num": 5928.91992188, "kd_den": 8813.96875, "gap_num": 2871.65820312, "gap_den": 8808.8046875, "kd_tokens": 3818.0, "teacher_topk_mass_num": 3620.66650391, "student_topk_mass_num": 3510.72509766, "mass_den": 3818.0, "lr": 2.625773066166979e-06, "difficulty": 9.601572036743164, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 169, "global_step": 170, "opt_step": 42, "combined": 0.28035346, "ce": 0.27071178, "kd": 0.31424689, "gap_energy": 0.05172772, "kd_alpha": 0.22146893, "proof_weight": 1.73979592, "ce_num": 2396.01489258, "ce_den": 8850.796875, "kd_num": 2781.33544922, "kd_den": 8850.796875, "gap_num": 457.55905151, "gap_den": 8845.52832031, "kd_tokens": 3818.0, "teacher_topk_mass_num": 3658.04345703, "student_topk_mass_num": 3594.16357422, "mass_den": 3818.0, "lr": 2.5829345761322675e-06, "difficulty": 9.6195068359375, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 179, "global_step": 180, "opt_step": 45, "combined": 0.25831455, "ce": 0.2564809, "kd": 0.26408181, "gap_energy": 0.02563814, "kd_alpha": 0.24124294, "proof_weight": 1.7244898, "ce_num": 2221.45703125, "ce_den": 8661.29589844, "kd_num": 2287.29077148, "kd_den": 8661.29589844, "gap_num": 221.94287109, "gap_den": 8656.74804688, "kd_tokens": 3748.0, "teacher_topk_mass_num": 3553.6015625, "student_topk_mass_num": 3502.31494141, "mass_den": 3748.0, "lr": 2.5151820788797823e-06, "difficulty": 9.612282752990723, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 1, "batch_idx": 189, "global_step": 190, "opt_step": 47, "combined": 0.4030064, "ce": 0.38660949, "kd": 0.44942892, "gap_energy": 0.06206001, "kd_alpha": 0.26101695, "proof_weight": 1.70918367, "ce_num": 3204.91943359, "ce_den": 8289.81054688, "kd_num": 3725.68066406, "kd_den": 8289.81054688, "gap_num": 514.19836426, "gap_den": 8285.50292969, "kd_tokens": 3865.0, "teacher_topk_mass_num": 3518.7331543, "student_topk_mass_num": 3433.62817383, "mass_den": 3865.0, "lr": 2.4678165591851067e-06, "difficulty": 9.535543441772461, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 3, "global_step": 200, "opt_step": 50, "combined": 1.24225593, "ce": 1.39534032, "kd": 0.85015059, "gap_energy": 0.269521, "kd_alpha": 0.28079096, "proof_weight": 1.69387755, "ce_num": 446.31286621, "ce_den": 319.85949707, "kd_num": 271.92874146, "kd_den": 319.85949707, "gap_num": 85.47875977, "gap_den": 317.15066528, "kd_tokens": 246.0, "teacher_topk_mass_num": 142.12185669, "student_topk_mass_num": 135.27420044, "mass_den": 246.0, "lr": 2.3937109249603387e-06, "difficulty": 5.464296340942383, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 13, "global_step": 210, "opt_step": 52, "combined": 0.66685486, "ce": 0.63092232, "kd": 0.75047231, "gap_energy": 0.27310866, "kd_alpha": 0.30056497, "proof_weight": 1.67857143, "ce_num": 396.93060303, "ce_den": 629.12756348, "kd_num": 472.14282227, "kd_den": 629.12756348, "gap_num": 171.09176636, "gap_den": 626.46044922, "kd_tokens": 479.0, "teacher_topk_mass_num": 378.57348633, "student_topk_mass_num": 363.54272461, "mass_den": 479.0, "lr": 2.3424140244979735e-06, "difficulty": 6.788113594055176, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 23, "global_step": 220, "opt_step": 55, "combined": 0.59748399, "ce": 0.58014715, "kd": 0.63426751, "gap_energy": 0.12397655, "kd_alpha": 0.32033898, "proof_weight": 1.66326531, "ce_num": 1008.77160645, "ce_den": 1738.8203125, "kd_num": 1102.87719727, "kd_den": 1738.8203125, "gap_num": 215.08178711, "gap_den": 1734.85864258, "kd_tokens": 1021.0, "teacher_topk_mass_num": 801.39776611, "student_topk_mass_num": 810.95361328, "mass_den": 1021.0, "lr": 2.2628871746483883e-06, "difficulty": 7.726119041442871, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 33, "global_step": 230, "opt_step": 57, "combined": 0.35832763, "ce": 0.3339847, "kd": 0.40555772, "gap_energy": 0.0712208, "kd_alpha": 0.34011299, "proof_weight": 1.64795918, "ce_num": 980.17529297, "ce_den": 2934.79101562, "kd_num": 1190.22717285, "kd_den": 2934.79101562, "gap_num": 208.73866272, "gap_den": 2930.86669922, "kd_tokens": 1484.0, "teacher_topk_mass_num": 1332.2479248, "student_topk_mass_num": 1324.51086426, "mass_den": 1484.0, "lr": 2.208303981704803e-06, "difficulty": 8.34626579284668, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 43, "global_step": 240, "opt_step": 60, "combined": 0.35486567, "ce": 0.33603364, "kd": 0.38983947, "gap_energy": 0.05347834, "kd_alpha": 0.35, "proof_weight": 1.63265306, "ce_num": 1235.95581055, "ce_den": 3678.07177734, "kd_num": 1433.85754395, "kd_den": 3678.07177734, "gap_num": 196.49060059, "gap_den": 3674.20898438, "kd_tokens": 1778.0, "teacher_topk_mass_num": 1631.93139648, "student_topk_mass_num": 1639.41564941, "mass_den": 1778.0, "lr": 2.124356012509813e-06, "difficulty": 8.55252456665039, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 53, "global_step": 250, "opt_step": 62, "combined": 0.43773556, "ce": 0.46106946, "kd": 0.39440125, "gap_energy": 0.04402284, "kd_alpha": 0.35, "proof_weight": 1.61734694, "ce_num": 2207.42407227, "ce_den": 4787.6171875, "kd_num": 1888.2421875, "kd_den": 4787.6171875, "gap_num": 210.58125305, "gap_den": 4783.45410156, "kd_tokens": 2192.0, "teacher_topk_mass_num": 1961.63867188, "student_topk_mass_num": 1959.3458252, "mass_den": 2192.0, "lr": 2.067172942406179e-06, "difficulty": 8.677350997924805, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 63, "global_step": 260, "opt_step": 65, "combined": 0.41169688, "ce": 0.39724419, "kd": 0.4385376, "gap_energy": 0.05188184, "kd_alpha": 0.35, "proof_weight": 1.60204082, "ce_num": 2263.84179688, "ce_den": 5698.8671875, "kd_num": 2499.16748047, "kd_den": 5698.8671875, "gap_num": 295.4510498, "gap_den": 5694.69091797, "kd_tokens": 2943.0, "teacher_topk_mass_num": 2676.89599609, "student_topk_mass_num": 2602.33740234, "mass_den": 2943.0, "lr": 1.979859548286157e-06, "difficulty": 9.147865295410156, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 73, "global_step": 270, "opt_step": 67, "combined": 0.31871417, "ce": 0.29050359, "kd": 0.37110531, "gap_energy": 0.03902761, "kd_alpha": 0.35, "proof_weight": 1.58673469, "ce_num": 1429.89501953, "ce_den": 4922.125, "kd_num": 1826.62670898, "kd_den": 4922.125, "gap_num": 191.9515686, "gap_den": 4918.35351562, "kd_tokens": 2449.0, "teacher_topk_mass_num": 2294.19091797, "student_topk_mass_num": 2247.55615234, "mass_den": 2449.0, "lr": 1.9207957113074184e-06, "difficulty": 9.025297164916992, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 83, "global_step": 280, "opt_step": 70, "combined": 0.31959426, "ce": 0.30900902, "kd": 0.33925256, "gap_energy": 0.03257714, "kd_alpha": 0.35, "proof_weight": 1.57142857, "ce_num": 1426.18310547, "ce_den": 4615.34472656, "kd_num": 1565.76757812, "kd_den": 4615.34472656, "gap_num": 150.23193359, "gap_den": 4611.57519531, "kd_tokens": 2419.0, "teacher_topk_mass_num": 2247.19628906, "student_topk_mass_num": 2205.96362305, "mass_den": 2419.0, "lr": 1.831214908853835e-06, "difficulty": 8.914824485778809, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 93, "global_step": 290, "opt_step": 72, "combined": 0.40945679, "ce": 0.40299642, "kd": 0.4214547, "gap_energy": 0.05730927, "kd_alpha": 0.35, "proof_weight": 1.55612245, "ce_num": 2152.30273438, "ce_den": 5340.74902344, "kd_num": 2250.88378906, "kd_den": 5340.74902344, "gap_num": 305.85800171, "gap_den": 5336.97265625, "kd_tokens": 2809.0, "teacher_topk_mass_num": 2531.46459961, "student_topk_mass_num": 2505.27807617, "mass_den": 2809.0, "lr": 1.7710130670196354e-06, "difficulty": 9.15400505065918, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 103, "global_step": 300, "opt_step": 75, "combined": 0.39351124, "ce": 0.38491118, "kd": 0.40948281, "gap_energy": 0.05109351, "kd_alpha": 0.35, "proof_weight": 1.54081633, "ce_num": 1915.125, "ce_den": 4975.49853516, "kd_num": 2037.38110352, "kd_den": 4975.49853516, "gap_num": 254.01828003, "gap_den": 4971.63525391, "kd_tokens": 2583.0, "teacher_topk_mass_num": 2381.25341797, "student_topk_mass_num": 2361.17797852, "mass_den": 2583.0, "lr": 1.6802913867993369e-06, "difficulty": 9.028308868408203, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 113, "global_step": 310, "opt_step": 77, "combined": 0.36131692, "ce": 0.30587906, "kd": 0.46427298, "gap_energy": 0.0437492, "kd_alpha": 0.35, "proof_weight": 1.5255102, "ce_num": 2284.91210938, "ce_den": 7469.98535156, "kd_num": 3468.11230469, "kd_den": 7469.98535156, "gap_num": 326.60665894, "gap_den": 7465.43115234, "kd_tokens": 3472.0, "teacher_topk_mass_num": 3023.33691406, "student_topk_mass_num": 3130.13110352, "mass_den": 3472.0, "lr": 1.6197086132006635e-06, "difficulty": 9.511463165283203, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 123, "global_step": 320, "opt_step": 80, "combined": 0.31258738, "ce": 0.30897042, "kd": 0.31930465, "gap_energy": 0.03223233, "kd_alpha": 0.35, "proof_weight": 1.51020408, "ce_num": 2031.16308594, "ce_den": 6573.97265625, "kd_num": 2099.10009766, "kd_den": 6573.97265625, "gap_num": 211.77005005, "gap_den": 6570.11425781, "kd_tokens": 3547.0, "teacher_topk_mass_num": 3221.05810547, "student_topk_mass_num": 3193.78173828, "mass_den": 3547.0, "lr": 1.5289869329803647e-06, "difficulty": 9.425006866455078, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 133, "global_step": 330, "opt_step": 82, "combined": 0.37523079, "ce": 0.36893314, "kd": 0.38692647, "gap_energy": 0.04796501, "kd_alpha": 0.35, "proof_weight": 1.49489796, "ce_num": 2777.39501953, "ce_den": 7528.1796875, "kd_num": 2912.85205078, "kd_den": 7528.1796875, "gap_num": 360.87277222, "gap_den": 7523.66699219, "kd_tokens": 3835.0, "teacher_topk_mass_num": 3558.80957031, "student_topk_mass_num": 3521.45751953, "mass_den": 3835.0, "lr": 1.4687850911461655e-06, "difficulty": 9.515018463134766, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 143, "global_step": 340, "opt_step": 85, "combined": 0.3591302, "ce": 0.36028352, "kd": 0.3569884, "gap_energy": 0.03716599, "kd_alpha": 0.35, "proof_weight": 1.47959184, "ce_num": 2544.3293457, "ce_den": 7062.01953125, "kd_num": 2521.05908203, "kd_den": 7062.01953125, "gap_num": 262.29891968, "gap_den": 7057.49853516, "kd_tokens": 3668.0, "teacher_topk_mass_num": 3338.10302734, "student_topk_mass_num": 3297.52075195, "mass_den": 3668.0, "lr": 1.3792042886925817e-06, "difficulty": 9.386213302612305, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 153, "global_step": 350, "opt_step": 87, "combined": 0.31356943, "ce": 0.28112417, "kd": 0.37382498, "gap_energy": 0.03802348, "kd_alpha": 0.35, "proof_weight": 1.46428571, "ce_num": 2047.1159668, "ce_den": 7281.89257812, "kd_num": 2722.15332031, "kd_den": 7281.89257812, "gap_num": 276.73950195, "gap_den": 7278.12158203, "kd_tokens": 3810.0, "teacher_topk_mass_num": 3609.49633789, "student_topk_mass_num": 3521.68920898, "mass_den": 3810.0, "lr": 1.3201404517138427e-06, "difficulty": 9.56966781616211, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 163, "global_step": 360, "opt_step": 90, "combined": 0.3101933, "ce": 0.32049137, "kd": 0.29106832, "gap_energy": 0.02370189, "kd_alpha": 0.35, "proof_weight": 1.44897959, "ce_num": 2269.39501953, "ce_den": 7080.98632812, "kd_num": 2061.05078125, "kd_den": 7080.98632812, "gap_num": 167.73965454, "gap_den": 7077.05957031, "kd_tokens": 3767.0, "teacher_topk_mass_num": 3466.20703125, "student_topk_mass_num": 3470.0456543, "mass_den": 3767.0, "lr": 1.2328270575938212e-06, "difficulty": 9.567719459533691, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 173, "global_step": 370, "opt_step": 92, "combined": 0.25587186, "ce": 0.24827541, "kd": 0.2699796, "gap_energy": 0.03437245, "kd_alpha": 0.35, "proof_weight": 1.43367347, "ce_num": 1768.06774902, "ce_den": 7121.39697266, "kd_num": 1922.63183594, "kd_den": 7121.39697266, "gap_num": 244.64147949, "gap_den": 7117.37207031, "kd_tokens": 3774.0, "teacher_topk_mass_num": 3600.57617188, "student_topk_mass_num": 3582.96337891, "mass_den": 3774.0, "lr": 1.1756439874901873e-06, "difficulty": 9.657493591308594, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 183, "global_step": 380, "opt_step": 95, "combined": 0.2495057, "ce": 0.23697412, "kd": 0.27277863, "gap_energy": 0.03516343, "kd_alpha": 0.35, "proof_weight": 1.41836735, "ce_num": 1761.13122559, "ce_den": 7431.74511719, "kd_num": 2027.22119141, "kd_den": 7431.74511719, "gap_num": 261.18389893, "gap_den": 7427.71386719, "kd_tokens": 3983.0, "teacher_topk_mass_num": 3788.17041016, "student_topk_mass_num": 3781.07714844, "mass_den": 3983.0, "lr": 1.0916960182951974e-06, "difficulty": 9.683958053588867, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 2, "batch_idx": 193, "global_step": 390, "opt_step": 97, "combined": 0.30295968, "ce": 0.31477621, "kd": 0.28101465, "gap_energy": 0.02040295, "kd_alpha": 0.35, "proof_weight": 1.40306122, "ce_num": 2344.95214844, "ce_den": 7449.58496094, "kd_num": 2093.44262695, "kd_den": 7449.58496094, "gap_num": 151.9080658, "gap_den": 7445.39550781, "kd_tokens": 3860.0, "teacher_topk_mass_num": 3460.72460938, "student_topk_mass_num": 3475.14306641, "mass_den": 3860.0, "lr": 1.0371128253516119e-06, "difficulty": 9.59878158569336, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 7, "global_step": 400, "opt_step": 100, "combined": 0.92327392, "ce": 0.99677551, "kd": 0.78677112, "gap_energy": 0.13165115, "kd_alpha": 0.35, "proof_weight": 1.3877551, "ce_num": 421.11520386, "ce_den": 422.47747803, "kd_num": 332.39306641, "kd_den": 422.47747803, "gap_num": 55.2678833, "gap_den": 419.80554199, "kd_tokens": 324.0, "teacher_topk_mass_num": 214.39503479, "student_topk_mass_num": 201.57331848, "mass_den": 324.0, "lr": 9.57585975502027e-07, "difficulty": 5.915072441101074, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 17, "global_step": 410, "opt_step": 102, "combined": 0.50501037, "ce": 0.44381145, "kd": 0.61866558, "gap_energy": 0.0960589, "kd_alpha": 0.35, "proof_weight": 1.37244898, "ce_num": 392.18963623, "ce_den": 883.6852417, "kd_num": 546.70562744, "kd_den": 883.6852417, "gap_num": 84.62545776, "gap_den": 880.97467041, "kd_tokens": 674.0, "teacher_topk_mass_num": 562.82067871, "student_topk_mass_num": 528.24304199, "mass_den": 674.0, "lr": 9.062890750396618e-07, "difficulty": 7.160126686096191, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 27, "global_step": 420, "opt_step": 105, "combined": 0.56388533, "ce": 0.51320404, "kd": 0.65800786, "gap_energy": 0.12683682, "kd_alpha": 0.35, "proof_weight": 1.35714286, "ce_num": 1160.43701172, "ce_den": 2261.16113281, "kd_num": 1487.86181641, "kd_den": 2261.16113281, "gap_num": 286.32626343, "gap_den": 2257.43798828, "kd_tokens": 1385.0, "teacher_topk_mass_num": 1162.08874512, "student_topk_mass_num": 1125.13244629, "mass_den": 1385.0, "lr": 8.321834408148934e-07, "difficulty": 8.027276992797852, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 37, "global_step": 430, "opt_step": 107, "combined": 0.34577429, "ce": 0.33484906, "kd": 0.36606401, "gap_energy": 0.08746573, "kd_alpha": 0.35, "proof_weight": 1.34183673, "ce_num": 1006.72387695, "ce_den": 3006.50048828, "kd_num": 1100.57165527, "kd_den": 3006.50048828, "gap_num": 262.65948486, "gap_den": 3002.99902344, "kd_tokens": 1761.0, "teacher_topk_mass_num": 1567.68347168, "student_topk_mass_num": 1570.17810059, "mass_den": 1761.0, "lr": 7.848179211202175e-07, "difficulty": 8.473159790039062, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 47, "global_step": 440, "opt_step": 110, "combined": 0.39065716, "ce": 0.39188409, "kd": 0.38837859, "gap_energy": 0.03742781, "kd_alpha": 0.35, "proof_weight": 1.32653061, "ce_num": 1219.81176758, "ce_den": 3112.68505859, "kd_num": 1208.90026855, "kd_den": 3112.68505859, "gap_num": 116.3790741, "gap_den": 3109.42822266, "kd_tokens": 1870.0, "teacher_topk_mass_num": 1658.35217285, "student_topk_mass_num": 1659.3380127, "mass_den": 1870.0, "lr": 7.170654238677327e-07, "difficulty": 8.60601806640625, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 57, "global_step": 450, "opt_step": 112, "combined": 0.16394296, "ce": 0.14618537, "kd": 0.19692138, "gap_energy": 0.02102127, "kd_alpha": 0.35, "proof_weight": 1.31122449, "ce_num": 476.72827148, "ce_den": 3261.12158203, "kd_num": 642.18457031, "kd_den": 3261.12158203, "gap_num": 68.48384857, "gap_den": 3257.83666992, "kd_tokens": 1909.0, "teacher_topk_mass_num": 1841.36376953, "student_topk_mass_num": 1828.80688477, "mass_den": 1909.0, "lr": 6.742269338330216e-07, "difficulty": 8.738836288452148, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 67, "global_step": 460, "opt_step": 115, "combined": 0.34792715, "ce": 0.37267017, "kd": 0.30197588, "gap_energy": 0.01995143, "kd_alpha": 0.35, "proof_weight": 1.29591837, "ce_num": 1611.00402832, "ce_den": 4322.86816406, "kd_num": 1305.40185547, "kd_den": 4322.86816406, "gap_num": 86.17908478, "gap_den": 4319.44433594, "kd_tokens": 2567.0, "teacher_topk_mass_num": 2339.13134766, "student_topk_mass_num": 2327.57617188, "mass_den": 2567.0, "lr": 6.13679600516685e-07, "difficulty": 9.008158683776855, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 77, "global_step": 470, "opt_step": 117, "combined": 0.21805955, "ce": 0.20472163, "kd": 0.24283001, "gap_energy": 0.02222797, "kd_alpha": 0.35, "proof_weight": 1.28061224, "ce_num": 1057.07836914, "ce_den": 5163.49121094, "kd_num": 1253.85058594, "kd_den": 5163.49121094, "gap_num": 114.66514587, "gap_den": 5158.59716797, "kd_tokens": 3050.0, "teacher_topk_mass_num": 2907.29199219, "student_topk_mass_num": 2896.83374023, "mass_den": 3050.0, "lr": 5.759068590095958e-07, "difficulty": 8.891244888305664, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 87, "global_step": 480, "opt_step": 120, "combined": 0.4542684, "ce": 0.48165402, "kd": 0.40340948, "gap_energy": 0.06115909, "kd_alpha": 0.35, "proof_weight": 1.26530612, "ce_num": 2448.62890625, "ce_den": 5083.79199219, "kd_num": 2050.84985352, "kd_den": 5083.79199219, "gap_num": 310.69360352, "gap_den": 5080.08886719, "kd_tokens": 3270.0, "teacher_topk_mass_num": 2836.52124023, "student_topk_mass_num": 2793.31738281, "mass_den": 3270.0, "lr": 5.233261074762414e-07, "difficulty": 9.195501327514648, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 97, "global_step": 490, "opt_step": 122, "combined": 0.35276407, "ce": 0.34306455, "kd": 0.37077755, "gap_energy": 0.03944525, "kd_alpha": 0.35, "proof_weight": 1.25, "ce_num": 1354.73242188, "ce_den": 3948.9140625, "kd_num": 1464.16870117, "kd_den": 3948.9140625, "gap_num": 155.63720703, "gap_den": 3945.65087891, "kd_tokens": 2435.0, "teacher_topk_mass_num": 2298.7253418, "student_topk_mass_num": 2252.71826172, "mass_den": 2435.0, "lr": 4.910941286374807e-07, "difficulty": 8.969837188720703, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 107, "global_step": 500, "opt_step": 125, "combined": 0.30729848, "ce": 0.30684832, "kd": 0.30813453, "gap_energy": 0.02581502, "kd_alpha": 0.35, "proof_weight": 1.23469388, "ce_num": 1901.15356445, "ce_den": 6195.74365234, "kd_num": 1909.12255859, "kd_den": 6195.74365234, "gap_num": 159.84867859, "gap_den": 6192.08105469, "kd_tokens": 3523.0, "teacher_topk_mass_num": 3198.31567383, "student_topk_mass_num": 3213.59204102, "mass_den": 3523.0, "lr": 4.471411923457035e-07, "difficulty": 9.413188934326172, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 117, "global_step": 510, "opt_step": 127, "combined": 0.31296164, "ce": 0.30628365, "kd": 0.32536358, "gap_energy": 0.04044421, "kd_alpha": 0.35, "proof_weight": 1.21938776, "ce_num": 1758.57641602, "ce_den": 5741.65917969, "kd_num": 1868.12670898, "kd_den": 5741.65917969, "gap_num": 232.08267212, "gap_den": 5738.34179688, "kd_tokens": 3478.0, "teacher_topk_mass_num": 3133.03076172, "student_topk_mass_num": 3117.89526367, "mass_den": 3478.0, "lr": 4.208553120105986e-07, "difficulty": 9.388130187988281, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 127, "global_step": 520, "opt_step": 130, "combined": 0.54961079, "ce": 0.57953328, "kd": 0.49404049, "gap_energy": 0.02451204, "kd_alpha": 0.35, "proof_weight": 1.20408163, "ce_num": 3456.52905273, "ce_den": 5964.33203125, "kd_num": 2946.62158203, "kd_den": 5964.33203125, "gap_num": 146.1060791, "gap_den": 5960.58496094, "kd_tokens": 3780.0, "teacher_topk_mass_num": 2950.45654297, "student_topk_mass_num": 3000.10839844, "mass_den": 3780.0, "lr": 3.860829246363548e-07, "difficulty": 9.353657722473145, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 137, "global_step": 530, "opt_step": 132, "combined": 0.25616327, "ce": 0.2474604, "kd": 0.27232572, "gap_energy": 0.03138196, "kd_alpha": 0.35, "proof_weight": 1.18877551, "ce_num": 1285.36254883, "ce_den": 5194.21533203, "kd_num": 1414.51843262, "kd_den": 5194.21533203, "gap_num": 162.90638733, "gap_den": 5191.08300781, "kd_tokens": 3460.0, "teacher_topk_mass_num": 3235.41064453, "student_topk_mass_num": 3186.61279297, "mass_den": 3460.0, "lr": 3.6607370300154275e-07, "difficulty": 9.462833404541016, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 147, "global_step": 540, "opt_step": 135, "combined": 0.2797212, "ce": 0.27460113, "kd": 0.28922996, "gap_energy": 0.02170261, "kd_alpha": 0.35, "proof_weight": 1.17346939, "ce_num": 1391.22668457, "ce_den": 5066.35449219, "kd_num": 1465.34155273, "kd_den": 5066.35449219, "gap_num": 109.88298035, "gap_den": 5063.12304688, "kd_tokens": 3281.0, "teacher_topk_mass_num": 3077.32519531, "student_topk_mass_num": 3067.69018555, "mass_den": 3281.0, "lr": 3.409191474914727e-07, "difficulty": 9.389362335205078, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 157, "global_step": 550, "opt_step": 137, "combined": 0.191145, "ce": 0.17197637, "kd": 0.22674394, "gap_energy": 0.02278897, "kd_alpha": 0.35, "proof_weight": 1.15816327, "ce_num": 1032.36450195, "ce_den": 6002.94384766, "kd_num": 1361.13110352, "kd_den": 6002.94384766, "gap_num": 136.73205566, "gap_den": 5999.92285156, "kd_tokens": 3806.0, "teacher_topk_mass_num": 3634.18554688, "student_topk_mass_num": 3607.42773438, "mass_den": 3806.0, "lr": 3.274382121286321e-07, "difficulty": 9.663820266723633, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 167, "global_step": 560, "opt_step": 140, "combined": 0.27715239, "ce": 0.26765925, "kd": 0.29478255, "gap_energy": 0.02763841, "kd_alpha": 0.35, "proof_weight": 1.14285714, "ce_num": 1546.05029297, "ce_den": 5776.18847656, "kd_num": 1702.71960449, "kd_den": 5776.18847656, "gap_num": 159.55589294, "gap_den": 5772.97558594, "kd_tokens": 3862.0, "teacher_topk_mass_num": 3626.77905273, "student_topk_mass_num": 3650.0546875, "mass_den": 3862.0, "lr": 3.1221782161328813e-07, "difficulty": 9.599081039428711, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 177, "global_step": 570, "opt_step": 142, "combined": 0.36908805, "ce": 0.41059637, "kd": 0.29200116, "gap_energy": 0.01557151, "kd_alpha": 0.35, "proof_weight": 1.12755102, "ce_num": 2398.20751953, "ce_den": 5840.79101562, "kd_num": 1705.51782227, "kd_den": 5840.79101562, "gap_num": 90.89710236, "gap_den": 5837.39697266, "kd_tokens": 3945.0, "teacher_topk_mass_num": 3590.37695312, "student_topk_mass_num": 3634.23754883, "mass_den": 3945.0, "lr": 3.054347031064272e-07, "difficulty": 9.548123359680176, "memory": "alloc=26.8GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}
{"epoch": 3, "batch_idx": 187, "global_step": 580, "opt_step": 145, "combined": 0.24322987, "ce": 0.23830642, "kd": 0.25237346, "gap_energy": 0.01909368, "kd_alpha": 0.35, "proof_weight": 1.1122449, "ce_num": 1339.30322266, "ce_den": 5620.08886719, "kd_num": 1418.36120605, "kd_den": 5620.08886719, "gap_num": 107.24766541, "gap_den": 5616.91796875, "kd_tokens": 3840.0, "teacher_topk_mass_num": 3661.22802734, "student_topk_mass_num": 3657.67651367, "mass_den": 3840.0, "lr": 3.0033988282761614e-07, "difficulty": 9.662643432617188, "memory": "alloc=20.4GiB reserved=39.8GiB peak=37.0GiB total=79.3GiB"}

356
training_artifacts/trainer_state.json Normal file
View File

@@ -0,0 +1,356 @@
{
"best_global_step": 142,
"best_metric": 0.22699420154094696,
"best_model_checkpoint": "/content/DualMind100_agentic/checkpoint-142",
"epoch": 2.0,
"eval_steps": 50,
"global_step": 142,
"is_hyper_param_search": false,
"is_local_process_zero": true,
"is_world_process_zero": true,
"log_history": [
{
"entropy": 0.16182193332642783,
"epoch": 0.07092198581560284,
"grad_norm": 0.5555469989776611,
"learning_rate": 5e-05,
"loss": 0.43506731986999514,
"mean_token_accuracy": 0.8946891106665135,
"num_tokens": 123430.0,
"step": 5
},
{
"entropy": 0.26333686951547863,
"epoch": 0.14184397163120568,
"grad_norm": 0.2982931435108185,
"learning_rate": 9.99862592554908e-05,
"loss": 0.3709733247756958,
"mean_token_accuracy": 0.898155527561903,
"num_tokens": 240929.0,
"step": 10
},
{
"entropy": 0.34238293119706215,
"epoch": 0.2127659574468085,
"grad_norm": 0.27323758602142334,
"learning_rate": 9.950612572673255e-05,
"loss": 0.3364511489868164,
"mean_token_accuracy": 0.906800027191639,
"num_tokens": 373968.0,
"step": 15
},
{
"entropy": 0.2675066887401044,
"epoch": 0.28368794326241137,
"grad_norm": 0.21929873526096344,
"learning_rate": 9.834648865751254e-05,
"loss": 0.2727418184280396,
"mean_token_accuracy": 0.9184959597885609,
"num_tokens": 494846.0,
"step": 20
},
{
"entropy": 0.2389053585124202,
"epoch": 0.3546099290780142,
"grad_norm": 0.19836418330669403,
"learning_rate": 9.652326481535435e-05,
"loss": 0.3025625705718994,
"mean_token_accuracy": 0.9214970305562019,
"num_tokens": 630306.0,
"step": 25
},
{
"entropy": 0.25811560889706014,
"epoch": 0.425531914893617,
"grad_norm": 0.19982172548770905,
"learning_rate": 9.406147912470143e-05,
"loss": 0.330981969833374,
"mean_token_accuracy": 0.9110702231526375,
"num_tokens": 758739.0,
"step": 30
},
{
"entropy": 0.24841026657959447,
"epoch": 0.49645390070921985,
"grad_norm": 0.16917800903320312,
"learning_rate": 9.099492118367123e-05,
"loss": 0.26140174865722654,
"mean_token_accuracy": 0.9228238843381404,
"num_tokens": 886920.0,
"step": 35
},
{
"entropy": 0.2569453418778721,
"epoch": 0.5673758865248227,
"grad_norm": 0.2460022270679474,
"learning_rate": 8.736568148000386e-05,
"loss": 0.28326830863952634,
"mean_token_accuracy": 0.9195088751614093,
"num_tokens": 1010775.0,
"step": 40
},
{
"entropy": 0.29926245249807837,
"epoch": 0.6382978723404256,
"grad_norm": 0.17467764019966125,
"learning_rate": 8.322357367194109e-05,
"loss": 0.3357896566390991,
"mean_token_accuracy": 0.9085973531007767,
"num_tokens": 1134373.0,
"step": 45
},
{
"entropy": 0.2604390908149071,
"epoch": 0.7092198581560284,
"grad_norm": 0.16242940723896027,
"learning_rate": 7.86254508636036e-05,
"loss": 0.2815252304077148,
"mean_token_accuracy": 0.9152372993528843,
"num_tokens": 1256155.0,
"step": 50
},
{
"epoch": 0.7092198581560284,
"eval_entropy": 0.22815538255963475,
"eval_loss": 0.23423044383525848,
"eval_mean_token_accuracy": 0.9300008341670036,
"eval_num_tokens": 1256155.0,
"eval_runtime": 5.3417,
"eval_samples_per_second": 8.986,
"eval_steps_per_second": 8.986,
"step": 50
},
{
"entropy": 0.2784213587117847,
"epoch": 0.7801418439716312,
"grad_norm": 0.1878730058670044,
"learning_rate": 7.363442525942826e-05,
"loss": 0.2845616340637207,
"mean_token_accuracy": 0.9107947684824467,
"num_tokens": 1378868.0,
"step": 55
},
{
"entropy": 0.29253384250914677,
"epoch": 0.851063829787234,
"grad_norm": 0.17584209144115448,
"learning_rate": 6.831900190841232e-05,
"loss": 0.31951193809509276,
"mean_token_accuracy": 0.9116400554776192,
"num_tokens": 1509648.0,
"step": 60
},
{
"entropy": 0.27282086764462293,
"epoch": 0.9219858156028369,
"grad_norm": 0.1782023161649704,
"learning_rate": 6.275213842808383e-05,
"loss": 0.28685288429260253,
"mean_token_accuracy": 0.9180420659482479,
"num_tokens": 1640179.0,
"step": 65
},
{
"entropy": 0.26427618057350627,
"epoch": 0.9929078014184397,
"grad_norm": 0.2283456027507782,
"learning_rate": 5.70102436140943e-05,
"loss": 0.29142701625823975,
"mean_token_accuracy": 0.9174253843724728,
"num_tokens": 1764592.0,
"step": 70
},
{
"entropy": 0.2524041742629682,
"epoch": 1.0567375886524824,
"grad_norm": 0.16881126165390015,
"learning_rate": 5.117212868016303e-05,
"loss": 0.276317834854126,
"mean_token_accuracy": 0.9248060741358333,
"num_tokens": 1874267.0,
"step": 75
},
{
"entropy": 0.29945559421321377,
"epoch": 1.127659574468085,
"grad_norm": 0.2186647355556488,
"learning_rate": 4.5317925523282464e-05,
"loss": 0.29511370658874514,
"mean_token_accuracy": 0.907706456631422,
"num_tokens": 1991985.0,
"step": 80
},
{
"entropy": 0.24618784408958164,
"epoch": 1.198581560283688,
"grad_norm": 0.15828007459640503,
"learning_rate": 3.952798686169279e-05,
"loss": 0.2731397867202759,
"mean_token_accuracy": 0.9240930497646331,
"num_tokens": 2118168.0,
"step": 85
},
{
"entropy": 0.2373207072203513,
"epoch": 1.2695035460992907,
"grad_norm": 0.20958004891872406,
"learning_rate": 3.388178334194232e-05,
"loss": 0.27325007915496824,
"mean_token_accuracy": 0.9249208129942417,
"num_tokens": 2240951.0,
"step": 90
},
{
"entropy": 0.22738294367445633,
"epoch": 1.3404255319148937,
"grad_norm": 0.19565178453922272,
"learning_rate": 2.8456812752951485e-05,
"loss": 0.25463979244232177,
"mean_token_accuracy": 0.9295149140059948,
"num_tokens": 2364856.0,
"step": 95
},
{
"entropy": 0.247153190523386,
"epoch": 1.4113475177304964,
"grad_norm": 0.1792992502450943,
"learning_rate": 2.3327536318821495e-05,
"loss": 0.2785398244857788,
"mean_token_accuracy": 0.9210605584084988,
"num_tokens": 2496977.0,
"step": 100
},
{
"epoch": 1.4113475177304964,
"eval_entropy": 0.2170733372719648,
"eval_loss": 0.22858719527721405,
"eval_mean_token_accuracy": 0.9310164228081703,
"eval_num_tokens": 2496977.0,
"eval_runtime": 5.44,
"eval_samples_per_second": 8.824,
"eval_steps_per_second": 8.824,
"step": 100
},
{
"entropy": 0.2818379683652893,
"epoch": 1.4822695035460993,
"grad_norm": 0.17297224700450897,
"learning_rate": 1.8564356670455767e-05,
"loss": 0.28323366641998293,
"mean_token_accuracy": 0.9150983460247517,
"num_tokens": 2625956.0,
"step": 105
},
{
"entropy": 0.25849783700541595,
"epoch": 1.5531914893617023,
"grad_norm": 0.17231720685958862,
"learning_rate": 1.4232651523993634e-05,
"loss": 0.26291708946228026,
"mean_token_accuracy": 0.9214990958571434,
"num_tokens": 2762411.0,
"step": 110
},
{
"entropy": 0.23584751789458097,
"epoch": 1.624113475177305,
"grad_norm": 0.19976992905139923,
"learning_rate": 1.0391876329443533e-05,
"loss": 0.24894940853118896,
"mean_token_accuracy": 0.9259149916470051,
"num_tokens": 2887514.0,
"step": 115
},
{
"entropy": 0.2430717541486956,
"epoch": 1.6950354609929077,
"grad_norm": 0.21702121198177338,
"learning_rate": 7.094748206242796e-06,
"loss": 0.2538208246231079,
"mean_token_accuracy": 0.9232472732663155,
"num_tokens": 3013243.0,
"step": 120
},
{
"entropy": 0.23513522319262847,
"epoch": 1.7659574468085106,
"grad_norm": 0.16074544191360474,
"learning_rate": 4.386522366755169e-06,
"loss": 0.22128725051879883,
"mean_token_accuracy": 0.9245993517339229,
"num_tokens": 3134144.0,
"step": 125
},
{
"entropy": 0.24327249150956048,
"epoch": 1.8368794326241136,
"grad_norm": 0.16139043867588043,
"learning_rate": 2.3043709592610485e-06,
"loss": 0.26496360301971433,
"mean_token_accuracy": 0.9262310199439525,
"num_tokens": 3260737.0,
"step": 130
},
{
"entropy": 0.23524304252350703,
"epoch": 1.9078014184397163,
"grad_norm": 0.16922682523727417,
"learning_rate": 8.768728562211947e-07,
"loss": 0.2534662961959839,
"mean_token_accuracy": 0.924710838496685,
"num_tokens": 3383649.0,
"step": 135
},
{
"entropy": 0.2577046745980624,
"epoch": 1.978723404255319,
"grad_norm": 0.18219797313213348,
"learning_rate": 1.236213907994943e-07,
"loss": 0.2727572679519653,
"mean_token_accuracy": 0.9200983986258506,
"num_tokens": 3511363.0,
"step": 140
},
{
"epoch": 2.0,
"eval_entropy": 0.21362574538215995,
"eval_loss": 0.22699420154094696,
"eval_mean_token_accuracy": 0.9318861390153567,
"eval_num_tokens": 3550882.0,
"eval_runtime": 5.4445,
"eval_samples_per_second": 8.816,
"eval_steps_per_second": 8.816,
"step": 142
},
{
"epoch": 2.0,
"step": 142,
"total_flos": 3.08394581409792e+16,
"train_loss": 0.2892465377357644,
"train_runtime": 1218.7991,
"train_samples_per_second": 1.851,
"train_steps_per_second": 0.117
}
],
"logging_steps": 5,
"max_steps": 142,
"num_input_tokens_seen": 0,
"num_train_epochs": 2,
"save_steps": 50,
"stateful_callbacks": {
"TrainerControl": {
"args": {
"should_epoch_stop": false,
"should_evaluate": false,
"should_log": false,
"should_save": true,
"should_training_stop": true
},
"attributes": {}
}
},
"total_flos": 3.08394581409792e+16,
"train_batch_size": 1,
"trial_name": null,
"trial_params": null
}