初始化项目，由ModelHub XC社区提供模型
Model: laion/a3-rl-DCAgent_exp_rpt_curriculum-medium-10-8B Source: Original Platform
This commit is contained in:
ModelHub XC
39
.gitattributes
vendored
Normal file
39
.gitattributes
vendored
Normal file
@@ -0,0 +1,39 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||||
|
training_logs/20260605_214218_reward_vs_steps.png filter=lfs diff=lfs merge=lfs -text
|
||||||
|
training_logs/20260605_214218_turn_count_distribution.png filter=lfs diff=lfs merge=lfs -text
|
||||||
|
training_logs/a3-rl-DCAgent_exp_rpt_curriculum-medium_591150.out filter=lfs diff=lfs merge=lfs -text
|
||||||
16
README.md
Normal file
16
README.md
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
# a3-rl-DCAgent_exp_rpt_curriculum-medium-10-8B
|
||||||
|
|
||||||
|
RL (SkyRL/terminus-2) fine-tune of `laion/GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink` on `DCAgent/exp_rpt_curriculum-medium` (512-task set, 2 epochs).
|
||||||
|
|
||||||
|
Checkpoint = **global_step_10**, selected by EMA-best of `reward/avg_raw_reward` (5-step EMA, α=1/3) across the full chain (job 591150). EMA at step 10 = 0.3106 (reward 0.2812). Data-limited run; training metrics emitted through step 16, checkpoints exported through step 21.
|
||||||
|
|
||||||
|
## Training Traces
|
||||||
|
|
||||||
|
Training-time Daytona/Harbor rollouts for this run are uploaded as a companion dataset:
|
||||||
|
**[penfever/a3-rl-DCAgent_exp_rpt_curriculum-medium](https://huggingface.co/datasets/penfever/a3-rl-DCAgent_exp_rpt_curriculum-medium)**
|
||||||
|
|
||||||
|
The dataset contains the `last` episode of each trial (per `make_and_upload_trace_dataset --episodes last`) — the same rollouts the policy was trained on after rollback / truncation.
|
||||||
|
|
||||||
|
## Training logs
|
||||||
|
|
||||||
|
See `training_logs/` for per-step metrics CSVs, the metrics report, and the reward-vs-steps plot (Jupiter has no W&B).
|
||||||
28
added_tokens.json
Normal file
28
added_tokens.json
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
{
|
||||||
|
"</think>": 151668,
|
||||||
|
"</tool_call>": 151658,
|
||||||
|
"</tool_response>": 151666,
|
||||||
|
"<think>": 151667,
|
||||||
|
"<tool_call>": 151657,
|
||||||
|
"<tool_response>": 151665,
|
||||||
|
"<|box_end|>": 151649,
|
||||||
|
"<|box_start|>": 151648,
|
||||||
|
"<|endoftext|>": 151643,
|
||||||
|
"<|file_sep|>": 151664,
|
||||||
|
"<|fim_middle|>": 151660,
|
||||||
|
"<|fim_pad|>": 151662,
|
||||||
|
"<|fim_prefix|>": 151659,
|
||||||
|
"<|fim_suffix|>": 151661,
|
||||||
|
"<|im_end|>": 151645,
|
||||||
|
"<|im_start|>": 151644,
|
||||||
|
"<|image_pad|>": 151655,
|
||||||
|
"<|object_ref_end|>": 151647,
|
||||||
|
"<|object_ref_start|>": 151646,
|
||||||
|
"<|quad_end|>": 151651,
|
||||||
|
"<|quad_start|>": 151650,
|
||||||
|
"<|repo_name|>": 151663,
|
||||||
|
"<|video_pad|>": 151656,
|
||||||
|
"<|vision_end|>": 151653,
|
||||||
|
"<|vision_pad|>": 151654,
|
||||||
|
"<|vision_start|>": 151652
|
||||||
|
}
|
||||||
89
chat_template.jinja
Normal file
89
chat_template.jinja
Normal file
@@ -0,0 +1,89 @@
|
|||||||
|
{%- if tools %}
|
||||||
|
{{- '<|im_start|>system\n' }}
|
||||||
|
{%- if messages[0].role == 'system' %}
|
||||||
|
{{- messages[0].content + '\n\n' }}
|
||||||
|
{%- endif %}
|
||||||
|
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
||||||
|
{%- for tool in tools %}
|
||||||
|
{{- "\n" }}
|
||||||
|
{{- tool | tojson }}
|
||||||
|
{%- endfor %}
|
||||||
|
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
||||||
|
{%- else %}
|
||||||
|
{%- if messages[0].role == 'system' %}
|
||||||
|
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
||||||
|
{%- for message in messages[::-1] %}
|
||||||
|
{%- set index = (messages|length - 1) - loop.index0 %}
|
||||||
|
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
|
||||||
|
{%- set ns.multi_step_tool = false %}
|
||||||
|
{%- set ns.last_query_index = index %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endfor %}
|
||||||
|
{%- for message in messages %}
|
||||||
|
{%- if message.content is string %}
|
||||||
|
{%- set content = message.content %}
|
||||||
|
{%- else %}
|
||||||
|
{%- set content = '' %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
||||||
|
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
||||||
|
{%- elif message.role == "assistant" %}
|
||||||
|
{%- set reasoning_content = '' %}
|
||||||
|
{%- if message.reasoning_content is string %}
|
||||||
|
{%- set reasoning_content = message.reasoning_content %}
|
||||||
|
{%- else %}
|
||||||
|
{%- if '</think>' in content %}
|
||||||
|
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
||||||
|
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- if loop.index0 > ns.last_query_index %}
|
||||||
|
{%- if loop.last or (not loop.last and reasoning_content) %}
|
||||||
|
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
|
||||||
|
{%- else %}
|
||||||
|
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- else %}
|
||||||
|
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- if message.tool_calls %}
|
||||||
|
{%- for tool_call in message.tool_calls %}
|
||||||
|
{%- if (loop.first and content) or (not loop.first) %}
|
||||||
|
{{- '\n' }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- if tool_call.function %}
|
||||||
|
{%- set tool_call = tool_call.function %}
|
||||||
|
{%- endif %}
|
||||||
|
{{- '<tool_call>\n{"name": "' }}
|
||||||
|
{{- tool_call.name }}
|
||||||
|
{{- '", "arguments": ' }}
|
||||||
|
{%- if tool_call.arguments is string %}
|
||||||
|
{{- tool_call.arguments }}
|
||||||
|
{%- else %}
|
||||||
|
{{- tool_call.arguments | tojson }}
|
||||||
|
{%- endif %}
|
||||||
|
{{- '}\n</tool_call>' }}
|
||||||
|
{%- endfor %}
|
||||||
|
{%- endif %}
|
||||||
|
{{- '<|im_end|>\n' }}
|
||||||
|
{%- elif message.role == "tool" %}
|
||||||
|
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
||||||
|
{{- '<|im_start|>user' }}
|
||||||
|
{%- endif %}
|
||||||
|
{{- '\n<tool_response>\n' }}
|
||||||
|
{{- content }}
|
||||||
|
{{- '\n</tool_response>' }}
|
||||||
|
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
||||||
|
{{- '<|im_end|>\n' }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endfor %}
|
||||||
|
{%- if add_generation_prompt %}
|
||||||
|
{{- '<|im_start|>assistant\n' }}
|
||||||
|
{%- if enable_thinking is defined and enable_thinking is false %}
|
||||||
|
{{- '<think>\n\n</think>\n\n' }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endif %}
|
||||||
68
config.json
Normal file
68
config.json
Normal file
@@ -0,0 +1,68 @@
|
|||||||
|
{
|
||||||
|
"architectures": [
|
||||||
|
"Qwen3ForCausalLM"
|
||||||
|
],
|
||||||
|
"attention_bias": false,
|
||||||
|
"attention_dropout": 0.0,
|
||||||
|
"dtype": "bfloat16",
|
||||||
|
"eos_token_id": 151645,
|
||||||
|
"head_dim": 128,
|
||||||
|
"hidden_act": "silu",
|
||||||
|
"hidden_size": 4096,
|
||||||
|
"initializer_range": 0.02,
|
||||||
|
"intermediate_size": 12288,
|
||||||
|
"layer_types": [
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention"
|
||||||
|
],
|
||||||
|
"max_position_embeddings": 40960,
|
||||||
|
"max_window_layers": 36,
|
||||||
|
"model_type": "qwen3",
|
||||||
|
"num_attention_heads": 32,
|
||||||
|
"num_hidden_layers": 36,
|
||||||
|
"num_key_value_heads": 8,
|
||||||
|
"pad_token_id": 151643,
|
||||||
|
"rms_norm_eps": 1e-06,
|
||||||
|
"rope_scaling": null,
|
||||||
|
"rope_theta": 1000000,
|
||||||
|
"sliding_window": null,
|
||||||
|
"tie_word_embeddings": false,
|
||||||
|
"transformers_version": "4.57.6",
|
||||||
|
"use_cache": false,
|
||||||
|
"use_sliding_window": false,
|
||||||
|
"vocab_size": 151936
|
||||||
|
}
|
||||||
12
generation_config.json
Normal file
12
generation_config.json
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
{
|
||||||
|
"do_sample": true,
|
||||||
|
"eos_token_id": [
|
||||||
|
151645,
|
||||||
|
151643
|
||||||
|
],
|
||||||
|
"pad_token_id": 151643,
|
||||||
|
"temperature": 0.6,
|
||||||
|
"top_k": 20,
|
||||||
|
"top_p": 0.95,
|
||||||
|
"transformers_version": "4.57.6"
|
||||||
|
}
|
||||||
151388
merges.txt
Normal file
151388
merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
model-00001-of-00004.safetensors
Normal file
3
model-00001-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:1e911ab206de493dd02085d97c615aa547f31defcf0a4ba0c60543a855a66f23
|
||||||
|
size 4902257696
|
||||||
3
model-00002-of-00004.safetensors
Normal file
3
model-00002-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:dba219f6efd68b23afcc80c408f35c1a519b1b30fb5ca61e1d587aec689a915e
|
||||||
|
size 4915960368
|
||||||
3
model-00003-of-00004.safetensors
Normal file
3
model-00003-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:02b783a314332ef6bb41e32e85081ee01cbbacc104972db23061ea45d830137f
|
||||||
|
size 4983068496
|
||||||
3
model-00004-of-00004.safetensors
Normal file
3
model-00004-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:c7e927c6178188b0e0c536afcbb31a0997dde99ef96c50091daa72ff5e4cfa78
|
||||||
|
size 1580230264
|
||||||
407
model.safetensors.index.json
Normal file
407
model.safetensors.index.json
Normal file
@@ -0,0 +1,407 @@
|
|||||||
|
{
|
||||||
|
"metadata": {
|
||||||
|
"total_parameters": 8190735360,
|
||||||
|
"total_size": 16381470720
|
||||||
|
},
|
||||||
|
"weight_map": {
|
||||||
|
"lm_head.weight": "model-00004-of-00004.safetensors",
|
||||||
|
"model.embed_tokens.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.0.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.0.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.1.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.1.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.10.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.10.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.10.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.10.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.10.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.10.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.11.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.11.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.11.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.11.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.11.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.12.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.12.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.12.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.12.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.12.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.12.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.13.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.13.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.13.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.13.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.13.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.13.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.13.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.13.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.13.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.14.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.14.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.14.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.14.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.14.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.15.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.15.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.15.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.15.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.15.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.15.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.16.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.16.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.16.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.16.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.16.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.16.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.16.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.16.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.16.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.16.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.16.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.17.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.17.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.17.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.17.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.17.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.17.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.17.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.17.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.17.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.17.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.17.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.18.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.18.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.18.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.18.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.18.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.18.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.18.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.18.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.18.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.18.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.18.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.19.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.19.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.19.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.19.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.19.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.19.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.19.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.19.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.19.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.19.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.19.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.2.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.2.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.20.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.20.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.20.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.20.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.20.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.20.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.20.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.20.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.20.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.20.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.20.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.21.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.21.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.21.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.21.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.21.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.21.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.21.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.21.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.21.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.21.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.21.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.22.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.22.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.22.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.22.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.22.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.22.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.23.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.23.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.23.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.23.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.24.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.24.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.24.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.24.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.24.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.24.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.24.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.24.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.24.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.24.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.25.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.25.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.25.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.25.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.25.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.25.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.25.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.25.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.25.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.25.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.25.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.26.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.26.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.26.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.26.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.26.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.26.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.26.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.26.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.26.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.26.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.26.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.27.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.27.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.27.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.27.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.27.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.27.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.27.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.27.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.27.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.27.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.27.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.28.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.28.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.28.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.28.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.28.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.28.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.28.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.28.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.28.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.28.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.28.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.29.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.29.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.29.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.29.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.29.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.29.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.29.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.29.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.29.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.29.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.29.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.3.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.3.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.30.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.30.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.30.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.30.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.30.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.30.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.30.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.30.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.30.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.30.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.30.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.31.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.31.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.31.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.31.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.31.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.31.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.31.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.31.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.31.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.31.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.31.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.32.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.32.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.32.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.32.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.32.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.32.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.32.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.32.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.32.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.32.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.32.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.33.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.33.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.33.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.33.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.33.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.33.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.33.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.33.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.33.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.33.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.33.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.34.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.34.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.34.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.34.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.34.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.34.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.34.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.34.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.34.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.34.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.34.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.35.input_layernorm.weight": "model-00004-of-00004.safetensors",
|
||||||
|
"model.layers.35.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
|
||||||
|
"model.layers.35.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
|
||||||
|
"model.layers.35.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
|
||||||
|
"model.layers.35.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
|
||||||
|
"model.layers.35.self_attn.k_norm.weight": "model-00004-of-00004.safetensors",
|
||||||
|
"model.layers.35.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.35.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
|
||||||
|
"model.layers.35.self_attn.q_norm.weight": "model-00004-of-00004.safetensors",
|
||||||
|
"model.layers.35.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.35.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||||
|
"model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.4.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.4.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.5.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.5.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.5.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.5.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.5.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.6.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.6.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.6.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.6.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.6.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.6.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.6.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.6.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.6.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.6.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.6.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.7.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.7.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.7.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.7.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.7.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.7.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.7.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.7.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.7.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.7.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.7.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.8.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.8.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.8.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.8.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.8.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.8.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.8.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.8.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.8.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.8.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.8.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.9.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.9.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||||
|
"model.layers.9.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.9.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.9.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.9.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.9.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.layers.9.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||||
|
"model.norm.weight": "model-00004-of-00004.safetensors"
|
||||||
|
}
|
||||||
|
}
|
||||||
316
rl_config.yaml
Normal file
316
rl_config.yaml
Normal file
@@ -0,0 +1,316 @@
|
|||||||
|
entrypoint: examples.terminal_bench.entrypoints.main_tbench
|
||||||
|
|
||||||
|
# Hydra config groups (+ prefix in CLI)
|
||||||
|
config_groups:
|
||||||
|
terminal_bench_config: terminal_bench
|
||||||
|
|
||||||
|
# Terminal bench / agentic environment settings
|
||||||
|
terminal_bench:
|
||||||
|
# trials_dir: Directory for Harbor trial artifacts (derived from experiments_dir if null)
|
||||||
|
trials_dir: null
|
||||||
|
|
||||||
|
# Harbor configuration - schema-driven mapping to TrialConfig
|
||||||
|
harbor:
|
||||||
|
# Agent settings
|
||||||
|
name: terminus-2
|
||||||
|
max_episodes: 999999
|
||||||
|
enable_summarize: false
|
||||||
|
store_all_messages: true
|
||||||
|
trajectory_config:
|
||||||
|
raw_content: true
|
||||||
|
enable_episode_logging: false
|
||||||
|
record_terminal_session: false
|
||||||
|
enable_pane_logging: false
|
||||||
|
|
||||||
|
# Strict JSON parser
|
||||||
|
strict_json_parser: true
|
||||||
|
|
||||||
|
# Interleaved Thinking Settings
|
||||||
|
interleaved_thinking: true
|
||||||
|
extra_body:
|
||||||
|
chat_template_kwargs:
|
||||||
|
enable_thinking: true
|
||||||
|
# 2026-05-27: 1800 → 900. The boundary-hugging a3 datasets (stack-junit,
|
||||||
|
# nemotron, methods2test) hit the 1800s agent-timeout wall on ~70-95% of
|
||||||
|
# trials (GLM-4_7-swesmith can't finish these in 1800s), so every group
|
||||||
|
# waited ~1800s for its slowest sample → generation buffer crawled (5/64
|
||||||
|
# in 4h19) and chains never reached training step 1. Halving to 900s
|
||||||
|
# ~2× the valid-group yield. Tradeoff: lower solve-rate on the hardest
|
||||||
|
# tasks (they fail faster) — acceptable vs. TIMEOUT-at-step-0. Fast-task
|
||||||
|
# chains (e2egit, median 7 turns / <900s) are unaffected. See
|
||||||
|
# agent_logs/2026-05-27_rl_chains_stuck_step0.md. AgentTimeoutError stays
|
||||||
|
# PASSTHROUGH (standing pref — partial trajectory+reward is meaningful).
|
||||||
|
override_timeout_sec: 900
|
||||||
|
|
||||||
|
# Environment settings
|
||||||
|
override_cpus: 1
|
||||||
|
override_memory_mb: 2048
|
||||||
|
override_storage_mb: 2048
|
||||||
|
|
||||||
|
# ==========================================================================
|
||||||
|
# AUTO SNAPSHOT: Reduce Daytona rate limits with hash-based snapshot caching
|
||||||
|
# ==========================================================================
|
||||||
|
# When true, automatically creates a snapshot from the Dockerfile on first use,
|
||||||
|
# then reuses it for all subsequent sandboxes with the same Dockerfile content.
|
||||||
|
# Snapshots are named: harbor__<sha256[:12]>__snapshot
|
||||||
|
auto_snapshot: true
|
||||||
|
|
||||||
|
# Verifier settings
|
||||||
|
verifier_override_timeout_sec: 120
|
||||||
|
|
||||||
|
# Retry settings
|
||||||
|
max_retries: 3
|
||||||
|
min_wait_sec: 60.0
|
||||||
|
max_wait_sec: 600.0
|
||||||
|
wait_multiplier: 2.0
|
||||||
|
|
||||||
|
exclude_exceptions:
|
||||||
|
- VerifierTimeoutError
|
||||||
|
- VerifierRuntimeError
|
||||||
|
- RewardFileNotFoundError
|
||||||
|
- RewardFileEmptyError
|
||||||
|
- VerifierOutputParseError
|
||||||
|
|
||||||
|
# 3x base (300 → 900): triple Daytona concurrency for higher throughput.
|
||||||
|
# Mirrors ALCC/56GPU_base.yaml — per-job sandbox count; doesn't affect
|
||||||
|
# the cross-cluster RUNNING-RL cap (≤ 6) which is about job count, not
|
||||||
|
# sandboxes-per-job.
|
||||||
|
# NOTE (2026-05-27): a temporary 900→500 walk-back was tried to mitigate
|
||||||
|
# the FD-exhaustion / uv__epoll_ctl_prep SIGABRT but REVERTED — the real
|
||||||
|
# fix is porting harbor's terminus-2 FD-taming features from
|
||||||
|
# penfever/temp-override into penfever/otagent-latest (that branch had the
|
||||||
|
# issue tamed at full concurrency). See agent_logs/2026-05-27_rl_chains_stuck_step0.md.
|
||||||
|
# 2026-05-28: 900 → 675 (-25%). Fresh a3 chains (#12/#13) abort with a
|
||||||
|
# driver SIGABRT that is NOT FD exhaustion (fd-monitor showed 1.6% of the
|
||||||
|
# 131072 limit) — suspected host-memory OOM from too many concurrent
|
||||||
|
# never-completing trials piling up resident state in skyrl_entrypoint on
|
||||||
|
# hard datasets. Cutting concurrent trial count reduces that resident
|
||||||
|
# memory. Paired with num_parallel_generation_workers -25%. See
|
||||||
|
# agent_logs/2026-05-28_fresh_a3_chain_crash_not_fd.md.
|
||||||
|
n_concurrent_trials: 675
|
||||||
|
|
||||||
|
# Logging settings
|
||||||
|
log_level: INFO
|
||||||
|
|
||||||
|
# Reward shaping (disabled - binary rewards)
|
||||||
|
enable_reward_shaping: false
|
||||||
|
|
||||||
|
# RLOO-N error classification
|
||||||
|
enable_error_classification: true
|
||||||
|
mask_exceptions:
|
||||||
|
- DaytonaError
|
||||||
|
- EnvironmentStartTimeoutError
|
||||||
|
- NetworkError
|
||||||
|
- ConnectionError
|
||||||
|
- RewardFileNotFoundError
|
||||||
|
- RewardFileEmptyError
|
||||||
|
- AgentEnvironmentTimeoutError
|
||||||
|
- ContextLengthExceededError
|
||||||
|
default_error_treatment: zero
|
||||||
|
# NOTE (2026-05-27): AgentTimeoutError / ContextLengthExceededError moved
|
||||||
|
# OUT of passthrough_exceptions into zero_exceptions. Passthrough routed
|
||||||
|
# these soft-limit trials through the *normal* (live-trajectory) path in
|
||||||
|
# terminal_bench_generator._process_trial_result, keeping the trial's
|
||||||
|
# dangling vLLM ObjectRefs + orphaned litellm async-callback closures
|
||||||
|
# alive in the training batch. That fed Ray's distributed-refcount race
|
||||||
|
# (reference_count.cc:1619 → SIGABRT / WorkerCrashedError) that killed the
|
||||||
|
# nemotron-junit chain (litellm +1 32769 VLLMValidationError +
|
||||||
|
# ContextLengthExceeded on near-budget prompts). Both mask + zero take the
|
||||||
|
# EARLY-RETURN path (response_ids=[0]) which never consumes rollout_details,
|
||||||
|
# so no dangling refs enter training. 2026-05-27: ContextLengthExceededError
|
||||||
|
# moved to mask_exceptions (above) — excluded from the RLOO-N baseline, since
|
||||||
|
# GLM-4_7-swesmith ~32k prompts overflow routinely and we don't want hard
|
||||||
|
# zeros dragging the baseline. AgentTimeoutError stays PASSTHROUGH (partial trajectory+reward is
|
||||||
|
# meaningful signal; infrequent vs ContextLengthExceeded so low refcount risk). Belt-and-suspenders w/ the _harbor_compat rollback_hook that
|
||||||
|
# already truncates these trials' dangling prompt-without-response.
|
||||||
|
passthrough_exceptions:
|
||||||
|
- AgentTimeoutError
|
||||||
|
zero_exceptions: []
|
||||||
|
|
||||||
|
# Model info for Harbor's hosted_vllm validation
|
||||||
|
model_info:
|
||||||
|
# Lowered 32767 -> 32000 (2026-05-27). Harbor's litellm token counter is
|
||||||
|
# +2 below vLLM's actual BPE tokenization on near-budget prompts (litellm
|
||||||
|
# +1 plus chat-template/special-token drift), so a 32767-capped prompt
|
||||||
|
# tokenizes to 32769 at the vLLM serving layer and is rejected with
|
||||||
|
# `VLLMValidationError: 32769 input tokens ... context length is only
|
||||||
|
# 32768`. vLLM returns this as an ErrorResponse (handled, not an engine
|
||||||
|
# crash), but harbor's litellm then retries the *deterministically*
|
||||||
|
# over-budget request (max_retries=3, exp backoff) across n_concurrent
|
||||||
|
# trials. On a boundary-heavy task distribution (e.g. nemotron-junit: 111+
|
||||||
|
# such overflows per chain link) the accumulating open sockets exhaust the
|
||||||
|
# skyrl_entrypoint actor's file descriptors -> libuv uv__epoll_ctl_prep
|
||||||
|
# aborts -> SIGABRT in the uvloop event loop -> ray.WorkerCrashedError kills
|
||||||
|
# the chain before training step 1 (job a3-rl ...nemotron-junit #11,
|
||||||
|
# chain 521442-448). A 768-token buffer below max_model_len=32768 ensures
|
||||||
|
# harbor truncates before vLLM ever rejects, eliminating the retry storm.
|
||||||
|
# Cost is ~2.3% usable context for all a3 chains; chains that don't hug the
|
||||||
|
# boundary (e2egit reached step 68 fine at 32767) are unaffected.
|
||||||
|
max_input_tokens: 32000
|
||||||
|
max_output_tokens: 4096
|
||||||
|
|
||||||
|
archiving:
|
||||||
|
# Enable trial archiving callback
|
||||||
|
enabled: false
|
||||||
|
|
||||||
|
# Post-training trace upload to HuggingFace
|
||||||
|
trace_upload:
|
||||||
|
enabled: true
|
||||||
|
repo_org: DCAgent
|
||||||
|
episodes: last
|
||||||
|
dataset_type: SFT
|
||||||
|
cleanup: true
|
||||||
|
|
||||||
|
# Trainer configuration
|
||||||
|
trainer:
|
||||||
|
strategy: fsdp2
|
||||||
|
algorithm:
|
||||||
|
advantage_estimator: rloo_n
|
||||||
|
use_kl_loss: false
|
||||||
|
kl_loss_coef: 0.0
|
||||||
|
eps_clip_low: 0.2
|
||||||
|
# eps_clip_high=0.05 mirrors 24GPU_base — midpoint between 0.2 default and
|
||||||
|
# 0.01 tight. Engages on collapse-onset ratios without over-clamping
|
||||||
|
# healthy updates. Asymmetric — only tightening upper bound.
|
||||||
|
eps_clip_high: 0.05
|
||||||
|
loss_reduction: token_mean
|
||||||
|
|
||||||
|
# Training loop settings
|
||||||
|
epochs: 2
|
||||||
|
max_steps: 80
|
||||||
|
update_epochs_per_batch: 1
|
||||||
|
|
||||||
|
# Batch sizes
|
||||||
|
train_batch_size: 64
|
||||||
|
policy_mini_batch_size: 64
|
||||||
|
eval_batch_size: 64
|
||||||
|
|
||||||
|
# Micro batch sizes (micro1x4 variant)
|
||||||
|
micro_forward_batch_size_per_gpu: 4
|
||||||
|
micro_train_batch_size_per_gpu: 1
|
||||||
|
|
||||||
|
max_prompt_length: 999999
|
||||||
|
|
||||||
|
# Evaluation and checkpointing
|
||||||
|
eval_interval: 999999
|
||||||
|
eval_before_train: false
|
||||||
|
# Resumable checkpointing
|
||||||
|
ckpt_interval: 2
|
||||||
|
resume_mode: latest
|
||||||
|
# HF upload-ready checkpoints
|
||||||
|
hf_save_interval: 5
|
||||||
|
# HuggingFace Hub upload (set via CLI: trainer.hf_hub_repo_id=org/repo)
|
||||||
|
hf_hub_repo_id: null
|
||||||
|
hf_hub_private: false
|
||||||
|
hf_hub_revision: main
|
||||||
|
|
||||||
|
# Database registration (auto-registers trained model to Supabase)
|
||||||
|
# Requires KEYS env var pointing to Supabase credentials file
|
||||||
|
enable_db_registration: false
|
||||||
|
|
||||||
|
# Logging
|
||||||
|
project_name: OpenThoughts-Agent
|
||||||
|
log_level: INFO
|
||||||
|
tracker_commit_each_step: true
|
||||||
|
logger: console
|
||||||
|
|
||||||
|
# Paths
|
||||||
|
run_name: null
|
||||||
|
ckpt_path: null
|
||||||
|
export_path: null
|
||||||
|
|
||||||
|
# Policy optimizer
|
||||||
|
# max_grad_norm=0.9 mirrors 24GPU_base — guardrail against grad-norm spikes
|
||||||
|
# entering correlation-mode-collapse territory (>1.0). 0.9 is just above the
|
||||||
|
# natural healthy peak observed on this dataset+base (‖g‖ peak ~0.81).
|
||||||
|
policy:
|
||||||
|
optimizer_config:
|
||||||
|
lr: 8e-6
|
||||||
|
weight_decay: 0.0
|
||||||
|
adam_betas: [0.9, 0.999]
|
||||||
|
max_grad_norm: 0.9
|
||||||
|
fsdp_config:
|
||||||
|
cpu_offload: false
|
||||||
|
reshard_after_forward: true
|
||||||
|
fsdp_size: 4
|
||||||
|
# Reference model
|
||||||
|
ref:
|
||||||
|
fsdp_config:
|
||||||
|
cpu_offload: false
|
||||||
|
reshard_after_forward: true
|
||||||
|
fsdp_size: 4
|
||||||
|
|
||||||
|
# Model placement (async training) - 8 shared GPUs for policy/ref
|
||||||
|
placement:
|
||||||
|
colocate_all: false
|
||||||
|
policy_num_nodes: 2
|
||||||
|
ref_num_nodes: 2
|
||||||
|
policy_num_gpus_per_node: 4
|
||||||
|
ref_num_gpus_per_node: 4
|
||||||
|
|
||||||
|
# Fully async generation (settings from v2_maxconcurrent)
|
||||||
|
fully_async:
|
||||||
|
max_staleness_steps: 16
|
||||||
|
# Setting conservatively to 1 / 2 of total concurrency
|
||||||
|
# 2026-05-28: 450 → 338 (-25%), paired with n_concurrent_trials 900→675,
|
||||||
|
# to cut driver memory pressure (suspected OOM on fresh hard-dataset a3
|
||||||
|
# chains). See agent_logs/2026-05-28_fresh_a3_chain_crash_not_fd.md.
|
||||||
|
num_parallel_generation_workers: 338
|
||||||
|
|
||||||
|
# Generator configuration
|
||||||
|
generator:
|
||||||
|
backend: vllm
|
||||||
|
timeout_multiplier: 1.0
|
||||||
|
model_dtype: bfloat16
|
||||||
|
|
||||||
|
inference_engine_tensor_parallel_size: 1
|
||||||
|
# 3x base (16 → 48): triple vLLM engines to lift gen throughput above training rate.
|
||||||
|
# Goal: build a real surplus of completed groups so the trainer never waits on gen.
|
||||||
|
# Layout: 48 engines × TP=1 + 8 GPUs for policy/ref = 56 GPUs total = 14 nodes.
|
||||||
|
num_inference_engines: 48
|
||||||
|
|
||||||
|
n_samples_per_prompt: 8
|
||||||
|
eval_n_samples_per_prompt: 8
|
||||||
|
|
||||||
|
# Jupiter-specific gpu_memory_utilization: 0.75 (vs ALCC's 0.85).
|
||||||
|
# GH200's 96 GB HBM has more headroom than ALCC's A100-80, but Jupiter
|
||||||
|
# also runs hosted_vllm in the same Ray cluster as the engines and we
|
||||||
|
# keep 0.75 across all jupiter yamls for consistency / fragmentation
|
||||||
|
# safety. Bump to 0.85 only if KV-cache pressure justifies it.
|
||||||
|
gpu_memory_utilization: 0.75
|
||||||
|
|
||||||
|
max_num_seqs: 24
|
||||||
|
# Jupiter-specific 65536 (vs ALCC's 16384) — matches the existing
|
||||||
|
# 24GPU_base.yaml on Jupiter; larger batched-token budget keeps the
|
||||||
|
# engine fed when many concurrent requests arrive in bursts.
|
||||||
|
max_num_batched_tokens: 65536
|
||||||
|
|
||||||
|
enable_prefix_caching: true
|
||||||
|
enable_chunked_prefill: true
|
||||||
|
|
||||||
|
run_engines_locally: true
|
||||||
|
weight_sync_backend: nccl
|
||||||
|
async_engine: true
|
||||||
|
batched: false
|
||||||
|
enable_http_endpoint: true
|
||||||
|
enable_ray_prometheus_stats: false
|
||||||
|
vllm_stats_interval: 1
|
||||||
|
append_eos_token_after_stop_str_in_multi_turn: true
|
||||||
|
max_turns: 999999
|
||||||
|
|
||||||
|
sampling_params:
|
||||||
|
max_generate_length: 4096
|
||||||
|
temperature: 0.7
|
||||||
|
top_p: 0.95
|
||||||
|
top_k: 20
|
||||||
|
|
||||||
|
engine_init_kwargs:
|
||||||
|
max_model_len: 32768
|
||||||
|
# Interleaved thinking chat template: preserves <think> blocks on ALL
|
||||||
|
# historical assistant turns (stock Qwen3 template strips them).
|
||||||
|
custom_chat_template_chat_completion_path: chat_templates/qwen3_thinking_acc.jinja2
|
||||||
|
|
||||||
|
# Data paths
|
||||||
|
data:
|
||||||
|
train_data: []
|
||||||
|
val_data: []
|
||||||
31
special_tokens_map.json
Normal file
31
special_tokens_map.json
Normal file
@@ -0,0 +1,31 @@
|
|||||||
|
{
|
||||||
|
"additional_special_tokens": [
|
||||||
|
"<|im_start|>",
|
||||||
|
"<|im_end|>",
|
||||||
|
"<|object_ref_start|>",
|
||||||
|
"<|object_ref_end|>",
|
||||||
|
"<|box_start|>",
|
||||||
|
"<|box_end|>",
|
||||||
|
"<|quad_start|>",
|
||||||
|
"<|quad_end|>",
|
||||||
|
"<|vision_start|>",
|
||||||
|
"<|vision_end|>",
|
||||||
|
"<|vision_pad|>",
|
||||||
|
"<|image_pad|>",
|
||||||
|
"<|video_pad|>"
|
||||||
|
],
|
||||||
|
"eos_token": {
|
||||||
|
"content": "<|im_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false
|
||||||
|
},
|
||||||
|
"pad_token": {
|
||||||
|
"content": "<|endoftext|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false
|
||||||
|
}
|
||||||
|
}
|
||||||
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
Binary file not shown.
240
tokenizer_config.json
Normal file
240
tokenizer_config.json
Normal file
@@ -0,0 +1,240 @@
|
|||||||
|
{
|
||||||
|
"add_bos_token": false,
|
||||||
|
"add_prefix_space": false,
|
||||||
|
"added_tokens_decoder": {
|
||||||
|
"151643": {
|
||||||
|
"content": "<|endoftext|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151644": {
|
||||||
|
"content": "<|im_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151645": {
|
||||||
|
"content": "<|im_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151646": {
|
||||||
|
"content": "<|object_ref_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151647": {
|
||||||
|
"content": "<|object_ref_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151648": {
|
||||||
|
"content": "<|box_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151649": {
|
||||||
|
"content": "<|box_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151650": {
|
||||||
|
"content": "<|quad_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151651": {
|
||||||
|
"content": "<|quad_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151652": {
|
||||||
|
"content": "<|vision_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151653": {
|
||||||
|
"content": "<|vision_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151654": {
|
||||||
|
"content": "<|vision_pad|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151655": {
|
||||||
|
"content": "<|image_pad|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151656": {
|
||||||
|
"content": "<|video_pad|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151657": {
|
||||||
|
"content": "<tool_call>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151658": {
|
||||||
|
"content": "</tool_call>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151659": {
|
||||||
|
"content": "<|fim_prefix|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151660": {
|
||||||
|
"content": "<|fim_middle|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151661": {
|
||||||
|
"content": "<|fim_suffix|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151662": {
|
||||||
|
"content": "<|fim_pad|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151663": {
|
||||||
|
"content": "<|repo_name|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151664": {
|
||||||
|
"content": "<|file_sep|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151665": {
|
||||||
|
"content": "<tool_response>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151666": {
|
||||||
|
"content": "</tool_response>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151667": {
|
||||||
|
"content": "<think>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151668": {
|
||||||
|
"content": "</think>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additional_special_tokens": [
|
||||||
|
"<|im_start|>",
|
||||||
|
"<|im_end|>",
|
||||||
|
"<|object_ref_start|>",
|
||||||
|
"<|object_ref_end|>",
|
||||||
|
"<|box_start|>",
|
||||||
|
"<|box_end|>",
|
||||||
|
"<|quad_start|>",
|
||||||
|
"<|quad_end|>",
|
||||||
|
"<|vision_start|>",
|
||||||
|
"<|vision_end|>",
|
||||||
|
"<|vision_pad|>",
|
||||||
|
"<|image_pad|>",
|
||||||
|
"<|video_pad|>"
|
||||||
|
],
|
||||||
|
"bos_token": null,
|
||||||
|
"clean_up_tokenization_spaces": false,
|
||||||
|
"eos_token": "<|im_end|>",
|
||||||
|
"errors": "replace",
|
||||||
|
"extra_special_tokens": {},
|
||||||
|
"model_max_length": 32768,
|
||||||
|
"pad_token": "<|endoftext|>",
|
||||||
|
"padding_side": "left",
|
||||||
|
"split_special_tokens": false,
|
||||||
|
"tokenizer_class": "Qwen2Tokenizer",
|
||||||
|
"unk_token": null
|
||||||
|
}
|
||||||
17
training_logs/20260605_214218_metrics_job_591150.csv
Normal file
17
training_logs/20260605_214218_metrics_job_591150.csv
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
async/discard_rate,async/discarded_count,async/effective_batch_groups,async/effective_batch_samples,async/staleness_max,async/staleness_mean,async/staleness_min,async/staleness_ratio,generate/avg_num_tokens,generate/avg_tokens_non_zero_rewards,generate/avg_tokens_zero_rewards,generate/max_num_tokens,generate/min_num_tokens,generate/std_num_tokens,loss/avg_final_rewards,loss/avg_raw_advantages,loss/avg_raw_advantages_abs,policy/final_loss,policy/log_ratio_abs_max,policy/log_ratio_abs_mean,policy/log_ratio_abs_p99,policy/log_ratio_abs_pos00,policy/log_ratio_abs_pos10,policy/log_ratio_abs_pos20,policy/log_ratio_abs_pos30,policy/log_ratio_abs_pos40,policy/log_ratio_abs_pos50,policy/log_ratio_abs_pos60,policy/log_ratio_abs_pos70,policy/log_ratio_abs_pos80,policy/log_ratio_abs_pos90,policy/n_tokens_dp_gt_10pct,policy/n_tokens_dp_gt_1pct,policy/n_tokens_dp_gt_50pct,policy/policy_entropy,policy/policy_loss,policy/policy_lr,policy/policy_update_steps,policy/ppo_clip_ratio,policy/raw_grad_norm,reward/avg_pass_at_8,reward/avg_raw_reward,system/process_rss_gb,system/process_vms_gb,system/ram_available_gb,system/ram_percent,system/ram_total_gb,system/ram_used_gb,timing/compute_advantages_and_returns,timing/convert_to_training_input,timing/fwd_logprobs_values_reward,timing/policy_train,timing/run_training,timing/step,timing/sync_weights,timing/train_critic_and_policy,timing/wait_for_generation_buffer,trainer/epoch,trainer/global_step,batch_errors/total_batches,batch_errors/total_instances,batch_errors/total_successful,batch_errors/total_failed,batch_errors/total_masked,batch_errors/avg_ContextLengthExceededError,batch_errors/total_ContextLengthExceededError,timing/cleanup_old_checkpoints,timing/save_checkpoints,batch_errors/avg_DaytonaError,batch_errors/total_DaytonaError,batch_errors/avg_VerifierTimeoutError,batch_errors/total_VerifierTimeoutError,batch_errors/avg_InvalidChatHistory,batch_errors/total_InvalidChatHistory,batch_errors/avg_AgentTimeoutError,batch_errors/total_AgentTimeoutError,timing/save_hf_model,batch_errors/avg_RuntimeError,batch_errors/total_RuntimeError
|
||||||
|
0.0,0,64,512,0,0.0,0,0.0,8795.541,7557.8707,9821.0393,28234,1862,5511.4955,0.4531,0.0154,0.1383,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1147,-0.0,0.0,1.0,0.0,0.0154,0.6094,0.4531,12.2192,50.7338,643.6898,25.0,857.9676,214.2778,0.1433,4.903,67.5598,331.9725,399.8923,2717.3975,19.1362,332.1889,2293.4656,0,1,127.0,1016.0,978.0,16.0,38.0,0.31496062992125984,40.0,,,,,,,,,,,,,
|
||||||
|
0.0,0,64,512,1,0.8906,0,0.8906,10265.4648,9975.6566,10448.2102,30188,1,7251.4743,0.3867,0.0046,0.1482,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1058,0.0,0.0,1.0,0.0,0.0194,0.5312,0.3867,12.8623,51.2017,641.1227,25.3,857.9676,216.8448,0.0832,5.4115,69.2781,417.5714,487.2024,1580.8868,19.4255,417.8407,1068.8446,0,2,61.0,488.0,289.0,42.0,199.0,3.278688524590164,200.0,10.894,19.6772,0.04918032786885246,3.0,0.03278688524590164,2.0,0.03278688524590164,2.0,,,,,
|
||||||
|
0.0,0,64,512,2,1.7188,1,1.0,6598.4668,9521.9474,6088.8693,29419,1,7508.721,0.1484,0.014,0.1069,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0709,0.0,0.0,1.0,0.0,0.0109,0.2969,0.1484,12.9957,51.341,637.1793,25.7,857.9676,220.7883,0.0935,4.8247,53.3349,340.9789,394.6829,675.1796,19.7755,341.254,255.893,0,3,60.0,480.0,411.0,22.0,69.0,1.05,63.0,,,,,,,0.016666666666666666,1.0,0.016666666666666666,1.0,,,
|
||||||
|
0.0,0,64,512,3,2.4219,1,1.0,9999.6543,9601.1462,10199.4927,29478,1,7901.9526,0.334,0.0083,0.0995,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1012,0.0,0.0,1.0,0.0,0.0119,0.4531,0.334,13.3472,51.7056,629.7285,26.6,857.9676,228.2391,0.0901,5.533,67.9116,401.1539,469.4069,1064.6399,19.7968,401.4046,569.8905,0,4,65.0,520.0,384.0,31.0,136.0,2.0153846153846153,131.0,0.0057,12.0885,,,,,0.07692307692307693,5.0,,,,,
|
||||||
|
0.0,0,64,512,4,3.0625,2,1.0,9386.873,11132.7622,8710.2818,29967,1,8316.5867,0.2793,-0.0005,0.1511,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0843,-0.0,0.0,1.0,0.0,0.0126,0.4375,0.2793,13.3903,51.7529,626.9168,26.9,857.9676,231.0508,0.0708,5.2362,79.0552,433.729,513.1297,1514.2234,19.1602,434.0033,976.6598,0,5,62.0,496.0,384.0,32.0,112.0,1.9193548387096775,119.0,,,,,,,,,,,6.6576,,
|
||||||
|
0.0,0,64,512,4,3.7656,3,1.0,10051.041,12819.4872,8837.9017,29419,1,8180.7526,0.3047,0.0255,0.2105,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0891,0.0,0.0,1.0,0.0,0.0202,0.5469,0.3047,13.8707,51.8625,639.0039,25.5,857.9676,218.9637,0.077,5.2138,69.1587,408.4466,477.9141,961.7158,18.673,408.6779,459.8809,0,6,63.0,504.0,316.0,42.0,187.0,2.9365079365079363,185.0,0.0083,9.9275,,,,,0.031746031746031744,2.0,,,,0.015873015873015872,1.0
|
||||||
|
0.0,0,64,512,5,4.7812,4,1.0,8289.875,10832.8,7553.2594,31013,1,8640.4314,0.2246,-0.0009,0.0888,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0742,0.0,0.0,1.0,0.0,0.0102,0.3281,0.2246,13.6701,51.6724,636.2298,25.8,857.9676,221.7378,0.0697,5.0448,66.0801,402.6137,469.0069,1264.3273,18.9369,402.8567,771.3001,0,7,60.0,480.0,268.0,53.0,211.0,3.45,207.0,,,0.03333333333333333,2.0,,,0.06666666666666667,4.0,,,,0.016666666666666666,1.0
|
||||||
|
0.0,0,64,512,6,5.7656,5,1.0,8230.7988,14968.2024,6908.5047,29456,1,9244.7451,0.1641,0.0138,0.1045,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0659,-0.0,0.0,1.0,0.0,0.0139,0.3438,0.1641,14.0063,52.0513,628.7164,26.7,857.9676,229.2512,0.065,4.8541,69.1673,417.4759,486.978,2605.5652,11.9483,417.7453,2101.7529,0,8,64.0,512.0,487.0,7.0,25.0,0.390625,25.0,0.008,10.0222,,,,,,,,,,,
|
||||||
|
0.0,0,64,512,0,0.0,0,0.0,10444.8711,9636.092,11216.6069,30180,1,6434.7455,0.4883,0.0112,0.1585,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1128,0.0,0.0,1.0,0.0,0.0149,0.6562,0.4883,13.9713,52.0759,624.9913,27.2,857.9676,232.9763,0.0687,5.276,68.7787,378.0224,447.1007,3884.0127,18.4616,378.2529,3413.142,1,9,58.0,464.0,301.0,36.0,163.0,3.0689655172413794,178.0,,,0.05172413793103448,3.0,,,,,,,,,
|
||||||
|
0.0,0,64,512,1,1.0,1,1.0,8861.4961,11804.4167,7709.9185,29409,1,8814.3882,0.2812,0.0029,0.0973,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0734,0.0,0.0,1.0,0.0,0.0095,0.4062,0.2812,14.7997,52.574,627.1608,26.9,857.9676,230.8068,0.0644,5.1711,72.0844,417.3814,489.7789,926.0357,19.0802,417.6298,412.0023,1,10,64.0,512.0,332.0,36.0,179.0,2.640625,169.0,0.0076,9.5829,0.046875,3.0,0.015625,1.0,0.015625,1.0,0.03125,2.0,6.3468,,
|
||||||
|
0.0,0,64,512,2,1.7031,0,0.9531,7327.1992,9194.4762,6845.4693,28487,1,7502.5285,0.2051,0.0026,0.0657,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0804,0.0,0.0,1.0,0.0,0.01,0.2812,0.2051,15.1613,52.9576,624.4334,27.2,857.9676,233.5342,0.0601,4.715,63.8687,351.7918,415.97,995.055,19.1877,352.0408,555.1789,1,11,62.0,496.0,358.0,30.0,138.0,2.193548387096774,136.0,,,0.03225806451612903,2.0,,,,,,,,,
|
||||||
|
0.0,0,64,512,3,2.0938,1,1.0,9918.1191,11967.1758,8943.7839,30360,1,8420.1283,0.3223,0.0095,0.1672,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0848,0.0,0.0,1.0,0.0,0.0144,0.4844,0.3223,15.4885,53.2342,634.7371,26.0,857.9676,223.2305,0.0831,5.3606,76.0698,426.1412,502.5426,1539.2977,19.1177,426.3893,1012.2655,1,12,62.0,496.0,314.0,40.0,181.0,2.8870967741935485,179.0,0.0072,9.3904,0.06451612903225806,4.0,0.016129032258064516,1.0,,,,,,,
|
||||||
|
0.0,0,64,512,4,2.5938,2,1.0,9382.0039,12392.648,8409.5736,30099,1,9042.8211,0.2441,-0.0046,0.1564,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0751,-0.002,0.0,1.0,0.0,0.0148,0.4219,0.2441,15.3411,53.0607,632.1736,26.3,857.9676,225.7939,0.0688,5.2259,70.3939,431.8179,502.543,1232.3428,19.0569,432.0799,705.4787,1,13,64.0,512.0,303.0,44.0,209.0,3.203125,205.0,,,0.0625,4.0,,,,,,,,,
|
||||||
|
0.0,0,64,512,4,3.0469,3,1.0,8458.4961,12622.1583,7183.9056,30096,1,8927.1653,0.2344,-0.0072,0.1074,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0716,0.0,0.0,1.0,0.0,0.0087,0.375,0.2344,15.6031,53.179,628.9873,26.7,857.9676,228.9803,0.0787,5.1752,73.5778,416.5122,490.4141,1429.1938,19.0701,416.7571,914.4966,1,14,60.0,480.0,334.0,33.0,146.0,2.716666666666667,163.0,0.0072,9.1814,0.03333333333333333,2.0,,,,,,,,,
|
||||||
|
0.0,0,64,512,4,4.0,4,1.0,9658.8066,12490.4551,8288.1246,29820,1,8982.0547,0.3262,0.0033,0.186,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0761,0.0,0.0,1.0,0.0,0.0122,0.4688,0.3262,15.4732,53.0505,626.9476,26.9,857.9676,231.02,0.0661,5.096,76.3654,444.7385,521.4854,1171.3137,18.8645,445.0534,625.8348,1,15,63.0,504.0,225.0,54.0,278.0,4.349206349206349,274.0,,,0.06349206349206349,4.0,,,,,,,5.8824,0.015873015873015872,1.0
|
||||||
|
0.0,0,64,512,5,5.0,5,1.0,7392.418,16034.6364,6113.5247,29621,1,9341.5481,0.1289,-0.0013,0.1187,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0553,0.0,0.0,1.0,0.0,0.0095,0.2812,0.1289,15.7165,53.3323,624.8472,27.2,857.9676,233.1204,0.0635,4.8164,68.4974,418.1021,486.9176,2531.0981,13.3106,418.3564,2026.0146,1,16,,,,,,,,0.0074,9.3875,,,,,,,,,,,
|
||||||
|
258
training_logs/20260605_214218_metrics_report.md
Normal file
258
training_logs/20260605_214218_metrics_report.md
Normal file
@@ -0,0 +1,258 @@
|
|||||||
|
# SkyRL Training Metrics Analysis
|
||||||
|
|
||||||
|
Generated from 1 log files
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
| Log File | Total Steps | Metric Blocks | Final Reward (mean) | Final Reward (max) | Total Time (s) |
|
||||||
|
|----------|-------------|---------------|---------------------|-------------------|----------------|
|
||||||
|
| job_591150 | 16 | 16 | 0.2828 | 0.4883 | 26092.3 |
|
||||||
|
|
||||||
|
## Async Metrics
|
||||||
|
|
||||||
|
| | Mean | Std | Min | Max | Count |
|
||||||
|
|:------------------------------|-----------:|---------:|------:|---------:|--------:|
|
||||||
|
| async/discard_rate | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| async/discarded_count | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| async/effective_batch_groups | 64 | 0 | 64 | 64 | 16 |
|
||||||
|
| async/effective_batch_samples | 512 | 0 | 512 | 512 | 16 |
|
||||||
|
| async/staleness_max | 3 | 1.82574 | 0 | 6 | 16 |
|
||||||
|
| async/staleness_mean | 2.61524 | 1.73471 | 0 | 5.7656 | 16 |
|
||||||
|
| async/staleness_min | 2 | 1.78885 | 0 | 5 | 16 |
|
||||||
|
| async/staleness_ratio | 0.865231 | 0.338976 | 0 | 1 | 16 |
|
||||||
|
|
||||||
|
## Generate Metrics
|
||||||
|
|
||||||
|
| | Mean | Std | Min | Max | Count |
|
||||||
|
|:-------------------------------------|----------:|---------:|---------:|---------:|--------:|
|
||||||
|
| generate/avg_num_tokens | 8941.32 | 1152.12 | 6598.47 | 10444.9 | 16 |
|
||||||
|
| generate/avg_tokens_non_zero_rewards | 11409.5 | 2195.47 | 7557.87 | 16034.6 | 16 |
|
||||||
|
| generate/avg_tokens_zero_rewards | 8329.9 | 1544.59 | 6088.87 | 11216.6 | 16 |
|
||||||
|
| generate/max_num_tokens | 29702.9 | 683.514 | 28234 | 31013 | 16 |
|
||||||
|
| generate/min_num_tokens | 117.312 | 465.25 | 1 | 1862 | 16 |
|
||||||
|
| generate/std_num_tokens | 8126.35 | 1067.38 | 5511.5 | 9341.55 | 16 |
|
||||||
|
|
||||||
|
## Loss Metrics
|
||||||
|
|
||||||
|
| | Mean | Std | Min | Max | Count |
|
||||||
|
|:----------------------------|----------:|-----------:|--------:|-------:|--------:|
|
||||||
|
| loss/avg_final_rewards | 0.282837 | 0.102531 | 0.1289 | 0.4883 | 16 |
|
||||||
|
| loss/avg_raw_advantages | 0.0060375 | 0.00855639 | -0.0072 | 0.0255 | 16 |
|
||||||
|
| loss/avg_raw_advantages_abs | 0.131562 | 0.0391317 | 0.0657 | 0.2105 | 16 |
|
||||||
|
|
||||||
|
## Policy Metrics
|
||||||
|
|
||||||
|
| | Mean | Std | Min | Max | Count |
|
||||||
|
|:----------------------------|-----------:|-----------:|--------:|-------:|--------:|
|
||||||
|
| policy/final_loss | 0 | 0 | -0 | 0 | 16 |
|
||||||
|
| policy/log_ratio_abs_max | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/log_ratio_abs_mean | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/log_ratio_abs_p99 | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/log_ratio_abs_pos00 | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/log_ratio_abs_pos10 | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/log_ratio_abs_pos20 | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/log_ratio_abs_pos30 | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/log_ratio_abs_pos40 | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/log_ratio_abs_pos50 | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/log_ratio_abs_pos60 | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/log_ratio_abs_pos70 | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/log_ratio_abs_pos80 | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/log_ratio_abs_pos90 | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/n_tokens_dp_gt_10pct | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/n_tokens_dp_gt_1pct | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/n_tokens_dp_gt_50pct | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/policy_entropy | 0.083475 | 0.0171367 | 0.0553 | 0.1147 | 16 |
|
||||||
|
| policy/policy_loss | -0.000125 | 0.0005 | -0.002 | 0 | 16 |
|
||||||
|
| policy/policy_lr | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/policy_update_steps | 1 | 0 | 1 | 1 | 16 |
|
||||||
|
| policy/ppo_clip_ratio | 0 | 0 | 0 | 0 | 16 |
|
||||||
|
| policy/raw_grad_norm | 0.0130313 | 0.00340993 | 0.0087 | 0.0202 | 16 |
|
||||||
|
|
||||||
|
## Reward Metrics
|
||||||
|
|
||||||
|
| | Mean | Std | Min | Max | Count |
|
||||||
|
|:----------------------|---------:|---------:|-------:|-------:|--------:|
|
||||||
|
| reward/avg_pass_at_8 | 0.432613 | 0.114214 | 0.2812 | 0.6562 | 16 |
|
||||||
|
| reward/avg_raw_reward | 0.282837 | 0.102531 | 0.1289 | 0.4883 | 16 |
|
||||||
|
|
||||||
|
## System Metrics
|
||||||
|
|
||||||
|
| | Mean | Std | Min | Max | Count |
|
||||||
|
|:------------------------|---------:|---------:|---------:|---------:|--------:|
|
||||||
|
| system/process_rss_gb | 14.2448 | 1.12927 | 12.2192 | 15.7165 | 16 |
|
||||||
|
| system/process_vms_gb | 52.2366 | 0.825632 | 50.7338 | 53.3323 | 16 |
|
||||||
|
| system/ram_available_gb | 631.679 | 6.2014 | 624.433 | 643.69 | 16 |
|
||||||
|
| system/ram_percent | 26.3687 | 0.72454 | 25 | 27.2 | 16 |
|
||||||
|
| system/ram_total_gb | 857.968 | 0 | 857.968 | 857.968 | 16 |
|
||||||
|
| system/ram_used_gb | 226.288 | 6.20141 | 214.278 | 233.534 | 16 |
|
||||||
|
|
||||||
|
## Timing Metrics
|
||||||
|
|
||||||
|
| | Mean | Std | Min | Max | Count |
|
||||||
|
|:--------------------------------------|------------:|------------:|---------:|----------:|--------:|
|
||||||
|
| timing/compute_advantages_and_returns | 0.077875 | 0.0200385 | 0.0601 | 0.1433 | 16 |
|
||||||
|
| timing/convert_to_training_input | 5.11608 | 0.237222 | 4.715 | 5.533 | 16 |
|
||||||
|
| timing/fwd_logprobs_values_reward | 69.4489 | 5.87917 | 53.3349 | 79.0552 | 16 |
|
||||||
|
| timing/policy_train | 402.403 | 33.9185 | 331.973 | 444.738 | 16 |
|
||||||
|
| timing/run_training | 472.185 | 38.58 | 394.683 | 521.485 | 16 |
|
||||||
|
| timing/step | 1630.77 | 862.939 | 675.18 | 3884.01 | 16 |
|
||||||
|
| timing/sync_weights | 18.3126 | 2.25802 | 11.9483 | 19.7968 | 16 |
|
||||||
|
| timing/train_critic_and_policy | 402.658 | 33.9295 | 332.189 | 445.053 | 16 |
|
||||||
|
| timing/wait_for_generation_buffer | 1135.13 | 869.349 | 255.893 | 3413.14 | 16 |
|
||||||
|
| timing/cleanup_old_checkpoints | 1.36818 | 3.84901 | 0.0057 | 10.894 | 8 |
|
||||||
|
| timing/save_checkpoints | 11.1572 | 3.56358 | 9.1814 | 19.6772 | 8 |
|
||||||
|
| timing/save_hf_model | 6.2956 | 0.390128 | 5.8824 | 6.6576 | 3 |
|
||||||
|
|
||||||
|
## Trainer Metrics
|
||||||
|
|
||||||
|
| | Mean | Std | Min | Max | Count |
|
||||||
|
|:--------------------|-------:|---------:|------:|------:|--------:|
|
||||||
|
| trainer/epoch | 0.5 | 0.516398 | 0 | 1 | 16 |
|
||||||
|
| trainer/global_step | 8.5 | 4.76095 | 1 | 16 | 16 |
|
||||||
|
|
||||||
|
## Batch_Errors Metrics
|
||||||
|
|
||||||
|
| | Mean | Std | Min | Max | Count |
|
||||||
|
|:----------------------------------------------|------------:|--------------:|------------:|-------------:|--------:|
|
||||||
|
| batch_errors/total_batches | 66.3333 | 16.8932 | 58 | 127 | 15 |
|
||||||
|
| batch_errors/total_instances | 530.667 | 135.146 | 464 | 1016 | 15 |
|
||||||
|
| batch_errors/total_successful | 378.933 | 177.363 | 225 | 978 | 15 |
|
||||||
|
| batch_errors/total_failed | 34.5333 | 12.6935 | 7 | 54 | 15 |
|
||||||
|
| batch_errors/total_masked | 151.4 | 68.5709 | 25 | 278 | 15 |
|
||||||
|
| batch_errors/avg_ContextLengthExceededError | 2.42765 | 1.13601 | 0.314961 | 4.34921 | 15 |
|
||||||
|
| batch_errors/total_ContextLengthExceededError | 151.6 | 67.7725 | 25 | 274 | 15 |
|
||||||
|
| batch_errors/avg_DaytonaError | 0.0485792 | 0.0132974 | 0.0322581 | 0.0645161 | 9 |
|
||||||
|
| batch_errors/total_DaytonaError | 3 | 0.866025 | 2 | 4 | 9 |
|
||||||
|
| batch_errors/avg_VerifierTimeoutError | 0.0215136 | 0.00976617 | 0.015625 | 0.0327869 | 3 |
|
||||||
|
| batch_errors/total_VerifierTimeoutError | 1.33333 | 0.57735 | 1 | 2 | 3 |
|
||||||
|
| batch_errors/avg_InvalidChatHistory | 0.0400691 | 0.0258191 | 0.015625 | 0.0769231 | 6 |
|
||||||
|
| batch_errors/total_InvalidChatHistory | 2.5 | 1.64317 | 1 | 5 | 6 |
|
||||||
|
| batch_errors/avg_AgentTimeoutError | 0.0239583 | 0.010312 | 0.0166667 | 0.03125 | 2 |
|
||||||
|
| batch_errors/total_AgentTimeoutError | 1.5 | 0.707107 | 1 | 2 | 2 |
|
||||||
|
| batch_errors/avg_RuntimeError | 0.0161376 | 0.000458214 | 0.015873 | 0.0166667 | 3 |
|
||||||
|
| batch_errors/total_RuntimeError | 1 | 0 | 1 | 1 | 3 |
|
||||||
|
|
||||||
|
## Training Progression by Log
|
||||||
|
|
||||||
|
### job_591150
|
||||||
|
|
||||||
|
| Step | Reward | Pass@8 | KL | Loss | Step Time (s) | Gen Wait (s) |
|
||||||
|
|------|--------|--------|-----|------|---------------|-------------|
|
||||||
|
| 1 | 0.4531 | 0.6094 | 0.000000 | -0.0000 | 2717.4 | 2293.5 |
|
||||||
|
| 2 | 0.3867 | 0.5312 | 0.000000 | 0.0000 | 1580.9 | 1068.8 |
|
||||||
|
| 3 | 0.1484 | 0.2969 | 0.000000 | 0.0000 | 675.2 | 255.9 |
|
||||||
|
| 4 | 0.3340 | 0.4531 | 0.000000 | 0.0000 | 1064.6 | 569.9 |
|
||||||
|
| 5 | 0.2793 | 0.4375 | 0.000000 | -0.0000 | 1514.2 | 976.7 |
|
||||||
|
| 6 | 0.3047 | 0.5469 | 0.000000 | 0.0000 | 961.7 | 459.9 |
|
||||||
|
| 7 | 0.2246 | 0.3281 | 0.000000 | 0.0000 | 1264.3 | 771.3 |
|
||||||
|
| 8 | 0.1641 | 0.3438 | 0.000000 | -0.0000 | 2605.6 | 2101.8 |
|
||||||
|
| 9 | 0.4883 | 0.6562 | 0.000000 | 0.0000 | 3884.0 | 3413.1 |
|
||||||
|
| 10 | 0.2812 | 0.4062 | 0.000000 | 0.0000 | 926.0 | 412.0 |
|
||||||
|
| 11 | 0.2051 | 0.2812 | 0.000000 | 0.0000 | 995.1 | 555.2 |
|
||||||
|
| 12 | 0.3223 | 0.4844 | 0.000000 | 0.0000 | 1539.3 | 1012.3 |
|
||||||
|
| 13 | 0.2441 | 0.4219 | 0.000000 | -0.0000 | 1232.3 | 705.5 |
|
||||||
|
| 14 | 0.2344 | 0.3750 | 0.000000 | 0.0000 | 1429.2 | 914.5 |
|
||||||
|
| 15 | 0.3262 | 0.4688 | 0.000000 | 0.0000 | 1171.3 | 625.8 |
|
||||||
|
| 16 | 0.1289 | 0.2812 | 0.000000 | 0.0000 | 2531.1 | 2026.0 |
|
||||||
|
|
||||||
|
## Timing Analysis
|
||||||
|
|
||||||
|
### Average Time Breakdown (% of step time)
|
||||||
|
|
||||||
|
| Component | Avg % of Step Time |
|
||||||
|
|-----------|-------------------|
|
||||||
|
| wait_for_generation_buffer | 62.9% |
|
||||||
|
| run_training | 35.3% |
|
||||||
|
| train_critic_and_policy | 30.2% |
|
||||||
|
| policy_train | 30.1% |
|
||||||
|
| fwd_logprobs_values_reward | 5.2% |
|
||||||
|
| sync_weights | 1.4% |
|
||||||
|
| save_checkpoints | 0.8% |
|
||||||
|
| save_hf_model | 0.5% |
|
||||||
|
| convert_to_training_input | 0.4% |
|
||||||
|
| cleanup_old_checkpoints | 0.1% |
|
||||||
|
| compute_advantages_and_returns | 0.0% |
|
||||||
|
|
||||||
|
## vLLM Inference Engine Analysis
|
||||||
|
|
||||||
|
Metrics from vLLM stat loggers (V1LoggingStatLoggerFixed).
|
||||||
|
|
||||||
|
> **Note**: Ray deduplicates similar log messages with `[repeated Nx across cluster]`,
|
||||||
|
> so we typically capture stats from one engine per timestamp. The stats shown are
|
||||||
|
> **per-engine** values. Multiply by num_inference_engines for cluster-wide estimates.
|
||||||
|
|
||||||
|
### Summary by Log (Per-Engine Stats)
|
||||||
|
|
||||||
|
| Log | Avg Running/Engine | Avg Waiting/Engine | Avg Gen Throughput/Engine | Avg KV Cache % | Avg Prefix Hit % |
|
||||||
|
|-----|-------------------|-------------------|--------------------------|----------------|------------------|
|
||||||
|
| job_591150 | 6.4 | 0.0 | 287.6 tok/s | 29.4% | 90.9% |
|
||||||
|
|
||||||
|
### Utilization Analysis (Per-Engine)
|
||||||
|
|
||||||
|
Key indicators of inference engine utilization:
|
||||||
|
|
||||||
|
- **Running requests/engine**: Concurrent requests being processed by each engine
|
||||||
|
- **Waiting requests**: Requests queued (0 = engine not saturated, has spare capacity)
|
||||||
|
- **Generation throughput**: Decode tokens/sec per engine
|
||||||
|
- 8B model on H100 can do **1000+ tok/s** when saturated
|
||||||
|
- If seeing <300 tok/s with 0 waiting, engine is **starved for requests**
|
||||||
|
|
||||||
|
#### job_591150
|
||||||
|
|
||||||
|
- **Running requests/engine**: avg=6.4, max=19
|
||||||
|
- **Waiting requests**: avg=0.0, max=0
|
||||||
|
- **Generation throughput/engine**: avg=287.6 tok/s, max=720.3 tok/s
|
||||||
|
- **KV cache usage**: avg=29.4%
|
||||||
|
- **Prefix cache hit rate**: avg=90.9%
|
||||||
|
- ⚠️ **Low throughput**: 288 tok/s << expected 1000+ tok/s for saturated 8B model
|
||||||
|
|
||||||
|
## Trial-Level Analysis (from result.json)
|
||||||
|
|
||||||
|
Total trials parsed: 8192
|
||||||
|
|
||||||
|
### Turn Count Statistics
|
||||||
|
|
||||||
|
| Metric | Value |
|
||||||
|
|--------|-------|
|
||||||
|
| Mean | 11.8 |
|
||||||
|
| Median | 10.0 |
|
||||||
|
| Std | 6.7 |
|
||||||
|
| Min | 1 |
|
||||||
|
| Max | 151 |
|
||||||
|
| Count | 8185 |
|
||||||
|
|
||||||
|
### Exception Distribution
|
||||||
|
|
||||||
|
| Exception Type | Count | % |
|
||||||
|
|---------------|-------|---|
|
||||||
|
| No exception | 5561 | 67.9% |
|
||||||
|
| ContextLengthExceededError | 2311 | 28.2% |
|
||||||
|
| AgentTimeoutError | 286 | 3.5% |
|
||||||
|
| DaytonaError | 27 | 0.3% |
|
||||||
|
| VerifierTimeoutError | 4 | 0.0% |
|
||||||
|
| RuntimeError | 3 | 0.0% |
|
||||||
|
|
||||||
|
### Turn Count by Exception Type
|
||||||
|
|
||||||
|
| Exception Type | Mean Turns | Median Turns | Count |
|
||||||
|
|---------------|-----------|-------------|-------|
|
||||||
|
| VerifierTimeoutError | 17.5 | 15.5 | 4 |
|
||||||
|
| ContextLengthExceededError | 13.9 | 12.0 | 2311 |
|
||||||
|
| No exception | 11.0 | 10.0 | 5561 |
|
||||||
|
| RuntimeError | 10.3 | 10.0 | 3 |
|
||||||
|
| AgentTimeoutError | 10.2 | 7.0 | 286 |
|
||||||
|
| DaytonaError | 10.1 | 9.0 | 20 |
|
||||||
|
|
||||||
|
### Turn Count by Outcome
|
||||||
|
|
||||||
|
| Outcome | Mean Turns | Median Turns | Count |
|
||||||
|
|---------|-----------|-------------|-------|
|
||||||
|
| Success | 10.0 | 9.0 | 2317 |
|
||||||
|
| Failure | 11.6 | 10.0 | 3527 |
|
||||||
|
|
||||||
|
### Reward Summary
|
||||||
|
|
||||||
|
- Mean reward: 0.3965
|
||||||
|
- Success rate: 39.6%
|
||||||
|
- Trials with reward data: 5844
|
||||||
|
|
||||||
17
training_logs/20260605_214218_metrics_table.csv
Normal file
17
training_logs/20260605_214218_metrics_table.csv
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
log_file,async/discard_rate,async/discarded_count,async/effective_batch_groups,async/effective_batch_samples,async/staleness_max,async/staleness_mean,async/staleness_min,async/staleness_ratio,generate/avg_num_tokens,generate/avg_tokens_non_zero_rewards,generate/avg_tokens_zero_rewards,generate/max_num_tokens,generate/min_num_tokens,generate/std_num_tokens,loss/avg_final_rewards,loss/avg_raw_advantages,loss/avg_raw_advantages_abs,policy/final_loss,policy/log_ratio_abs_max,policy/log_ratio_abs_mean,policy/log_ratio_abs_p99,policy/log_ratio_abs_pos00,policy/log_ratio_abs_pos10,policy/log_ratio_abs_pos20,policy/log_ratio_abs_pos30,policy/log_ratio_abs_pos40,policy/log_ratio_abs_pos50,policy/log_ratio_abs_pos60,policy/log_ratio_abs_pos70,policy/log_ratio_abs_pos80,policy/log_ratio_abs_pos90,policy/n_tokens_dp_gt_10pct,policy/n_tokens_dp_gt_1pct,policy/n_tokens_dp_gt_50pct,policy/policy_entropy,policy/policy_loss,policy/policy_lr,policy/policy_update_steps,policy/ppo_clip_ratio,policy/raw_grad_norm,reward/avg_pass_at_8,reward/avg_raw_reward,system/process_rss_gb,system/process_vms_gb,system/ram_available_gb,system/ram_percent,system/ram_total_gb,system/ram_used_gb,timing/compute_advantages_and_returns,timing/convert_to_training_input,timing/fwd_logprobs_values_reward,timing/policy_train,timing/run_training,timing/step,timing/sync_weights,timing/train_critic_and_policy,timing/wait_for_generation_buffer,trainer/epoch,trainer/global_step,batch_errors/total_batches,batch_errors/total_instances,batch_errors/total_successful,batch_errors/total_failed,batch_errors/total_masked,batch_errors/avg_ContextLengthExceededError,batch_errors/total_ContextLengthExceededError,timing/cleanup_old_checkpoints,timing/save_checkpoints,batch_errors/avg_DaytonaError,batch_errors/total_DaytonaError,batch_errors/avg_VerifierTimeoutError,batch_errors/total_VerifierTimeoutError,batch_errors/avg_InvalidChatHistory,batch_errors/total_InvalidChatHistory,batch_errors/avg_AgentTimeoutError,batch_errors/total_AgentTimeoutError,timing/save_hf_model,batch_errors/avg_RuntimeError,batch_errors/total_RuntimeError,global_step
|
||||||
|
job_591150,0.0,0,64,512,0,0.0,0,0.0,8795.541,7557.8707,9821.0393,28234,1862,5511.4955,0.4531,0.0154,0.1383,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1147,-0.0,0.0,1.0,0.0,0.0154,0.6094,0.4531,12.2192,50.7338,643.6898,25.0,857.9676,214.2778,0.1433,4.903,67.5598,331.9725,399.8923,2717.3975,19.1362,332.1889,2293.4656,0,1,127.0,1016.0,978.0,16.0,38.0,0.31496062992125984,40.0,,,,,,,,,,,,,,1
|
||||||
|
job_591150,0.0,0,64,512,1,0.8906,0,0.8906,10265.4648,9975.6566,10448.2102,30188,1,7251.4743,0.3867,0.0046,0.1482,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1058,0.0,0.0,1.0,0.0,0.0194,0.5312,0.3867,12.8623,51.2017,641.1227,25.3,857.9676,216.8448,0.0832,5.4115,69.2781,417.5714,487.2024,1580.8868,19.4255,417.8407,1068.8446,0,2,61.0,488.0,289.0,42.0,199.0,3.278688524590164,200.0,10.894,19.6772,0.04918032786885246,3.0,0.03278688524590164,2.0,0.03278688524590164,2.0,,,,,,2
|
||||||
|
job_591150,0.0,0,64,512,2,1.7188,1,1.0,6598.4668,9521.9474,6088.8693,29419,1,7508.721,0.1484,0.014,0.1069,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0709,0.0,0.0,1.0,0.0,0.0109,0.2969,0.1484,12.9957,51.341,637.1793,25.7,857.9676,220.7883,0.0935,4.8247,53.3349,340.9789,394.6829,675.1796,19.7755,341.254,255.893,0,3,60.0,480.0,411.0,22.0,69.0,1.05,63.0,,,,,,,0.016666666666666666,1.0,0.016666666666666666,1.0,,,,3
|
||||||
|
job_591150,0.0,0,64,512,3,2.4219,1,1.0,9999.6543,9601.1462,10199.4927,29478,1,7901.9526,0.334,0.0083,0.0995,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1012,0.0,0.0,1.0,0.0,0.0119,0.4531,0.334,13.3472,51.7056,629.7285,26.6,857.9676,228.2391,0.0901,5.533,67.9116,401.1539,469.4069,1064.6399,19.7968,401.4046,569.8905,0,4,65.0,520.0,384.0,31.0,136.0,2.0153846153846153,131.0,0.0057,12.0885,,,,,0.07692307692307693,5.0,,,,,,4
|
||||||
|
job_591150,0.0,0,64,512,4,3.0625,2,1.0,9386.873,11132.7622,8710.2818,29967,1,8316.5867,0.2793,-0.0005,0.1511,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0843,-0.0,0.0,1.0,0.0,0.0126,0.4375,0.2793,13.3903,51.7529,626.9168,26.9,857.9676,231.0508,0.0708,5.2362,79.0552,433.729,513.1297,1514.2234,19.1602,434.0033,976.6598,0,5,62.0,496.0,384.0,32.0,112.0,1.9193548387096775,119.0,,,,,,,,,,,6.6576,,,5
|
||||||
|
job_591150,0.0,0,64,512,4,3.7656,3,1.0,10051.041,12819.4872,8837.9017,29419,1,8180.7526,0.3047,0.0255,0.2105,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0891,0.0,0.0,1.0,0.0,0.0202,0.5469,0.3047,13.8707,51.8625,639.0039,25.5,857.9676,218.9637,0.077,5.2138,69.1587,408.4466,477.9141,961.7158,18.673,408.6779,459.8809,0,6,63.0,504.0,316.0,42.0,187.0,2.9365079365079363,185.0,0.0083,9.9275,,,,,0.031746031746031744,2.0,,,,0.015873015873015872,1.0,6
|
||||||
|
job_591150,0.0,0,64,512,5,4.7812,4,1.0,8289.875,10832.8,7553.2594,31013,1,8640.4314,0.2246,-0.0009,0.0888,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0742,0.0,0.0,1.0,0.0,0.0102,0.3281,0.2246,13.6701,51.6724,636.2298,25.8,857.9676,221.7378,0.0697,5.0448,66.0801,402.6137,469.0069,1264.3273,18.9369,402.8567,771.3001,0,7,60.0,480.0,268.0,53.0,211.0,3.45,207.0,,,0.03333333333333333,2.0,,,0.06666666666666667,4.0,,,,0.016666666666666666,1.0,7
|
||||||
|
job_591150,0.0,0,64,512,6,5.7656,5,1.0,8230.7988,14968.2024,6908.5047,29456,1,9244.7451,0.1641,0.0138,0.1045,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0659,-0.0,0.0,1.0,0.0,0.0139,0.3438,0.1641,14.0063,52.0513,628.7164,26.7,857.9676,229.2512,0.065,4.8541,69.1673,417.4759,486.978,2605.5652,11.9483,417.7453,2101.7529,0,8,64.0,512.0,487.0,7.0,25.0,0.390625,25.0,0.008,10.0222,,,,,,,,,,,,8
|
||||||
|
job_591150,0.0,0,64,512,0,0.0,0,0.0,10444.8711,9636.092,11216.6069,30180,1,6434.7455,0.4883,0.0112,0.1585,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1128,0.0,0.0,1.0,0.0,0.0149,0.6562,0.4883,13.9713,52.0759,624.9913,27.2,857.9676,232.9763,0.0687,5.276,68.7787,378.0224,447.1007,3884.0127,18.4616,378.2529,3413.142,1,9,58.0,464.0,301.0,36.0,163.0,3.0689655172413794,178.0,,,0.05172413793103448,3.0,,,,,,,,,,9
|
||||||
|
job_591150,0.0,0,64,512,1,1.0,1,1.0,8861.4961,11804.4167,7709.9185,29409,1,8814.3882,0.2812,0.0029,0.0973,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0734,0.0,0.0,1.0,0.0,0.0095,0.4062,0.2812,14.7997,52.574,627.1608,26.9,857.9676,230.8068,0.0644,5.1711,72.0844,417.3814,489.7789,926.0357,19.0802,417.6298,412.0023,1,10,64.0,512.0,332.0,36.0,179.0,2.640625,169.0,0.0076,9.5829,0.046875,3.0,0.015625,1.0,0.015625,1.0,0.03125,2.0,6.3468,,,10
|
||||||
|
job_591150,0.0,0,64,512,2,1.7031,0,0.9531,7327.1992,9194.4762,6845.4693,28487,1,7502.5285,0.2051,0.0026,0.0657,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0804,0.0,0.0,1.0,0.0,0.01,0.2812,0.2051,15.1613,52.9576,624.4334,27.2,857.9676,233.5342,0.0601,4.715,63.8687,351.7918,415.97,995.055,19.1877,352.0408,555.1789,1,11,62.0,496.0,358.0,30.0,138.0,2.193548387096774,136.0,,,0.03225806451612903,2.0,,,,,,,,,,11
|
||||||
|
job_591150,0.0,0,64,512,3,2.0938,1,1.0,9918.1191,11967.1758,8943.7839,30360,1,8420.1283,0.3223,0.0095,0.1672,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0848,0.0,0.0,1.0,0.0,0.0144,0.4844,0.3223,15.4885,53.2342,634.7371,26.0,857.9676,223.2305,0.0831,5.3606,76.0698,426.1412,502.5426,1539.2977,19.1177,426.3893,1012.2655,1,12,62.0,496.0,314.0,40.0,181.0,2.8870967741935485,179.0,0.0072,9.3904,0.06451612903225806,4.0,0.016129032258064516,1.0,,,,,,,,12
|
||||||
|
job_591150,0.0,0,64,512,4,2.5938,2,1.0,9382.0039,12392.648,8409.5736,30099,1,9042.8211,0.2441,-0.0046,0.1564,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0751,-0.002,0.0,1.0,0.0,0.0148,0.4219,0.2441,15.3411,53.0607,632.1736,26.3,857.9676,225.7939,0.0688,5.2259,70.3939,431.8179,502.543,1232.3428,19.0569,432.0799,705.4787,1,13,64.0,512.0,303.0,44.0,209.0,3.203125,205.0,,,0.0625,4.0,,,,,,,,,,13
|
||||||
|
job_591150,0.0,0,64,512,4,3.0469,3,1.0,8458.4961,12622.1583,7183.9056,30096,1,8927.1653,0.2344,-0.0072,0.1074,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0716,0.0,0.0,1.0,0.0,0.0087,0.375,0.2344,15.6031,53.179,628.9873,26.7,857.9676,228.9803,0.0787,5.1752,73.5778,416.5122,490.4141,1429.1938,19.0701,416.7571,914.4966,1,14,60.0,480.0,334.0,33.0,146.0,2.716666666666667,163.0,0.0072,9.1814,0.03333333333333333,2.0,,,,,,,,,,14
|
||||||
|
job_591150,0.0,0,64,512,4,4.0,4,1.0,9658.8066,12490.4551,8288.1246,29820,1,8982.0547,0.3262,0.0033,0.186,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0761,0.0,0.0,1.0,0.0,0.0122,0.4688,0.3262,15.4732,53.0505,626.9476,26.9,857.9676,231.02,0.0661,5.096,76.3654,444.7385,521.4854,1171.3137,18.8645,445.0534,625.8348,1,15,63.0,504.0,225.0,54.0,278.0,4.349206349206349,274.0,,,0.06349206349206349,4.0,,,,,,,5.8824,0.015873015873015872,1.0,15
|
||||||
|
job_591150,0.0,0,64,512,5,5.0,5,1.0,7392.418,16034.6364,6113.5247,29621,1,9341.5481,0.1289,-0.0013,0.1187,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0553,0.0,0.0,1.0,0.0,0.0095,0.2812,0.1289,15.7165,53.3323,624.8472,27.2,857.9676,233.1204,0.0635,4.8164,68.4974,418.1021,486.9176,2531.0981,13.3106,418.3564,2026.0146,1,16,,,,,,,,0.0074,9.3875,,,,,,,,,,,,16
|
||||||
|
3
training_logs/20260605_214218_reward_vs_steps.png
Normal file
3
training_logs/20260605_214218_reward_vs_steps.png
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:ee3b133494ebb52044eccc1fd7ff5123af14d20a88f24159d2958c56c01d038b
|
||||||
|
size 228873
|
||||||
8193
training_logs/20260605_214218_trial_results.csv
Normal file
8193
training_logs/20260605_214218_trial_results.csv
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b8d0854eae5bcb5a92b2988276676bc1e75f6e2e75e1ce042397ab65007e975a
|
||||||
|
size 109734
|
||||||
4997
training_logs/20260605_214218_vllm_metrics_job_591150.csv
Normal file
4997
training_logs/20260605_214218_vllm_metrics_job_591150.csv
Normal file
File diff suppressed because it is too large
Load Diff
4997
training_logs/20260605_214218_vllm_metrics_table.csv
Normal file
4997
training_logs/20260605_214218_vllm_metrics_table.csv
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b168199ac61c8652f91da79b196a1fde395387c5abecf67e85a416253de1d5b1
|
||||||
|
size 48862777
|
||||||
1417
training_logs/a3-rl-DCAgent_exp_rpt_curriculum-medium_591151.out
Normal file
1417
training_logs/a3-rl-DCAgent_exp_rpt_curriculum-medium_591151.out
Normal file
File diff suppressed because one or more lines are too long
1243
training_logs/a3-rl-DCAgent_exp_rpt_curriculum-medium_591152.out
Normal file
1243
training_logs/a3-rl-DCAgent_exp_rpt_curriculum-medium_591152.out
Normal file
File diff suppressed because one or more lines are too long
1487
training_logs/a3-rl-DCAgent_exp_rpt_curriculum-medium_591153.out
Normal file
1487
training_logs/a3-rl-DCAgent_exp_rpt_curriculum-medium_591153.out
Normal file
File diff suppressed because one or more lines are too long
1226
training_logs/a3-rl-DCAgent_exp_rpt_curriculum-medium_591154.out
Normal file
1226
training_logs/a3-rl-DCAgent_exp_rpt_curriculum-medium_591154.out
Normal file
File diff suppressed because one or more lines are too long
1221
training_logs/a3-rl-DCAgent_exp_rpt_curriculum-medium_591155.out
Normal file
1221
training_logs/a3-rl-DCAgent_exp_rpt_curriculum-medium_591155.out
Normal file
File diff suppressed because one or more lines are too long
1
vocab.json
Normal file
1
vocab.json
Normal file
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user