初始化项目，由ModelHub XC社区提供模型

Model: harsha070/expfinal-qwen-island-s42-lambda-0p50 Source: Original Platform
2026-06-16 05:14:16 +08:00
commit 804993290c
33 changed files with 983 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,67 @@
+---
+base_model: Qwen/Qwen2.5-3B-Instruct
+library_name: transformers
+model_name: expfinal-qwen-island-s42-lambda-0p50
+tags:
+- generated_from_trainer
+- grpo
+- trl
+licence: license
+---
+
+# Model Card for expfinal-qwen-island-s42-lambda-0p50
+
+This model is a fine-tuned version of [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="harsha070/expfinal-qwen-island-s42-lambda-0p50", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/models-self5933/obfuscation-early-warning/runs/rottuhnw) 
+
+
+
+This model was trained with GRPO, a method introduced in [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://huggingface.co/papers/2402.03300).
+
+### Framework versions
+
+- TRL: 1.3.0
+- Transformers: 5.7.0
+- Pytorch: 2.11.0
+- Datasets: 4.8.5
+- Tokenizers: 0.22.2
+
+## Citations
+
+Cite GRPO as:
+
+```bibtex
+@article{shao2024deepseekmath,
+    title        = {{DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models}},
+    author       = {Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo},
+    year         = 2024,
+    eprint       = {arXiv:2402.03300},
+}
+```
+
+Cite TRL as:
+    
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```
--- a/chat_template.jinja
+++ b/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}
--- a/completions/completions_00010.parquet
+++ b/completions/completions_00010.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7bdd0558ee31f69dcef0e047620e18b2b89c28dcfc0ed153e2559d4668f446df
+size 24317
--- a/completions/completions_00020.parquet
+++ b/completions/completions_00020.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4654c956d4acd8bef59680224087f061d8a93f1c85995e3cd4ee71518836524c
+size 21310
--- a/completions/completions_00030.parquet
+++ b/completions/completions_00030.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ec451cb47393fbc2087668544da5e65149d57b9ec7ee25cb365007467e610b3
+size 23320
--- a/completions/completions_00040.parquet
+++ b/completions/completions_00040.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd835d1e4bdcdadc21180c21da6634a5333853f82dc2a22f96569cf97335fbf
+size 21463
--- a/completions/completions_00050.parquet
+++ b/completions/completions_00050.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3b51cf9c72102c43b2d33ab45a47b3f18f0b394fad9dacf5149301789855248
+size 22222
--- a/completions/completions_00060.parquet
+++ b/completions/completions_00060.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3d833af4522b6d246fd99b8048d12144feb2e9970d6380ad2fed8de86195329
+size 23570
--- a/completions/completions_00070.parquet
+++ b/completions/completions_00070.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d07dd653bfa6ce0ddc96303c5f50fde6a9dcc1ea388bd43b43cdb7b72774ae6f
+size 18972
--- a/completions/completions_00080.parquet
+++ b/completions/completions_00080.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:517309bce7ad9fa7859b87ecb9cdf0c3c8cb1193a74f0dbc249410b38d4e9d44
+size 19963
--- a/completions/completions_00090.parquet
+++ b/completions/completions_00090.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f978e6b612ac321a31291def0eee55a4aed7d408d86c81a42330f6844ec8c78a
+size 19253
--- a/completions/completions_00100.parquet
+++ b/completions/completions_00100.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6edbb2223e4a42eeb886cc7bf875db5c300ef042c6fe9677fa39f661aca778f
+size 21716
--- a/completions/completions_00110.parquet
+++ b/completions/completions_00110.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db04d88b34288f7d8cc99822a749d160bddabc3401096ddcd9ea3235da977e9b
+size 21866
--- a/completions/completions_00120.parquet
+++ b/completions/completions_00120.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c552fa033b8ad870dea41dd8c5a64b96b782ee6060ad1d6751b16e34fecbc7eb
+size 20930
--- a/completions/completions_00130.parquet
+++ b/completions/completions_00130.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c282ede81636a52bc38248cba7fc6977cbc9f139a79821680c05fd17ad6d6a5
+size 21673
--- a/completions/completions_00140.parquet
+++ b/completions/completions_00140.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04552e3b9ba6747e9fbf97f651cfbf041b16ca653b28df9aca318e56287bc5b8
+size 17355
--- a/completions/completions_00150.parquet
+++ b/completions/completions_00150.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a73a0c4351d15173bdb7b22117abf071f0338d397c0676246b7a623a2b13b7f
+size 24100
--- a/completions/completions_00160.parquet
+++ b/completions/completions_00160.parquet
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:57f36e5114b168437d03b6ec58afc042f893a85ac2db91d92178c0aacc504ed1
+size 19065
--- a/config.json
+++ b/config.json
@@ -0,0 +1,69 @@
+{
+  "architectures": [
+    "Qwen2ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": null,
+  "dtype": "bfloat16",
+  "eos_token_id": 151645,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 11008,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "max_window_layers": 70,
+  "model_type": "qwen2",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 36,
+  "num_key_value_heads": 2,
+  "pad_token_id": 151643,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "rope_theta": 1000000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.7.0",
+  "use_cache": false,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}
--- a/generation_config.json
+++ b/generation_config.json
@@ -0,0 +1,13 @@
+{
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "repetition_penalty": 1.05,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "5.7.0"
+}
--- a/last-checkpoint/chat_template.jinja
+++ b/last-checkpoint/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}
--- a/last-checkpoint/config.json
+++ b/last-checkpoint/config.json
@@ -0,0 +1,69 @@
+{
+  "architectures": [
+    "Qwen2ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": null,
+  "dtype": "bfloat16",
+  "eos_token_id": 151645,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 11008,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "max_window_layers": 70,
+  "model_type": "qwen2",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 36,
+  "num_key_value_heads": 2,
+  "pad_token_id": 151643,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "rope_theta": 1000000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.7.0",
+  "use_cache": false,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}
--- a/last-checkpoint/generation_config.json
+++ b/last-checkpoint/generation_config.json
@@ -0,0 +1,13 @@
+{
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "repetition_penalty": 1.05,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "5.7.0"
+}
--- a/last-checkpoint/model.safetensors
+++ b/last-checkpoint/model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6032762bcfdf97d0ffef0d438d20af4d7f344890e0668ef6b790396c6cafa1f9
+size 6171927112
--- a/last-checkpoint/tokenizer.json
+++ b/last-checkpoint/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892
--- a/last-checkpoint/tokenizer_config.json
+++ b/last-checkpoint/tokenizer_config.json
@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}
--- a/last-checkpoint/trainer_state.json
+++ b/last-checkpoint/trainer_state.json
@@ -0,0 +1,482 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.28,
+  "eval_steps": 500,
+  "global_step": 160,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.05,
+      "completions/max_length": 899.7,
+      "completions/max_terminated_length": 834.6,
+      "completions/mean_length": 599.55,
+      "completions/mean_terminated_length": 580.2101257324218,
+      "completions/min_length": 345.7,
+      "completions/min_terminated_length": 345.7,
+      "entropy": 0.9067775651812553,
+      "epoch": 0.08,
+      "frac_reward_zero_std": 0.15,
+      "grad_norm": 2.203125,
+      "kl": 0.034341241162474036,
+      "learning_rate": 9.4375e-06,
+      "loss": -0.016357305645942687,
+      "num_tokens": 68608.0,
+      "reward": 0.5690673828125,
+      "reward_std": 0.5320930480957031,
+      "rewards/JointRewardFunction/mean": 0.5690673828125,
+      "rewards/JointRewardFunction/std": 0.5320930480957031,
+      "step": 10,
+      "step_time": 36.65178473630149
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.075,
+      "completions/max_length": 915.4,
+      "completions/max_terminated_length": 890.0,
+      "completions/mean_length": 654.2625,
+      "completions/mean_terminated_length": 633.4842956542968,
+      "completions/min_length": 382.9,
+      "completions/min_terminated_length": 382.9,
+      "entropy": 1.2692338980734348,
+      "epoch": 0.16,
+      "frac_reward_zero_std": 0.175,
+      "grad_norm": 2.0625,
+      "kl": 0.08574351686984301,
+      "learning_rate": 8.8125e-06,
+      "loss": 0.011555161327123642,
+      "num_tokens": 141593.0,
+      "reward": 0.55255126953125,
+      "reward_std": 0.5202917702496052,
+      "rewards/JointRewardFunction/mean": 0.55255126953125,
+      "rewards/JointRewardFunction/std": 0.5202917769551277,
+      "step": 20,
+      "step_time": 38.578124190003294
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.025,
+      "completions/max_length": 831.7,
+      "completions/max_terminated_length": 792.0,
+      "completions/mean_length": 567.075,
+      "completions/mean_terminated_length": 556.1625061035156,
+      "completions/min_length": 353.2,
+      "completions/min_terminated_length": 353.2,
+      "entropy": 1.2077870965003967,
+      "epoch": 0.24,
+      "frac_reward_zero_std": 0.15,
+      "grad_norm": 1.4921875,
+      "kl": 0.07180349011905492,
+      "learning_rate": 8.1875e-06,
+      "loss": 0.021731069684028624,
+      "num_tokens": 207403.0,
+      "reward": 0.68575439453125,
+      "reward_std": 0.4835183143615723,
+      "rewards/JointRewardFunction/mean": 0.68575439453125,
+      "rewards/JointRewardFunction/std": 0.48351832032203673,
+      "step": 30,
+      "step_time": 35.385694373202206
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.025,
+      "completions/max_length": 899.7,
+      "completions/max_terminated_length": 841.7,
+      "completions/mean_length": 571.8875,
+      "completions/mean_terminated_length": 558.7678588867187,
+      "completions/min_length": 322.2,
+      "completions/min_terminated_length": 322.2,
+      "entropy": 1.0670640360563994,
+      "epoch": 0.32,
+      "frac_reward_zero_std": 0.15,
+      "grad_norm": 1.0859375,
+      "kl": 0.07311075136531145,
+      "learning_rate": 7.5625e-06,
+      "loss": -0.013997538387775421,
+      "num_tokens": 273728.0,
+      "reward": 0.74111328125,
+      "reward_std": 0.5125789776444435,
+      "rewards/JointRewardFunction/mean": 0.74111328125,
+      "rewards/JointRewardFunction/std": 0.512578995525837,
+      "step": 40,
+      "step_time": 38.20040096230041
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.025,
+      "completions/max_length": 842.8,
+      "completions/max_terminated_length": 821.3,
+      "completions/mean_length": 610.425,
+      "completions/mean_terminated_length": 600.9160766601562,
+      "completions/min_length": 379.3,
+      "completions/min_terminated_length": 379.3,
+      "entropy": 1.1711551163345575,
+      "epoch": 0.4,
+      "frac_reward_zero_std": 0.2,
+      "grad_norm": 1.3203125,
+      "kl": 0.08198881428688765,
+      "learning_rate": 6.9375e-06,
+      "loss": 0.0175897479057312,
+      "num_tokens": 343392.0,
+      "reward": 0.69986572265625,
+      "reward_std": 0.47377224564552306,
+      "rewards/JointRewardFunction/mean": 0.69986572265625,
+      "rewards/JointRewardFunction/std": 0.47377225160598757,
+      "step": 50,
+      "step_time": 35.86526439490153
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.0,
+      "completions/max_length": 783.2,
+      "completions/max_terminated_length": 783.2,
+      "completions/mean_length": 560.7,
+      "completions/mean_terminated_length": 560.7,
+      "completions/min_length": 375.6,
+      "completions/min_terminated_length": 375.6,
+      "entropy": 1.1601736910641194,
+      "epoch": 0.48,
+      "frac_reward_zero_std": 0.15,
+      "grad_norm": 2.0625,
+      "kl": 0.07123464278411121,
+      "learning_rate": 6.3125e-06,
+      "loss": 0.005981885641813278,
+      "num_tokens": 408588.0,
+      "reward": 0.77021484375,
+      "reward_std": 0.45352104529738424,
+      "rewards/JointRewardFunction/mean": 0.77021484375,
+      "rewards/JointRewardFunction/std": 0.4535210572183132,
+      "step": 60,
+      "step_time": 33.62324385899883
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.0375,
+      "completions/max_length": 859.6,
+      "completions/max_terminated_length": 791.1,
+      "completions/mean_length": 576.6875,
+      "completions/mean_terminated_length": 558.1005981445312,
+      "completions/min_length": 331.1,
+      "completions/min_terminated_length": 331.1,
+      "entropy": 1.0957016348838806,
+      "epoch": 0.56,
+      "frac_reward_zero_std": 0.225,
+      "grad_norm": 1.578125,
+      "kl": 0.06286744237877429,
+      "learning_rate": 5.6875e-06,
+      "loss": 0.02311268150806427,
+      "num_tokens": 475179.0,
+      "reward": 0.87257080078125,
+      "reward_std": 0.4612266093492508,
+      "rewards/JointRewardFunction/mean": 0.87257080078125,
+      "rewards/JointRewardFunction/std": 0.4612266182899475,
+      "step": 70,
+      "step_time": 36.65166022339836
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.0,
+      "completions/max_length": 801.1,
+      "completions/max_terminated_length": 801.1,
+      "completions/mean_length": 529.3,
+      "completions/mean_terminated_length": 529.3,
+      "completions/min_length": 325.9,
+      "completions/min_terminated_length": 325.9,
+      "entropy": 0.9475126106292009,
+      "epoch": 0.64,
+      "frac_reward_zero_std": 0.175,
+      "grad_norm": 1.6484375,
+      "kl": 0.0728621585527435,
+      "learning_rate": 5.0625e-06,
+      "loss": -0.00028136512264609337,
+      "num_tokens": 538137.0,
+      "reward": 0.92275390625,
+      "reward_std": 0.4701471716165543,
+      "rewards/JointRewardFunction/mean": 0.92275390625,
+      "rewards/JointRewardFunction/std": 0.47014717757701874,
+      "step": 80,
+      "step_time": 34.54745226030063
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.0125,
+      "completions/max_length": 809.0,
+      "completions/max_terminated_length": 774.7,
+      "completions/mean_length": 550.9125,
+      "completions/mean_terminated_length": 544.0785736083984,
+      "completions/min_length": 338.7,
+      "completions/min_terminated_length": 338.7,
+      "entropy": 0.8775507573038339,
+      "epoch": 0.72,
+      "frac_reward_zero_std": 0.225,
+      "grad_norm": 1.6328125,
+      "kl": 0.0652636706130579,
+      "learning_rate": 4.4375e-06,
+      "loss": 0.00958292931318283,
+      "num_tokens": 602830.0,
+      "reward": 0.9910400390625,
+      "reward_std": 0.49945615530014037,
+      "rewards/JointRewardFunction/mean": 0.9910400390625,
+      "rewards/JointRewardFunction/std": 0.49945615530014037,
+      "step": 90,
+      "step_time": 34.542127643501225
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.025,
+      "completions/max_length": 854.6,
+      "completions/max_terminated_length": 811.1,
+      "completions/mean_length": 556.0125,
+      "completions/mean_terminated_length": 544.571435546875,
+      "completions/min_length": 342.6,
+      "completions/min_terminated_length": 342.6,
+      "entropy": 0.9368259467184543,
+      "epoch": 0.8,
+      "frac_reward_zero_std": 0.1,
+      "grad_norm": 1.5703125,
+      "kl": 0.06576599276158959,
+      "learning_rate": 3.8125e-06,
+      "loss": 0.025794655084609985,
+      "num_tokens": 667977.0,
+      "reward": 0.95867919921875,
+      "reward_std": 0.45399501323699953,
+      "rewards/JointRewardFunction/mean": 0.95867919921875,
+      "rewards/JointRewardFunction/std": 0.45399501025676725,
+      "step": 100,
+      "step_time": 36.289319235399674
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.0,
+      "completions/max_length": 818.4,
+      "completions/max_terminated_length": 818.4,
+      "completions/mean_length": 521.6875,
+      "completions/mean_terminated_length": 521.6875,
+      "completions/min_length": 269.7,
+      "completions/min_terminated_length": 269.7,
+      "entropy": 0.8984692271798849,
+      "epoch": 0.88,
+      "frac_reward_zero_std": 0.15,
+      "grad_norm": 1.7578125,
+      "kl": 0.06471977110486478,
+      "learning_rate": 3.1875e-06,
+      "loss": 0.011397648602724075,
+      "num_tokens": 730256.0,
+      "reward": 0.99234619140625,
+      "reward_std": 0.42927705943584443,
+      "rewards/JointRewardFunction/mean": 0.99234619140625,
+      "rewards/JointRewardFunction/std": 0.4292770564556122,
+      "step": 110,
+      "step_time": 35.05004594640013
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.0,
+      "completions/max_length": 848.3,
+      "completions/max_terminated_length": 848.3,
+      "completions/mean_length": 537.95,
+      "completions/mean_terminated_length": 537.95,
+      "completions/min_length": 328.4,
+      "completions/min_terminated_length": 328.4,
+      "entropy": 0.886419289931655,
+      "epoch": 0.96,
+      "frac_reward_zero_std": 0.325,
+      "grad_norm": 1.4609375,
+      "kl": 0.06785837954375892,
+      "learning_rate": 2.5625e-06,
+      "loss": 0.01289355307817459,
+      "num_tokens": 793664.0,
+      "reward": 0.94119873046875,
+      "reward_std": 0.4706839919090271,
+      "rewards/JointRewardFunction/mean": 0.94119873046875,
+      "rewards/JointRewardFunction/std": 0.47068399786949155,
+      "step": 120,
+      "step_time": 36.370545529201266
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.0375,
+      "completions/max_length": 827.7,
+      "completions/max_terminated_length": 761.1,
+      "completions/mean_length": 534.6625,
+      "completions/mean_terminated_length": 516.6375061035156,
+      "completions/min_length": 321.3,
+      "completions/min_terminated_length": 321.3,
+      "entropy": 0.954836780950427,
+      "epoch": 1.04,
+      "frac_reward_zero_std": 0.275,
+      "grad_norm": 1.7265625,
+      "kl": 0.06585919852368534,
+      "learning_rate": 1.9375e-06,
+      "loss": 0.011952327191829681,
+      "num_tokens": 857055.0,
+      "reward": 0.96031494140625,
+      "reward_std": 0.44789557456970214,
+      "rewards/JointRewardFunction/mean": 0.96031494140625,
+      "rewards/JointRewardFunction/std": 0.44789558053016665,
+      "step": 130,
+      "step_time": 35.311843357800896
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.025,
+      "completions/max_length": 826.3,
+      "completions/max_terminated_length": 738.5,
+      "completions/mean_length": 522.15,
+      "completions/mean_terminated_length": 506.9553619384766,
+      "completions/min_length": 309.7,
+      "completions/min_terminated_length": 309.7,
+      "entropy": 0.8662954691797495,
+      "epoch": 1.12,
+      "frac_reward_zero_std": 0.25,
+      "grad_norm": 2.125,
+      "kl": 0.07243931000120937,
+      "learning_rate": 1.3125000000000001e-06,
+      "loss": 0.03181460499763489,
+      "num_tokens": 919475.0,
+      "reward": 0.86949462890625,
+      "reward_std": 0.5099265575408936,
+      "rewards/JointRewardFunction/mean": 0.86949462890625,
+      "rewards/JointRewardFunction/std": 0.5099265664815903,
+      "step": 140,
+      "step_time": 35.292809389702235
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.0,
+      "completions/max_length": 779.8,
+      "completions/max_terminated_length": 779.8,
+      "completions/mean_length": 524.8875,
+      "completions/mean_terminated_length": 524.8875,
+      "completions/min_length": 323.7,
+      "completions/min_terminated_length": 323.7,
+      "entropy": 0.9163745902478695,
+      "epoch": 1.2,
+      "frac_reward_zero_std": 0.325,
+      "grad_norm": 1.3359375,
+      "kl": 0.07890935295727104,
+      "learning_rate": 6.875000000000001e-07,
+      "loss": 0.0025221250951290132,
+      "num_tokens": 982042.0,
+      "reward": 1.10718994140625,
+      "reward_std": 0.4580986052751541,
+      "rewards/JointRewardFunction/mean": 1.10718994140625,
+      "rewards/JointRewardFunction/std": 0.45809862315654754,
+      "step": 150,
+      "step_time": 33.446236442600274
+    },
+    {
+      "clip_ratio/high_max": 0.0,
+      "clip_ratio/high_mean": 0.0,
+      "clip_ratio/low_mean": 0.0,
+      "clip_ratio/low_min": 0.0,
+      "clip_ratio/region_mean": 0.0,
+      "completions/clipped_ratio": 0.0125,
+      "completions/max_length": 720.2,
+      "completions/max_terminated_length": 691.7,
+      "completions/mean_length": 492.3125,
+      "completions/mean_terminated_length": 485.6000030517578,
+      "completions/min_length": 343.0,
+      "completions/min_terminated_length": 343.0,
+      "entropy": 0.8506938068196177,
+      "epoch": 1.28,
+      "frac_reward_zero_std": 0.225,
+      "grad_norm": 1.71875,
+      "kl": 0.06682636064942926,
+      "learning_rate": 6.250000000000001e-08,
+      "loss": 0.03128217160701752,
+      "num_tokens": 1042329.0,
+      "reward": 0.963037109375,
+      "reward_std": 0.4769674390554428,
+      "rewards/JointRewardFunction/mean": 0.963037109375,
+      "rewards/JointRewardFunction/std": 0.47696744799613955,
+      "step": 160,
+      "step_time": 31.293741717099692
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 160,
+  "num_input_tokens_seen": 1042329,
+  "num_train_epochs": 2,
+  "save_steps": 10,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}
--- a/last-checkpoint/training_args.bin
+++ b/last-checkpoint/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41733497170620207c2a02121b3cf2b43ec76ffc1db127fa6366ce5f1d1ccfde
+size 7249
--- a/model.safetensors
+++ b/model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6032762bcfdf97d0ffef0d438d20af4d7f344890e0668ef6b790396c6cafa1f9
+size 6171927112
--- a/tokenizer.json
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}
--- a/training_args.bin
+++ b/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41733497170620207c2a02121b3cf2b43ec76ffc1db127fa6366ce5f1d1ccfde
+size 7249