初始化项目，由ModelHub XC社区提供模型
Model: Nanbeige/CoSineVerifier-Tool-4B Source: Original Platform
This commit is contained in:
ModelHub XC
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
228
README.md
Normal file
228
README.md
Normal file
@@ -0,0 +1,228 @@
|
||||
---
|
||||
license: mit
|
||||
---
|
||||
|
||||
# CoSineVerifier: Tool-Augmented Answer Verification for Computation-Oriented Scientific Questions.
|
||||
|
||||
# 📖 Introduction
|
||||
|
||||
We present **CoSineVerifier-Tool**, a compact, tool-augmented verifier for **C**omputation-**O**riented **S**c**I**e**N**tific s**E**nario answer verification. It equips LLM reasoning with external tools—e.g., Python interpreter,—to accurately verify answers in **computation-oriented scenarios** such as algebraic equivalence and physical-constant alignment. We also release the **CoSineVerifier** series: efficient single-token labeling verifiers with performance comparable to CoSineVerifier-Tool. Our methods achieve state-of-the-art results on VerifyBench and SCI-VerifyBench. They also show clear improvements on RLVR tasks over other verification methods.
|
||||
|
||||
<figure style="text-align:center;">
|
||||
<img src="./pics/RLVR_Experiments.png" alt="RLVR experiments benchmark results">
|
||||
<figcaption style="font-size:0.95em; color:#666; margin-top:6px;">
|
||||
Picture 1: RLVR performance with different verification methods.
|
||||
</figcaption>
|
||||
</figure>
|
||||
|
||||
|
||||
|
||||
# 🌟 Key Features:
|
||||
|
||||
- **Tool-augmented verification for computation-oriented scientific scenarios**:
|
||||
|
||||
CoSineVerifier-Tool evaluates multi-step reasoning in math and science where correctness depends on precise intermediate calculations. It also normalizes answers to a common dimension/unit for consistent numeric checks. It integrates:
|
||||
|
||||
- 🧮Python Interpreter: executes snippets to validate algebra/calculus steps, algorithmic logic, and data operations.
|
||||
- 🔬Scientific Unit Converter: verifies and normalizes unit conversions (e.g., km/h → m/s; joule → calorie).
|
||||
|
||||
- **Concise, low-latency reasoning**:
|
||||
|
||||
A 4B-parameter CoSinVerifier-Tool with ≤100 output tokens enables real-time evaluation and large-scale batch processing.
|
||||
|
||||
- **General** **applicability across tasks and domains**:
|
||||
|
||||
Accurately assesses equivalence across math, physics, chemistry, biology, and logical reasoning; supports short-answer and multiple-choice formats; and handles both brief answers and long-form responses.
|
||||
|
||||
|
||||
|
||||
# 🧪 Quick Start
|
||||
|
||||
This repository provides a **minimal working demo** for playing around the verifier conveniently.
|
||||
|
||||
### **⚙️ Installation**
|
||||
|
||||
```
|
||||
pip install torch transformers pint
|
||||
```
|
||||
|
||||
### **🚀 Run the Demo**
|
||||
|
||||
You can run the verifier with local or Hugging Face model checkpoint:
|
||||
|
||||
```
|
||||
python demo.py --model /path/to/the/verifier
|
||||
```
|
||||
|
||||
The script will:
|
||||
|
||||
1. Load the model and tokenizer.
|
||||
2. Run several **evaluation cases** (e.g., numerical equivalence, physics derivations, symbolic math).
|
||||
3. Print intermediate reasoning, any tool calls, and the final judgement for each case.
|
||||
|
||||
### **😊 Note**
|
||||
|
||||
- Use the **exact** system and user prompt templates. **Do** **not** change whitespace; preserve line breaks **identical**.
|
||||
- The **final judgement** is parsed from the token `\boxed{Correct}` or `\boxed{Incorrect}` in the model’s reply.
|
||||
|
||||
### **🧩 Example**
|
||||
|
||||
Hamilton–Jacobi Projectile Motion Verification
|
||||
|
||||
<div style="max-height:480px; overflow:auto; border:2px solid #ccc; border-radius:8px; padding:10px; background:#fafafa; font-family:monospace; line-height:1.3; font-size:12px;">
|
||||
<hr style="border:none;border-top:1px solid #ddd;margin:6px 0"/>
|
||||
<span style="font-weight:1000;">User:</span>
|
||||
<pre style="white-space:pre-wrap; margin:6px 0; font-size:15px;">
|
||||
{
|
||||
"Question":"A particle m undergoes projectile motion with an initial velocity v₀ and a horizontal angle of elevation α. Solve using the Hamilton–Jacobi equation.",
|
||||
"Ground_Truth":"['y = x tan α - (g x²) / (2 v₀² cos² α)']",
|
||||
"Model_Prediction":"y = x tan α - (g x² tan² α) / (2 v₀²) - (g x²) / (2 v₀²)"
|
||||
}
|
||||
</pre>
|
||||
<hr style="border:none;border-top:1px solid #ddd;margin:6px 0"/>
|
||||
<span style="font-weight:1000;">Assistant:</span>
|
||||
<pre style="white-space:pre-wrap; margin:6px 0; font-size:15px;"><tool_call>
|
||||
{
|
||||
"name":"python_interpreter",
|
||||
"arguments":{
|
||||
"code":"```python
|
||||
def main():
|
||||
import math
|
||||
a,g,v0,x=1.0,9.8,10.0,1.0
|
||||
y_candidate=x*math.tan(a)-(g*x**2*math.tan(a)**2)/(2*v0**2)-(g*x**2)/(2*v0**2)
|
||||
y_gold=x*math.tan(a)-(g*x**2)/(2*v0**2*(math.cos(a)**2))
|
||||
print(y_candidate,y_gold)
|
||||
if __name__=='__main__':
|
||||
main()```"
|
||||
}
|
||||
}
|
||||
</tool_call></pre>
|
||||
<hr style="border:none;border-top:1px solid #ddd;margin:6px 0"/>
|
||||
<span style="font-weight:1000;">User:</span>
|
||||
<pre style="white-space:pre-wrap; margin:6px 0; font-size:15px;"><tool_response>
|
||||
{
|
||||
"name":"python_interpreter",
|
||||
"status":"Success",
|
||||
"run_result":{
|
||||
"stdout":"1.3895573024349792 1.3895573024349792",
|
||||
"stderr":"",
|
||||
"exit_success":true,
|
||||
"return_code":0,
|
||||
"status":"Finished"
|
||||
}
|
||||
}
|
||||
</tool_response></pre>
|
||||
<hr style="border:none;border-top:1px solid #ddd;margin:6px 0"/>
|
||||
<span style="font-weight:1000;">Assistant:</span>
|
||||
<pre style="white-space:pre-wrap; margin:6px 0; font-size:15px;">
|
||||
Gold: \(y=x\tan\alpha-\frac{g x^2}{2v_0^2\cos^2\alpha}\).
|
||||
Using \(\sec^2\alpha=1+\tan^2\alpha\): \(\frac{g x^2}{2v_0^2\cos^2\alpha}=\frac{g x^2}{2v_0^2}+\frac{g x^2\tan^2\alpha}{2v_0^2}\).
|
||||
Hence \(y=x\tan\alpha-\frac{g x^2}{2v_0^2}-\frac{g x^2\tan^2\alpha}{2v_0^2}\equiv\texttt{pred}\).
|
||||
Final Verdict:</b> <code>[Correct]</code>
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
### **🔍 Performance**
|
||||
|
||||
We conduct a systematic evaluation on [VerifyBench](https://github.com/ZJU-REAL/VerifyBench) and [SCI-VerifyBench] for **CoSineVerifier-Tool-4B** and **CoSineVerifier-32B**, trained from **Qwen3-4B-Instruct-2507** and **Qwen3-32B**, respectively. We report accuracy as mean@3 and efficiency as average output tokens per verdict on these benchmarks.
|
||||
|
||||
<table style="margin:0 auto;">
|
||||
<thead>
|
||||
<tr>
|
||||
<th style="text-align:center;">Model</th>
|
||||
<th style="text-align:center;">VerifyBench</th>
|
||||
<th style="text-align:center;">VerifyBench (Hard)</th>
|
||||
<th style="text-align:center;">SCI-VerifyBench</th>
|
||||
<th style="text-align:center;">Avg Tokens</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<!-- Group: CoT Verifier -->
|
||||
<tr>
|
||||
<th colspan="5" style="text-align:center;font-weight:800;color:#000;">CoT Verifier</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:center;">o3</td><td style="text-align:center;">96.1</td><td style="text-align:center;">88.7</td><td style="text-align:center;">87.5</td><td style="text-align:center;">206.7</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:center;">GPT-oss-20B</td><td style="text-align:center;">92.2</td><td style="text-align:center;">84.7</td><td style="text-align:center;">85.0</td><td style="text-align:center;">221</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:center;">Qwen3-4B</td><td style="text-align:center;">92.6</td><td style="text-align:center;">80.3</td><td style="text-align:center;">82.0</td><td style="text-align:center;">1156.7</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:center;">Qwen3-8B</td><td style="text-align:center;">93.7</td><td style="text-align:center;">83.6</td><td style="text-align:center;">83.9</td><td style="text-align:center;">926.6</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:center;">Qwen3-32B</td><td style="text-align:center;">94.7</td><td style="text-align:center;">85.2</td><td style="text-align:center;">83.5</td><td style="text-align:center;">798.8</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:center;">Qwen3-4B-Instruct-2507</td><td style="text-align:center;">94.7</td><td style="text-align:center;">84.1</td><td style="text-align:center;">82.4</td><td style="text-align:center;">869.7</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:center;">Qwen3-235B-A22B-2507</td><td style="text-align:center;">94.4</td><td style="text-align:center;">87.7</td><td style="text-align:center;">82.6</td><td style="text-align:center;">1885.3</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:center;"><strong>CoSineVerifier-Tool-4B</strong></td><td style="text-align:center;"><strong>96.6</strong></td><td style="text-align:center;"><strong>91.9</strong></td><td style="text-align:center;"><strong>89.7</strong></td><td style="text-align:center;"><strong>95.3</strong></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th colspan="5" style="text-align:center;font-weight:800;color:#000;">Labeling Verifier</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:center;">XVerify-8B-I</td><td style="text-align:center;">92.5</td><td style="text-align:center;">83.3</td><td style="text-align:center;">78.1</td><td style="text-align:center;">1.0</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:center;">CompassVerifier-7B</td><td style="text-align:center;">93.5</td><td style="text-align:center;">85.2</td><td style="text-align:center;">85.7</td><td style="text-align:center;">1.0</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:center;">CompassVerifier-32B</td><td style="text-align:center;">96.3</td><td style="text-align:center;">88.9</td><td style="text-align:center;">85.3</td><td style="text-align:center;">1.0</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:center;">CoSineVerifier-4B</td><td style="text-align:center;">95.0</td><td style="text-align:center;">84.6</td><td style="text-align:center;">86.2</td><td style="text-align:center;">3.0</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:center;"><strong>CoSineVerifier-32B</strong></td><td style="text-align:center;"><strong>94.2</strong></td><td style="text-align:center;"><strong>89.7</strong></td><td style="text-align:center;"><strong>86.6</strong></td><td style="text-align:center;"><strong>3.0</strong></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
We further evaluate answer-verification methods in an RLVR setting to demonstrate the efficacy of the CoSineVerifier series. Using an on-policy GRPO algorithm, we train **Qwen3-4B-Instruct-2507** on competition-math problems with 42K training data drawn from [DAPO-Math-17k](https://huggingface.co/datasets/BytedTsinghua-SIA/DAPO-Math-17k), [OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k), and [DeepScaleR-Preview](https://huggingface.co/datasets/agentica-org/DeepScaleR-Preview-Dataset). We train 3 epochs for each experiment, with batch size of 128 and rollout_num = 8.
|
||||
|
||||
We compare CoSineVerifier-Tool-4B and CoSineVerifier-32B against Math-verify, CompassVerifier, and Xverify. We report **mean@32 accuracy** on AIME 2024 and AIME 2025 as reported in Figure 1.
|
||||
|
||||
|
||||
# 🤖 Limitations
|
||||
|
||||
Despite its high accuracy, **CoSinVerifier-Tool-4B** invokes external tools in only ~10% of cases and still struggles on the hardest verification items. We will improve its capability by increasing tool-use coverage and developing a unified external-tool suite. Current CoSineVerifier Labeling models output three tokens instead of one token and we will update it into 1 token in next version.
|
||||
|
||||
|
||||
|
||||
# ✏️ Citation
|
||||
|
||||
If you find our verifiers useful or want to use it in your projects, please kindly cite this Huggingface project.
|
||||
|
||||
<pre><code>
|
||||
@misc{feng2025cosineverifiertoolaugmentedanswerverification,
|
||||
title={CoSineVerifier: Tool-Augmented Answer Verification for Computation-Oriented Scientific Questions},
|
||||
author={Ruixiang Feng and Zhenwei An and Yuntao Wen and Ran Le and Yiming Jia and Chen Yang and Zongchao Chen and Lisi Chen and Shen Gao and Shuo Shang and Yang Song and Tao Zhang},
|
||||
year={2025},
|
||||
eprint={2512.01224},
|
||||
archivePrefix={arXiv},
|
||||
primaryClass={cs.LG},
|
||||
url={https://arxiv.org/abs/2512.01224},
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
# 📮 Contact
|
||||
|
||||
If you have any questions, please raise an issue or contact us at [nanbeige@126.com](mailto:nanbeige@126.com).
|
||||
28
added_tokens.json
Normal file
28
added_tokens.json
Normal file
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"</think>": 151668,
|
||||
"</tool_call>": 151658,
|
||||
"</tool_response>": 151666,
|
||||
"<think>": 151667,
|
||||
"<tool_call>": 151657,
|
||||
"<tool_response>": 151665,
|
||||
"<|box_end|>": 151649,
|
||||
"<|box_start|>": 151648,
|
||||
"<|endoftext|>": 151643,
|
||||
"<|file_sep|>": 151664,
|
||||
"<|fim_middle|>": 151660,
|
||||
"<|fim_pad|>": 151662,
|
||||
"<|fim_prefix|>": 151659,
|
||||
"<|fim_suffix|>": 151661,
|
||||
"<|im_end|>": 151645,
|
||||
"<|im_start|>": 151644,
|
||||
"<|image_pad|>": 151655,
|
||||
"<|object_ref_end|>": 151647,
|
||||
"<|object_ref_start|>": 151646,
|
||||
"<|quad_end|>": 151651,
|
||||
"<|quad_start|>": 151650,
|
||||
"<|repo_name|>": 151663,
|
||||
"<|video_pad|>": 151656,
|
||||
"<|vision_end|>": 151653,
|
||||
"<|vision_pad|>": 151654,
|
||||
"<|vision_start|>": 151652
|
||||
}
|
||||
86
chat_template.jinja
Normal file
86
chat_template.jinja
Normal file
@@ -0,0 +1,86 @@
|
||||
{%- if tools %}
|
||||
{{- '<|im_start|>system\n' }}
|
||||
{%- if messages[0].role == 'system' %}
|
||||
{{- messages[0].content + '\n\n' }}
|
||||
{%- endif %}
|
||||
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
||||
{%- for tool in tools %}
|
||||
{{- "\n" }}
|
||||
{{- tool | tojson }}
|
||||
{%- endfor %}
|
||||
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
||||
{%- else %}
|
||||
{%- if messages[0].role == 'system' %}
|
||||
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
||||
{%- for message in messages[::-1] %}
|
||||
{%- set index = (messages|length - 1) - loop.index0 %}
|
||||
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
|
||||
{%- set ns.multi_step_tool = false %}
|
||||
{%- set ns.last_query_index = index %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- for message in messages %}
|
||||
{%- if message.content is string %}
|
||||
{%- set content = message.content %}
|
||||
{%- else %}
|
||||
{%- set content = '' %}
|
||||
{%- endif %}
|
||||
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
||||
{%- elif message.role == "assistant" %}
|
||||
{%- set reasoning_content = '' %}
|
||||
{%- if message.reasoning_content is string %}
|
||||
{%- set reasoning_content = message.reasoning_content %}
|
||||
{%- else %}
|
||||
{%- if '</think>' in content %}
|
||||
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
||||
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- if loop.index0 > ns.last_query_index %}
|
||||
{%- if loop.last or (not loop.last and reasoning_content) %}
|
||||
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||
{%- endif %}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||
{%- endif %}
|
||||
{%- if message.tool_calls %}
|
||||
{%- for tool_call in message.tool_calls %}
|
||||
{%- if (loop.first and content) or (not loop.first) %}
|
||||
{{- '\n' }}
|
||||
{%- endif %}
|
||||
{%- if tool_call.function %}
|
||||
{%- set tool_call = tool_call.function %}
|
||||
{%- endif %}
|
||||
{{- '<tool_call>\n{"name": "' }}
|
||||
{{- tool_call.name }}
|
||||
{{- '", "arguments": ' }}
|
||||
{%- if tool_call.arguments is string %}
|
||||
{{- tool_call.arguments }}
|
||||
{%- else %}
|
||||
{{- tool_call.arguments | tojson }}
|
||||
{%- endif %}
|
||||
{{- '}\n</tool_call>' }}
|
||||
{%- endfor %}
|
||||
{%- endif %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- elif message.role == "tool" %}
|
||||
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
||||
{{- '<|im_start|>user' }}
|
||||
{%- endif %}
|
||||
{{- '\n<tool_response>\n' }}
|
||||
{{- content }}
|
||||
{{- '\n</tool_response>' }}
|
||||
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- if add_generation_prompt %}
|
||||
{{- '<|im_start|>assistant\n' }}
|
||||
{%- endif %}
|
||||
71
config.json
Normal file
71
config.json
Normal file
@@ -0,0 +1,71 @@
|
||||
{
|
||||
"_attn_implementation_autoset": true,
|
||||
"architectures": [
|
||||
"Qwen3ForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"dtype": "bfloat16",
|
||||
"embd_pdrop": 0.0,
|
||||
"eos_token_id": 151645,
|
||||
"head_dim": 128,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 2560,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 9728,
|
||||
"layer_types": [
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention"
|
||||
],
|
||||
"max_position_embeddings": 262144,
|
||||
"max_window_layers": 36,
|
||||
"model_type": "qwen3",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 36,
|
||||
"num_key_value_heads": 8,
|
||||
"pad_token_id": 151643,
|
||||
"resid_pdrop": 0.0,
|
||||
"rms_norm_eps": 1e-06,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 5000000,
|
||||
"sliding_window": null,
|
||||
"tie_word_embeddings": true,
|
||||
"transformers_version": "4.56.2",
|
||||
"use_cache": false,
|
||||
"use_sliding_window": false,
|
||||
"vocab_size": 151936
|
||||
}
|
||||
342
demo.py
Normal file
342
demo.py
Normal file
@@ -0,0 +1,342 @@
|
||||
import argparse, json, os, re, subprocess, sys, tempfile
|
||||
from typing import Dict, Any, List, Optional, Tuple
|
||||
import torch
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig
|
||||
import json
|
||||
|
||||
|
||||
|
||||
YOUR_SYSTEM_PROMPT = """# Tools
|
||||
|
||||
You may call one or more functions to assist with the user query.
|
||||
|
||||
You are provided with function signatures within <tools></tools> XML tags:
|
||||
<tools>
|
||||
{"type": "function", "function": {"name": "python_interpreter", "description": "Executes a complete, runnable Python code. The code must be well-structured, preferably with a 'main()' function, and wrapped in a Python markdown block as shown in the parameter description.\\nNote: Common packages (math, collections, re, etc.) are pre-imported, so do NOT include standard 'import' statements.", "parameters": {"type": "object", "properties": {"code": {"type": "string", "description": "A string containing a full Python code.\\nThe code should define its logic within a 'main()' function and call it using an 'if __name__ == \\\\\"__main__\\\\":' block. \\nThe final, concise result must be printed to the console using the 'print()' function.\\n\\nFor example:\\ndef main():\\n # Your code logic here\\n result = 10 * 5\\n print(result)\\n\\nif __name__ == \\\\\"__main__\\\\":\\n main()"}}, "required": ["code"]}}}
|
||||
{"type": "function", "function": {"name": "unit_conversion", "description": "Performs precise physical unit conversions. Handles length, mass, time, pressure, energy, and complex compound units (e.g., velocity, acceleration).\\nIMPORTANT: For temperature, use 'degC', 'degF', 'K'. For scientific notation like '1.8 * 10^5 Pa', the value must be fully resolved to a number.", "parameters": {"type": "object", "properties": {"value": {"type": "number", "description": "The complete numerical value to be converted, as a single float or integer. If the value is in scientific notation (e.g., 1.8e5 or 1.8 * 10**5), calculate and pass the final number. Example: for '1.8 * 10^5', pass 180000."}, "source_unit": {"type": "string", "description": "The unit of the input 'value'. Must be a standard physical unit symbol ONLY. Do NOT include numbers or scientific notation like '10^5' or 'e5'. Supports compound units with '/' for division and '^' or '**' for powers. Examples: 'kg', 'km/h', 'm/s^2', 'Pa', 'MJ', 'degC'."}, "target_unit": {"type": "string", "description": "The desired unit to convert to. Must be dimensionally compatible with the source unit. Examples: 'g', 'm/s', 'ft/s^2', 'atm', 'kWh', 'degF'."}}, "required": ["value", "source_unit", "target_unit"]}}}
|
||||
</tools>
|
||||
|
||||
For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
|
||||
<tool_call>
|
||||
{"name": <function-name>, "arguments": <args-json-object>}
|
||||
</tool_call>"""
|
||||
|
||||
USER_TMPL = '''You are a diligent and precise assistant tasked with evaluating the correctness of responses. You will receive a question, an output sentence, and the correct answer. Your task is to determine if the output sentence corectly answers the question based on the provided correct answer. You can perform a short tool call and a short reasoning process. After a short reasoing process, put your response in the final with either [Correct] or [Incorrect] wrapped in \\boxed{{}}
|
||||
|
||||
Evaluation Protocol:
|
||||
1. Reference Standard:
|
||||
- The standard (gold) answer is definitive and always correct.
|
||||
- The question is always valid — never challenge it.
|
||||
- Allow equivalenct meaning answers.
|
||||
- Do not regenerate answers; only compare candidate's final answer with the gold answer.
|
||||
- You only need to compare correct answer and output sentence, do not regenerate or judge correct answer.
|
||||
|
||||
2. Comparison Method:
|
||||
- Analyze the question's requirements and the gold answer's structure.
|
||||
- Determine if the question requires exact matching or allows equivalence.
|
||||
- Compare ONLY the candidate's final answer. Ignore reasoning errors.
|
||||
- Ignore differences in formatting or style.
|
||||
- For math expressions: check algebraic equivalence step by step; if uncertain, test numerically at multiple points.
|
||||
- For multiple-choice: only compare the final choice and its content.
|
||||
|
||||
3. Multi-part Answers:
|
||||
- All parts must match the gold answer exactly.
|
||||
- Partial matches are incorrect.
|
||||
- If not specified, answer order may vary. For example, \\frac{{27}}{{7}}, -\\frac{{8}}{{7}} and -\\frac{{8}}{{7}}, \\frac{{27}}{{7}} are equivalent.
|
||||
|
||||
-
|
||||
Special considerations:
|
||||
|
||||
1. **Mathematical Problems**:
|
||||
- If the formats differ but the answers are mathematically equivalent after simplfying or rounding to two decimal places(e.g. 2.909 vs \\frac{{32}}{{11}}, \\frac{{32}}{{11}} vs \\frac{{96}}{{33}}), respond with [Correct].
|
||||
- You only need to verify the correctness of the mathematical expression, not values unrelated to the overall expression, such as the domain or units(e.g. 16 vs 16km, 20 vs 20db), these cases will be considered as [Correct].
|
||||
- You may need to calculate the value or converse the value to different unit when needed to match the reference answer.
|
||||
|
||||
2. **Multi-choice questions**:
|
||||
- If the question provides explicit candidate answers(e.g. multi-choice questions), the output will be considered correct if it clearly indicates the correct option's content or the correct option's code.
|
||||
|
||||
3. **Fact quuestions**:
|
||||
- If the question provides fact-seeking answers, the output must align with the correct answer in content to be considered [Correct].
|
||||
|
||||
4. **Multiple Reference Answers**:
|
||||
-If multiple reference answers are equivalent, just matching one answer will be considered [Correct].
|
||||
-If multiple reference answers are inequivalent, only mathcing all answers will be considered [Correct].
|
||||
|
||||
5. **Ohter conditions**:
|
||||
- If incomplete (cut off, unfinished sentence) → Label as [Incorrect].
|
||||
- If repetitive (looping words/phrases) → Label as [Incorrect].
|
||||
- Gives an answer but then negates it at the end. → Label as [Incorrect].
|
||||
- Numerically correct but without units. → Label as [Correct].
|
||||
-
|
||||
|
||||
-
|
||||
You can use following tools to help your verification process:
|
||||
|
||||
1. **Python Intepreter**: When you feel needed, you can use a python inteperter to help you determine your verification result.
|
||||
|
||||
2. **Unit Conversion Tool**: When faced with different physical units, you can use a unit conversion tool to convert them into the same unit.
|
||||
-
|
||||
|
||||
Question: """{question}"""
|
||||
|
||||
Output sentence: """{pred}"""
|
||||
|
||||
Correct answer: {reference}
|
||||
|
||||
Judgement:'''
|
||||
|
||||
# ======= Tool execution (demo) =======
|
||||
def run_python_interpreter(args: Dict[str, Any]) -> str:
|
||||
code = str(args.get("code", ""))
|
||||
# SECURITY NOTE: This executes arbitrary code. Demo only.
|
||||
with tempfile.NamedTemporaryFile("w", suffix=".py", delete=False) as tf:
|
||||
tf.write(code)
|
||||
path = tf.name
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
[sys.executable, path],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=float(args.get("timeout", 5.0)),
|
||||
)
|
||||
return json.dumps(
|
||||
{"stdout": proc.stdout.strip(), "stderr": proc.stderr.strip(), "returncode": proc.returncode},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
return "ERROR: Timeout"
|
||||
except Exception as e:
|
||||
return f"ERROR: {e}"
|
||||
finally:
|
||||
try: os.remove(path)
|
||||
except Exception: pass
|
||||
|
||||
def run_unit_conversion(args: Dict[str, Any]) -> str:
|
||||
try:
|
||||
import pint
|
||||
except Exception:
|
||||
return "ERROR: pint not installed. pip install pint"
|
||||
try:
|
||||
value = float(args["value"])
|
||||
src = str(args["source_unit"])
|
||||
tgt = str(args["target_unit"])
|
||||
except Exception as e:
|
||||
return f"ERROR: bad args: {e}"
|
||||
|
||||
ureg = pint.UnitRegistry(system="mks")
|
||||
try:
|
||||
q = ureg.Quantity(value, src)
|
||||
res = q.to(tgt)
|
||||
return f"{float(res.magnitude)} {res.units}"
|
||||
except pint.errors.DimensionalityError as e:
|
||||
return f"ERROR: Dimensionality mismatch: {e}"
|
||||
except pint.errors.UndefinedUnitError as e:
|
||||
return f"ERROR: Undefined unit: {e}"
|
||||
except Exception as e:
|
||||
return f"ERROR: {e}"
|
||||
|
||||
TOOL_IMPLS = {
|
||||
"python_interpreter": run_python_interpreter,
|
||||
"unit_conversion": run_unit_conversion,
|
||||
}
|
||||
|
||||
# ======= Helpers =======
|
||||
TC_RE = re.compile(r"<tool_call>\s*(\{.*?\})\s*</tool_call>", re.DOTALL)
|
||||
|
||||
def parse_tool_call(text: str) -> Optional[Dict[str, Any]]:
|
||||
m = TC_RE.search(text)
|
||||
if not m:
|
||||
return None
|
||||
try:
|
||||
obj = json.loads(m.group(1))
|
||||
name = obj.get("name")
|
||||
args = obj.get("arguments", {})
|
||||
if isinstance(name, str) and isinstance(args, dict):
|
||||
return {"name": name, "arguments": args}
|
||||
except Exception:
|
||||
return None
|
||||
return None
|
||||
|
||||
def apply_chat(tokenizer, messages: List[Dict[str, str]]):
|
||||
if hasattr(tokenizer, "apply_chat_template"):
|
||||
txt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||||
return tokenizer(txt, return_tensors="pt")
|
||||
txt = ""
|
||||
for m in messages:
|
||||
txt += f"<|im_start|>{m['role']}\n{m['content']}<|im_end|>\n"
|
||||
return tokenizer(txt, return_tensors="pt")
|
||||
|
||||
def parse_verdict(text: str) -> str:
|
||||
import re
|
||||
m = re.search(r"\\boxed\{\s*\[?(Correct|Incorrect)\]?\s*\}", text, flags=re.IGNORECASE)
|
||||
if not m:
|
||||
return "INCORRECT"
|
||||
return "CORRECT" if m.group(1).lower() == "correct" else "INCORRECT"
|
||||
|
||||
|
||||
def load_model(model_path: str):
|
||||
if torch.cuda.is_available():
|
||||
dtype = torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16
|
||||
device_map = "auto"
|
||||
else:
|
||||
dtype = torch.float32
|
||||
device_map = None
|
||||
|
||||
tok = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
|
||||
if tok.pad_token is None:
|
||||
tok.pad_token = tok.eos_token
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
model_path,
|
||||
torch_dtype=dtype,
|
||||
low_cpu_mem_usage=True,
|
||||
trust_remote_code=True,
|
||||
).eval().cuda()
|
||||
|
||||
return model, tok
|
||||
|
||||
|
||||
@torch.inference_mode()
|
||||
def generate_reply(model, tokenizer, messages, max_new_tokens=128) -> str:
|
||||
inputs = apply_chat(tokenizer, messages)
|
||||
|
||||
if torch.cuda.is_available():
|
||||
inputs = {k: v.to("cuda") for k, v in inputs.items()}
|
||||
|
||||
cfg = GenerationConfig(
|
||||
temperature=0.6,
|
||||
top_p=0.85,
|
||||
max_new_tokens=max_new_tokens,
|
||||
eos_token_id=tokenizer.eos_token_id,
|
||||
pad_token_id=tokenizer.eos_token_id,
|
||||
)
|
||||
|
||||
out = model.generate(
|
||||
**inputs,
|
||||
do_sample=True,
|
||||
temperature=0.6,
|
||||
top_p=0.85,
|
||||
max_new_tokens=max_new_tokens,
|
||||
eos_token_id=tokenizer.eos_token_id,
|
||||
pad_token_id=tokenizer.eos_token_id,
|
||||
)
|
||||
gen_ids = out[0][inputs["input_ids"].shape[1]:]
|
||||
return tokenizer.decode(gen_ids, skip_special_tokens=True)
|
||||
|
||||
|
||||
def run_case(model, tok, question: str, pred: str, ref: str, allow_tool_rounds=5) -> Tuple[str, List[Dict[str,str]], str]:
|
||||
msgs = [
|
||||
{"role": "system", "content": YOUR_SYSTEM_PROMPT},
|
||||
{"role": "user", "content": USER_TMPL.format(question=question, pred=pred, reference=ref)},
|
||||
]
|
||||
# First response (may include <tool_call>)
|
||||
assistant = generate_reply(model, tok, msgs, max_new_tokens=2048)
|
||||
msgs.append({"role": "assistant", "content": assistant})
|
||||
print(assistant)
|
||||
for _ in range(allow_tool_rounds):
|
||||
call = parse_tool_call(assistant)
|
||||
if not call:
|
||||
break
|
||||
|
||||
name = call["name"]
|
||||
args = call["arguments"]
|
||||
impl = TOOL_IMPLS.get(name)
|
||||
|
||||
if impl is None:
|
||||
result_payload = {
|
||||
"name": name,
|
||||
"status": "Error",
|
||||
"error": f"Unknown tool '{name}'"
|
||||
}
|
||||
else:
|
||||
try:
|
||||
raw = impl(args)
|
||||
if isinstance(raw, str):
|
||||
result_payload = {
|
||||
"name": name,
|
||||
"status": "Success",
|
||||
"run_result": {
|
||||
"stdout": raw,
|
||||
"stderr": "",
|
||||
"exit_success": True,
|
||||
"return_code": 0,
|
||||
"status": "Finished"
|
||||
}
|
||||
}
|
||||
else:
|
||||
result_payload = {"name": name, **raw}
|
||||
result_payload.setdefault("status", "Success")
|
||||
except Exception as e:
|
||||
result_payload = {
|
||||
"name": name,
|
||||
"status": "Error",
|
||||
"error": repr(e)
|
||||
}
|
||||
|
||||
tool_response_msg = {
|
||||
"role": "user",
|
||||
"content": "<tool_response>\n"
|
||||
+ json.dumps(result_payload, ensure_ascii=False)
|
||||
+ "\n</tool_response>"
|
||||
}
|
||||
msgs.append(tool_response_msg)
|
||||
print(tool_response_msg)
|
||||
|
||||
assistant = generate_reply(model, tok, msgs, max_new_tokens=2048)
|
||||
print(assistant)
|
||||
msgs.append({"role": "assistant", "content": assistant})
|
||||
|
||||
verdict = parse_verdict(assistant)
|
||||
return verdict, msgs, assistant
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--model", required=True, help="Local path or HF id of your verifier")
|
||||
ap.add_argument("--tool_turns", required=False, help="Number of Turns", default="5")
|
||||
args = ap.parse_args()
|
||||
|
||||
|
||||
model, tok = load_model(args.model)
|
||||
|
||||
cases = [
|
||||
{
|
||||
"question": "A car accelerates at 0.02 km/s². Convert this acceleration to m/s².",
|
||||
"pred": "20 m/s^2",
|
||||
"ref": "20 m/s²",
|
||||
"gt": "[Correct]"
|
||||
},
|
||||
{
|
||||
"question": "Please predict the dot-bracket notation of the secondary structure directly from the RNA sequence: CAACUAAAUCCACCCUUGCGGGUGGGUGAAAUAUUGCUUCGCAAUAUGAAAUACGCUUUCAGCGUAUGAAAUCGCUG",
|
||||
"ref": ".......((((((((....))))))))...((((((.....))))))...(((((((...)))))))..........",
|
||||
"pred": "......((((((((....))))))))...((((((.....))))))...(((((((...)))))))..........",
|
||||
"gt": "[Incorrect]"
|
||||
},
|
||||
{
|
||||
"question": "The number of particles in a classical ideal gas of monoatomic molecules is $N$, the volume is $V$, and the temperature is $T$. Each atom has two internal energy levels $\\varepsilon_1 = 0$ and $\\varepsilon_2 = \\Delta$. Find the chemical potential $\\mu$?",
|
||||
"ref": "['\\\\boxed{\\\\mu = kT \\\\ln(n V_q) - kT \\\\ln z_{\\\\text{int}}}']",
|
||||
"pred": "\\mu = kT \\ln\\left( \\frac{N V_q}{V z_{\\text{int}}}",
|
||||
"gt": "[Correct]",
|
||||
},
|
||||
{
|
||||
"question": "A particle $m$ undergoes projectile motion with an initial velocity of $v_{0}$ and a horizontal angle of elevation $\\alpha$. Solve using the Hamilton-Jacobi equation.",
|
||||
"ref": "['y=x \\\\tan \\\\alpha-\\\\frac{g x^{2}}{2 v_{0}^{2} \\\\cos ^{2} \\\\alpha}']",
|
||||
"pred": "y = x \\tan \\alpha - \\frac{g x^2 \\tan^2 \\alpha}{2 v_0^2} - \\frac{g x^2}{2 v_0^2}",
|
||||
"gt": "[Correct]"
|
||||
},
|
||||
{
|
||||
"question": """以下是中国数学竞赛中的解答题，答案类型为表达式。请根据题目的要求和所提供的信息计算得出答案。解答过程和结果中使用的变量和公式请使用LaTeX格式表示。请在最后以\"所以最终答案是\\\\boxed{答案}。\"显式给出结果。\\n$已知a_n为等差数列，前n项和为S_n(n\\\\in N^)，b_n是首项为2的等比数列，且公比大于0，b_2+b_3=12，b_3=a_4-2a_1，S_{11}=11b_4.$\\n$求数列{a_{2n}b_{n}}的前n项和(n\\\\in N^{}).$\\n\\n请通过逐步推理来解答问题，并把最终答案放置于\\\\boxed{}中。""",
|
||||
"ref": "$(3n-4)2^{n+2}+16$",
|
||||
"pred": "16 + (12n - 16)2^n",
|
||||
"gt": "[Correct]"
|
||||
},
|
||||
|
||||
]
|
||||
|
||||
|
||||
for i, c in enumerate(cases, 1):
|
||||
print(f"==================Case {i}==================")
|
||||
print(c)
|
||||
verdict, msgs, final_output = run_case(model, tok, c["question"], c["pred"], c["ref"])
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
14
generation_config.json
Normal file
14
generation_config.json
Normal file
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"bos_token_id": 151643,
|
||||
"do_sample": true,
|
||||
"eos_token_id": [
|
||||
2,
|
||||
151645,
|
||||
151643
|
||||
],
|
||||
"pad_token_id": 151643,
|
||||
"temperature": 0.7,
|
||||
"top_k": 20,
|
||||
"top_p": 0.8,
|
||||
"transformers_version": "4.56.2"
|
||||
}
|
||||
151388
merges.txt
Normal file
151388
merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
model-00001-of-00002.safetensors
Normal file
3
model-00001-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b8169668c60c7a6203602599cfe671f370a9a55bb0d51559456cc51f6c7abad3
|
||||
size 4963923168
|
||||
3
model-00002-of-00002.safetensors
Normal file
3
model-00002-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:866ec2af913b3ec6dd8e37ca3361e06d678b2d7db54ce50824c138ea314b31ba
|
||||
size 3858971264
|
||||
407
model.safetensors.index.json
Normal file
407
model.safetensors.index.json
Normal file
@@ -0,0 +1,407 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_parameters": 4411424256,
|
||||
"total_size": 8822848512
|
||||
},
|
||||
"weight_map": {
|
||||
"lm_head.weight": "model-00001-of-00002.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.norm.weight": "model-00002-of-00002.safetensors"
|
||||
}
|
||||
}
|
||||
BIN
pics/RLVR_Experiments.png
Normal file
BIN
pics/RLVR_Experiments.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 79 KiB |
31
special_tokens_map.json
Normal file
31
special_tokens_map.json
Normal file
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"eos_token": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
Binary file not shown.
243
tokenizer_config.json
Normal file
243
tokenizer_config.json
Normal file
@@ -0,0 +1,243 @@
|
||||
{
|
||||
"add_bos_token": false,
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"151643": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151644": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151645": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151646": {
|
||||
"content": "<|object_ref_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151647": {
|
||||
"content": "<|object_ref_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151648": {
|
||||
"content": "<|box_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151649": {
|
||||
"content": "<|box_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151650": {
|
||||
"content": "<|quad_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151651": {
|
||||
"content": "<|quad_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151652": {
|
||||
"content": "<|vision_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151653": {
|
||||
"content": "<|vision_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151654": {
|
||||
"content": "<|vision_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151655": {
|
||||
"content": "<|image_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151656": {
|
||||
"content": "<|video_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151657": {
|
||||
"content": "<tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151658": {
|
||||
"content": "</tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151659": {
|
||||
"content": "<|fim_prefix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151660": {
|
||||
"content": "<|fim_middle|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151661": {
|
||||
"content": "<|fim_suffix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151662": {
|
||||
"content": "<|fim_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151663": {
|
||||
"content": "<|repo_name|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151664": {
|
||||
"content": "<|file_sep|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151665": {
|
||||
"content": "<tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151666": {
|
||||
"content": "</tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151667": {
|
||||
"content": "<think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151668": {
|
||||
"content": "</think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"bos_token": null,
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|im_end|>",
|
||||
"errors": "replace",
|
||||
"extra_special_tokens": {},
|
||||
"max_length": 16000,
|
||||
"model_max_length": 262144,
|
||||
"pad_token": "<|endoftext|>",
|
||||
"split_special_tokens": false,
|
||||
"stride": 0,
|
||||
"tokenizer_class": "Qwen2Tokenizer",
|
||||
"truncation_side": "right",
|
||||
"truncation_strategy": "longest_first",
|
||||
"unk_token": null
|
||||
}
|
||||
1
vocab.json
Normal file
1
vocab.json
Normal file
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user