初始化项目，由ModelHub XC社区提供模型
Model: KOREAson/KO-REAson-AX3_1-7B-0831 Source: Original Platform
This commit is contained in:
ModelHub XC
57
.gitattributes
vendored
Normal file
57
.gitattributes
vendored
Normal file
@@ -0,0 +1,57 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zstandard filter=lfs diff=lfs merge=lfs -text
|
||||
*.tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
*.db* filter=lfs diff=lfs merge=lfs -text
|
||||
*.ark* filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.gguf* filter=lfs diff=lfs merge=lfs -text
|
||||
*.ggml filter=lfs diff=lfs merge=lfs -text
|
||||
*.llamafile* filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
model-00005-of-00006.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
merges.txt filter=lfs diff=lfs merge=lfs -text
|
||||
model-00002-of-00006.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
vocab.json filter=lfs diff=lfs merge=lfs -text
|
||||
model-00003-of-00006.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
model-00004-of-00006.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
model-00001-of-00006.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
model-00006-of-00006.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
242
README.md
Normal file
242
README.md
Normal file
@@ -0,0 +1,242 @@
|
||||
---
|
||||
library_name: transformers
|
||||
tags: []
|
||||
---
|
||||
|
||||
# KO-REAson
|
||||
|
||||
**KO-REAson** is a series of Korean-centric reasoning language models developed in collaboration with [OneLineAI](https://onelineai.com/), [KISTI-KONI](https://huggingface.co/KISTI-KONI), [HAE-RAE](https://huggingface.co/HAERAE-HUB) and ORACLE.
|
||||
|
||||
We use the **Language-Mixed Chain-of-Thought (CoT)** approach, which allows the model to alternate between English and Korean during the “Think” stage of reasoning, preserving key Korean terms while leveraging English for logical scaffolding.
|
||||
|
||||
Top-performing models of our series [KO-REAson-AX3_1-7B-0831 (KONI-7B-R-20250831)](https://huggingface.co/KISTI-KONI/KONI-7B-R-20250831) and [KO-REAson-7B-Q2_5-0831](https://huggingface.co/KoReason/KO-REASon-7B-Q2_5-0831) show performance comparable to models trained on closed-source datasets such as Exaone-Deep-7.8B.
|
||||
|
||||
|
||||
<p align="left">
|
||||
<img src="https://cdn-uploads.huggingface.co/production/uploads/60d3e619b8448e1785bbda2a/uqrKdxbQEqAFknYBmuH7Y.png"
|
||||
alt="Model Comparison" width="750"/>
|
||||
<br>
|
||||
<em style="display:inline-block; max-width:750px; text-align:cener; white-space:normal; word-wrap:break-word; line-height:1.5;">
|
||||
<b>Left:</b> Average performance (Held-out-Ko) of open models trained on closed or open data;
|
||||
our models are highlighted in green.
|
||||
</em>
|
||||
</p>
|
||||
|
||||
## Model Details
|
||||
|
||||
The **KO-REAson-0831** family comes in six variants based on the base model used.
|
||||
|
||||
| Model (link) | Base | Notes |
|
||||
| -------------------------------------------------------------------------------------------- | -------------------- | --------------------------- |
|
||||
| [KO-REAson-L3_1-8B-0831](https://huggingface.co/KoReason/KO-REASon-L3_1-8B-0831) | [Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) | `L3_1` → Llama-3.1-8B |
|
||||
| [KO-REAson-KL3_1-8B-0831](https://huggingface.co/KOREAson/KO-REAson-KL3_1-8B-0831) | [Koni-Llama-3.1-8B](https://huggingface.co/KISTI-KONI/KONI-Llama3.1-8B-Instruct-20241024) | `KL3_1` → Koni-Llama-3.1-8B; also called [KONI-Llama3.1-8B-R-20250831](https://huggingface.co/KISTI-KONI/KONI-Llama3.1-8B-R-20250831) |
|
||||
| [KO-REAson-G3-4B-0831](https://huggingface.co/KoReason/KO-REASon-G3-4B-0831) | [Gemma-3 4B](https://huggingface.co/google/gemma-3-4b-it) | `G3` → Gemma-3-4B |
|
||||
| [KO-REAson-AX3_1-7B-0831](https://huggingface.co/KOREAson/KO-REAson-7B-AX3_1-0831) | [A.X.-3.1-Light (≈7B)](https://huggingface.co/skt/A.X-3.1-Light) | `AX3_1` → A.X.-3.1-Light; also called [KONI-7B-R-20250831](https://huggingface.co/KISTI-KONI/KONI-7B-R-20250831) |
|
||||
| [KO-REAson-K2505_8B-0831](https://huggingface.co/KoReason/KO-REASon-K2505_8B-0831) | [Kanana-2505 (8B)](https://huggingface.co/kakaocorp/kanana-1.5-8b-instruct-2505) | `K2505` → Kanana-2505 |
|
||||
| [KO-REAson-7B-Q2_5-0831](https://huggingface.co/KoReason/KO-REASon-7B-Q2_5-0831) | [Qwen-2.5 (7B)](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) | `Q2_5` → Qwen-2.5 |
|
||||
|
||||
|
||||
|
||||
# Performance
|
||||
|
||||
**Evaluation Datasets**
|
||||
|
||||
The model's performance was evaluated across a total of 11 benchmarks, and the evaluation suite is divided into two parts: (You can check these benchmarks in [HAERAE-HUB/KoSimpleEval](https://huggingface.co/datasets/HAERAE-HUB/KoSimpleEval))
|
||||
|
||||
- **Held-in**: This set of benchmarks is used for routine monitoring of the model's performance during the training and ablation study phases.
|
||||
- **Held-out**: This set is used only once to evaluate the final model after all training and ablations are complete.
|
||||
|
||||
This separation is designed to prevent inadvertent overfitting to the benchmarks during the iterative training process and to provide a more accurate measure of the model's generalization capabilities.
|
||||
|
||||
|**Category**|**Held-in**|**Held-out**|
|
||||
|---|---|---|
|
||||
|**General Knowledge**|KMMLU-Redux|KMMLU-HARD, KMMLU-Pro|
|
||||
|**Reasoning**|MCLM|KSM, GPQA, AIME2024, AIME2025|
|
||||
|**Korean-specific**|HAE-RAE Bench|CLIcK, KoBALT-700|
|
||||
|
||||
|
||||
**Comparison with models trained on public datasets**
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Models</th>
|
||||
<th># Instances</th>
|
||||
<th>Methodology</th>
|
||||
<th>Held-Out (Ko)</th>
|
||||
<th>Held-Out (En)</th>
|
||||
<th>Total</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<th>KO-REASon-AX3_1-7B-0831(KONI-7B-R-20250831; Ours)</th>
|
||||
<td>260k</td>
|
||||
<td>SFT</td>
|
||||
<td><b>44.6</b></td>
|
||||
<td>41.2</td>
|
||||
<td><u>43.3</u></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>KO-REASon-7B-Q2_5-0831(Ours)</th>
|
||||
<td>260k</td>
|
||||
<td>SFT</td>
|
||||
<td><b>45.10</b></td>
|
||||
<td>38.75</td>
|
||||
<td><u>49.95</u></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>KO-REAson-KL3_1-8B-0831(KONI-Llama3.1-8B-R-20250831)</th>
|
||||
<td>260k</td>
|
||||
<td>SFT</td>
|
||||
<td>40.13</td>
|
||||
<td>30.57</td>
|
||||
<td>43.66</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td colspan="6" style="text-align:center; font-weight:bold;">Open Recipe (En)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>OpenThinker3-7B</th>
|
||||
<td>1.2M</td>
|
||||
<td>SFT</td>
|
||||
<td>33.6</td>
|
||||
<td><b>55.5</b></td>
|
||||
<td>41.8</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>s1.1-7B</th>
|
||||
<td>1k</td>
|
||||
<td>SFT</td>
|
||||
<td>35.6</td>
|
||||
<td>23.4</td>
|
||||
<td>31.1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Llama-3.1-Nemotron-Nano-8B-v1</th>
|
||||
<td>>3M</td>
|
||||
<td>SFT & RL</td>
|
||||
<td>27.0</td>
|
||||
<td>44.1</td>
|
||||
<td>33.4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td colspan="6" style="text-align:center; font-weight:bold;">Open Recipe (Ko)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Ko-R1-14B</th>
|
||||
<td>45k</td>
|
||||
<td>SFT</td>
|
||||
<td><u>43.7</u></td>
|
||||
<td><u>46.3</u></td>
|
||||
<td><b>44.7</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Ko-R1-7B</th>
|
||||
<td>45k</td>
|
||||
<td>SFT</td>
|
||||
<td>27.3</td>
|
||||
<td>36.1</td>
|
||||
<td>30.6</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>LLaMa-3.1-Ko-Reasoning-8B</th>
|
||||
<td>63k</td>
|
||||
<td>SFT</td>
|
||||
<td>17.7</td>
|
||||
<td>7.7</td>
|
||||
<td>14.0</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
**Held-out benchmark performance**
|
||||
|
||||
<table border="1" cellspacing="0" cellpadding="6">
|
||||
<thead>
|
||||
<tr>
|
||||
<th rowspan="2">Model</th>
|
||||
<th rowspan="2">Model Size</th>
|
||||
<th colspan="2">General</th>
|
||||
<th colspan="4">Reasoning</th>
|
||||
<th colspan="2">Korean-Specific</th>
|
||||
<th rowspan="2">Average<br>(Held-out)</th>
|
||||
<th rowspan="2">Average<br>(Held-out-Ko)</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>KMMLU-HARD</th>
|
||||
<th>KMMLU-Pro</th>
|
||||
<th>KSM</th>
|
||||
<th>AIME 2024</th>
|
||||
<th>AIME 2025</th>
|
||||
<th>GPQA</th>
|
||||
<th>CLIcK</th>
|
||||
<th>KoBALT-700</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><b>Llama-3.1-Nemotron-Nano-8B</b></td>
|
||||
<td>8.03</td><td>21.47</td><td>22.89</td><td>47.06</td><td>56.67</td><td>43.33</td><td>32.32</td><td>34.54</td><td>9.29</td><td>33.45</td><td>27.05</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>Llama-3.1-Korean-Reasoning-8B-Instruct</b></td>
|
||||
<td>8.03</td><td>14.91</td><td>21.72</td><td>6.09</td><td>0.00</td><td>0.00</td><td>23.23</td><td>39.65</td><td>6.14</td><td>13.97</td><td>17.70</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>EXAONE-Deep-7.8B</b></td>
|
||||
<td>7.82</td><td><u>40.96</u></td><td>37.35</td><td><b>70.80</b></td><td><b>70.00</b></td><td><b>63.33</b></td><td><b>64.65</b></td><td>54.24</td><td>18.86</td><td><b>52.52</b></td><td>44.44</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>DeepSeek-R1-Distill-Qwen-7B</b></td>
|
||||
<td>7.62</td><td>0.00</td><td>23.00</td><td>56.09</td><td>60.00</td><td>40.00</td><td>43.43</td><td>0.00</td><td>8.29</td><td>28.85</td><td>17.48</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>DeepSeek-R1-Distill-Llama-8B</b></td>
|
||||
<td>8.03</td><td>23.22</td><td>26.26</td><td>29.97</td><td>33.33</td><td>20.00</td><td><U>46.46</u></td><td>39.05</td><td>13.29</td><td>28.95</td><td>26.36</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>s1.1-7B</b></td>
|
||||
<td>7.62</td><td>31.16</td><td><u>37.70</u></td><td>30.60</td><td>16.67</td><td>23.33</td><td>30.30</td><td><u>56.84</u></td><td><u>21.86</u></td><td>31.06</td><td>35.63</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>OpenThinker3-7B</b></td>
|
||||
<td>7.62</td><td>30.31</td><td>26.26</td><td><u>63.59</u></td><td><u>66.67</u></td><td><u>53.33</u></td><td><u>46.46</u></td><td>47.69</td><td>10.14</td><td>35.63</td><td>30.60</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>Ko-R1-7B</b></td>
|
||||
<td>7.61</td><td>28.46</td><td>19.31</td><td>51.61</td><td>46.67</td><td>33.33</td><td>28.28</td><td>32.48</td><td>4.71</td><td>30.61</td><td>27.31</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>KO-REAson-KL3_1-8B-0831(KONI-Llama3.1-8B-R-20250831)</b></td>
|
||||
<td>8.03</td><td>44.64</td><td>40.08</td><td>37.96</td><td>23.33</td><td>30.00</td><td>38.38</td><td>56.39</td><td>21.57</td><td>30.57</td><td>40.13</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>KO-REASon-AX3_1-7B-0831 (KONI-7B-R-20250831)</b></td>
|
||||
<td>7.26</td><td>45.57</td><td>38.13</td><td>52.80</td><td>53.33</td><td>33.33</td><td>36.87</td><td><b>62.86</b></td><td>23.43</td><td><u>43.29</u></td><td><u>44.56</u></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>KO-REASon-7B-Q2_5-0831</b></td>
|
||||
<td>7.26</td><td><b>46.81</b></td><td><b>44.93</b></td><td>48.11</td><td>43.33</td><td>30.00</td><td>42.93</td><td>60.65</td><td><b>25.00</b></td><td>42.72</td><td><b>45.10</b></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## Citation
|
||||
```
|
||||
The paper will be released soon!
|
||||
```
|
||||
|
||||
|
||||
## Contact
|
||||
|
||||
For any questions contact us via the following email :)
|
||||
|
||||
```
|
||||
spthsrbwls123@yonsei.ac.kr
|
||||
```
|
||||
|
||||
|
||||
## Acknowlegments
|
||||
This research was supported by the Korea Institute of Science and Technology Information (KISTI) (No.(KISTI) K25L1M1C1), aimed at developing KONI (KISTI Open Neural Intelligence), a large language model specialized in science and technology.
|
||||
72
chat_template.jinja
Normal file
72
chat_template.jinja
Normal file
@@ -0,0 +1,72 @@
|
||||
{%- if tools is iterable and tools | length > 0 %}
|
||||
{{- '<|im_start|><|system|>'}}
|
||||
{{- '당신은 도구 호출 기능을 갖춘 유용한 도우미입니다. 사용자의 요청을 처리하기 위해서 필요한 도구가 주어진 목록에 있는 경우 도구 호출로 응답하세요.
|
||||
필요한 도구가 목록에 없는 경우에는 도구 호출 없이 사용자가 요구한 정보를 제공하세요.
|
||||
필요한 도구가 목록에 있지만 해당 도구를 호출하는데 필요한 argument 정보가 부족한 경우 해당 정보를 사용자에게 요청하세요.
|
||||
사용자의 요청을 처리하기 위해 여러번 도구를 호출할 수 있어야 합니다.
|
||||
도구 호출 이후 도구 실행 결과를 입력으로 받으면 해당 결과를 활용하여 답변을 생성하세요.
|
||||
|
||||
다음은 접근할 수 있는 도구들의 목록 입니다:
|
||||
<tools>
|
||||
'}}
|
||||
{%- for t in tools %}
|
||||
{{- t | tojson }}
|
||||
{{- '
|
||||
' }}
|
||||
{%- endfor %}
|
||||
{{- '</tools>' }}
|
||||
{{- '
|
||||
|
||||
도구를 호출하려면 아래의 JSON으로 응답하세요.
|
||||
도구 호출 형식: <tool_call>{"name": 도구 이름, "arguments": dictionary 형태의 도구 인자값}</tool_call>' }}
|
||||
{{- '<|im_end|>' }}
|
||||
{%- endif %}
|
||||
|
||||
{%- for message in messages %}
|
||||
{%- if message.role == 'system' %}
|
||||
{{- '<|im_start|><|system|>' + message.content + '<|im_end|>'}}
|
||||
{%- elif message.role == 'user' %}
|
||||
{{- '<|im_start|><|user|>' + message.content + '<|im_end|>'}}
|
||||
{%- elif message.role == 'assistant' %}
|
||||
{{- '<|im_start|><|assistant|>'}}
|
||||
{%- set content = '' %}
|
||||
{%- if message.content is defined %}
|
||||
{%- set content = message.content %}
|
||||
{%- endif %}
|
||||
|
||||
{%- if add_generation_prompt and not (message.reasoning_content is defined and message.reasoning_content is not none) %}
|
||||
{%- if '</think>' in message.content %}
|
||||
{%- set content = message.content.split('</think>'.strip())[-1].lstrip('\n') %}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
|
||||
{{- content}}
|
||||
{%- if message.tool_calls is defined %}
|
||||
{%- for tool_call in message.tool_calls %}
|
||||
{%- if tool_call.function is defined %}
|
||||
{%- set tool_call = tool_call.function %}
|
||||
{%- endif %}
|
||||
{{- '<tool_call>' }}
|
||||
{{- '{' }}
|
||||
{{- '"name": "' }}
|
||||
{{- tool_call.name }}
|
||||
{{- '"' }}
|
||||
{%- if tool_call.arguments is defined %}
|
||||
{{- ', ' }}
|
||||
{{- '"arguments": ' }}
|
||||
{{- tool_call.arguments|tojson }}
|
||||
{%- endif %}
|
||||
{{- '}' }}
|
||||
{{- '</tool_call>' }}
|
||||
{%- endfor %}
|
||||
{%- endif %}
|
||||
{{- '<|im_end|>'}}
|
||||
|
||||
{%- elif message.role == 'tool' %}
|
||||
{{- '<|im_start|><|extra_id_13|><tool_output>' + message.content + '</tool_output><|im_end|>'}}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
|
||||
{%- if add_generation_prompt %}
|
||||
{{- '<|im_start|><|assistant|>' }}
|
||||
{%- endif %}
|
||||
29
config.json
Normal file
29
config.json
Normal file
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.1,
|
||||
"bos_token_id": 0,
|
||||
"eos_token_id": 0,
|
||||
"head_dim": 128,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 4096,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 10880,
|
||||
"max_position_embeddings": 32768,
|
||||
"mlp_bias": false,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 32,
|
||||
"num_key_value_heads": 32,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 500000,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "float32",
|
||||
"transformers_version": "4.55.4",
|
||||
"use_cache": false,
|
||||
"vocab_size": 102400
|
||||
}
|
||||
1
configuration.json
Normal file
1
configuration.json
Normal file
@@ -0,0 +1 @@
|
||||
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}
|
||||
7
generation_config.json
Normal file
7
generation_config.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"bos_token_id": 0,
|
||||
"eos_token_id": 27,
|
||||
"max_new_tokens": 32768,
|
||||
"pad_token_id": 1,
|
||||
"transformers_version": "4.55.4"
|
||||
}
|
||||
3
merges.txt
Normal file
3
merges.txt
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d33bd322a424d0144e5bd123dbeef33f2538ac57adabf639984599ee9b9ca9b3
|
||||
size 1538148
|
||||
3
model-00001-of-00006.safetensors
Normal file
3
model-00001-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:983e2d0b15b189c99e93f4ed296ba57408cefac3cfa81ad4157c7e01872efc1e
|
||||
size 4957802784
|
||||
3
model-00002-of-00006.safetensors
Normal file
3
model-00002-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:30b2117cada0e6a62c9d930558614a13113685b0b673bf65baf7f4a9c6ecc1de
|
||||
size 4953676072
|
||||
3
model-00003-of-00006.safetensors
Normal file
3
model-00003-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:ec378fdedc1b03b36a9d47d78511a1e1a43fdf7018ab9aa8a2bb707be02c9bfc
|
||||
size 4886567144
|
||||
3
model-00004-of-00006.safetensors
Normal file
3
model-00004-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:771b22ae7c556a9b452eda7498d45efbe78eca8f7cb3340792ab0176a223b624
|
||||
size 4997716200
|
||||
3
model-00005-of-00006.safetensors
Normal file
3
model-00005-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e89d46643533ec0a14e3d2f122b29b67dca68425e4a825f26fa1aeffb20ab0a6
|
||||
size 4997716192
|
||||
3
model-00006-of-00006.safetensors
Normal file
3
model-00006-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:562a20eb61c449b8208ebdef4c5637f6e267143e800e794bf69d5ab9314c2ccf
|
||||
size 4265758312
|
||||
299
model.safetensors.index.json
Normal file
299
model.safetensors.index.json
Normal file
@@ -0,0 +1,299 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_parameters": 7264800768,
|
||||
"total_size": 29059203072
|
||||
},
|
||||
"weight_map": {
|
||||
"lm_head.weight": "model-00006-of-00006.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00003-of-00006.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00004-of-00006.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.28.mlp.down_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.28.mlp.gate_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.28.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.28.post_attention_layernorm.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.28.self_attn.o_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00005-of-00006.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.29.mlp.down_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.29.mlp.gate_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.29.mlp.up_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.29.post_attention_layernorm.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.29.self_attn.o_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.30.mlp.down_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.30.mlp.gate_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.30.mlp.up_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.30.post_attention_layernorm.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.30.self_attn.o_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.31.mlp.down_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.31.mlp.gate_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.31.mlp.up_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.31.post_attention_layernorm.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.31.self_attn.o_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00006-of-00006.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00006.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00002-of-00006.safetensors",
|
||||
"model.norm.weight": "model-00006-of-00006.safetensors"
|
||||
}
|
||||
}
|
||||
88
special_tokens_map.json
Normal file
88
special_tokens_map.json
Normal file
@@ -0,0 +1,88 @@
|
||||
{
|
||||
"additional_special_tokens": [
|
||||
"<|endoftext|>",
|
||||
"<|pad|>",
|
||||
"<|unk|>",
|
||||
"<|sep|>",
|
||||
"<|mask|>",
|
||||
"<|cls|>",
|
||||
"<|image|>",
|
||||
"<|audio|>",
|
||||
"<|user|>",
|
||||
"<|system|>",
|
||||
"<|assistant|>",
|
||||
"<|extra_id_0|>",
|
||||
"<|extra_id_1|>",
|
||||
"<|extra_id_2|>",
|
||||
"<|extra_id_3|>",
|
||||
"<|extra_id_4|>",
|
||||
"<|extra_id_5|>",
|
||||
"<|extra_id_6|>",
|
||||
"<|extra_id_7|>",
|
||||
"<|extra_id_8|>",
|
||||
"<|extra_id_9|>",
|
||||
"<|extra_id_10|>",
|
||||
"<|extra_id_13|>",
|
||||
"<|im_start|>",
|
||||
"<|im_sep|>",
|
||||
"<|im_end|>",
|
||||
"<|resident_reg|>",
|
||||
"<|foreigner_reg|>",
|
||||
"<|business_reg|>",
|
||||
"<|credit_card|>",
|
||||
"<|passport|>",
|
||||
"<|driver_license|>",
|
||||
"<|telephone|>",
|
||||
"<|health_insurance|>",
|
||||
"<|bank_account|>"
|
||||
],
|
||||
"bos_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"cls_token": {
|
||||
"content": "<|cls|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"mask_token": {
|
||||
"content": "<|mask|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<|pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"sep_token": {
|
||||
"content": "<|sep|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"unk_token": {
|
||||
"content": "<|unk|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e3973eceeed921e72d54fbbf597e1bea0a219112a0809ad8eaf62a1754f8ea44
|
||||
size 8495125
|
||||
386
tokenizer_config.json
Normal file
386
tokenizer_config.json
Normal file
@@ -0,0 +1,386 @@
|
||||
{
|
||||
"add_bos_token": false,
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"0": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"1": {
|
||||
"content": "<|pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"2": {
|
||||
"content": "<|unk|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"3": {
|
||||
"content": "<|sep|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"4": {
|
||||
"content": "<|mask|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"5": {
|
||||
"content": "<|cls|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"6": {
|
||||
"content": "<|image|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"7": {
|
||||
"content": "<|audio|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"8": {
|
||||
"content": "<|user|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"9": {
|
||||
"content": "<|system|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"10": {
|
||||
"content": "<|assistant|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"11": {
|
||||
"content": "<|extra_id_0|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"12": {
|
||||
"content": "<|extra_id_1|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"13": {
|
||||
"content": "<|extra_id_2|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"14": {
|
||||
"content": "<|extra_id_3|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"15": {
|
||||
"content": "<|extra_id_4|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"16": {
|
||||
"content": "<|extra_id_5|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"17": {
|
||||
"content": "<|extra_id_6|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"18": {
|
||||
"content": "<|extra_id_7|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"19": {
|
||||
"content": "<|extra_id_8|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"20": {
|
||||
"content": "<|extra_id_9|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"21": {
|
||||
"content": "<|extra_id_10|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"22": {
|
||||
"content": "</think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"23": {
|
||||
"content": "<think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"24": {
|
||||
"content": "<|extra_id_13|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"25": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"26": {
|
||||
"content": "<|im_sep|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"27": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"28": {
|
||||
"content": "<|resident_reg|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"29": {
|
||||
"content": "<|foreigner_reg|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"30": {
|
||||
"content": "<|business_reg|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"31": {
|
||||
"content": "<|credit_card|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"32": {
|
||||
"content": "<|passport|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"33": {
|
||||
"content": "<|driver_license|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"34": {
|
||||
"content": "<|telephone|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"35": {
|
||||
"content": "<|health_insurance|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"36": {
|
||||
"content": "<|bank_account|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"37": {
|
||||
"content": "</tool_output>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"38": {
|
||||
"content": "<tool_output>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"39": {
|
||||
"content": "</tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"40": {
|
||||
"content": "<tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|endoftext|>",
|
||||
"<|pad|>",
|
||||
"<|unk|>",
|
||||
"<|sep|>",
|
||||
"<|mask|>",
|
||||
"<|cls|>",
|
||||
"<|image|>",
|
||||
"<|audio|>",
|
||||
"<|user|>",
|
||||
"<|system|>",
|
||||
"<|assistant|>",
|
||||
"<|extra_id_0|>",
|
||||
"<|extra_id_1|>",
|
||||
"<|extra_id_2|>",
|
||||
"<|extra_id_3|>",
|
||||
"<|extra_id_4|>",
|
||||
"<|extra_id_5|>",
|
||||
"<|extra_id_6|>",
|
||||
"<|extra_id_7|>",
|
||||
"<|extra_id_8|>",
|
||||
"<|extra_id_9|>",
|
||||
"<|extra_id_10|>",
|
||||
"<|extra_id_13|>",
|
||||
"<|im_start|>",
|
||||
"<|im_sep|>",
|
||||
"<|im_end|>",
|
||||
"<|resident_reg|>",
|
||||
"<|foreigner_reg|>",
|
||||
"<|business_reg|>",
|
||||
"<|credit_card|>",
|
||||
"<|passport|>",
|
||||
"<|driver_license|>",
|
||||
"<|telephone|>",
|
||||
"<|health_insurance|>",
|
||||
"<|bank_account|>"
|
||||
],
|
||||
"bos_token": "<|endoftext|>",
|
||||
"clean_up_tokenization_spaces": true,
|
||||
"cls_token": "<|cls|>",
|
||||
"eod_token": "<|endoftext|>",
|
||||
"eos_token": "<|im_end|>",
|
||||
"errors": "replace",
|
||||
"extra_special_tokens": {},
|
||||
"mask_token": "<|mask|>",
|
||||
"max_length": 7680,
|
||||
"model_max_length": 32768,
|
||||
"pad_token": "<|pad|>",
|
||||
"sep_token": "<|sep|>",
|
||||
"tokenizer_class": "GPT2Tokenizer",
|
||||
"unk_token": "<|unk|>",
|
||||
"vocab_size": 102400
|
||||
}
|
||||
3
vocab.json
Normal file
3
vocab.json
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:fe985752b212eccc3e58ac96ae9ffe85fc842d6ff6c182ee3ff7d7bec9bb170c
|
||||
size 2248633
|
||||
Reference in New Issue
Block a user