初始化项目，由ModelHub XC社区提供模型
Model: ibm-granite/granite-4.1-3b-base Source: Original Platform
This commit is contained in:
ModelHub XC
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
475
README.md
Normal file
475
README.md
Normal file
@@ -0,0 +1,475 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
library_name: transformers
|
||||
tags:
|
||||
- language
|
||||
- granite-4.1
|
||||
---
|
||||
|
||||
[](https://mot.isitopen.ai/model/1163)
|
||||
|
||||
# Granite-4.1-3B-Base
|
||||
|
||||
**Model Summary:**
|
||||
Granite‑4.1‑3B‑Base is a decoder‑only language model with long‑context capabilities, designed to support a broad range of general text‑to‑text generation tasks, as well as fill‑in‑the‑Middle (FIM) code completion. This model shares the same underlying architecture and weights as Granite 4.0 3B Micro, which is trained from scratch on approximately 15 trillion tokens following a four-stage training strategy: 10 trillion tokens in the first stage, 2 trillion in the second, another 2 trillion in the third, and 0.5 trillion in the final stage. An additional training phase is applied to expand the model’s context window to 512K tokens.
|
||||
|
||||
<!--
|
||||
TO DO: Don't it only applies to the 3B model card?
|
||||
Grante-4.1-3B base is same base model as Granite-4.0-3B-Micro.
|
||||
-->
|
||||
- **Developers:** Granite Team, IBM
|
||||
- **HF Collection:** [Granite 4.1 Language Models HF Collection](https://huggingface.co/collections/ibm-granite/granite-41-language-models)
|
||||
- **Technical Blog:** [Granite-4.1 Blog](https://huggingface.co/blog/ibm-granite/granite-4-1)
|
||||
- **GitHub Repository:** [ibm-granite/granite-4.1-language-models](https://github.com/ibm-granite/granite-4.1-language-models)
|
||||
- **Website**: [Granite Docs](https://www.ibm.com/granite/docs/)
|
||||
- **Release Date**: April 29th, 2026
|
||||
- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
|
||||
|
||||
**Supported Languages:**
|
||||
English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese. Users may finetune Granite 4.1 models for languages beyond these languages.
|
||||
|
||||
**Intended Use:**
|
||||
Prominent use cases of LLMs in text-to-text generation include summarization, text classification, extraction, question-answering, code-completion (including FIM), and long-context generation tasks. All Granite Base models are able to handle these tasks as they were trained on a large amount of data from various domains. Moreover, they can serve as baseline to create specialized models for specific application scenarios.
|
||||
|
||||
**Generation:**
|
||||
This is a simple example of how to use Granite-4.1-3B-Base model.
|
||||
|
||||
Install the following libraries:
|
||||
|
||||
```shell
|
||||
pip install torch torchvision torchaudio
|
||||
pip install accelerate
|
||||
pip install transformers
|
||||
```
|
||||
Then, copy the code snippet below to run the example.
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
device = "cuda"
|
||||
|
||||
model_path = "ibm-granite/granite-4.1-3b-base"
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_path)
|
||||
# drop device_map if running on CPU
|
||||
model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
|
||||
model.eval()
|
||||
# change input text as desired
|
||||
input_text = "The capital of France is"
|
||||
# tokenize the text
|
||||
input_tokens = tokenizer(input_text, return_tensors="pt").to(device)
|
||||
# generate output tokens
|
||||
output = model.generate(**input_tokens, max_length=10)
|
||||
# decode output tokens into text
|
||||
output = tokenizer.batch_decode(output)
|
||||
# print output
|
||||
print(output[0])
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```shell
|
||||
The capital of France is Paris.
|
||||
```
|
||||
|
||||
**Evaluation Results:**
|
||||
|
||||
<table>
|
||||
<!-- <caption><b> All Results</b></caption> -->
|
||||
<thead>
|
||||
<tr>
|
||||
<th style="text-align:left; background-color: #001d6c; color: white;">Benchmarks</th>
|
||||
<th style="text-align:left; background-color: #001d6c; color: white;">Metric</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">3B Dense</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">8B Dense</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">30B Dense</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td colspan="5" style="text-align:center; background-color: #FFFFFF; color: #2D2D2D; font-style:italic;">
|
||||
General Tasks
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MMLU</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">5-shot</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">66.47</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">73.60</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">78.44</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MMLU-Pro</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">5-shot,CoT</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">37.16</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">44.58</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">49.51</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">BBH</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">3-shot, CoT</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">63.84</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">73.83</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">80.66</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">AGI EVAL</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">3-shot</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">54.32</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">61.68</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">69.20</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">DROP</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">5-shot</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">66.04</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">72.36</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">78.57</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">SimpleQA</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">no-judge-short-form</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">6.85</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">7.92</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">10.54</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<!-- <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D; font-weight: bold;">General Avg</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D; font-weight: bold;"></td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D; font-weight: bold;">49.11</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D; font-weight: bold;">55.64</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D; font-weight: bold;">61.15</td>
|
||||
</tr> -->
|
||||
<tr>
|
||||
<td colspan="7" style="text-align:center; background-color: #FFFFFF; color: #2D2D2D; font-style:italic;">
|
||||
Math Tasks
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">GSM8K</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">8-shot</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">72.93</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">73.54</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">83.78</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">Minerva Math</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">4-shot</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">38.00</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">43.42</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">45.66</td>
|
||||
</tr>
|
||||
<!-- <tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">GPQA-Main</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">CoT</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">27.23</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">29.46</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">24.55</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">GPQA-Extended</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">CoT</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">26.19</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">22.34</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">25.64</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">GPQA-Diamond</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">CoT</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">22.22</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">25.25</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">29.29</td>
|
||||
</tr> -->
|
||||
<!-- <tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D; ">GPQA-Weighted-Average</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D; ">CoT</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D; ">26.43</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D; ">25.50</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">25.84</td>
|
||||
</tr> -->
|
||||
<tr>
|
||||
<td colspan="5" style="text-align:center; background-color: #FFFFFF; color: #2D2D2D; font-style:italic;">
|
||||
Code Tasks
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">HumanEval</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">pass@1 [StarCoder Prompt]</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">76.19</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">79.24</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">81.52</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">HumanEval</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">pass@1</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">59.76</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">68.29</td>
|
||||
<!-- <td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">69.50</td> -->
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">67.68</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">HumanEval+</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">pass@1</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">54.27</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">62.80</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">62.20</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MBPP</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">pass@1</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">81.48</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">63.76</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">83.60</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MBPP+</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">pass@1</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">68.25</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">53.97</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">69.58</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D; ">Eval+ Avg</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D; "></td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D; ">65.94</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D; ">62.21</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D; ">70.76</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td colspan="5" style="text-align:center; background-color: #FFFFFF; color: #2D2D2D; font-style:italic;">
|
||||
Multilingual Tasks
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MMMLU</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">5-shot</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">56.59</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">64.73</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">73.36</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">INCLUDE</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">5-shot</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">51.77</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">57.60</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">67.07</td>
|
||||
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MGSM</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">8-shot</td>
|
||||
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">58.48</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">63.68</td>
|
||||
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">74.40</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
|
||||
<table>
|
||||
<caption><b>Multilingual Benchmarks and the included languages:</b></caption>
|
||||
<thead>
|
||||
<tr>
|
||||
<th style="text-align:left; background-color: #001d6c; color: white;">Benchmarks</th>
|
||||
<th style="text-align:left; background-color: #001d6c; color: white;"># Langs</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">Languages</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MMMLU</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">11</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">ar, de, en, es, fr, ja, ko, pt, zh, bn, hi</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">INCLUDE</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">14</td>
|
||||
<!-- <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">hindi, bengali, tamil, telugu, arabic, german, spanish, french, italian, japanese, korean, dutch, portuguese, chinese</td> -->
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">hi, bn, ta, te, ar, de, es, fr, it, ja, ko, nl, pt, zh</td>
|
||||
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MGSM</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">5</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">en, es, fr, ja, zh</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
**Model Architecture:**
|
||||
|
||||
Granite-4.1-3B-Base is based on a decoder-only dense transformer architecture. Core components of this architecture are: GQA, RoPE, MLP with SwiGLU, RMSNorm, and shared input/output embeddings.
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th style="text-align:left; background-color: #001d6c; color: white;">Model</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">3B Dense</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">8B Dense</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">30B Dense</th>
|
||||
</tr></thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">Embedding size</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">2560</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">4096</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">4096</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">Number of layers</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">40</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">40</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">64</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">Attention head size</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">64</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">128</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">128</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">Number of attention heads</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">40</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">32</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">32</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">Number of KV heads</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">8</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">8</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">8</td>
|
||||
</tr>
|
||||
<!--<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">Mamba2 state size</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">-</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;"></td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">Number of Mamba2 heads</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;"></td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
|
||||
</tr>-->
|
||||
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">MLP / Shared expert hidden size</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">8192</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">12800</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">32768</td>
|
||||
</tr>
|
||||
<!--<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">Num. Experts</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;"></td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">Num. active Experts</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;"></td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">Expert hidden size</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;"></td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
|
||||
</tr>-->
|
||||
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">MLP activation</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">SwiGLU</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">SwiGLU</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">SwiGLU</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">Sequence length</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">131072</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">131072</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">131072</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">Position embedding</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">RoPE</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">RoPE</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">RoPE</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;"># Parameters</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">3B</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">8B</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">30B</td>
|
||||
</tr>
|
||||
<!-- <tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;"># Active parameters</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;"></td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
|
||||
</tr>-->
|
||||
</tbody></table>
|
||||
|
||||
|
||||
**Training Data:** This model is trained on a mix of open source and proprietary data following a five-phase training strategy. We refer to phase-1 and phase-2 as pre-training and phase-3, phase-4, and phase-5 as mid-training.
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th style="text-align:left; background-color: #001d6c; color: white;">Stage</th>
|
||||
<th style="text-align:left; background-color: #001d6c; color: white;">Characteristics</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">3B Dense</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">8B Dense</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">30B Dense</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">I</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">General mixture of training data, warmup, and power scheduler for learning rate.</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">10T</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">10T</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">10T</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">II</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">General mixture of training data with higher percentages of code and math with power scheduler for learning rate.</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">2T</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">2T</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">2T</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">III</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">High quality training data, exponential decay of learning rate.</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">2T</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">2T</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">2T</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">IV</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">High quality training data, linear decay to zero for learning rate.</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">500B</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">500B</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">500B</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">V</td>
|
||||
<td style="text-align:left; background-color: #FFFFFF; color: black;">Long Context Extension with exponential learning rate schedule.</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF; color: black;">396B</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">396B</td>
|
||||
<td style="text-align:center; background-color: #FFFFFF; color: black;">396B</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
|
||||
**Infrastructure:**
|
||||
We trained the Granite 4.1 Language Models utilizing an NVIDIA GB200 NVL72 cluster hosted in CoreWeave. Intra-rack communication occurs via the 72-GPU NVLink domain, and a non-blocking, full Fat-Tree NDR 400 Gb/s InfiniBand network provides inter-rack communication. This cluster provides a scalable and efficient infrastructure for training our models over thousands of GPUs.
|
||||
|
||||
**Ethical Considerations and Limitations:**
|
||||
The use of Large Language Models involves risks and ethical considerations people must be aware of, including but not limited to: bias and fairness, misinformation, and autonomous decision-making. Granite-4.1-3B-Base model is not an exception in this regard. Even though this model is suited for multiple generative AI tasks, it has not undergone any safety alignment and it may produce problematic outputs. Additionally, it remains uncertain whether smaller models might exhibit increased susceptibility to hallucination in generation scenarios by copying text verbatim from the training dataset due to their reduced sizes and memorization capacities. This aspect is currently an active area of research, and we anticipate more rigorous exploration, comprehension, and mitigations in this domain. Regarding ethics, a latent risk associated with all Large Language Models is their malicious utilization. We urge the community to use Granite-4.1-3B-Base model with ethical intentions and in a responsible way. To enhance safety in enterprise deployments, we recommend using Granite 4.1 Language models alongside [Granite Guardian](https://huggingface.co/ibm-granite/granite-guardian-4.1-8b), a model designed to detect and flag risks in inputs and outputs across key dimensions outlined in the IBM AI Risk Atlas.
|
||||
|
||||
**Resources**
|
||||
- ⭐️ Learn about the latest updates with Granite: https://www.ibm.com/granite
|
||||
- 📄 Get started with tutorials, best practices, and prompt engineering advice: https://www.ibm.com/granite/docs/
|
||||
- 💡 Learn about the latest Granite learning resources: https://github.com/ibm-granite-community/
|
||||
- [PRISM: Demystifying Retention and Interaction in Mid-Training](https://huggingface.co/papers/2603.17074)
|
||||
32
config.json
Normal file
32
config.json
Normal file
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"architectures": [
|
||||
"GraniteForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"attention_multiplier": 0.015625,
|
||||
"bos_token_id": 100257,
|
||||
"embedding_multiplier": 12.0,
|
||||
"eos_token_id": 100257,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 2560,
|
||||
"initializer_range": 0.1,
|
||||
"intermediate_size": 8192,
|
||||
"logits_scaling": 10.0,
|
||||
"max_position_embeddings": 131072,
|
||||
"mlp_bias": false,
|
||||
"model_type": "granite",
|
||||
"num_attention_heads": 40,
|
||||
"num_hidden_layers": 40,
|
||||
"num_key_value_heads": 8,
|
||||
"pad_token_id": 100256,
|
||||
"residual_multiplier": 0.22,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 10000000,
|
||||
"tie_word_embeddings": true,
|
||||
"torch_dtype": "bfloat16",
|
||||
"transformers_version": "4.53.3",
|
||||
"use_cache": true,
|
||||
"vocab_size": 100352
|
||||
}
|
||||
7
generation_config.json
Normal file
7
generation_config.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 100257,
|
||||
"eos_token_id": 100257,
|
||||
"pad_token_id": 100256,
|
||||
"transformers_version": "4.53.3"
|
||||
}
|
||||
100001
merges.txt
Normal file
100001
merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
model-00001-of-00002.safetensors
Normal file
3
model-00001-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6f689cdf8d8bcde2d8ea60eb1d767d56f8dcbdb243cecde92ff9f8836de2645f
|
||||
size 4918153128
|
||||
3
model-00002-of-00002.safetensors
Normal file
3
model-00002-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f9e80cae16101f1c6dd83a746056334286d82cf260e5f0020601bd304ec73b7e
|
||||
size 1887561680
|
||||
369
model.safetensors.index.json
Normal file
369
model.safetensors.index.json
Normal file
@@ -0,0 +1,369 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 6805672960
|
||||
},
|
||||
"weight_map": {
|
||||
"model.embed_tokens.weight": "model-00001-of-00002.safetensors",
|
||||
"model.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.36.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.36.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.36.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.36.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.36.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.36.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.36.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.36.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.37.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.37.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.37.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.37.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.37.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.37.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.37.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.37.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.38.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.38.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.38.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.38.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.38.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.38.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.38.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.38.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.39.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.39.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.39.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.39.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.39.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.39.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.39.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.39.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.36.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.37.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.38.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.39.mlp.gate_proj.weight": "model-00002-of-00002.safetensors"
|
||||
}
|
||||
}
|
||||
30
special_tokens_map.json
Normal file
30
special_tokens_map.json
Normal file
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<|end_of_text|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<|end_of_text|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<|pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"unk_token": {
|
||||
"content": "<|unk|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
501264
tokenizer.json
Normal file
501264
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
783
tokenizer_config.json
Normal file
783
tokenizer_config.json
Normal file
@@ -0,0 +1,783 @@
|
||||
{
|
||||
"add_bos_token": false,
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"100256": {
|
||||
"content": "<|pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100257": {
|
||||
"content": "<|end_of_text|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100258": {
|
||||
"content": "<|fim_prefix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100259": {
|
||||
"content": "<|fim_middle|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100260": {
|
||||
"content": "<|fim_suffix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100261": {
|
||||
"content": "<|fim_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100262": {
|
||||
"content": "<|filename|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100263": {
|
||||
"content": "<|reponame|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100264": {
|
||||
"content": "<|start_of_role|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100265": {
|
||||
"content": "<|end_of_role|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100266": {
|
||||
"content": "<|unused_1|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100267": {
|
||||
"content": "<|start_of_plugin|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100268": {
|
||||
"content": "<|end_of_plugin|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100269": {
|
||||
"content": "<|unk|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100270": {
|
||||
"content": "<tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100271": {
|
||||
"content": "</tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100272": {
|
||||
"content": "<tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100273": {
|
||||
"content": "</tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100274": {
|
||||
"content": "<think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100275": {
|
||||
"content": "</think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100276": {
|
||||
"content": "<think_on>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100277": {
|
||||
"content": "<think_off>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100278": {
|
||||
"content": "<schema>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100279": {
|
||||
"content": "</schema>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100280": {
|
||||
"content": "<tools>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100281": {
|
||||
"content": "</tools>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100282": {
|
||||
"content": "<documents>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100283": {
|
||||
"content": "</documents>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100284": {
|
||||
"content": "<|unused_15|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100285": {
|
||||
"content": "<|unused_16|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100286": {
|
||||
"content": "<|unused_17|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100287": {
|
||||
"content": "<|unused_18|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100288": {
|
||||
"content": "<|unused_19|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100289": {
|
||||
"content": "<|unused_20|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100290": {
|
||||
"content": "<|unused_21|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100291": {
|
||||
"content": "<|unused_22|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100292": {
|
||||
"content": "<|unused_23|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100293": {
|
||||
"content": "<|unused_24|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100294": {
|
||||
"content": "<|unused_25|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100295": {
|
||||
"content": "<|unused_26|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100296": {
|
||||
"content": "<|unused_27|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100297": {
|
||||
"content": "<|unused_28|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100298": {
|
||||
"content": "<|unused_29|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100299": {
|
||||
"content": "<|unused_30|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100300": {
|
||||
"content": "<|unused_31|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100301": {
|
||||
"content": "<|unused_32|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100302": {
|
||||
"content": "<|unused_33|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100303": {
|
||||
"content": "<|unused_34|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100304": {
|
||||
"content": "<|unused_35|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100305": {
|
||||
"content": "<|unused_36|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100306": {
|
||||
"content": "<|unused_37|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100307": {
|
||||
"content": "<|unused_38|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100308": {
|
||||
"content": "<|unused_39|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100309": {
|
||||
"content": "<|unused_40|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100310": {
|
||||
"content": "<|unused_41|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100311": {
|
||||
"content": "<|unused_42|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100312": {
|
||||
"content": "<|unused_43|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100313": {
|
||||
"content": "<|unused_44|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100314": {
|
||||
"content": "<|unused_45|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100315": {
|
||||
"content": "<|unused_46|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100316": {
|
||||
"content": "<|unused_47|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100317": {
|
||||
"content": "<|unused_48|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100318": {
|
||||
"content": "<|unused_49|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100319": {
|
||||
"content": "<|unused_50|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100320": {
|
||||
"content": "<|unused_51|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100321": {
|
||||
"content": "<|unused_52|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100322": {
|
||||
"content": "<|unused_53|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100323": {
|
||||
"content": "<|unused_54|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100324": {
|
||||
"content": "<|unused_55|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100325": {
|
||||
"content": "<|unused_56|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100326": {
|
||||
"content": "<|unused_57|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100327": {
|
||||
"content": "<|unused_58|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100328": {
|
||||
"content": "<|unused_59|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100329": {
|
||||
"content": "<|unused_60|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100330": {
|
||||
"content": "<|unused_61|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100331": {
|
||||
"content": "<|unused_62|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100332": {
|
||||
"content": "<|unused_63|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100333": {
|
||||
"content": "<|unused_64|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100334": {
|
||||
"content": "<|unused_65|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100335": {
|
||||
"content": "<|unused_66|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100336": {
|
||||
"content": "<|unused_67|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100337": {
|
||||
"content": "<|unused_68|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100338": {
|
||||
"content": "<|unused_69|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100339": {
|
||||
"content": "<|unused_70|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100340": {
|
||||
"content": "<|unused_71|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100341": {
|
||||
"content": "<|unused_72|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100342": {
|
||||
"content": "<|unused_73|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100343": {
|
||||
"content": "<|unused_74|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100344": {
|
||||
"content": "<|unused_75|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100345": {
|
||||
"content": "<|unused_76|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100346": {
|
||||
"content": "<|unused_77|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100347": {
|
||||
"content": "<|unused_78|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100348": {
|
||||
"content": "<|unused_79|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100349": {
|
||||
"content": "<|unused_80|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100350": {
|
||||
"content": "<|unused_81|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100351": {
|
||||
"content": "<|unused_82|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
}
|
||||
},
|
||||
"bos_token": "<|end_of_text|>",
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|end_of_text|>",
|
||||
"extra_special_tokens": {},
|
||||
"model_max_length": 1000000000000000019884624838656,
|
||||
"pad_token": "<|pad|>",
|
||||
"padding_side": "left",
|
||||
"tokenizer_class": "GPT2Tokenizer",
|
||||
"unk_token": "<|unk|>"
|
||||
}
|
||||
1
vocab.json
Normal file
1
vocab.json
Normal file
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user