初始化项目，由ModelHub XC社区提供模型
Model: patrickliu28/Nile-Chat-12B Source: Original Platform
This commit is contained in:
ModelHub XC
37
.gitattributes
vendored
Normal file
37
.gitattributes
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
overall_benchmark_scores.png filter=lfs diff=lfs merge=lfs -text
|
||||
642
README.md
Normal file
642
README.md
Normal file
@@ -0,0 +1,642 @@
|
||||
---
|
||||
base_model:
|
||||
- google/gemma-3-12b-pt
|
||||
datasets:
|
||||
- MBZUAI-Paris/Egyptian-SFT
|
||||
language:
|
||||
- arz
|
||||
library_name: transformers
|
||||
license: gemma
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- conversational
|
||||
extra_gated_button_content: Acknowledge license
|
||||
---
|
||||
|
||||
# JAIS Initiative: Nile-Chat Models
|
||||
|
||||
## Model Overview
|
||||
|
||||
Nile-Chat is a family of open instruction-tuned models for Egyptian dialect, developed to handle both scripts commonly used in Egypt: Arabic script and Latin-based Arabizi. As part of the [Jais](https://arxiv.org/abs/2308.16149) project for standard Arabic and its extensions to dialectal Arabic, Nile-Chat is designed to support natural language generation in a way that reflects the script-diverse nature of Egyptian communication. These models are effective for a variety of tasks including question answering, translation and transliteration. Their range of sizes ensures accessibility, from lightweight personal deployments to more powerful setups, enabling broader use of AI technologies for Egyptian Arabic speakers. The family includes two versions:
|
||||
|
||||
* [Nile-Chat-4B](https://huggingface.co/MBZUAI-Paris/Nile-Chat-4B): A compact 4B parameter model that balances efficiency and fluency, well-suited for generating Egyptian Arabic in both Arabic and Latin scripts.
|
||||
* [Nile-Chat-2x4B-A6B](https://huggingface.co/MBZUAI-Paris/Nile-Chat-2x4B-A6B): A 2x4B MoE parameter model with 6B Activated (soft-merging/1 Expert per script) that balances efficiency and fluency, well-suited for generating Egyptian Arabic in both Arabic and Latin scripts.
|
||||
* [Nile-Chat-3x4B-A6B](https://huggingface.co/MBZUAI-Paris/Nile-Chat-3x4B-A6B): A 3x4B MoE parameter model with 6B Activated (2 experts activated) providing high-capacity generation in Egyptian Arabic in both Arabic and Latin scripts and English.
|
||||
* [Nile-Chat-12B](https://huggingface.co/MBZUAI-Paris/Nile-Chat-12B): A dense 12B parameter model providing high-capacity generation capabilities, ideal for complex, multi-turn interactions and nuanced understanding in both Egyptian scripts.
|
||||
|
||||
**Paper:** [Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts](https://huggingface.co/papers/2507.04569)
|
||||
|
||||
## 👥 Our Team
|
||||
|
||||
The model is developed by MBZUAI France Lab, an AI research center in Paris affiliated with the [Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)](https://mbzuai.ac.ae/) headquartered in Abu Dhabi.
|
||||
|
||||
|
||||
## Usage
|
||||
|
||||
Below we share some code snippets on how to get quickly started with running the model. First, install the Transformers library with:
|
||||
|
||||
```sh
|
||||
pip install -U transformers sentencepiece
|
||||
```
|
||||
|
||||
Then, copy the snippet from the section below.
|
||||
|
||||
#### Running with the `pipeline` API
|
||||
```python
|
||||
import torch
|
||||
from transformers import pipeline
|
||||
|
||||
pipe = pipeline(
|
||||
"text-generation",
|
||||
model="MBZUAI-Paris/Nile-Chat-12B",
|
||||
model_kwargs={"torch_dtype": torch.bfloat16},
|
||||
device="cuda" # replace with "mps" to run on a Mac device
|
||||
)
|
||||
|
||||
```
|
||||
Q1:
|
||||
```
|
||||
messages = [
|
||||
{"role": "user", "content": 'اسمك ايه؟'},
|
||||
]
|
||||
|
||||
outputs = pipe(messages, max_new_tokens=256)
|
||||
assistant_response = outputs[0]["generated_text"][-1]["content"].strip()
|
||||
print(assistant_response)
|
||||
```
|
||||
A1:
|
||||
- Response:
|
||||
|
||||
>اسمي نايل-شات، على اسم نهر النيل، اطول نهر في العالم، اللي من زمان كان عامل مهم في تطور مصر، وبيساعد في معيشة الناس وأثر على التراث والثقافة بتاعتنا. وعشان انا موديل لغة، الباحثين بتوع جامعة محمد بن زايد للذكاء الاصطناعي دربوني باستخدام مجموعة من المصادر المفتوحة، وعشان كده سموني نايل-شات. لو عندك اي سؤال او محتاج مساعدة في حاجة، اتفضل اسأل!
|
||||
|
||||
Q2:
|
||||
```python
|
||||
messages = [
|
||||
{"role": "user", "content": 'Esmak eh?'},
|
||||
]
|
||||
outputs = pipe(messages, max_new_tokens=256)
|
||||
assistant_response = outputs[0]["generated_text"][-1]["content"].strip()
|
||||
print(assistant_response)
|
||||
```
|
||||
A2:
|
||||
- Response:
|
||||
|
||||
>Esmi Nile-Chat, 3ala esm nahr el Nile, atwal nahr fel 3alam w mahmo kter men el hadarat 3ala tool el taree5. Ana masroor gedan bek! A2dar asa3dak ezay el naharda?
|
||||
|
||||
## Training Data
|
||||
Nile-Chat models were trained on diverse datasets focusing on Egyptian dialect consisting of approximately 3.3B tokens during continual pre-training phase, 1.9M instructions during instruction finetuning and 0.2M samples for DPO, with a maximum length of 2048 tokens, including:
|
||||
|
||||
* Web documents: A diverse collection of Egyptian web text ensures the model is exposed to a broad range of linguistic styles, topics, and vocabulary.
|
||||
* Instruction samples created from publicly available Egyptian Arabic datasets including translation and transliteration.
|
||||
* Translated English and multi-lingual pretraining and instruction-tuning datasets using Claude 3.5 Sonnet (v2).
|
||||
|
||||
The dataset covers both Egyptian Arabic and Latin scripts. Our instruction tuning dataset [Egyptian-SFT-Mixture](https://huggingface.co/datasets/MBZUAI-Paris/Egyptian-SFT-Mixture) is publicly available.
|
||||
|
||||
|
||||
## Implementation Information
|
||||
Nile-Chat models are based on Gemma 3 models. The Nile-Chat models were trained using 8 NVIDIA A100 80 GB GPUs in parallel using FSDP on AWS Sagemaker. The model is trained using HuggingFace transformers and parameter-efficient fine-tuning with LoRA rank of 256 for both continual pre-training and instruction finetuning, while performing full finetuning for DPO. The continual pre-training is divided into two phases: (i) general pre-training on 2.8B tokens from the Egyptian web and (ii) annealing phase with 0.5B high quality Egyptian text.
|
||||
|
||||
# Evaluation
|
||||
Nile-Chat models were evaluated on a comprehensive suite of tasks using various datasets and benchmarks to assess their performance across multiple dimensions. These included tasks such as:
|
||||
|
||||
* **EgyptianMMLU:** An Egyptian version of ArabicMMLU and MMLU benchmarks.
|
||||
* **EgyptianHellaSwag:** An Egyptian version of HellaSwag (In both scripts Arabic and Latin).
|
||||
* **Belebele Arz_Arab:** Belebele is a multiple-choice machine reading comprehension dataset published by Facebook spanning 122 language variants. The Evaluation is done on the Arz_Arab part of Belebele that refers to Egyptian Arabic.
|
||||
* **Translation:** Including four directions and three languages: Arabic script Egyptian, MSA and English.
|
||||
* **Transliteration:** Transforming a sentence from Egyptian (written in Arabic script) to Arabizi (Written in Latin script) and vice-versa.
|
||||
* **EgyptianPIQA:** An Egyptian version of PIQA benchmark (In both scripts Arabic and Latin).
|
||||
* **EgyptianWinoGrande:** An Egyptian version of WinoGrande benchmark (In both scripts Arabic and Latin).
|
||||
* **EgyptianRACE:** An Egyptian version of RACE benchmark (In both scripts Arabic and Latin).
|
||||
* **EgyptianOpenBookQA:** An Egyptian version of OpenBookQA benchmark.
|
||||
* **EgyptianAlpacaEval:** An Egyptian adaptation of AlpacaEval to assess LLM instruction-following and cultural alignment.
|
||||
|
||||
The models were compared against a collection of existing open-source Arabic models to gauge their effectiveness, with a particular focus on performance in Egyptian. All scores are based on zero-shot performance. The prompts are written mainly in Egyptian. We used [Language Model Evaluation Harness](https://github.com/MBZUAI-Paris/lm-evaluation-harness-nile-chat) to conduct these evaluations. All evaluations are done with applying chat template except for EgyptianWinoGrande.
|
||||
|
||||
## Benchmarks:
|
||||
### Arabic Script Benchmarks
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th><a href="#">Model</a></th>
|
||||
<th>Average</th>
|
||||
<th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianMMLU_dev" target="_blank">EgyptianMMLU</a></th>
|
||||
<th><a href="https://huggingface.co/datasets/facebook/belebele/viewer/ary_Arab" target="_blank">Belebele Arz</a></th>
|
||||
<th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianHellaSwag" target="_blank">EgyptianHellaSwag</a></th>
|
||||
<th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianPIQA" target="_blank">EgyptianPIQA</a></th>
|
||||
<th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianWinoGrande" target="_blank">EgyptianWinoGrande</a></th>
|
||||
<th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianOpenBookQA" target="_blank">EgyptianOpenBookQA</a></th>
|
||||
<th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianRACE" target="_blank">EgyptianRACE High</a></th>
|
||||
<th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianRACE" target="_blank">EgyptianRACE Middle</a></th>
|
||||
<th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianAlpacaEval" target="_blank">EgyptianAlpacaEval</a></th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/google/gemma-3-4b-it" target="_blank">gemma-3-4b-it</a></td>
|
||||
<td>48.76</td>
|
||||
<td>46.08</td><td>38.56</td><td>42.56</td><td>60.32</td><td>56.49</td><td>35.79</td><td>33.68</td><td>40.06</td><td>85.30</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/inceptionai/jais-family-6p7b-chat" target="_blank">jais-family-6p7b-chat</a></td>
|
||||
<td>46.64</td>
|
||||
<td>42.60</td><td>57.33</td><td>49.18</td><td>62.23</td><td>57.04</td><td>33.33</td><td>34.72</td><td>37.50</td><td>45.86</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/inceptionai/jais-adapted-7b-chat" target="_blank">jais-adapted-7b-chat</a></td>
|
||||
<td>42.18</td>
|
||||
<td>40.96</td><td>55.67</td><td>40.85</td><td>56.50</td><td>54.35</td><td>32.89</td><td>34.62</td><td>42.33</td><td>21.45</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/Qwen/Qwen2.5-7B-Instruct" target="_blank">Qwen2.5-7B-Instruct</a></td>
|
||||
<td>49.40</td>
|
||||
<td>45.74</td><td>64.22</td><td>45.47</td><td>58.02</td><td>56.41</td><td>38.70</td><td>35.45</td><td>41.76</td><td>58.80</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/ALLaM-AI/ALLaM-7B-Instruct-preview" target="_blank">ALLaM-7B-Instruct-preview</a></td>
|
||||
<td>56.40</td>
|
||||
<td>60.08</td><td>67.67</td><td>57.29</td><td>66.10</td><td>62.18</td><td>40.04</td><td>39.50</td><td>45.17</td><td>69.55</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/CohereLabs/c4ai-command-r7b-arabic-02-2025" target="_blank">c4ai-command-r7b-arabic-02-2025</a></td>
|
||||
<td>53.36</td>
|
||||
<td>50.97</td><td>70.67</td><td>50.39</td><td>61.84</td><td>57.20</td><td>36.91</td><td>41.89</td><td>46.02</td><td>73.36</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct" target="_blank">Llama-3.1-8B-Instruct</a></td>
|
||||
<td>46.31</td>
|
||||
<td>42.88</td><td>55.89</td><td>43.10</td><td>57.97</td><td>54.27</td><td>35.57</td><td>34.41</td><td>40.34</td><td>52.35</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/FreedomIntelligence/AceGPT-v2-8B-chat" target="_blank">AceGPT-v2-8b-chat</a></td>
|
||||
<td>58.33</td>
|
||||
<td>55.25</td><td>73.33</td><td>53.14</td><td>62.50</td><td>58.39</td><td>39.82</td><td>41.06</td><td>47.16</td><td>93.33</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/google/gemma-2-9b-it" target="_blank">gemma-2-9b-it</a></td>
|
||||
<td>53.17</td>
|
||||
<td>50.72</td><td>49.44</td><td>49.53</td><td>61.35</td><td>61.79</td><td>35.79</td><td>40.23</td><td>48.01</td><td>81.66</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/google/gemma-3-12b-it" target="_blank">gemma-3-12b-it</a></td>
|
||||
<td>59.70</td>
|
||||
<td>61.55</td><td>77.00</td><td>49.49</td><td>64.96</td><td>63.53</td><td>38.03</td><td>41.27</td><td>48.86</td><td>92.61</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/inceptionai/jais-family-13b-chat" target="_blank">jais-family-13b-chat</a></td>
|
||||
<td>49.81</td>
|
||||
<td>44.85</td><td>66.33</td><td>52.99</td><td>64.85</td><td>57.91</td><td>36.91</td><td>33.26</td><td>38.64</td><td>52.52</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/inceptionai/jais-adapted-13b-chat" target="_blank">jais-adapted-13b-chat</a></td>
|
||||
<td>49.80</td>
|
||||
<td>50.03</td><td>65.33</td><td>47.53</td><td>61.30</td><td>56.72</td><td>37.14</td><td>35.45</td><td>41.76</td><td>52.91</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/Qwen/Qwen2.5-14B-Instruct" target="_blank">Qwen2.5-14B-Instruct</a></td>
|
||||
<td>57.34</td>
|
||||
<td>60.81</td><td>72.33</td><td>55.84</td><td>63.97</td><td>59.97</td><td>38.26</td><td>43.25</td><td>50.28</td><td>71.35</td>
|
||||
</tr>
|
||||
<tr style="border-top: 4px solid;"></tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-4B" target="_blank"><strong>Nile-Chat-4B</strong></a></td>
|
||||
<td>57.85</td>
|
||||
<td>50.25</td><td>68.56</td><td>55.92</td><td>67.30</td><td>61.87</td><td>40.94</td><td>42.10</td><td>46.02</td><td>87.65</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-2x4B-A6B" target="_blank"><strong>Nile-Chat-2x4B-A6B</strong></a></td>
|
||||
<td>60.89</td>
|
||||
<td>52.05</td><td>73.89</td><td>59.69</td><td>68.67</td><td>62.26</td><td>41.61</td><td>44.07</td><td>51.14</td><td>94.58</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-3x4B-A6B" target="_blank"><strong>Nile-Chat-3x4B-A6B</strong></a></td>
|
||||
<td>60.25</td>
|
||||
<td>52.13</td><td>75.44</td><td>59.3</td><td>69.27</td><td>57.91</td><td>41.16</td><td>44.59</td><td>48.3</td><td>94.18</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-12B" target="_blank"><strong>Nile-Chat-12B</strong></a></td>
|
||||
<td>64.11</td>
|
||||
<td>62.59</td><td>79.44</td><td>64.04</td><td>70.69</td><td>63.53</td><td>42.06</td><td>48.02</td><td>53.13</td><td>93.50</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
### Latin Script Benchmarks
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th><a href="#">Model</a></th>
|
||||
<th>Average</th>
|
||||
<th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianHellaSwag" target="_blank">EgyptianHellaSwag</a></th>
|
||||
<th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianPIQA" target="_blank">EgyptianPIQA</a></th>
|
||||
<th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianWinoGrande" target="_blank">EgyptianWinoGrande</a></th>
|
||||
<th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianRACE" target="_blank">EgyptianRACE High</a></th>
|
||||
<th><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianRACE" target="_blank">EgyptianRACE Middle</a></th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/google/gemma-3-4b-it" target="_blank">gemma-3-4b-it</a></td>
|
||||
<td>36.93</td>
|
||||
<td>30.90</td><td>52.76</td><td>48.57</td><td>25.47</td><td>26.94</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/inceptionai/jais-family-6p7b-chat" target="_blank">jais-family-6p7b-chat</a></td>
|
||||
<td>37.58</td>
|
||||
<td>30.27</td><td>53.25</td><td>52.14</td><td>24.18</td><td>28.06</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/inceptionai/jais-adapted-7b-chat" target="_blank">jais-adapted-7b-chat</a></td>
|
||||
<td>37.06</td>
|
||||
<td>30.81</td><td>51.67</td><td>50.40</td><td>24.38</td><td>28.06</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/Qwen/Qwen2.5-7B-Instruct" target="_blank">Qwen2.5-7B-Instruct</a></td>
|
||||
<td>36.87</td>
|
||||
<td>30.51</td><td>51.88</td><td>50.95</td><td>24.88</td><td>26.11</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/ALLaM-AI/ALLaM-7B-Instruct-preview" target="_blank">ALLaM-7B-Instruct-preview</a></td>
|
||||
<td>38.58</td>
|
||||
<td>32.17</td><td>53.09</td><td>50.63</td><td>25.07</td><td>31.94</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/CohereLabs/c4ai-command-r7b-arabic-02-2025" target="_blank">c4ai-command-r7b-arabic-02-2025</a></td>
|
||||
<td>37.38</td>
|
||||
<td>30.88</td><td>52.32</td><td>51.43</td><td>25.07</td><td>27.22</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct" target="_blank">Llama-3.1-8B-Instruct</a></td>
|
||||
<td>37.62</td>
|
||||
<td>31.77</td><td>53.30</td><td>50.24</td><td>24.48</td><td>28.33</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/FreedomIntelligence/AceGPT-v2-8B-chat" target="_blank">AceGPT-v2-8b-chat</a></td>
|
||||
<td>38.77</td>
|
||||
<td>33.16</td><td>53.80</td><td>50.24</td><td>26.07</td><td>30.56</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/google/gemma-2-9b-it" target="_blank">gemma-2-9b-it</a></td>
|
||||
<td>38.70</td>
|
||||
<td>33.75</td><td>53.69</td><td>50.79</td><td>26.66</td><td>28.61</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/google/gemma-3-12b-it" target="_blank">gemma-3-12b-it</a></td>
|
||||
<td>41.63</td>
|
||||
<td>37.52</td><td>53.14</td><td>51.19</td><td>31.02</td><td>35.28</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/inceptionai/jais-family-13b-chat" target="_blank">jais-family-13b-chat</a></td>
|
||||
<td>36.96</td>
|
||||
<td>30.46</td><td>53.09</td><td>48.18</td><td>25.28</td><td>27.78</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/inceptionai/jais-adapted-13b-chat" target="_blank">jais-adapted-13b-chat</a></td>
|
||||
<td>36.98</td>
|
||||
<td>31.14</td><td>52.87</td><td>50.79</td><td>23.98</td><td>26.11</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/Qwen/Qwen2.5-14B-Instruct" target="_blank">Qwen2.5-14B-Instruct</a></td>
|
||||
<td>39.48</td>
|
||||
<td>33.49</td><td>52.87</td><td>53.41</td><td>27.35</td><td>30.28</td>
|
||||
</tr>
|
||||
<tr style="border-top: 4px solid;"></tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-4B" target="_blank"><strong>Nile-Chat-4B</strong></a></td>
|
||||
<td>51.38</td>
|
||||
<td>50.55</td><td>65.32</td><td>60.62</td><td>37.36</td><td>43.06</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-2x4B-A6B" target="_blank"><strong>Nile-Chat-2x4B-A6B</strong></a></td>
|
||||
<td>54.12</td>
|
||||
<td>55.49</td><td>68</td><td>61.33</td><td>40.24</td><td>45.56</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-3x4B-A6B" target="_blank"><strong>Nile-Chat-3x4B-A6B</strong></a></td>
|
||||
<td>52.26</td>
|
||||
<td>55</td><td>66.68</td><td>56.42</td><td>40.44</td><td>42.78</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-12B" target="_blank"><strong>Nile-Chat-12B</strong></a></td>
|
||||
<td>53.88</td>
|
||||
<td>53.71</td><td>65.10</td><td>59.98</td><td>41.72</td><td>48.89</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
### Translation and Transliteration Tasks:
|
||||
<table>
|
||||
<tr>
|
||||
<td rowspan="2">Model</td>
|
||||
<td colspan="3"><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianBench" target="_blank">Long Translation</a></td>
|
||||
<td colspan="3"><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianBench" target="_blank">Short Translation</a></td>
|
||||
<td colspan="3"><a href="https://huggingface.co/datasets/MBZUAI-Paris/EgyptianBench" target="_blank">Transliteration</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>BLEU</td>
|
||||
<td>chrF</td>
|
||||
<td>BERTScore</td>
|
||||
<td>BLEU</td>
|
||||
<td>chrF</td>
|
||||
<td>BERTScore</td>
|
||||
<td>BLEU</td>
|
||||
<td>chrF</td>
|
||||
<td>BERTScore</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/google/gemma-3-4b-it" target="_blank">gemma-3-4b-it</a></td>
|
||||
<td>20.67</td>
|
||||
<td>44.75</td>
|
||||
<td>73.03</td>
|
||||
<td>04.76</td>
|
||||
<td>31.15</td>
|
||||
<td>52.98</td>
|
||||
<td>01.44</td>
|
||||
<td>20.36</td>
|
||||
<td>47.54</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/inceptionai/jais-family-6p7b-chat" target="_blank">jais-family-6p7b-chat</a></td>
|
||||
<td>12.71</td>
|
||||
<td>36.53</td>
|
||||
<td>68.07</td>
|
||||
<td>08.73</td>
|
||||
<td>31.52</td>
|
||||
<td>56.78</td>
|
||||
<td>00.70</td>
|
||||
<td>10.64</td>
|
||||
<td>42.51</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/inceptionai/jais-adapted-7b-chat" target="_blank">jais-adapted-7b-chat</a></td>
|
||||
<td>10.61</td>
|
||||
<td>27.56</td>
|
||||
<td>63.48</td>
|
||||
<td>09.19</td>
|
||||
<td>24.85</td>
|
||||
<td>53.52</td>
|
||||
<td>01.11</td>
|
||||
<td>06.14</td>
|
||||
<td>40.45</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/Qwen/Qwen2.5-7B-Instruct" target="_blank">Qwen2.5-7B-Instruct</a></td>
|
||||
<td>19.89</td>
|
||||
<td>44.80</td>
|
||||
<td>73.64</td>
|
||||
<td>11.34</td>
|
||||
<td>36.31</td>
|
||||
<td>54.96</td>
|
||||
<td>02.74</td>
|
||||
<td>20.63</td>
|
||||
<td>49.32</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/ALLaM-AI/ALLaM-7B-Instruct-preview" target="_blank">ALLaM-7B-Instruct-preview</a></td>
|
||||
<td>26.57</td>
|
||||
<td>52.59</td>
|
||||
<td>78.34</td>
|
||||
<td>25.20</td>
|
||||
<td>48.12</td>
|
||||
<td>65.97</td>
|
||||
<td>02.10</td>
|
||||
<td>18.92</td>
|
||||
<td>49.42</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/CohereLabs/c4ai-command-r7b-arabic-02-2025" target="_blank">c4ai-command-r7b-arabic-02-2025</a></td>
|
||||
<td>25.18</td>
|
||||
<td>50.26</td>
|
||||
<td>77.97</td>
|
||||
<td>23.30</td>
|
||||
<td>45.34</td>
|
||||
<td>65.20</td>
|
||||
<td>03.52</td>
|
||||
<td>24.57</td>
|
||||
<td>50.49</td>
|
||||
</tr>
|
||||
<!-- <tr style="border-top: 4px solid;"></tr> -->
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct" target="_blank">Llama-3.1-8B-Instruct</a></td>
|
||||
<td>12.90</td>
|
||||
<td>32.58</td>
|
||||
<td>68.76</td>
|
||||
<td>09.06</td>
|
||||
<td>28.56</td>
|
||||
<td>54.19</td>
|
||||
<td>03.26</td>
|
||||
<td>17.55</td>
|
||||
<td>48.71</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/FreedomIntelligence/AceGPT-v2-8B-chat" target="_blank">AceGPT-v2-8b-chat</a></td>
|
||||
<td>24.59</td>
|
||||
<td>49.39</td>
|
||||
<td>77.57</td>
|
||||
<td>22.47</td>
|
||||
<td>44.97</td>
|
||||
<td>66.30</td>
|
||||
<td>04.80</td>
|
||||
<td>23.52</td>
|
||||
<td>49.33</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/google/gemma-2-9b-it" target="_blank">gemma-2-9b-it</a></td>
|
||||
<td>23.09</td>
|
||||
<td>46.98</td>
|
||||
<td>75.42</td>
|
||||
<td>11.73</td>
|
||||
<td>39.00</td>
|
||||
<td>60.42</td>
|
||||
<td>02.68</td>
|
||||
<td>24.28</td>
|
||||
<td>48.26</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/google/gemma-3-12b-it" target="_blank">gemma-3-12b-it</a></td>
|
||||
<td>22.90</td>
|
||||
<td>45.97</td>
|
||||
<td>73.46</td>
|
||||
<td>05.24</td>
|
||||
<td>32.82</td>
|
||||
<td>54.34</td>
|
||||
<td>02.77</td>
|
||||
<td>26.16</td>
|
||||
<td>50.47</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/inceptionai/jais-family-13b-chat" target="_blank">jais-family-13b-chat</a></td>
|
||||
<td>10.41</td>
|
||||
<td>31.98</td>
|
||||
<td>64.15</td>
|
||||
<td>08.64</td>
|
||||
<td>30.10</td>
|
||||
<td>57.00</td>
|
||||
<td>00.84</td>
|
||||
<td>11.35</td>
|
||||
<td>44.71</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/inceptionai/jais-adapted-13b-chat" target="_blank">jais-adapted-13b-chat</a></td>
|
||||
<td>15.53</td>
|
||||
<td>41.48</td>
|
||||
<td>70.86</td>
|
||||
<td>15.96</td>
|
||||
<td>38.81</td>
|
||||
<td>63.52</td>
|
||||
<td>01.00</td>
|
||||
<td>13.33</td>
|
||||
<td>46.08</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://huggingface.co/Qwen/Qwen2.5-14B-Instruct" target="_blank">Qwen2.5-14B-Instruct</a></td>
|
||||
<td>21.71</td>
|
||||
<td>45.55</td>
|
||||
<td>73.36</td>
|
||||
<td>09.26</td>
|
||||
<td>34.21</td>
|
||||
<td>53.89</td>
|
||||
<td>04.07</td>
|
||||
<td>25.83</td>
|
||||
<td>51.41</td>
|
||||
</tr>
|
||||
<tr style="border-top: 4px solid;"></tr>
|
||||
<tr>
|
||||
<td><strong><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-4B" target="_blank">Nile-Chat-4B</a></td>
|
||||
<td>37.49</td>
|
||||
<td>58.40</td>
|
||||
<td>84.30</td>
|
||||
<td>30.35</td>
|
||||
<td>52.01</td>
|
||||
<td>74.07</td>
|
||||
<td>51.46</td>
|
||||
<td>80.44</td>
|
||||
<td>89.59</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-2x4B-A6B" target="_blank">Nile-Chat-2x4B-A6B</a></td>
|
||||
<td>41.98</td>
|
||||
<td>61.59</td>
|
||||
<td>86.11</td>
|
||||
<td>33.4</td>
|
||||
<td>53.71</td>
|
||||
<td>76.78</td>
|
||||
<td>57.75</td>
|
||||
<td>83.89</td>
|
||||
<td>91.05</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-3x4B-A6B" target="_blank">Nile-Chat-3x4B-A6B</a></td>
|
||||
<td>42.43</td>
|
||||
<td>61.9</td>
|
||||
<td>86.26</td>
|
||||
<td>34.56</td>
|
||||
<td>55.37</td>
|
||||
<td>76.97</td>
|
||||
<td>57.79</td>
|
||||
<td>83.97</td>
|
||||
<td>91.13</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong><a href="https://huggingface.co/MBZUAI-Paris/Nile-Chat-12B" target="_blank">Nile-Chat-12B</a></td>
|
||||
<td>40.53</td>
|
||||
<td>60.61</td>
|
||||
<td>85.45</td>
|
||||
<td>32.2</td>
|
||||
<td>53.53</td>
|
||||
<td>74.72</td>
|
||||
<td>52.21</td>
|
||||
<td>80.97</td>
|
||||
<td>89.71</td>
|
||||
</tr>
|
||||
|
||||
|
||||
</table>
|
||||
|
||||
|
||||
## Usage and Limitations
|
||||
|
||||
These models have certain limitations that users should be aware of.
|
||||
<details>
|
||||
<summary>Intended Usage</summary>
|
||||
|
||||
Open Large Language Models (LLMs) have a wide range of applications across
|
||||
various industries and domains. The following list of potential uses is not
|
||||
comprehensive. The purpose of this list is to provide contextual information
|
||||
about the possible use-cases that the model creators considered as part of model
|
||||
training and development.
|
||||
|
||||
* Content Creation and Communication
|
||||
* Text Generation: These models can be used to generate creative text formats
|
||||
such as poems, scripts, code, marketing copy, and email drafts.
|
||||
* Chatbots and Conversational AI: Power conversational interfaces for customer
|
||||
service, virtual assistants, or interactive applications.
|
||||
* Text Summarization: Generate concise summaries of a text corpus, research
|
||||
papers, or reports.
|
||||
* Research and Education
|
||||
* Natural Language Processing (NLP) Research: These models can serve as a
|
||||
foundation for researchers to experiment with NLP techniques, develop
|
||||
algorithms, and contribute to the advancement of the field.
|
||||
* Language Learning Tools: Support interactive language learning experiences,
|
||||
aiding in grammar correction or providing writing practice.
|
||||
* Knowledge Exploration: Assist researchers in exploring large bodies of text
|
||||
by generating summaries or answering questions about specific topics.
|
||||
</details>
|
||||
<details>
|
||||
<summary>Limitations</summary>
|
||||
|
||||
* Training Data
|
||||
* The quality and diversity of the training data significantly influence the
|
||||
model's capabilities. Biases or gaps in the training data can lead to
|
||||
limitations in the model's responses.
|
||||
* The scope of the training dataset determines the subject areas the model can
|
||||
handle effectively.
|
||||
* Context and Task Complexity
|
||||
* LLMs perform better on tasks framed with clear prompts and
|
||||
instructions. Open-ended or highly complex tasks might be challenging.
|
||||
* A model's performance can be influenced by the amount of context provided
|
||||
(longer context generally leads to better outputs, up to a certain point).
|
||||
* Language Ambiguity and Nuance
|
||||
* Natural language is inherently complex. LLMs might struggle to grasp subtle
|
||||
nuances, sarcasm, or figurative language.
|
||||
* Factual Accuracy
|
||||
* LLMs generate responses based on information they learned from their
|
||||
training datasets, but they are not knowledge bases. They may generate
|
||||
incorrect or outdated factual statements.
|
||||
* Common Sense
|
||||
* LLMs rely on statistical patterns in language. They might lack the ability
|
||||
to apply common sense reasoning in certain situations.
|
||||
</details>
|
||||
<details>
|
||||
<summary> Ethical Considerations and Risks</summary>
|
||||
|
||||
The development of large language models (LLMs) raises several ethical concerns.
|
||||
In creating an open model, we have carefully considered the following:
|
||||
|
||||
* Bias and Fairness
|
||||
* LLMs trained on large-scale, real-world text data can reflect socio-cultural
|
||||
biases embedded in the training material.
|
||||
* Misinformation and Misuse
|
||||
* LLMs can be misused to generate text that is false, misleading, or harmful.
|
||||
* Guidelines are provided for responsible use with the model, see the
|
||||
[Responsible Generative AI Toolkit][rai-toolkit].
|
||||
* Transparency and Accountability:
|
||||
* This model card summarizes details on the models' architecture,
|
||||
capabilities, limitations, and evaluation processes.
|
||||
* A responsibly developed open model offers the opportunity to share
|
||||
innovation by making LLM technology accessible to developers and researchers
|
||||
across the AI ecosystem.
|
||||
|
||||
Risks identified and mitigations:
|
||||
|
||||
* Perpetuation of biases: It's encouraged to perform continuous monitoring
|
||||
(using evaluation metrics, human review) and the exploration of de-biasing
|
||||
techniques during model training, fine-tuning, and other use cases.
|
||||
* Generation of harmful content: Mechanisms and guidelines for content safety
|
||||
are essential. Developers are encouraged to exercise caution and implement
|
||||
appropriate content safety safeguards based on their specific product policies
|
||||
and application use cases.
|
||||
* Privacy violations: Models were trained on data filtered for removal of PII
|
||||
(Personally Identifiable Information). Developers are encouraged to adhere to
|
||||
privacy regulations with privacy-preserving techniques.
|
||||
|
||||
</details>
|
||||
3
added_tokens.json
Normal file
3
added_tokens.json
Normal file
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"<image_soft_token>": 262144
|
||||
}
|
||||
37
config.json
Normal file
37
config.json
Normal file
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"architectures": [
|
||||
"Gemma3ForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"attn_logit_softcapping": null,
|
||||
"bos_token_id": 2,
|
||||
"cache_implementation": "hybrid",
|
||||
"eos_token_id": 1,
|
||||
"final_logit_softcapping": null,
|
||||
"head_dim": 256,
|
||||
"hidden_activation": "gelu_pytorch_tanh",
|
||||
"hidden_size": 3840,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 15360,
|
||||
"max_position_embeddings": 131072,
|
||||
"model_type": "gemma3_text",
|
||||
"num_attention_heads": 16,
|
||||
"num_hidden_layers": 48,
|
||||
"num_key_value_heads": 8,
|
||||
"pad_token_id": 0,
|
||||
"query_pre_attn_scalar": 256,
|
||||
"rms_norm_eps": 1e-06,
|
||||
"rope_local_base_freq": 10000.0,
|
||||
"rope_scaling": {
|
||||
"factor": 8.0,
|
||||
"rope_type": "linear"
|
||||
},
|
||||
"rope_theta": 1000000.0,
|
||||
"sliding_window": 1024,
|
||||
"sliding_window_pattern": 6,
|
||||
"torch_dtype": "bfloat16",
|
||||
"transformers_version": "4.51.0.dev0",
|
||||
"use_cache": true,
|
||||
"vocab_size": 262208
|
||||
}
|
||||
16
generation_config.json
Normal file
16
generation_config.json
Normal file
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"_attn_implementation": "eager",
|
||||
"attn_implementation": "eager",
|
||||
"bos_token_id": 2,
|
||||
"cache_implementation": "hybrid",
|
||||
"do_sample": true,
|
||||
"eos_token_id": [
|
||||
1,
|
||||
106
|
||||
],
|
||||
"pad_token_id": 0,
|
||||
"top_k": 64,
|
||||
"top_p": 0.95,
|
||||
"transformers_version": "4.51.0.dev0",
|
||||
"repetition_penalty": 1.1
|
||||
}
|
||||
3
model-00001-of-00005.safetensors
Normal file
3
model-00001-of-00005.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:63f9214281d54e2f005b97319851a6b3ca17a2d6e6b02985a177a828f71a35e2
|
||||
size 4915892992
|
||||
3
model-00002-of-00005.safetensors
Normal file
3
model-00002-of-00005.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5b124701fab96a4d714a8ca3cb0cfb753c4d8c1e67ceddf634489d8e8a6b7fab
|
||||
size 4931294472
|
||||
3
model-00003-of-00005.safetensors
Normal file
3
model-00003-of-00005.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c6ef71c2300cabe5479364c28b1fc8944a0a243f41977f233af98b3e83719693
|
||||
size 4931294528
|
||||
3
model-00004-of-00005.safetensors
Normal file
3
model-00004-of-00005.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b6049f380614cea7d6f5fdfeca3b40bdf2c472b08a621ef8a9e214df99fba0cb
|
||||
size 4931294528
|
||||
3
model-00005-of-00005.safetensors
Normal file
3
model-00005-of-00005.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a2d7f22e5332b8fc9a816f5d6370f3f71c9c157b3f143f886a75f7944f2b7e3d
|
||||
size 3822364808
|
||||
633
model.safetensors.index.json
Normal file
633
model.safetensors.index.json
Normal file
@@ -0,0 +1,633 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 23532068352
|
||||
},
|
||||
"weight_map": {
|
||||
"model.embed_tokens.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.0.post_feedforward_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.0.pre_feedforward_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.0.self_attn.k_norm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.0.self_attn.q_norm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.1.post_feedforward_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.1.pre_feedforward_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.1.self_attn.k_norm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.1.self_attn.q_norm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.10.post_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.10.pre_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.10.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.10.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.11.post_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.11.pre_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.11.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.11.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.12.post_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.12.pre_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.12.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.12.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.13.post_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.13.pre_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.13.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.13.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.14.post_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.14.pre_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.14.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.14.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.15.post_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.15.pre_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.15.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.15.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.16.post_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.16.pre_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.16.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.16.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.17.post_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.17.pre_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.17.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.17.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.18.post_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.18.pre_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.18.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.18.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.19.post_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.19.pre_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.19.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.19.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.2.post_feedforward_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.2.pre_feedforward_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.2.self_attn.k_norm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.2.self_attn.q_norm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.20.post_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.20.pre_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.20.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.20.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.21.post_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.21.pre_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.21.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.21.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.22.post_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.22.pre_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.22.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.22.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.23.post_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.23.pre_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.23.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.23.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.24.post_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.24.pre_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.24.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.24.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.25.post_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.25.pre_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.25.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.25.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.26.post_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.26.pre_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.26.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.26.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.27.post_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.27.pre_feedforward_layernorm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.27.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.27.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.28.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.28.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.28.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.28.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.28.post_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.28.pre_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.28.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.28.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.28.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.29.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.29.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.29.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.29.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.29.post_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.29.pre_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.29.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.29.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.29.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.3.post_feedforward_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.3.pre_feedforward_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.3.self_attn.k_norm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.3.self_attn.q_norm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.30.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.30.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.30.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.30.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.30.post_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.30.pre_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.30.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.30.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.30.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.31.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.31.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.31.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.31.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.31.post_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.31.pre_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.31.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.31.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.31.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.32.input_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.32.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.32.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.32.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.32.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.32.post_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.32.pre_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.32.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.32.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.32.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.32.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.32.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.32.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.33.input_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.33.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.33.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.33.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.33.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.33.post_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.33.pre_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.33.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.33.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.33.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.33.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.33.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.33.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.34.input_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.34.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.34.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.34.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.34.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.34.post_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.34.pre_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.34.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.34.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.34.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.34.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.34.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.34.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.35.input_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.35.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.35.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.35.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.35.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.35.post_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.35.pre_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.35.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.35.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.35.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.35.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.35.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.35.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.36.input_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.36.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.36.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.36.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.36.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.36.post_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.36.pre_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.36.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.36.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.36.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.36.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.36.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.36.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.37.input_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.37.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.37.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.37.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.37.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.37.post_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.37.pre_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.37.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.37.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.37.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.37.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.37.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.37.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.38.input_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.38.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.38.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.38.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.38.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.38.post_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.38.pre_feedforward_layernorm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.38.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.38.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.38.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.38.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.38.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.38.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.39.input_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.39.mlp.down_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.39.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.39.mlp.up_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.39.post_attention_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.39.post_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.39.pre_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.39.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.39.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.39.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.39.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.39.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.39.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.4.post_feedforward_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.4.pre_feedforward_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.4.self_attn.k_norm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.4.self_attn.q_norm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.40.input_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.40.mlp.down_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.40.mlp.gate_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.40.mlp.up_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.40.post_attention_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.40.post_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.40.pre_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.40.self_attn.k_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.40.self_attn.k_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.40.self_attn.o_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.40.self_attn.q_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.40.self_attn.q_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.40.self_attn.v_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.41.input_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.41.mlp.down_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.41.mlp.gate_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.41.mlp.up_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.41.post_attention_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.41.post_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.41.pre_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.41.self_attn.k_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.41.self_attn.k_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.41.self_attn.o_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.41.self_attn.q_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.41.self_attn.q_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.41.self_attn.v_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.42.input_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.42.mlp.down_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.42.mlp.gate_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.42.mlp.up_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.42.post_attention_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.42.post_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.42.pre_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.42.self_attn.k_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.42.self_attn.k_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.42.self_attn.o_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.42.self_attn.q_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.42.self_attn.q_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.42.self_attn.v_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.43.input_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.43.mlp.down_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.43.mlp.gate_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.43.mlp.up_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.43.post_attention_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.43.post_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.43.pre_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.43.self_attn.k_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.43.self_attn.k_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.43.self_attn.o_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.43.self_attn.q_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.43.self_attn.q_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.43.self_attn.v_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.44.input_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.44.mlp.down_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.44.mlp.gate_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.44.mlp.up_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.44.post_attention_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.44.post_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.44.pre_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.44.self_attn.k_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.44.self_attn.k_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.44.self_attn.o_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.44.self_attn.q_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.44.self_attn.q_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.44.self_attn.v_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.45.input_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.45.mlp.down_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.45.mlp.gate_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.45.mlp.up_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.45.post_attention_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.45.post_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.45.pre_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.45.self_attn.k_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.45.self_attn.k_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.45.self_attn.o_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.45.self_attn.q_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.45.self_attn.q_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.45.self_attn.v_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.46.input_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.46.mlp.down_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.46.mlp.gate_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.46.mlp.up_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.46.post_attention_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.46.post_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.46.pre_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.46.self_attn.k_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.46.self_attn.k_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.46.self_attn.o_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.46.self_attn.q_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.46.self_attn.q_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.46.self_attn.v_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.47.input_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.47.mlp.down_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.47.mlp.gate_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.47.mlp.up_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.47.post_attention_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.47.post_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.47.pre_feedforward_layernorm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.47.self_attn.k_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.47.self_attn.k_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.47.self_attn.o_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.47.self_attn.q_norm.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.47.self_attn.q_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.47.self_attn.v_proj.weight": "model-00005-of-00005.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.5.post_feedforward_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.5.pre_feedforward_layernorm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.5.self_attn.k_norm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.5.self_attn.q_norm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.6.post_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.6.pre_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.6.self_attn.k_norm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.6.self_attn.q_norm.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.7.post_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.7.pre_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.7.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.7.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.8.post_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.8.pre_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.8.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.8.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.9.post_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.9.pre_feedforward_layernorm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.9.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.9.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
|
||||
"model.norm.weight": "model-00005-of-00005.safetensors"
|
||||
}
|
||||
}
|
||||
1
notice.txt
Normal file
1
notice.txt
Normal file
@@ -0,0 +1 @@
|
||||
Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms
|
||||
3
overall_benchmark_scores.png
Normal file
3
overall_benchmark_scores.png
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:37c64ce1e1afba9d7a71e97d1cd386028c4111d0600b15c501c0537d2ca518d4
|
||||
size 1703247
|
||||
33
special_tokens_map.json
Normal file
33
special_tokens_map.json
Normal file
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"boi_token": "<start_of_image>",
|
||||
"bos_token": {
|
||||
"content": "<bos>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eoi_token": "<end_of_image>",
|
||||
"eos_token": {
|
||||
"content": "<eos>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"image_token": "<image_soft_token>",
|
||||
"pad_token": {
|
||||
"content": "<pad>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"unk_token": {
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:33753cc9825494361904313ed469063a8b3e05f1648c18e4b2936f5aa3c78202
|
||||
size 33384124
|
||||
3
tokenizer.model
Normal file
3
tokenizer.model
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
|
||||
size 4689074
|
||||
51346
tokenizer_config.json
Normal file
51346
tokenizer_config.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user