commit 5c940e6b6c3e9475ca9c94a8fb4b977e1f37fe42
Author: ModelHub XC <noreply@modelhub.org.cn>
Date:   Sat Sep 26 20:04:29 2026 +0800

    初始化项目，由ModelHub XC社区提供模型
    
    Model: third-intelligence/llm-jp-4-kappa-32b-a3b-v0.1
    Source: Original Platform

diff --git a/.gitattributes b/.gitattributes
new file mode 100644
index 0000000..52373fe
--- /dev/null
+++ b/.gitattributes
@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..ec032eb
--- /dev/null
+++ b/README.md
@@ -0,0 +1,331 @@
+---
+license: apache-2.0
+language:
+- en
+- ja
+base_model:
+- llm-jp/llm-jp-4-32b-a3b-thinking
+pipeline_tag: text-generation
+library_name: transformers
+---
+
+# llm-jp-4-kappa-32b-a3b-v0.1
+
+![kappa-images.003 (1)](https://cdn-uploads.huggingface.co/production/uploads/65f01b5235c5424c262c8be8/DjZxFJ3ic19pm4rkiHuOO.jpeg)
+
+ 
+**llm-jp-4-kappa-32b-a3b-v0.1** は、**Third-intelligence** が研究開発の一環で試作した日本語・英語対応の Thinking 型大規模言語モデルです。[`llm-jp/llm-jp-4-32b-a3b-thinking`](https://huggingface.co/llm-jp/llm-jp-4-32b-a3b-thinking) をベースに、
+ 
+- Reasoning(思考プロセスの生成)能力を強化するための **SFT (Supervised Fine-Tuning)**
+- 数学・コード領域における **強化学習 (Reinforcement Learning)**
+の 2 段階の追加学習を行い、ベースモデルが持つ日本語性能を維持しつつ、推論・数学・コーディング能力の向上を目指しました。
+ 
+ベースモデルと同様、**Mixture-of-Experts (MoE) アーキテクチャ**(総パラメータ約 32B / activated 約 3B)を採用しているため、同規模の dense モデルと比較して推論時の計算コストを抑えつつ、高い性能を発揮します。
+
+詳細については[技術ブログ](https://third-intelligence.com/ja/posts/kappa-v01-reasoning-post-training)を参照ください。
+
+---
+
+## モデル概要
+
+| 項目 | 内容 |
+|---|---|
+| ベースモデル | [`llm-jp/llm-jp-4-32b-a3b-thinking`](https://huggingface.co/llm-jp/llm-jp-4-32b-a3b-thinking) |
+| アーキテクチャ | Mixture-of-Experts (MoE), Thinking モデル |
+| 総パラメータ数 | 約 32B |
+| Activated パラメータ数 | 約 3B |
+| 学習手法 | SFT(Reasoning 強化) + 強化学習(数学・コード) |
+| 対応言語 | 日本語 / 英語 |
+| 最大コンテキスト長 | 65,536 tokens |
+| ライセンス | Apache License 2.0(ベースモデルの条件も継承) |
+
+---
+
+## ベンチマーク
+
+評価には [swallow-llm/swallow-evaluation-instruct](https://github.com/swallow-llm/swallow-evaluation-instruct) を使用しました(OpenRouter 経由でモデルが使用できるよう、一部修正を加えました。）
+
+また、OpenRouter でデプロイされているモデルについては、serving しているプロバイダーの設定により量子化などが行われており、本来のフルプレシジョン版から性能が劣化している可能性があります。ただし、評価にあたっては OpenRouter 側の設定で可能な限り高いパフォーマンスとなるよう **Exacto** の設定を有効にしました。
+
+LLM-as-a-judgeが必要になるベンチマークについては評価モデルにopenai/gpt-4.1を使用しました。（プロバイダーはAzureに固定しています。）
+
+### 比較対象モデル
+
+| 略称 | モデル ID | 備考 |
+|---|---|---|
+| **llm-jp-4-kappa (本モデル)** | `third-intelligence/llm-jp-4-kappa-32b-a3b-v0.1` | 本モデル |
+| llm-jp-4 | `llm-jp/llm-jp-4-32b-a3b-thinking` | ベースモデル |
+| gpt-oss-20b | `openai/gpt-oss-20b` (via OpenRouter) | |
+| nemotron-3-nano | `nvidia/nemotron-3-nano-30b-a3b` (via OpenRouter) | |
+| qwen3-30b | `qwen/qwen3-30b-a3b-thinking-2507` (via OpenRouter) | |
+
+
+### サマリー(主要スコア)
+
+> **太字** は各行の最高スコアを示します。
+
+| ベンチマーク | **llm-jp-4-kappa (本モデル)** | llm-jp-4 | gpt-oss-20b | nemotron-3-nano | qwen3-30b |
+|---|---:|---:|---:|---:|---:|
+| MMLU ProX Japanese (avg) | 0.6724 | 0.6759 | 0.6886 | 0.5740 | **0.7547** |
+| MMLU Pro English (avg) | 0.7025 | 0.6956 | 0.7343 | 0.7180 | **0.8038** |
+| Japanese MT-Bench (avg) | **0.9339** | 0.9270 | 0.8749 | 0.8900 | 0.8693 |
+| English MT-Bench (avg) | 0.9226 | 0.9220 | 0.8899 | 0.9216 | **0.9304** |
+| JEMHopQA F1 | 0.6329 | **0.6391** | 0.4737 | 0.2329 | 0.5597 |
+| MCLM Math 100 (Japanese) | 0.8788 | 0.8687 | 0.9394 | 0.8081 | **0.9697** |
+| GPQA Diamond | 0.5505 | 0.4899 | 0.6364 | 0.5909 | **0.7121** |
+| AIME 2024/2025 | 0.4333 | 0.3833 | 0.5667 | 0.3667 | **0.8833** |
+
+### Japanese MT-Bench(カテゴリ別)
+
+| カテゴリ | **llm-jp-4-kappa (本モデル)** | llm-jp-4 | gpt-oss-20b | nemotron-3-nano | qwen3-30b |
+|---|---:|---:|---:|---:|---:|
+| coding | **0.9480** | 0.9420 | 0.9340 | 0.9450 | 0.8680 |
+| extraction | **0.8730** | 0.8650 | 0.8380 | 0.8030 | 0.8400 |
+| humanities | **0.9870** | 0.9770 | 0.8300 | 0.9550 | 0.9540 |
+| math | **0.9950** | 0.9800 | 0.9920 | 0.9070 | 0.9470 |
+| reasoning | **0.8470** | 0.8310 | 0.8090 | 0.8050 | 0.7590 |
+| roleplay | **0.9480** | 0.9340 | 0.8310 | 0.8720 | 0.8130 |
+| stem | 0.9670 | **0.9770** | 0.9120 | 0.9610 | 0.9130 |
+| writing | 0.9060 | **0.9100** | 0.8530 | 0.8720 | 0.8600 |
+| 1st turn | **0.9545** | 0.9520 | 0.9145 | 0.8888 | 0.8930 |
+| 2nd turn | **0.9133** | 0.9020 | 0.8353 | 0.8913 | 0.8455 |
+| **平均** | **0.9339** | 0.9270 | 0.8749 | 0.8900 | 0.8693 |
+
+### English MT-Bench(カテゴリ別)
+
+| カテゴリ | **llm-jp-4-kappa (本モデル)** | llm-jp-4 | gpt-oss-20b | nemotron-3-nano | qwen3-30b |
+|---|---:|---:|---:|---:|---:|
+| coding | 0.9240 | **0.9340** | 0.9040 | 0.8910 | 0.8860 |
+| extraction | 0.8590 | 0.8220 | 0.8610 | 0.8430 | **0.8980** |
+| humanities | 0.9570 | **0.9800** | 0.9250 | 0.9410 | 0.9600 |
+| math | 0.9990 | 0.9990 | 0.9720 | 0.9960 | **1.0000** |
+| reasoning | 0.8460 | 0.8260 | 0.7590 | 0.8990 | **0.9030** |
+| roleplay | 0.9290 | **0.9360** | 0.8710 | 0.9340 | 0.9090 |
+| stem | **0.9910** | 0.9780 | 0.9380 | 0.9410 | 0.9730 |
+| writing | 0.8760 | 0.9010 | 0.8890 | **0.9280** | 0.9140 |
+| 1st turn | 0.9468 | 0.9368 | 0.9208 | **0.9523** | 0.9515 |
+| 2nd turn | 0.8985 | 0.9073 | 0.8590 | 0.8910 | **0.9093** |
+| **平均** | 0.9226 | 0.9220 | 0.8899 | 0.9216 | **0.9304** |
+
+### MMLU ProX Japanese / MMLU Pro English(科目別)
+
+<details>
+<summary>📊 MMLU ProX Japanese 科目別スコアを開く</summary>
+
+| 科目 | **llm-jp-4-kappa (本モデル)** | llm-jp-4 | gpt-oss-20b | nemotron-3-nano | qwen3-30b |
+|---|---:|---:|---:|---:|---:|
+| business | 0.7338 | 0.7326 | 0.7782 | 0.5741 | **0.8238** |
+| law | 0.3326 | 0.3472 | 0.3034 | 0.2899 | **0.4067** |
+| psychology | 0.6454 | 0.6654 | 0.6654 | 0.5890 | **0.7180** |
+| biology | 0.7950 | 0.7880 | 0.7964 | 0.6820 | **0.8522** |
+| chemistry | 0.7686 | 0.7712 | 0.8012 | 0.6793 | **0.8640** |
+| history | 0.5039 | 0.5223 | 0.5118 | 0.4173 | **0.5984** |
+| other | 0.6006 | 0.5898 | 0.5931 | 0.5011 | **0.6656** |
+| health | 0.6332 | 0.6332 | 0.6332 | 0.5109 | **0.6594** |
+| economics | 0.7370 | 0.7322 | 0.7464 | 0.6351 | **0.8128** |
+| math | 0.8586 | 0.8608 | 0.8993 | 0.7631 | **0.9023** |
+| physics | 0.7929 | 0.7968 | 0.8075 | 0.6659 | **0.8676** |
+| computer science | 0.7512 | 0.7537 | 0.8073 | 0.6634 | **0.8195** |
+| philosophy | 0.5050 | 0.5170 | 0.5010 | 0.3928 | **0.5892** |
+| engineering | 0.5160 | 0.5222 | 0.5470 | 0.4314 | **0.7379** |
+| **平均** | 0.6724 | 0.6759 | 0.6886 | 0.5740 | **0.7547** |
+
+</details>
+
+<details>
+<summary>📊 MMLU Pro English 科目別スコアを開く</summary>
+
+| 科目 | **llm-jp-4-kappa (本モデル)** | llm-jp-4 | gpt-oss-20b | nemotron-3-nano | qwen3-30b |
+|---|---:|---:|---:|---:|---:|
+| business | 0.7529 | 0.7452 | 0.8137 | 0.7617 | **0.8631** |
+| law | 0.4532 | 0.4405 | 0.4142 | 0.4714 | **0.5450** |
+| psychology | 0.7306 | 0.7168 | 0.7206 | 0.7556 | **0.7757** |
+| biology | 0.8410 | 0.8536 | 0.8536 | 0.8438 | **0.8870** |
+| chemistry | 0.7420 | 0.7482 | 0.8269 | 0.7889 | **0.8763** |
+| history | 0.6273 | 0.6010 | 0.5906 | 0.6089 | **0.6483** |
+| other | 0.6504 | 0.6353 | 0.6461 | 0.6526 | **0.7370** |
+| health | 0.6883 | 0.6601 | 0.7237 | 0.7152 | **0.7665** |
+| economics | 0.7820 | 0.7761 | 0.7998 | 0.7773 | **0.8483** |
+| math | 0.8653 | 0.8564 | 0.9112 | 0.8756 | **0.9445** |
+| physics | 0.7852 | 0.7814 | 0.8276 | 0.7921 | **0.8961** |
+| computer science | 0.7268 | 0.7341 | 0.8146 | 0.7756 | **0.8439** |
+| philosophy | 0.5892 | 0.6132 | 0.6192 | 0.5972 | **0.6854** |
+| engineering | 0.5057 | 0.4902 | 0.5944 | 0.5304 | **0.7678** |
+| **平均** | 0.7025 | 0.6956 | 0.7343 | 0.7180 | **0.8038** |
+
+</details>
+
+---
+
+## 使い方
+
+### 必要環境
+
+- GPU: 推奨 **A100 80GB ×2 以上 / H100 80GB ×2 以上**(bfloat16・MoE expert parallel 動作時)
+  - 単一 80GB GPU でも `tensor-parallel-size=1` で動作可能ですが、KV キャッシュ確保のため `--max-model-len` を下げる必要がある場合があります。
+- CUDA 12.4+
+- Python 3.10+
+
+### 1. transformers での推論
+
+HuggingFace transformers ベースのパイプラインに組み込みたい場合は、以下のように直接ロードして使用できます。なお transformers から直接生成すると、出力には思考プロセス用の特殊トークン(`<|channel|>analysis` や `<|channel|>final` など)がそのまま含まれるため、思考と最終回答を分離して扱いたい場合は前述の vLLM + cookbook 構成を利用するか、出力をパースしてください。
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "third-intelligence/llm-jp-4-kappa-32b-a3b-v0.1"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+    trust_remote_code=True,
+)
+
+messages = [
+    {"role": "user", "content": "日本の宇宙開発の歴史について教えてください。"},
+]
+
+# chat_template により思考用プロンプトが付与されます
+input_ids = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt",
+).to(model.device)
+
+with torch.no_grad():
+    output_ids = model.generate(
+        input_ids,
+        max_new_tokens=4096,
+        do_sample=True,
+        temperature=0.7,
+        top_p=0.95,
+        pad_token_id=tokenizer.eos_token_id,
+    )
+
+generated = output_ids[0][input_ids.shape[-1]:]
+print(tokenizer.decode(generated, skip_special_tokens=False))
+```
+
+### 2. vLLM での推論(推奨)
+
+LLM-jp-4 系のモデルは、独自の Reasoning フォーマット(`<|channel|>final` などの特殊トークンによる思考プロセス出力)を採用しているため、素の vLLM ではそのまま正しく serve することができません。公式の [`llm-jp/llm-jp-4-cookbook`](https://github.com/llm-jp/llm-jp-4-cookbook) を利用することが推奨されています。
+
+#### セットアップ
+
+```bash
+# 1. vLLM 0.17 以降をインストール
+pip install "vllm>=0.17.0"
+
+# 2. LLM-jp-4 cookbook を clone
+git clone https://github.com/llm-jp/llm-jp-4-cookbook.git
+cd llm-jp-4-cookbook/llmjp4_vllm
+```
+
+#### vLLM サーバの起動
+
+```bash
+python example_cli.py serve third-intelligence/llm-jp-4-kappa-32b-a3b-v0.1 \
+  --host 0.0.0.0 \
+  --port 8000 \
+  --served-model-name llm-jp-4-kappa \
+  --trust-remote-code \
+  --dtype bfloat16 \
+  --tensor-parallel-size 2 \
+  --gpu-memory-utilization 0.85 \
+  --max-model-len 65536 \
+  --reasoning-parser llmjp4 \
+  --enable-expert-parallel
+```
+
+主なオプションの意味:
+
+| オプション | 説明 |
+|---|---|
+| `--reasoning-parser llmjp4` | LLM-jp-4 thinking 用の reasoning parser を有効化。**必須** |
+| `--enable-expert-parallel` | MoE の expert parallel を有効化(マルチ GPU で推奨) |
+| `--tensor-parallel-size` | テンソル並列数。GPU 枚数に合わせて変更してください(1, 2, 4, 8 など) |
+| `--max-model-len` | 最大コンテキスト長(最大 65,536) |
+| `--gpu-memory-utilization` | GPU メモリ使用率の上限。OOM になる場合は下げてください |
+
+> **トラブルシュート: `RuntimeError: Already borrowed`**
+> 環境によっては、reasoning parser の初期化時に tokenizer の競合により上記エラーが発生することがあります。その場合は cookbook 内 `llmjp4_reasoning_parser.py` の以下 2 行を、tokenizer.encode を経由しない形に置き換えてください。
+>
+> ```python
+> # Before
+> self._reasoning_end_prefix = tokenizer.encode("<|channel|>final")
+> self._reasoning_prefill = tokenizer.encode("<|start|>assistant")
+>
+> # After
+> self._reasoning_end_prefix = [9, 2520]
+> self._reasoning_prefill = [10, 12811]
+> ```
+
+#### OpenAI 互換 API で叩く
+
+```bash
+curl http://localhost:8000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "llm-jp-4-kappa",
+    "messages": [
+      {"role": "user", "content": "整数 a, b が a^2 + b^2 = 50 を満たすとき、(a, b) の組み合わせを全て求めてください。"}
+    ],
+    "max_tokens": 4096,
+    "temperature": 0.7
+  }'
+```
+
+Python (OpenAI SDK) からも同様に呼び出せます。`--reasoning-parser llmjp4` を有効にしているため、思考プロセスは `reasoning_content` フィールドに、最終回答は `content` フィールドに分離されて返ります。
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
+
+resp = client.chat.completions.create(
+    model="llm-jp-4-kappa",
+    messages=[
+        {"role": "user", "content": "再帰関数を使ってフィボナッチ数列の n 番目の値を返す Python 関数を書いてください。"}
+    ],
+    max_tokens=4096,
+    temperature=0.7,
+)
+
+msg = resp.choices[0].message
+print("=== 思考プロセス ===")
+print(getattr(msg, "reasoning_content", ""))
+print("=== 最終回答 ===")
+print(msg.content)
+```
+
+---
+
+## 注意事項
+
+本モデルは **研究目的で公開されているモデル** です。出力の正確性・安全性・倫理性は保証されず、事実と異なる内容(ハルシネーション)や不適切な内容を生成する可能性があります。商用利用やミッションクリティカルな用途での使用は想定していません。本モデルの利用により生じたいかなる損害についても開発者は責任を負いません。ライセンスはベースモデル(`llm-jp/llm-jp-4-32b-a3b-thinking`)の条件も併せて確認・遵守してください。
+
+## ライセンス
+
+本モデルは **Apache License 2.0** のもとで公開されます。ベースモデル `llm-jp/llm-jp-4-32b-a3b-thinking` のライセンス・利用規約も併せて遵守してください。
+
+## 謝辞
+
+- ベースモデルを公開してくださっている [LLM-jp プロジェクト](https://llm-jp.nii.ac.jp/)
+- データを公開してくださっている[NVIDIA](https://huggingface.co/nvidia)
+- 評価フレームワーク [`swallow-llm/swallow-evaluation-instruct`](https://github.com/swallow-llm/swallow-evaluation-instruct) を公開してくださっている [Swallow プロジェクト](https://swallow-llm.github.io/index.ja.html)
+
+をはじめとする皆様に最大の感謝を申し上げます。
+
+## 引用
+
+```bibtex
+@misc{llm-jp-4-kappa-32b-a3b-v0_1,
+  title  = {llm-jp-4-kappa-32b-a3b-v0.1},
+  author = {third-intelligence},
+  year   = {2026},
+  howpublished = {\url{https://huggingface.co/third-intelligence/llm-jp-4-kappa-32b-a3b-v0.1}}
+}
+```
\ No newline at end of file
diff --git a/chat_template.jinja b/chat_template.jinja
new file mode 100644
index 0000000..90daff8
--- /dev/null
+++ b/chat_template.jinja
@@ -0,0 +1,63 @@
+{#-
+  Simplified Harmony chat template for GRPO training (no tool support).
+  Based on the official llm-jp-4 chat_template.jinja.
+-#}
+{#- System Message Construction -#}
+{%- macro build_system_message() -%}
+{%- if model_identity is not defined %}
+{%- set model_identity = "You are LLM-jp-4, a large language model trained by LLM-jp." %}
+{%- endif %}
+{{- model_identity + "\n" -}}
+{%- if knowledge_cutoff is not defined %}
+{%- set knowledge_cutoff = "2025-12" %}
+{%- endif %}
+{{- "Knowledge cutoff: " + knowledge_cutoff + "\n" -}}
+{%- if conversation_start_date is not defined %}
+{%- set conversation_start_date = "2026-04-14" %}
+{%- endif %}
+{{- "Current date: " + conversation_start_date + "\n\n" }}
+{%- if reasoning_effort is not defined %}
+{%- set reasoning_effort = "medium" %}
+{%- endif %}
+{{- "Reasoning: " + reasoning_effort + "\n\n" }}
+{{- "# Valid channels: analysis, commentary, final. Channel must be included for every message." }}
+{%- endmacro -%}
+
+{#- Main Template Logic -#}
+{{- "<|start|>system<|message|>" }}
+{{- build_system_message() }}
+{{- "<|end|>" }}
+
+{%- if messages[0].role == "developer" or messages[0].role == "system" %}
+{%- set developer_message = messages[0].content %}
+{%- set loop_messages = messages[1:] %}
+{%- else %}
+{%- set developer_message = "" %}
+{%- set loop_messages = messages %}
+{%- endif %}
+
+{%- if developer_message %}
+{{- "<|start|>developer<|message|>" }}
+{{- "# Instructions\n\n" }}
+{{- developer_message }}
+{{- "<|end|>" }}
+{%- endif %}
+
+{%- for message in loop_messages -%}
+{%- if message.role == 'assistant' -%}
+{%- if loop.last and not add_generation_prompt %}
+{%- if "thinking" in message %}
+{{- "<|start|>assistant<|channel|>analysis<|message|>" + message.thinking + "<|end|>" }}
+{%- endif %}
+{{- "<|start|>assistant<|channel|>final<|message|>" + message.content + "<|return|>" }}
+{%- else %}
+{{- "<|start|>assistant<|channel|>final<|message|>" + message.content + "<|end|>" }}
+{%- endif %}
+{%- elif message.role == 'user' -%}
+{{- "<|start|>user<|message|>" + message.content + "<|end|>" }}
+{%- endif -%}
+{%- endfor -%}
+
+{%- if add_generation_prompt -%}
+<|start|>assistant
+{%- endif -%}
diff --git a/config.json b/config.json
new file mode 100644
index 0000000..96a808c
--- /dev/null
+++ b/config.json
@@ -0,0 +1,39 @@
+{
+  "architectures": [
+    "Qwen3MoeForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "decoder_sparse_step": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2560,
+  "initializer_range": 0.006,
+  "intermediate_size": 7680,
+  "max_position_embeddings": 65536,
+  "max_window_layers": 32,
+  "mlp_only_layers": [],
+  "model_type": "qwen3_moe",
+  "moe_intermediate_size": 960,
+  "norm_topk_prob": true,
+  "num_attention_heads": 40,
+  "num_experts": 128,
+  "num_experts_per_tok": 8,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 4,
+  "output_router_logits": false,
+  "pad_token_id": 4,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 500000,
+  "router_aux_loss_coef": 0.01,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "4.57.6",
+  "use_cache": false,
+  "use_sliding_window": false,
+  "vocab_size": 196608
+}
diff --git a/generation_config.json b/generation_config.json
new file mode 100644
index 0000000..9dd7eb8
--- /dev/null
+++ b/generation_config.json
@@ -0,0 +1,8 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "pad_token_id": 4,
+  "transformers_version": "4.57.6",
+  "use_cache": false
+}
diff --git a/llmjp4_harmony.py b/llmjp4_harmony.py
new file mode 100644
index 0000000..6d23fd8
--- /dev/null
+++ b/llmjp4_harmony.py
@@ -0,0 +1,129 @@
+# Generic parser for OpenAI Harmony format.
+
+from dataclasses import dataclass
+from enum import Enum
+from typing import Iterator, Sequence
+
+from transformers import PreTrainedTokenizerBase as TokenizerLike
+
+
+class HarmonyMessageEndType(Enum):
+    INCOMPLETE = 0
+    END = 1
+    CALL = 2
+
+
+@dataclass(frozen=True)
+class HarmonySequence:
+    """A data class representing a sequence of tokens in the Harmony format."""
+    token_ids: list[int]
+    start: int  # Start position of the sequence in the original token sequence
+
+
+@dataclass(frozen=True)
+class HarmonyMessage:
+    """A data class representing a message in the Harmony format."""
+    end: HarmonyMessageEndType
+    role: HarmonySequence | None = None
+    channel: HarmonySequence | None = None
+    constrain: HarmonySequence | None = None
+    content: HarmonySequence | None = None
+
+
+class HarmonyMessageParser:
+    """A parser that performs lexical analysis to extract Harmony messages."""
+
+    def __init__(self, tokenizer: TokenizerLike):
+        vocab = tokenizer.get_vocab()
+        self._begin_map = {
+            vocab["<|start|>"]: "role",
+            vocab["<|channel|>"]: "channel",
+            vocab["<|constrain|>"]: "constrain",
+            vocab["<|message|>"]: "content",
+        }
+        self._end_map = {
+            vocab["<|end|>"]: HarmonyMessageEndType.END,
+            vocab["<|return|>"]: HarmonyMessageEndType.END,
+            vocab["<|call|>"]: HarmonyMessageEndType.CALL,
+        }
+
+    def iter_messages(self, token_ids: Sequence[int]) -> Iterator[HarmonyMessage]:
+        """
+        Parse given token ids into messages.
+
+        Args:
+            token_ids: A sequence of token ids to be parsed.
+        
+        Yields:
+            Detected HarmonyMessages.
+        """
+
+        message_dict: dict[str, HarmonySequence] = {}
+        section: str | None = None  # None indicates out-of-message.
+        text_ids: list[int] = []
+        text_start: int | None = None
+
+        for token_position, token_id in enumerate(token_ids):
+            if token_id in self._begin_map:
+                if section is not None:
+                    message_dict[section] = HarmonySequence(
+                        token_ids=text_ids,
+                        start=text_start,
+                    )
+                section = self._begin_map[token_id]
+                text_ids = []
+                text_start = token_position + 1
+
+            elif token_id in self._end_map:
+                if section is not None:
+                    message_dict[section] = HarmonySequence(
+                        token_ids=text_ids,
+                        start=text_start,
+                    )
+
+                yield HarmonyMessage(**message_dict, end=self._end_map[token_id])
+
+                message_dict = {}
+                section = None
+                text_ids = []
+                text_start = None
+    
+            else:
+                if section is not None:
+                    text_ids.append(token_id)
+        
+        if section is not None:
+            message_dict[section] = HarmonySequence(
+                token_ids=text_ids,
+                start=text_start,
+            )
+            yield HarmonyMessage(**message_dict, end=HarmonyMessageEndType.INCOMPLETE)
+
+    def get_all_messages(self, token_ids: Sequence[int]) -> list[HarmonyMessage]:
+        """
+        Parse given token ids into messages.
+
+        Args:
+            token_ids: A sequence of token ids to be parsed.
+        
+        Returns:
+            A list of detected HarmonyMessages.
+        """
+        return list(self.iter_messages(token_ids))
+    
+    def reverse_iter_messages(self, token_ids: Sequence[int]) -> Iterator[HarmonyMessage]:
+        """
+        Parse given token ids into messages in reverse order.
+
+        Args:
+            token_ids: A sequence of token ids to be parsed.
+        
+        Yields:
+            Detected HarmonyMessages in reverse order.
+        """
+        end_position = len(token_ids)
+
+        for i in range(len(token_ids) - 1, -1, -1):
+            if token_ids[i] == self._start_id:
+                yield next(self.iter_messages(token_ids[i:end_position]))
+                end_position = i
diff --git a/llmjp4_tokenizer.py b/llmjp4_tokenizer.py
new file mode 100644
index 0000000..60f99e0
--- /dev/null
+++ b/llmjp4_tokenizer.py
@@ -0,0 +1,101 @@
+# llm-jp-4 tokenizer
+
+from collections.abc import Sequence
+import os
+
+from transformers import LlamaTokenizerFast
+from tokenizers import Tokenizer
+
+from .llmjp4_harmony import HarmonyMessageParser, HarmonyMessage
+
+
+class Llmjp4Tokenizer(LlamaTokenizerFast):
+    _HARMONY_TOKENS: set[str] = {
+        "<|start|>",
+        "<|message|>",
+        "<|channel|>",
+        "<|constrain|>",
+        "<|end|>",
+        "<|return|>",
+        "<|call|>",
+    }
+
+    # NOTE(odashi):
+    # Response schemas are not recognized automatically.
+    # We need to define them manually.
+    # https://github.com/huggingface/trl/issues/4609
+    _RESPONSE_SCHEMA = {
+        "type": "object",
+        "properties": {
+            "role": {"const": "assistant"},
+            "content": {"type": "string", "x-regex": r"<\|channel\|>final<\|message\|>(.*?)(?:<\|end\|>|<\|return\|>|$)"},
+            "thinking": {"type": "string", "x-regex": r"<\|channel\|>analysis<\|message\|>(.*?)<\|end\|>"},
+            "tool_calls": {
+                "x-regex-iterator": r"<\|channel\|>commentary (to=functions\..*?<\|message\|>.*?)(?:<\|call\|>|$)",
+                "type": "array",
+                "items": {
+                    "type": "object",
+                    "properties": {
+                        "type": {"const": "function"},
+                        "function": {
+                            "type": "object",
+                            "properties": {
+                                "name": {"type": "string", "x-regex": r"^to=functions\.(\w+)"},
+                                "arguments": {
+                                    "type": "object",
+                                    "x-regex": r"<\|message\|>(.*)",
+                                    "x-parser": "json",
+                                    "additionalProperties": {"type": "any"},
+                                },
+                            },
+                        },
+                    },
+                },
+            },
+        },
+    }
+
+    @classmethod
+    def convert_to_native_format(cls, **kwargs):
+        # NOTE(odashi):
+        # Workaround for transformers 5.x.
+        # Guaranteeing the same inner behavior with TokenizersBackend.
+        # https://github.com/huggingface/transformers/blob/7d9754a05193eb79b1d86aa744b622b8068008cd/src/transformers/tokenization_utils_tokenizers.py#L110-L116
+        local_kwargs = dict(kwargs)
+        fast_tokenizer_file = local_kwargs.pop("tokenizer_file", None)
+        if fast_tokenizer_file is None or not os.path.isfile(fast_tokenizer_file):
+            raise ValueError("Tokenizer file must exist.")
+
+        local_kwargs["tokenizer_object"] = Tokenizer.from_file(fast_tokenizer_file)
+        return local_kwargs
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+
+        self.response_schema = self._RESPONSE_SCHEMA
+
+        self._harmony_token_ids = {
+            self.convert_tokens_to_ids(token)
+            for token in self._HARMONY_TOKENS
+        }
+
+    def _decode(self, token_ids: int | list[int], *args, **kwargs):
+        if isinstance(token_ids, int):
+            token_ids = [token_ids]
+
+        result: list[str] = []
+        prev_pos = 0
+
+        # NOTE(odashi):
+        # Ensure that text tokens are decoded without preceding Harmony tokens
+        # to avoid incorrect addition of whitespaces.
+        for pos, token_id in enumerate(token_ids, start=1):
+            if token_id in self._harmony_token_ids or pos == len(token_ids):
+                result.append(super()._decode(token_ids[prev_pos:pos], *args, **kwargs))
+                prev_pos = pos
+
+        return "".join(result)
+
+    def parse_harmony_message(self, token_ids: Sequence[int]) -> list[HarmonyMessage]:
+        """Helper function to parse token IDs into Harmony messages."""
+        return HarmonyMessageParser(self).get_all_messages(token_ids)
diff --git a/model-00001-of-00013.safetensors b/model-00001-of-00013.safetensors
new file mode 100644
index 0000000..e926e59
--- /dev/null
+++ b/model-00001-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ebd2e6ac46ed407ec5917f44d1b83997b3fe57f8aabf0be51070a2bc41e5d9c
+size 4995959408
diff --git a/model-00002-of-00013.safetensors b/model-00002-of-00013.safetensors
new file mode 100644
index 0000000..191d341
--- /dev/null
+++ b/model-00002-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e2f6ba09fda33d11c72efb03c1e93c11e02f469c8e9467b55cdead6b191c962
+size 4997602792
diff --git a/model-00003-of-00013.safetensors b/model-00003-of-00013.safetensors
new file mode 100644
index 0000000..44b15f4
--- /dev/null
+++ b/model-00003-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6556aefafda3e0d76756b77ea8ec4c8387521649a9788b85197bdeed552c40a4
+size 4999858104
diff --git a/model-00004-of-00013.safetensors b/model-00004-of-00013.safetensors
new file mode 100644
index 0000000..be22c4a
--- /dev/null
+++ b/model-00004-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e39228d108526bcdaf20f4bd8152dafdef7c316ef2666385e3c845c290b29292
+size 4999242896
diff --git a/model-00005-of-00013.safetensors b/model-00005-of-00013.safetensors
new file mode 100644
index 0000000..4ab0f33
--- /dev/null
+++ b/model-00005-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b1a351d88a7a4e33d47f24f4bf509f48a487ac73c60450690835251f362ff9ee
+size 4998255672
diff --git a/model-00006-of-00013.safetensors b/model-00006-of-00013.safetensors
new file mode 100644
index 0000000..b1f2cc9
--- /dev/null
+++ b/model-00006-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49f4d27b835ce5ff68b5e57e38f04d7e16a54f5bfe60e644e6a651061d2c6282
+size 4998904016
diff --git a/model-00007-of-00013.safetensors b/model-00007-of-00013.safetensors
new file mode 100644
index 0000000..32f0c07
--- /dev/null
+++ b/model-00007-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6dd612b07216adc92a5ec26e310d84ffabbd9ac95953f779787b347e77ced0c6
+size 4998569752
diff --git a/model-00008-of-00013.safetensors b/model-00008-of-00013.safetensors
new file mode 100644
index 0000000..d779299
--- /dev/null
+++ b/model-00008-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5523ccfdd3ff8909749cbf8623d2f855d249a143070c8201c3f4e7c128fa601
+size 4999213704
diff --git a/model-00009-of-00013.safetensors b/model-00009-of-00013.safetensors
new file mode 100644
index 0000000..19e47df
--- /dev/null
+++ b/model-00009-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd493e9ddb2e0458821b01b6fcb89296ede0dd0d426efb756e63acd23be772d1
+size 4998245056
diff --git a/model-00010-of-00013.safetensors b/model-00010-of-00013.safetensors
new file mode 100644
index 0000000..282b5e7
--- /dev/null
+++ b/model-00010-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:62c4133cf9d1c9b92901d0505ce46c142a3fe3d2c9ef67a7c6f0cf0603048b6e
+size 4998278984
diff --git a/model-00011-of-00013.safetensors b/model-00011-of-00013.safetensors
new file mode 100644
index 0000000..067bd6f
--- /dev/null
+++ b/model-00011-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5b18eeaf5272fab1d983048423b3ad4c8593c94ebc10a4431fea14960d31654
+size 4996293096
diff --git a/model-00012-of-00013.safetensors b/model-00012-of-00013.safetensors
new file mode 100644
index 0000000..903b951
--- /dev/null
+++ b/model-00012-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb9e8cdf080598a22f94e0a8c134a15eb662e1d1ec44ba0125c5622108d8fd0b
+size 4989398160
diff --git a/model-00013-of-00013.safetensors b/model-00013-of-00013.safetensors
new file mode 100644
index 0000000..ddd404d
--- /dev/null
+++ b/model-00013-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6af20dfcb6ea88ec9e7dcb5ef3e201b1ad7ccedf2e84b66bd0415e01d417a164
+size 4309790912
diff --git a/model.safetensors.index.json b/model.safetensors.index.json
new file mode 100644
index 0000000..ee54e39
--- /dev/null
+++ b/model.safetensors.index.json
@@ -0,0 +1,12587 @@
+{
+  "metadata": {
+    "total_parameters": 32139028992,
+    "total_size": 64278057984
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00003-of-00013.safetensors",
+    "model.embed_tokens.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.0.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.0.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.0.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.1.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.1.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.1.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.10.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.10.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.10.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.100.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.100.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.100.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.101.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.101.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.101.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.102.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.102.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.102.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.103.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.103.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.103.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.104.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.104.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.104.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.105.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.105.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.105.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.106.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.106.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.106.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.107.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.107.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.107.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.108.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.108.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.108.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.109.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.109.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.109.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.11.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.11.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.11.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.110.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.110.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.110.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.111.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.111.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.111.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.112.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.112.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.112.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.113.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.113.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.113.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.114.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.114.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.114.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.115.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.115.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.115.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.116.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.116.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.116.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.117.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.117.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.117.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.118.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.118.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.118.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.119.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.119.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.119.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.12.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.12.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.12.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.120.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.120.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.120.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.121.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.121.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.121.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.122.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.122.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.122.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.123.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.123.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.123.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.124.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.124.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.124.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.125.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.125.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.125.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.126.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.126.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.126.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.127.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.127.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.127.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.13.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.13.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.13.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.14.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.14.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.14.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.15.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.15.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.15.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.16.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.16.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.16.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.17.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.17.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.17.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.18.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.18.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.18.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.19.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.19.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.19.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.2.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.2.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.2.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.20.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.20.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.20.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.21.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.21.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.21.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.22.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.22.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.22.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.23.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.23.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.23.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.24.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.24.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.24.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.25.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.25.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.25.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.26.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.26.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.26.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.27.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.27.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.27.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.28.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.28.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.28.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.29.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.29.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.29.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.3.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.3.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.3.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.30.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.30.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.30.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.31.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.31.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.31.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.32.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.32.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.32.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.33.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.33.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.33.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.34.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.34.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.34.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.35.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.35.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.35.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.36.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.36.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.36.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.37.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.37.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.37.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.38.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.38.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.38.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.39.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.39.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.39.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.4.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.4.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.4.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.40.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.40.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.40.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.41.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.41.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.41.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.42.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.42.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.42.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.43.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.43.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.43.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.44.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.44.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.44.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.45.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.45.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.45.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.46.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.46.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.46.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.47.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.47.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.47.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.48.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.48.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.48.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.49.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.49.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.49.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.5.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.5.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.5.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.50.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.50.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.50.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.51.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.51.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.51.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.52.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.52.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.52.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.53.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.53.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.53.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.54.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.54.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.54.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.55.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.55.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.55.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.56.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.56.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.56.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.57.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.57.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.57.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.58.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.58.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.58.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.59.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.59.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.59.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.6.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.6.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.6.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.60.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.60.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.60.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.61.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.61.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.61.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.62.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.62.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.62.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.63.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.63.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.63.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.64.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.64.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.64.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.65.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.65.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.65.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.66.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.66.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.66.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.67.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.67.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.67.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.68.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.68.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.68.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.69.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.69.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.69.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.7.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.7.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.7.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.70.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.70.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.70.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.71.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.71.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.71.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.72.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.72.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.72.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.73.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.73.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.73.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.74.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.74.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.74.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.75.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.75.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.75.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.76.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.76.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.76.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.77.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.77.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.77.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.78.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.78.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.78.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.79.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.79.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.mlp.experts.79.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.8.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.8.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.8.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.80.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.80.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.80.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.81.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.81.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.81.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.82.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.82.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.82.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.83.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.83.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.83.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.84.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.84.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.84.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.85.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.85.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.85.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.86.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.86.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.86.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.87.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.87.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.87.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.88.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.88.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.88.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.89.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.89.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.0.mlp.experts.89.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.9.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.9.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.9.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.90.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.90.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.90.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.91.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.91.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.91.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.92.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.92.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.92.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.93.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.mlp.experts.93.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.93.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.94.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.0.mlp.experts.94.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.94.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.95.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.95.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.95.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.96.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.96.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.0.mlp.experts.96.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.mlp.experts.97.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.0.mlp.experts.97.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.0.mlp.experts.97.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.mlp.experts.98.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.98.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.0.mlp.experts.98.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.experts.99.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.0.mlp.experts.99.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.mlp.experts.99.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.0.mlp.gate.weight": "model-00002-of-00013.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.self_attn.k_norm.weight": "model-00008-of-00013.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.0.self_attn.q_norm.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.0.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.0.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.0.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.1.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.1.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.1.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.10.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.10.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.10.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.100.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.100.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.100.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.101.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.101.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.101.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.102.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.102.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.102.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.103.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.103.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.103.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.104.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.104.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.104.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.105.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.105.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.105.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.106.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.106.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.106.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.107.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.107.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.107.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.108.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.108.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.108.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.109.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.109.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.109.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.11.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.11.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.11.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.110.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.110.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.110.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.111.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.111.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.111.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.112.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.112.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.112.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.113.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.113.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.113.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.114.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.114.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.114.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.115.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.115.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.115.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.116.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.116.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.116.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.117.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.117.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.117.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.118.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.118.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.118.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.119.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.119.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.119.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.12.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.12.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.12.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.120.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.120.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.120.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.121.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.121.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.121.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.122.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.122.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.122.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.123.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.123.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.123.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.124.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.124.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.124.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.125.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.125.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.125.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.126.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.126.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.126.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.127.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.127.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.127.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.13.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.13.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.13.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.14.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.14.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.14.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.15.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.15.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.15.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.16.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.16.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.16.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.17.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.17.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.17.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.18.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.18.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.18.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.19.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.19.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.19.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.2.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.2.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.2.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.20.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.20.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.20.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.21.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.21.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.21.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.22.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.22.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.22.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.23.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.23.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.23.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.24.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.24.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.24.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.25.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.25.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.25.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.26.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.26.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.26.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.27.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.27.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.27.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.28.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.28.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.28.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.29.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.29.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.29.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.3.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.3.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.3.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.30.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.30.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.30.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.31.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.31.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.31.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.32.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.32.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.32.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.33.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.33.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.33.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.34.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.34.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.34.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.35.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.35.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.35.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.36.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.36.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.36.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.37.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.37.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.37.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.38.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.38.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.38.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.39.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.39.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.39.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.4.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.4.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.4.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.40.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.40.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.40.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.41.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.41.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.41.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.42.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.42.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.42.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.43.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.43.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.43.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.44.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.44.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.44.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.45.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.45.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.45.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.46.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.46.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.46.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.47.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.47.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.47.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.48.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.48.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.48.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.49.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.49.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.49.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.5.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.5.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.5.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.50.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.50.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.50.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.51.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.51.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.51.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.52.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.52.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.52.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.53.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.53.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.53.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.54.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.54.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.54.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.55.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.55.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.55.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.56.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.56.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.56.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.57.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.57.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.57.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.58.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.58.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.58.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.59.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.59.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.59.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.6.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.6.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.6.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.60.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.60.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.60.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.61.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.61.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.61.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.62.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.62.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.62.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.63.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.63.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.63.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.64.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.64.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.64.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.65.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.65.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.65.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.66.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.66.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.66.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.67.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.67.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.67.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.68.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.68.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.68.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.69.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.69.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.69.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.7.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.7.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.7.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.70.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.70.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.70.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.71.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.71.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.71.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.72.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.72.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.72.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.73.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.73.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.73.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.74.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.74.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.74.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.75.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.75.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.75.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.76.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.76.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.76.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.77.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.77.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.77.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.78.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.78.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.78.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.79.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.79.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.79.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.8.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.8.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.8.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.80.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.80.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.80.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.81.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.81.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.81.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.82.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.82.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.82.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.83.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.83.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.83.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.84.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.84.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.84.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.85.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.85.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.85.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.86.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.86.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.86.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.87.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.87.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.87.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.88.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.88.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.88.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.89.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.89.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.89.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.9.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.9.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.9.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.1.mlp.experts.90.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.90.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.90.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.91.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.mlp.experts.91.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.91.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.92.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.92.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.1.mlp.experts.92.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.1.mlp.experts.93.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.mlp.experts.93.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.93.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.94.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.94.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.94.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.95.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.95.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.95.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.96.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.96.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.96.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.mlp.experts.97.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.1.mlp.experts.97.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.97.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.1.mlp.experts.98.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.mlp.experts.98.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.1.mlp.experts.98.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.mlp.experts.99.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.experts.99.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.1.mlp.experts.99.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.1.mlp.gate.weight": "model-00009-of-00013.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.self_attn.k_norm.weight": "model-00012-of-00013.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.1.self_attn.q_norm.weight": "model-00011-of-00013.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.0.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.0.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.0.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.1.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.1.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.1.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.10.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.10.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.10.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.100.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.100.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.100.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.101.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.101.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.101.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.102.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.102.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.102.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.103.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.103.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.103.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.104.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.104.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.104.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.105.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.105.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.105.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.106.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.106.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.106.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.107.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.107.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.107.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.108.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.108.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.108.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.109.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.109.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.109.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.11.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.11.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.11.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.110.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.110.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.110.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.111.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.111.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.111.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.112.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.112.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.112.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.113.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.113.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.113.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.114.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.114.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.114.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.115.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.115.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.115.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.116.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.116.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.116.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.117.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.117.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.117.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.118.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.118.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.118.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.119.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.119.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.119.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.12.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.12.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.12.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.120.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.120.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.120.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.121.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.121.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.121.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.122.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.122.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.122.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.123.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.123.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.123.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.124.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.124.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.124.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.125.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.125.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.125.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.126.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.126.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.126.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.127.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.127.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.127.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.13.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.13.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.13.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.14.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.14.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.14.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.15.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.15.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.15.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.16.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.16.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.16.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.17.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.17.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.17.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.18.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.18.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.18.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.19.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.19.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.19.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.2.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.2.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.2.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.20.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.20.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.20.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.21.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.21.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.21.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.22.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.22.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.22.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.23.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.23.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.23.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.24.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.24.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.24.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.25.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.25.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.25.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.26.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.26.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.26.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.27.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.27.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.27.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.28.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.28.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.28.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.29.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.29.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.29.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.3.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.3.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.3.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.30.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.30.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.30.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.31.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.31.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.31.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.32.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.32.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.32.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.33.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.33.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.33.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.34.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.34.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.34.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.35.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.35.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.35.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.36.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.36.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.36.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.37.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.37.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.37.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.38.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.38.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.38.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.39.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.39.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.39.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.4.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.4.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.4.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.40.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.40.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.40.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.41.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.41.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.41.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.42.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.42.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.42.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.43.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.43.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.43.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.44.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.44.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.44.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.45.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.45.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.45.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.46.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.46.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.46.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.47.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.47.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.47.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.48.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.48.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.48.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.49.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.49.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.49.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.5.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.5.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.5.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.50.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.50.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.50.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.51.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.51.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.51.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.52.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.52.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.52.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.53.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.53.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.53.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.54.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.54.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.54.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.55.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.55.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.55.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.56.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.56.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.56.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.57.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.57.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.57.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.58.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.58.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.58.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.59.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.59.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.59.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.6.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.6.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.6.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.60.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.60.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.60.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.61.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.61.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.61.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.62.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.62.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.62.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.63.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.63.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.63.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.64.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.64.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.64.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.65.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.65.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.65.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.66.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.66.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.66.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.67.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.67.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.67.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.68.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.68.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.68.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.69.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.69.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.69.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.7.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.7.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.7.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.70.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.70.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.70.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.71.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.71.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.71.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.72.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.72.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.72.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.73.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.73.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.73.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.74.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.74.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.74.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.75.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.75.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.75.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.76.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.76.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.76.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.77.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.77.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.77.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.78.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.78.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.78.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.79.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.79.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.79.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.8.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.8.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.8.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.80.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.80.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.80.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.81.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.81.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.81.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.82.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.82.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.82.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.83.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.83.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.83.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.84.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.84.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.84.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.85.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.85.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.85.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.86.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.86.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.86.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.87.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.87.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.87.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.88.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.88.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.88.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.89.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.mlp.experts.89.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.89.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.9.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.9.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.9.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.90.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.90.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.90.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.mlp.experts.91.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.10.mlp.experts.91.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.91.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.92.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.92.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.92.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.93.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.93.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.10.mlp.experts.93.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.94.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.94.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.94.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.10.mlp.experts.95.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.95.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.95.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.mlp.experts.96.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.96.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.96.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.10.mlp.experts.97.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.10.mlp.experts.97.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.10.mlp.experts.97.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.98.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.10.mlp.experts.98.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.10.mlp.experts.98.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.experts.99.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.99.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.10.mlp.experts.99.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.mlp.gate.weight": "model-00005-of-00013.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.self_attn.k_norm.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.10.self_attn.q_norm.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.0.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.0.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.0.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.1.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.1.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.1.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.10.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.10.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.10.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.100.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.100.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.100.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.101.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.101.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.101.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.102.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.102.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.102.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.103.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.103.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.103.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.104.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.104.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.104.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.105.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.105.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.105.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.106.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.106.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.106.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.107.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.107.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.107.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.108.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.108.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.108.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.109.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.109.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.109.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.11.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.11.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.11.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.110.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.110.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.110.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.111.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.111.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.111.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.112.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.112.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.112.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.113.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.113.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.113.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.114.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.114.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.114.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.115.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.115.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.115.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.116.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.116.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.116.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.117.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.117.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.117.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.118.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.118.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.118.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.119.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.119.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.119.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.12.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.12.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.12.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.120.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.120.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.120.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.121.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.121.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.121.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.122.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.122.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.122.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.123.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.123.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.123.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.124.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.124.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.124.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.125.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.125.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.125.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.126.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.126.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.126.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.127.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.127.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.127.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.13.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.13.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.13.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.14.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.14.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.14.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.15.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.15.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.15.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.16.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.16.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.16.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.17.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.17.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.17.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.18.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.18.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.18.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.19.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.19.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.19.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.2.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.2.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.2.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.20.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.20.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.20.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.21.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.21.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.21.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.22.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.22.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.22.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.23.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.23.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.23.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.24.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.24.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.24.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.25.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.25.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.25.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.26.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.26.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.26.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.27.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.27.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.27.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.28.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.28.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.28.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.29.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.29.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.29.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.3.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.3.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.3.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.30.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.30.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.30.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.31.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.31.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.31.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.32.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.32.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.32.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.33.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.33.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.33.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.34.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.34.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.34.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.35.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.35.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.35.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.36.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.36.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.36.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.37.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.37.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.37.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.38.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.38.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.38.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.39.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.39.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.39.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.4.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.4.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.4.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.40.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.40.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.40.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.41.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.41.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.41.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.42.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.42.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.42.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.43.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.43.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.43.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.44.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.44.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.44.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.45.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.45.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.45.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.46.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.46.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.46.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.47.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.47.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.47.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.48.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.48.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.48.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.49.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.49.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.49.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.5.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.5.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.5.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.50.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.50.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.50.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.51.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.51.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.51.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.52.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.52.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.52.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.53.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.53.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.53.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.54.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.54.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.54.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.55.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.55.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.55.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.56.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.56.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.56.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.57.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.57.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.57.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.58.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.58.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.58.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.59.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.59.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.59.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.6.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.6.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.6.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.60.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.60.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.60.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.61.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.61.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.61.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.62.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.62.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.62.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.63.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.63.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.63.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.64.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.64.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.64.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.65.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.65.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.65.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.66.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.66.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.66.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.67.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.67.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.67.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.68.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.68.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.68.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.69.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.69.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.69.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.7.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.7.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.7.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.70.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.70.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.70.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.71.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.71.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.71.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.72.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.72.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.72.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.73.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.73.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.73.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.74.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.74.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.74.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.75.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.75.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.75.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.76.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.76.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.76.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.77.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.77.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.77.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.78.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.78.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.78.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.79.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.79.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.79.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.8.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.8.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.8.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.80.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.80.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.80.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.81.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.11.mlp.experts.81.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.81.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.82.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.82.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.82.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.83.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.83.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.83.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.84.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.84.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.84.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.85.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.85.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.85.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.86.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.86.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.86.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.87.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.87.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.87.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.88.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.88.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.88.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.89.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.89.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.89.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.11.mlp.experts.9.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.9.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.9.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.90.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.90.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.90.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.91.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.mlp.experts.91.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.91.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.92.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.92.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.92.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.93.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.93.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.93.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.94.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.94.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.experts.94.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.mlp.experts.95.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.mlp.experts.95.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.95.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.96.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.mlp.experts.96.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.11.mlp.experts.96.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.11.mlp.experts.97.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.97.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.97.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.98.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.98.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.11.mlp.experts.98.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.11.mlp.experts.99.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.mlp.experts.99.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.mlp.experts.99.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.11.mlp.gate.weight": "model-00009-of-00013.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00003-of-00013.safetensors",
+    "model.layers.11.self_attn.k_norm.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.11.self_attn.q_norm.weight": "model-00008-of-00013.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.0.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.0.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.0.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.1.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.1.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.1.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.10.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.10.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.10.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.100.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.100.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.100.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.101.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.101.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.101.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.102.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.102.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.102.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.103.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.103.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.103.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.104.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.104.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.104.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.105.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.105.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.105.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.106.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.106.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.106.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.107.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.107.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.107.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.108.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.108.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.108.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.109.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.109.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.109.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.11.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.11.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.11.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.110.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.110.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.110.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.111.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.111.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.111.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.112.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.112.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.112.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.113.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.113.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.113.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.114.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.114.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.114.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.115.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.115.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.115.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.116.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.116.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.116.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.117.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.117.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.117.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.118.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.118.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.118.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.119.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.119.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.119.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.12.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.12.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.12.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.120.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.120.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.120.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.121.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.121.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.121.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.122.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.122.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.122.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.123.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.123.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.123.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.124.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.124.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.124.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.125.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.125.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.125.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.126.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.126.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.126.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.127.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.127.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.127.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.13.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.13.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.13.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.14.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.14.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.14.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.15.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.15.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.15.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.16.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.16.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.16.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.17.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.17.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.17.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.18.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.18.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.18.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.19.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.19.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.19.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.2.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.2.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.2.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.20.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.20.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.20.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.21.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.21.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.21.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.22.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.22.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.22.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.23.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.23.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.23.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.24.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.24.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.24.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.25.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.25.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.25.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.26.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.26.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.26.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.27.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.27.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.27.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.28.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.28.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.28.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.29.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.29.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.29.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.3.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.3.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.3.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.30.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.30.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.30.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.31.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.31.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.31.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.32.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.32.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.32.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.33.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.33.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.33.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.34.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.34.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.34.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.35.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.35.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.35.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.36.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.36.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.36.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.37.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.37.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.37.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.38.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.38.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.38.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.39.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.39.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.39.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.4.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.4.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.4.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.40.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.40.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.40.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.41.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.41.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.41.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.42.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.42.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.42.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.43.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.43.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.43.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.44.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.44.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.44.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.45.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.45.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.45.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.46.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.46.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.46.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.47.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.47.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.47.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.48.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.48.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.48.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.49.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.49.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.49.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.5.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.5.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.5.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.50.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.50.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.50.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.51.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.51.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.51.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.52.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.52.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.52.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.53.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.53.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.53.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.54.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.54.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.54.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.55.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.55.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.55.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.56.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.56.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.56.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.57.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.57.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.57.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.58.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.58.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.58.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.59.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.59.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.59.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.6.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.6.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.6.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.60.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.60.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.60.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.61.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.61.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.61.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.62.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.62.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.62.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.63.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.63.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.63.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.64.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.64.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.64.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.65.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.65.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.65.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.66.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.66.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.66.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.67.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.67.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.67.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.68.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.68.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.68.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.69.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.69.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.69.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.7.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.7.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.7.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.70.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.70.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.70.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.71.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.71.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.71.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.72.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.72.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.72.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.73.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.73.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.73.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.74.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.74.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.74.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.75.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.75.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.75.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.76.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.76.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.76.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.77.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.77.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.77.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.78.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.78.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.78.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.79.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.79.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.79.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.8.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.8.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.8.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.80.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.80.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.80.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.81.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.81.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.81.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.82.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.82.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.82.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.83.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.83.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.83.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.84.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.84.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.mlp.experts.84.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.85.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.85.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.85.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.86.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.86.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.86.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.87.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.87.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.87.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.88.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.88.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.88.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.89.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.89.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.89.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.9.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.9.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.9.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.experts.90.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.90.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.90.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.91.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.91.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.91.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.12.mlp.experts.92.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.92.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.92.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.93.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.93.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.93.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.94.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.94.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.12.mlp.experts.94.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.12.mlp.experts.95.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.95.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.mlp.experts.95.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.12.mlp.experts.96.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.12.mlp.experts.96.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.96.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.97.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.97.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.97.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.12.mlp.experts.98.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.98.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.mlp.experts.98.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.mlp.experts.99.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.mlp.experts.99.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.12.mlp.experts.99.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.mlp.gate.weight": "model-00007-of-00013.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00013-of-00013.safetensors",
+    "model.layers.12.self_attn.k_norm.weight": "model-00011-of-00013.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.12.self_attn.q_norm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.0.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.0.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.0.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.1.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.1.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.1.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.10.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.10.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.10.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.100.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.100.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.100.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.101.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.101.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.101.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.102.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.102.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.102.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.103.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.103.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.103.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.104.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.104.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.104.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.105.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.105.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.105.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.106.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.106.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.106.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.107.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.107.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.107.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.108.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.108.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.108.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.109.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.109.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.109.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.11.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.11.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.11.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.110.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.110.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.110.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.111.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.111.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.111.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.112.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.112.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.112.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.113.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.113.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.113.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.114.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.114.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.114.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.115.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.115.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.115.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.116.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.116.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.116.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.117.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.117.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.117.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.118.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.118.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.118.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.119.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.119.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.119.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.12.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.12.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.12.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.120.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.120.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.120.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.121.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.121.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.121.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.122.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.122.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.122.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.123.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.123.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.123.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.124.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.124.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.124.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.125.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.125.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.125.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.126.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.126.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.126.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.127.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.127.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.127.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.13.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.13.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.13.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.14.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.14.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.14.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.15.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.15.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.15.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.16.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.16.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.16.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.17.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.17.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.17.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.18.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.18.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.18.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.19.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.19.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.19.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.2.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.2.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.2.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.20.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.20.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.20.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.21.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.21.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.21.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.22.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.22.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.22.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.23.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.23.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.23.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.24.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.24.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.24.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.25.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.25.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.25.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.26.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.26.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.26.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.27.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.27.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.27.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.28.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.28.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.28.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.29.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.29.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.29.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.3.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.3.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.3.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.30.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.30.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.30.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.31.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.31.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.31.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.32.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.32.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.32.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.33.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.33.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.33.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.34.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.34.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.34.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.35.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.35.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.35.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.36.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.36.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.36.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.37.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.37.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.37.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.38.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.38.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.38.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.39.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.39.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.39.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.4.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.4.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.4.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.40.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.40.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.40.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.41.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.41.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.41.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.42.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.42.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.42.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.43.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.43.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.43.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.44.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.44.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.44.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.45.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.45.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.45.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.46.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.46.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.46.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.47.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.47.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.47.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.48.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.48.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.48.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.49.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.49.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.49.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.5.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.5.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.5.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.50.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.50.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.50.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.51.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.51.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.51.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.52.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.52.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.52.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.53.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.53.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.53.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.54.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.54.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.54.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.55.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.55.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.55.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.56.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.56.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.56.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.57.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.57.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.57.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.58.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.58.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.58.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.59.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.59.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.59.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.6.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.6.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.6.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.60.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.60.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.60.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.61.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.61.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.61.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.62.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.62.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.62.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.63.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.63.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.63.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.64.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.64.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.64.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.65.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.65.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.65.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.66.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.66.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.66.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.67.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.67.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.67.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.68.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.68.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.68.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.69.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.69.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.69.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.7.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.7.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.7.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.70.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.70.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.70.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.71.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.71.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.71.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.72.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.72.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.72.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.73.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.73.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.73.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.74.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.74.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.74.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.75.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.75.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.75.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.76.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.76.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.76.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.77.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.77.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.77.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.78.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.78.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.78.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.79.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.79.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.79.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.8.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.8.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.8.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.80.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.80.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.80.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.81.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.81.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.81.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.82.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.82.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.82.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.83.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.83.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.83.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.84.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.mlp.experts.84.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.84.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.85.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.85.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.85.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.86.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.86.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.86.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.87.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.87.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.87.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.88.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.88.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.88.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.89.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.89.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.89.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.9.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.9.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.9.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.90.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.90.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.90.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.91.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.91.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.91.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.92.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.92.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.92.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.93.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.93.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.mlp.experts.93.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.94.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.94.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.13.mlp.experts.94.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.95.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.13.mlp.experts.95.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.13.mlp.experts.95.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.96.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.96.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.96.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.13.mlp.experts.97.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.13.mlp.experts.97.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.13.mlp.experts.97.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.13.mlp.experts.98.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.98.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.mlp.experts.98.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.13.mlp.experts.99.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.experts.99.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.mlp.experts.99.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.mlp.gate.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.13.self_attn.k_norm.weight": "model-00005-of-00013.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.self_attn.q_norm.weight": "model-00008-of-00013.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.0.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.0.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.0.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.1.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.1.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.1.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.10.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.10.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.10.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.100.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.100.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.100.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.101.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.101.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.101.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.102.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.102.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.102.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.103.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.103.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.103.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.104.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.104.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.104.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.105.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.105.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.105.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.106.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.106.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.106.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.107.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.107.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.107.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.108.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.108.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.108.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.109.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.109.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.109.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.11.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.11.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.11.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.110.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.110.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.110.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.111.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.111.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.111.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.112.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.112.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.112.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.113.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.113.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.113.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.114.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.114.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.114.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.115.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.115.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.115.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.116.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.116.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.116.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.117.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.117.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.117.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.118.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.118.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.118.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.119.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.119.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.119.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.12.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.12.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.12.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.120.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.120.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.120.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.121.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.121.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.121.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.122.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.122.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.122.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.123.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.123.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.123.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.124.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.124.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.124.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.125.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.125.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.125.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.126.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.126.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.126.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.127.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.127.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.127.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.13.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.13.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.13.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.14.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.14.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.14.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.15.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.15.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.15.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.16.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.16.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.16.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.17.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.17.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.17.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.18.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.18.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.18.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.19.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.19.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.19.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.2.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.2.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.2.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.20.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.20.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.20.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.21.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.21.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.21.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.22.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.22.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.22.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.23.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.23.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.23.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.24.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.24.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.24.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.25.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.25.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.25.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.26.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.26.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.26.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.27.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.27.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.27.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.28.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.28.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.28.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.29.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.29.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.29.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.3.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.3.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.3.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.30.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.30.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.30.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.31.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.31.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.31.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.32.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.32.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.32.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.33.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.33.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.33.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.34.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.34.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.34.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.35.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.35.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.35.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.36.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.36.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.36.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.37.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.37.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.37.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.38.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.38.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.38.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.39.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.39.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.39.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.4.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.4.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.4.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.40.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.40.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.40.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.41.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.41.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.41.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.42.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.42.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.42.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.43.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.43.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.43.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.44.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.44.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.44.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.45.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.45.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.45.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.46.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.46.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.46.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.47.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.47.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.47.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.48.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.48.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.48.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.49.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.49.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.49.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.5.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.5.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.5.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.50.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.50.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.50.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.51.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.51.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.51.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.52.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.52.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.52.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.53.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.53.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.53.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.54.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.54.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.54.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.55.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.55.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.55.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.56.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.56.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.56.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.57.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.57.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.57.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.58.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.58.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.58.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.59.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.59.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.59.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.6.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.6.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.6.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.60.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.60.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.60.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.61.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.61.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.61.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.62.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.62.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.62.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.63.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.63.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.63.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.64.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.64.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.64.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.65.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.65.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.65.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.66.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.66.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.66.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.67.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.67.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.67.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.68.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.68.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.68.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.69.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.69.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.69.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.7.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.7.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.7.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.70.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.70.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.70.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.71.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.71.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.71.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.72.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.72.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.72.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.73.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.73.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.73.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.74.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.74.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.74.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.75.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.75.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.75.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.76.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.76.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.76.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.77.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.77.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.77.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.78.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.78.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.78.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.79.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.79.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.79.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.8.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.8.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.8.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.80.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.80.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.80.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.81.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.81.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.81.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.82.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.82.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.82.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.83.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.83.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.83.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.84.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.84.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.84.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.85.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.85.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.14.mlp.experts.85.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.86.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.86.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.86.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.87.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.87.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.87.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.88.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.88.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.88.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.89.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.89.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.89.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.9.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.9.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.9.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.90.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.14.mlp.experts.90.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.90.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.mlp.experts.91.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.91.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.91.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.92.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.92.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.14.mlp.experts.92.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.93.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.93.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.93.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.94.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.experts.94.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.94.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.14.mlp.experts.95.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.95.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.95.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.96.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.14.mlp.experts.96.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.96.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.mlp.experts.97.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.97.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.97.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.14.mlp.experts.98.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.mlp.experts.98.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.98.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.mlp.experts.99.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.mlp.experts.99.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.14.mlp.experts.99.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.14.mlp.gate.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00008-of-00013.safetensors",
+    "model.layers.14.self_attn.k_norm.weight": "model-00002-of-00013.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.14.self_attn.q_norm.weight": "model-00013-of-00013.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.0.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.0.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.0.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.1.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.1.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.1.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.10.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.10.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.10.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.100.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.100.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.100.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.101.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.101.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.101.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.102.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.102.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.102.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.103.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.103.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.103.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.104.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.104.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.104.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.105.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.105.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.105.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.106.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.106.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.106.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.107.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.107.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.107.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.108.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.108.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.108.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.109.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.109.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.109.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.11.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.11.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.11.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.110.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.110.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.110.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.111.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.111.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.111.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.112.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.112.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.112.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.113.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.113.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.113.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.114.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.114.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.114.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.115.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.115.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.115.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.116.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.116.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.116.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.117.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.117.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.117.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.118.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.118.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.118.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.119.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.119.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.119.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.12.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.12.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.12.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.120.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.120.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.120.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.121.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.121.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.121.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.122.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.122.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.122.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.123.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.123.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.123.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.124.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.124.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.124.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.125.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.125.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.125.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.126.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.126.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.126.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.127.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.127.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.127.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.13.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.13.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.13.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.14.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.14.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.14.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.15.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.15.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.15.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.16.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.16.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.16.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.17.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.17.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.17.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.18.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.18.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.18.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.19.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.19.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.19.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.2.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.2.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.2.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.20.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.20.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.20.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.21.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.21.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.21.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.22.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.22.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.22.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.23.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.23.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.23.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.24.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.24.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.24.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.25.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.25.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.25.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.26.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.26.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.26.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.27.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.27.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.27.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.28.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.28.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.28.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.29.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.29.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.29.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.3.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.3.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.3.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.30.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.30.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.30.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.31.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.31.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.31.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.32.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.32.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.32.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.33.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.33.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.33.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.34.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.34.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.34.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.35.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.35.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.35.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.36.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.36.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.36.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.37.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.37.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.37.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.38.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.38.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.38.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.39.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.39.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.39.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.4.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.4.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.4.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.40.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.40.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.40.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.41.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.41.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.41.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.42.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.42.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.42.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.43.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.43.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.43.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.44.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.44.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.44.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.45.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.45.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.45.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.46.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.46.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.46.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.47.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.47.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.47.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.48.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.48.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.48.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.49.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.49.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.49.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.5.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.5.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.5.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.50.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.50.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.50.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.51.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.51.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.51.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.52.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.52.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.52.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.53.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.53.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.53.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.54.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.54.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.54.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.55.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.55.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.55.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.56.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.56.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.56.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.57.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.57.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.57.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.58.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.58.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.58.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.59.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.59.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.59.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.6.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.6.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.6.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.60.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.60.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.60.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.61.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.61.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.61.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.62.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.62.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.62.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.63.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.63.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.63.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.64.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.64.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.64.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.65.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.65.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.65.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.66.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.66.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.66.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.67.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.67.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.67.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.68.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.68.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.68.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.69.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.69.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.69.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.7.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.7.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.7.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.70.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.70.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.70.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.71.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.71.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.71.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.72.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.72.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.72.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.73.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.73.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.73.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.74.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.74.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.74.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.75.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.75.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.75.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.76.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.76.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.76.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.77.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.77.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.77.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.78.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.78.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.78.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.79.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.79.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.79.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.8.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.8.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.15.mlp.experts.8.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.80.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.80.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.80.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.81.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.81.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.81.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.82.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.82.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.82.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.83.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.83.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.83.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.84.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.84.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.84.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.85.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.85.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.85.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.86.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.86.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.86.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.87.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.87.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.87.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.88.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.88.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.15.mlp.experts.88.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.89.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.89.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.89.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.15.mlp.experts.9.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.9.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.9.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.90.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.90.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.90.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.91.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.91.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.91.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.92.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.15.mlp.experts.92.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.15.mlp.experts.92.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.93.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.93.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.93.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.94.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.94.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.94.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.15.mlp.experts.95.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.95.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.95.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.experts.96.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.mlp.experts.96.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.96.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.97.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.97.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.97.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.mlp.experts.98.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.mlp.experts.98.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.mlp.experts.98.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.15.mlp.experts.99.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.99.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.15.mlp.experts.99.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.mlp.gate.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00012-of-00013.safetensors",
+    "model.layers.15.self_attn.k_norm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.15.self_attn.q_norm.weight": "model-00011-of-00013.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.0.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.0.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.0.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.1.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.1.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.1.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.10.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.10.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.10.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.100.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.100.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.100.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.101.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.101.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.101.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.102.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.102.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.102.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.103.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.103.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.103.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.104.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.104.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.104.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.105.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.105.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.105.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.106.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.106.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.106.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.107.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.107.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.107.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.108.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.108.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.108.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.109.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.109.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.109.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.11.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.11.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.11.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.110.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.110.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.110.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.111.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.111.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.111.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.112.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.112.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.112.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.113.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.113.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.113.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.114.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.114.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.114.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.115.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.115.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.115.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.116.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.116.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.116.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.117.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.117.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.117.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.118.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.118.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.118.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.119.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.119.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.119.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.12.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.12.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.12.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.120.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.120.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.120.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.121.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.121.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.121.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.122.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.122.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.122.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.123.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.123.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.123.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.124.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.124.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.124.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.125.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.125.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.125.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.126.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.126.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.126.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.127.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.127.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.127.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.13.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.13.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.13.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.14.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.14.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.14.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.15.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.15.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.15.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.16.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.16.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.16.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.17.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.17.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.17.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.18.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.18.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.18.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.19.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.19.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.19.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.2.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.2.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.2.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.20.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.20.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.20.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.21.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.21.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.21.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.22.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.22.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.22.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.23.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.23.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.23.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.24.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.24.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.24.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.25.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.25.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.25.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.26.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.26.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.26.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.27.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.27.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.27.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.28.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.28.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.28.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.29.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.29.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.29.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.3.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.3.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.3.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.30.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.30.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.30.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.31.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.31.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.31.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.32.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.32.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.32.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.33.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.33.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.33.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.34.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.34.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.34.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.35.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.35.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.35.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.36.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.36.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.36.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.37.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.37.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.37.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.38.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.38.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.38.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.39.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.39.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.39.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.4.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.4.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.4.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.40.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.40.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.40.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.41.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.41.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.41.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.42.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.42.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.42.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.43.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.43.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.43.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.44.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.44.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.44.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.45.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.45.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.45.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.46.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.46.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.46.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.47.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.47.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.47.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.48.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.48.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.48.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.49.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.49.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.49.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.5.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.5.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.5.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.50.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.50.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.50.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.51.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.51.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.51.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.52.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.52.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.52.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.53.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.53.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.53.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.54.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.54.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.54.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.55.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.55.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.55.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.56.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.56.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.56.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.57.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.57.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.57.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.58.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.58.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.58.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.59.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.59.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.59.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.6.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.6.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.6.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.60.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.60.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.60.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.61.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.61.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.61.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.62.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.62.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.62.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.63.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.63.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.63.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.64.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.64.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.64.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.65.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.65.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.65.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.66.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.66.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.66.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.67.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.67.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.67.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.68.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.68.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.68.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.69.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.69.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.69.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.7.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.7.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.7.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.70.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.70.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.70.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.71.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.71.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.71.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.72.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.72.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.72.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.73.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.73.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.73.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.74.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.74.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.74.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.75.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.75.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.75.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.76.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.76.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.76.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.77.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.77.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.77.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.78.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.78.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.78.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.79.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.79.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.79.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.8.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.8.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.8.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.80.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.80.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.80.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.81.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.81.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.81.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.82.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.82.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.82.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.83.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.83.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.83.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.84.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.84.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.84.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.85.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.85.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.85.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.86.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.86.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.86.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.87.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.87.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.87.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.88.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.88.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.88.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.89.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.89.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.89.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.16.mlp.experts.9.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.9.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.9.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.90.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.90.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.90.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.91.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.91.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.mlp.experts.91.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.92.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.92.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.92.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.93.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.93.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.mlp.experts.93.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.94.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.16.mlp.experts.94.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.94.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.95.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.16.mlp.experts.95.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.mlp.experts.95.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.16.mlp.experts.96.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.96.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.96.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.16.mlp.experts.97.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.16.mlp.experts.97.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.97.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.mlp.experts.98.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.98.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.mlp.experts.98.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.99.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.16.mlp.experts.99.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.experts.99.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.mlp.gate.weight": "model-00009-of-00013.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00011-of-00013.safetensors",
+    "model.layers.16.self_attn.k_norm.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.16.self_attn.q_norm.weight": "model-00004-of-00013.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.0.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.0.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.0.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.1.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.1.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.1.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.10.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.10.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.10.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.100.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.100.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.100.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.101.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.101.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.101.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.102.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.102.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.102.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.103.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.103.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.103.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.104.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.104.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.104.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.105.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.105.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.105.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.106.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.106.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.106.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.107.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.107.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.107.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.108.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.108.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.108.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.109.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.109.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.109.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.11.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.11.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.11.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.110.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.110.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.110.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.111.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.111.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.111.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.112.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.112.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.112.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.113.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.113.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.113.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.114.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.114.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.114.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.115.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.115.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.115.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.116.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.116.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.116.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.117.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.117.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.117.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.118.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.118.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.118.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.119.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.119.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.119.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.12.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.12.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.12.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.120.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.120.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.120.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.121.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.121.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.121.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.122.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.122.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.122.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.123.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.123.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.123.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.124.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.124.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.124.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.125.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.125.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.125.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.126.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.126.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.126.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.127.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.127.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.127.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.13.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.13.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.13.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.14.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.14.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.14.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.15.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.15.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.15.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.16.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.16.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.16.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.17.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.17.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.17.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.18.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.18.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.18.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.19.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.19.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.19.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.2.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.2.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.2.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.20.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.20.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.20.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.21.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.21.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.21.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.22.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.22.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.22.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.23.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.23.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.23.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.24.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.24.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.24.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.25.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.25.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.25.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.26.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.26.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.26.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.27.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.27.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.27.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.28.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.28.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.28.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.29.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.29.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.29.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.3.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.3.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.3.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.30.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.30.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.30.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.31.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.31.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.31.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.32.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.32.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.32.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.33.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.33.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.33.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.34.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.34.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.34.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.35.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.35.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.35.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.36.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.36.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.36.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.37.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.37.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.37.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.38.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.38.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.38.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.39.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.39.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.39.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.4.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.4.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.4.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.40.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.40.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.40.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.41.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.41.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.41.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.42.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.42.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.42.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.43.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.43.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.43.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.44.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.44.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.44.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.45.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.45.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.45.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.46.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.46.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.46.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.47.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.47.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.47.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.48.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.48.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.48.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.49.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.49.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.49.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.5.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.5.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.5.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.50.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.50.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.50.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.51.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.51.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.51.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.52.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.52.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.52.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.53.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.53.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.53.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.54.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.54.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.54.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.55.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.55.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.55.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.56.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.56.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.56.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.57.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.57.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.57.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.58.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.58.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.58.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.59.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.59.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.59.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.6.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.6.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.6.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.60.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.60.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.60.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.61.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.61.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.61.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.62.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.62.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.62.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.63.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.63.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.63.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.64.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.64.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.64.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.65.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.65.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.65.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.66.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.66.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.66.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.67.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.67.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.67.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.68.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.68.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.68.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.69.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.69.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.69.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.7.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.7.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.7.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.70.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.70.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.70.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.71.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.71.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.71.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.72.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.72.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.72.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.73.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.73.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.73.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.74.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.74.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.74.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.75.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.75.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.75.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.76.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.76.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.76.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.77.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.77.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.77.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.78.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.78.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.78.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.79.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.79.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.79.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.8.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.8.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.8.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.80.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.80.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.80.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.81.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.81.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.81.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.82.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.82.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.mlp.experts.82.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.83.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.83.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.83.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.84.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.84.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.84.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.85.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.85.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.85.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.86.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.17.mlp.experts.86.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.86.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.87.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.87.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.87.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.88.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.88.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.88.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.89.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.89.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.89.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.9.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.9.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.9.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.90.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.90.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.90.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.91.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.91.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.91.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.92.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.92.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.92.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.17.mlp.experts.93.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.93.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.93.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.94.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.94.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.94.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.95.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.95.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.95.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.17.mlp.experts.96.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.17.mlp.experts.96.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.17.mlp.experts.96.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.97.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.97.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.mlp.experts.97.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.17.mlp.experts.98.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.mlp.experts.98.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.17.mlp.experts.98.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.mlp.experts.99.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.experts.99.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.17.mlp.experts.99.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.17.mlp.gate.weight": "model-00007-of-00013.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.self_attn.k_norm.weight": "model-00013-of-00013.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.self_attn.q_norm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.0.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.0.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.0.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.1.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.1.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.1.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.10.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.10.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.10.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.100.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.100.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.100.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.101.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.101.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.101.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.102.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.102.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.102.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.103.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.103.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.103.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.104.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.104.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.104.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.105.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.105.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.105.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.106.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.106.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.106.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.107.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.107.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.107.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.108.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.108.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.108.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.109.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.109.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.109.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.11.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.11.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.11.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.110.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.110.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.110.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.111.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.111.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.111.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.112.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.112.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.112.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.113.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.113.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.113.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.114.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.114.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.114.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.115.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.115.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.115.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.116.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.116.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.116.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.117.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.117.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.117.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.118.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.118.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.118.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.119.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.119.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.119.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.12.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.12.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.12.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.120.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.120.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.120.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.121.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.121.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.121.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.122.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.122.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.122.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.123.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.123.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.123.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.124.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.124.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.124.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.125.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.125.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.125.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.126.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.126.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.126.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.127.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.127.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.127.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.13.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.13.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.13.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.14.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.14.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.14.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.15.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.15.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.15.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.16.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.16.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.16.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.17.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.17.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.17.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.18.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.18.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.18.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.19.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.19.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.19.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.2.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.2.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.2.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.20.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.20.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.20.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.21.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.21.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.21.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.22.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.22.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.22.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.23.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.23.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.23.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.24.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.24.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.24.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.25.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.25.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.25.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.26.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.26.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.26.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.27.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.27.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.27.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.28.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.28.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.28.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.29.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.29.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.29.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.3.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.3.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.3.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.30.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.30.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.30.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.31.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.31.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.31.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.32.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.32.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.32.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.33.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.33.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.33.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.34.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.34.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.34.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.35.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.35.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.35.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.36.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.36.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.36.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.37.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.37.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.37.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.38.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.38.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.38.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.39.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.39.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.39.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.4.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.4.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.4.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.40.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.40.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.40.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.41.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.41.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.41.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.42.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.42.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.42.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.43.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.43.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.43.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.44.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.44.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.44.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.45.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.45.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.45.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.46.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.46.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.46.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.47.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.47.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.47.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.48.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.48.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.48.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.49.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.49.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.49.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.5.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.5.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.5.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.50.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.50.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.50.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.51.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.51.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.51.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.52.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.52.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.52.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.53.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.53.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.53.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.54.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.54.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.54.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.55.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.55.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.55.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.56.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.56.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.56.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.57.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.57.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.57.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.58.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.58.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.58.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.59.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.59.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.59.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.6.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.6.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.6.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.60.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.60.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.60.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.61.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.61.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.61.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.62.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.62.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.62.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.63.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.63.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.63.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.64.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.64.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.64.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.65.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.65.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.65.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.66.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.66.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.66.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.67.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.67.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.67.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.68.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.68.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.68.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.69.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.69.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.69.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.7.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.7.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.7.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.70.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.70.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.70.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.71.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.71.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.71.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.72.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.72.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.72.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.73.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.73.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.73.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.74.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.74.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.74.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.75.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.75.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.75.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.76.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.76.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.76.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.77.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.77.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.77.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.78.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.78.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.78.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.79.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.79.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.79.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.8.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.8.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.8.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.80.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.80.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.80.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.81.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.81.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.81.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.18.mlp.experts.82.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.82.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.82.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.83.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.83.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.83.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.84.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.84.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.84.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.85.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.85.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.85.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.86.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.86.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.86.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.87.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.experts.87.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.87.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.88.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.88.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.88.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.89.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.89.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.89.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.9.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.9.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.9.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.90.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.90.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.90.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.91.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.91.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.91.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.92.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.92.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.92.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.93.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.93.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.18.mlp.experts.93.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.94.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.94.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.94.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.95.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.18.mlp.experts.95.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.mlp.experts.95.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.mlp.experts.96.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.18.mlp.experts.96.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.96.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.97.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.97.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.18.mlp.experts.97.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.mlp.experts.98.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.98.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.18.mlp.experts.98.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.18.mlp.experts.99.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.mlp.experts.99.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.18.mlp.experts.99.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.18.mlp.gate.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.self_attn.k_norm.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.18.self_attn.q_norm.weight": "model-00002-of-00013.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.0.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.0.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.0.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.1.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.1.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.1.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.10.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.10.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.10.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.100.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.100.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.100.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.101.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.101.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.101.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.102.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.102.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.102.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.103.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.103.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.103.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.104.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.104.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.104.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.105.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.105.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.105.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.106.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.106.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.106.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.107.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.107.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.107.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.108.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.108.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.108.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.109.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.109.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.109.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.11.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.11.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.11.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.110.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.110.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.110.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.111.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.111.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.111.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.112.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.112.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.112.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.113.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.113.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.113.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.114.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.114.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.114.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.115.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.115.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.115.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.116.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.116.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.116.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.117.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.117.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.117.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.118.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.118.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.118.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.119.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.119.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.119.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.12.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.12.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.12.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.120.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.120.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.120.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.121.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.121.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.121.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.122.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.122.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.122.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.123.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.123.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.123.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.124.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.124.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.124.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.125.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.125.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.125.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.126.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.126.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.126.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.127.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.127.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.127.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.13.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.13.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.13.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.14.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.14.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.14.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.15.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.15.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.15.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.16.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.16.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.16.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.17.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.17.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.17.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.18.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.18.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.18.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.19.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.19.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.19.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.2.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.2.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.2.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.20.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.20.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.20.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.21.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.21.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.21.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.22.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.22.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.22.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.23.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.23.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.23.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.24.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.24.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.24.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.25.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.25.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.25.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.26.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.26.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.26.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.27.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.27.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.27.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.28.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.28.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.28.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.29.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.29.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.29.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.3.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.3.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.3.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.30.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.30.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.30.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.31.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.31.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.31.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.32.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.32.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.32.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.33.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.33.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.33.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.34.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.34.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.34.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.35.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.35.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.35.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.36.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.36.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.36.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.37.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.37.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.37.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.38.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.38.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.38.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.39.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.39.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.39.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.4.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.4.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.4.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.40.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.40.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.40.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.41.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.41.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.41.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.42.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.42.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.42.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.43.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.43.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.43.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.44.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.44.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.44.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.45.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.45.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.45.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.46.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.46.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.46.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.47.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.47.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.47.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.48.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.48.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.48.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.49.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.49.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.49.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.5.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.5.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.5.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.50.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.50.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.50.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.51.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.51.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.51.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.52.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.52.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.52.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.53.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.53.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.53.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.54.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.54.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.54.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.55.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.55.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.55.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.56.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.56.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.56.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.57.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.57.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.57.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.58.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.58.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.58.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.59.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.59.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.59.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.6.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.6.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.6.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.60.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.60.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.60.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.61.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.61.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.61.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.62.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.62.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.62.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.63.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.63.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.63.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.64.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.64.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.64.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.65.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.65.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.65.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.66.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.66.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.66.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.67.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.67.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.67.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.68.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.68.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.68.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.69.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.69.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.69.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.7.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.7.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.7.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.70.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.70.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.70.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.71.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.71.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.71.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.72.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.72.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.72.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.73.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.73.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.73.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.74.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.74.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.74.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.75.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.75.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.75.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.76.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.76.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.76.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.77.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.77.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.77.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.78.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.78.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.78.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.79.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.79.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.79.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.8.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.8.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.8.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.80.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.80.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.80.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.81.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.81.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.81.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.82.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.82.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.82.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.83.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.83.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.83.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.84.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.84.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.84.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.85.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.85.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.85.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.86.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.86.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.86.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.87.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.87.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.87.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.88.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.88.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.88.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.89.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.89.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.89.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.9.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.9.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.9.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.90.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.19.mlp.experts.90.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.90.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.91.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.91.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.91.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.92.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.92.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.92.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.93.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.93.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.93.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.94.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.94.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.94.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.19.mlp.experts.95.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.mlp.experts.95.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.experts.95.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.96.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.96.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.19.mlp.experts.96.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.19.mlp.experts.97.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.19.mlp.experts.97.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.mlp.experts.97.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.mlp.experts.98.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.98.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.19.mlp.experts.98.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.mlp.experts.99.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.mlp.experts.99.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.19.mlp.experts.99.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.19.mlp.gate.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00013-of-00013.safetensors",
+    "model.layers.19.self_attn.k_norm.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.19.self_attn.q_norm.weight": "model-00002-of-00013.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.0.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.0.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.0.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.1.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.1.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.1.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.10.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.10.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.10.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.100.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.100.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.100.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.101.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.101.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.101.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.102.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.102.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.102.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.103.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.103.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.103.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.104.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.104.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.104.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.105.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.105.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.105.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.106.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.106.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.106.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.107.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.107.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.107.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.108.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.108.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.108.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.109.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.109.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.109.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.11.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.11.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.11.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.110.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.110.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.110.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.111.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.111.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.111.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.112.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.112.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.112.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.113.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.113.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.113.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.114.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.114.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.114.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.115.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.115.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.115.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.116.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.116.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.116.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.117.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.117.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.117.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.118.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.118.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.118.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.119.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.119.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.119.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.12.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.12.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.12.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.120.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.120.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.120.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.121.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.121.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.121.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.122.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.122.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.122.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.123.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.123.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.123.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.124.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.124.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.124.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.125.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.125.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.125.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.126.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.126.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.126.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.127.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.127.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.127.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.13.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.13.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.13.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.14.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.14.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.14.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.15.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.15.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.15.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.16.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.16.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.16.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.17.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.17.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.17.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.18.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.18.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.18.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.19.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.19.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.19.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.2.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.2.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.2.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.20.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.20.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.20.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.21.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.21.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.21.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.22.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.22.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.22.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.23.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.23.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.23.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.24.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.24.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.24.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.25.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.25.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.25.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.26.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.26.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.26.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.27.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.27.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.27.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.28.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.28.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.28.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.29.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.29.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.29.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.3.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.3.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.3.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.30.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.30.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.30.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.31.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.31.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.31.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.32.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.32.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.32.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.33.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.33.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.33.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.34.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.34.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.34.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.35.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.35.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.35.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.36.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.36.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.36.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.37.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.37.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.37.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.38.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.38.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.38.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.39.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.39.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.39.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.4.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.4.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.4.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.40.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.40.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.40.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.41.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.41.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.41.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.42.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.42.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.42.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.43.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.43.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.43.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.44.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.44.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.44.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.45.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.45.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.45.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.46.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.46.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.46.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.47.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.47.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.47.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.48.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.48.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.48.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.49.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.49.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.49.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.5.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.5.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.5.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.50.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.50.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.50.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.51.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.51.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.51.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.52.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.52.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.52.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.53.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.53.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.53.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.54.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.54.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.54.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.55.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.55.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.55.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.56.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.56.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.56.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.57.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.57.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.57.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.58.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.58.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.58.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.59.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.59.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.59.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.6.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.6.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.6.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.60.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.60.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.60.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.61.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.61.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.61.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.62.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.62.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.62.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.63.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.63.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.63.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.64.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.64.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.64.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.65.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.65.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.65.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.66.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.66.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.66.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.67.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.67.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.67.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.68.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.68.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.68.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.69.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.69.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.69.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.7.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.7.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.7.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.70.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.70.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.70.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.71.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.71.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.71.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.72.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.72.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.72.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.73.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.73.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.73.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.74.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.74.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.74.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.75.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.75.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.75.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.76.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.76.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.76.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.77.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.77.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.77.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.78.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.78.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.78.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.79.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.79.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.79.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.8.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.8.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.8.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.80.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.80.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.80.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.81.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.81.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.81.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.82.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.82.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.82.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.83.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.83.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.83.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.84.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.84.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.84.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.85.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.85.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.85.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.86.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.86.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.86.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.87.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.87.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.87.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.88.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.88.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.88.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.89.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.89.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.89.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.9.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.9.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.2.mlp.experts.9.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.mlp.experts.90.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.90.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.90.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.91.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.91.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.2.mlp.experts.91.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.92.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.92.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.92.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.93.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.93.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.2.mlp.experts.93.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.mlp.experts.94.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.94.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.94.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.95.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.2.mlp.experts.95.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.95.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.96.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.96.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.mlp.experts.96.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.mlp.experts.97.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.2.mlp.experts.97.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.97.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.98.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.mlp.experts.98.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.98.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.mlp.experts.99.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.experts.99.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.mlp.experts.99.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.2.mlp.gate.weight": "model-00010-of-00013.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00011-of-00013.safetensors",
+    "model.layers.2.self_attn.k_norm.weight": "model-00009-of-00013.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.2.self_attn.q_norm.weight": "model-00005-of-00013.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.0.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.0.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.0.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.1.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.1.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.1.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.10.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.10.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.10.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.100.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.100.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.100.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.101.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.101.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.101.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.102.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.102.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.102.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.103.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.103.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.103.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.104.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.104.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.104.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.105.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.105.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.105.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.106.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.106.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.106.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.107.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.107.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.107.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.108.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.108.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.108.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.109.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.109.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.109.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.11.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.11.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.11.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.110.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.110.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.110.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.111.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.111.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.111.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.112.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.112.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.112.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.113.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.113.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.113.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.114.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.114.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.114.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.115.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.115.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.115.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.116.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.116.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.116.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.117.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.117.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.117.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.118.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.118.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.118.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.119.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.119.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.119.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.12.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.12.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.12.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.120.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.120.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.120.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.121.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.121.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.121.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.122.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.122.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.122.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.123.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.123.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.123.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.124.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.124.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.124.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.125.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.125.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.125.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.126.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.126.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.126.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.127.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.127.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.127.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.13.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.13.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.13.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.14.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.14.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.14.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.15.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.15.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.15.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.16.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.16.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.16.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.17.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.17.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.17.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.18.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.18.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.18.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.19.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.19.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.19.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.2.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.2.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.2.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.20.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.20.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.20.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.21.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.21.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.21.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.22.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.22.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.22.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.23.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.23.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.23.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.24.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.24.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.24.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.25.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.25.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.25.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.26.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.26.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.26.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.27.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.27.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.27.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.28.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.28.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.28.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.29.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.29.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.29.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.3.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.3.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.3.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.30.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.30.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.30.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.31.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.31.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.31.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.32.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.32.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.32.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.33.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.33.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.33.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.34.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.34.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.34.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.35.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.35.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.35.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.36.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.36.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.36.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.37.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.37.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.37.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.38.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.38.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.38.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.39.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.39.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.39.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.4.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.4.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.4.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.40.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.40.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.40.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.41.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.41.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.41.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.42.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.42.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.42.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.43.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.43.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.43.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.44.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.44.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.44.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.45.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.45.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.45.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.46.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.46.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.46.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.47.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.47.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.47.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.48.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.48.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.48.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.49.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.49.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.49.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.5.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.5.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.5.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.50.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.50.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.50.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.51.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.51.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.51.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.52.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.52.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.52.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.53.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.53.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.53.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.54.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.54.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.54.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.55.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.55.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.55.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.56.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.56.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.56.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.57.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.57.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.57.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.58.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.58.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.58.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.59.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.59.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.59.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.6.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.6.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.6.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.60.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.60.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.60.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.61.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.61.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.61.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.62.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.62.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.62.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.63.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.63.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.63.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.64.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.64.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.64.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.65.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.65.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.65.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.66.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.66.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.66.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.67.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.67.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.67.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.68.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.68.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.68.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.69.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.69.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.69.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.7.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.7.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.7.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.70.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.70.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.70.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.71.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.71.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.71.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.72.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.72.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.72.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.73.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.73.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.73.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.74.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.74.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.74.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.75.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.75.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.75.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.76.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.76.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.76.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.77.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.77.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.77.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.78.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.78.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.78.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.79.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.79.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.79.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.8.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.8.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.8.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.80.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.80.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.80.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.81.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.81.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.81.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.82.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.82.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.82.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.83.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.83.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.83.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.84.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.84.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.84.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.85.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.85.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.85.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.86.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.86.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.86.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.87.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.87.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.mlp.experts.87.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.88.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.88.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.20.mlp.experts.88.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.89.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.89.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.89.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.9.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.9.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.9.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.20.mlp.experts.90.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.90.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.20.mlp.experts.90.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.91.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.91.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.91.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.92.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.92.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.20.mlp.experts.92.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.93.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.93.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.93.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.mlp.experts.94.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.20.mlp.experts.94.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.94.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.95.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.95.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.95.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.96.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.96.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.96.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.97.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.experts.97.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.97.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.20.mlp.experts.98.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.20.mlp.experts.98.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.mlp.experts.98.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.99.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.20.mlp.experts.99.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.mlp.experts.99.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.20.mlp.gate.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.self_attn.k_norm.weight": "model-00013-of-00013.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.self_attn.q_norm.weight": "model-00002-of-00013.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.0.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.0.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.0.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.1.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.1.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.1.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.10.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.10.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.10.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.100.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.100.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.100.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.101.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.101.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.101.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.102.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.102.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.102.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.103.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.103.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.103.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.104.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.104.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.104.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.105.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.105.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.105.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.106.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.106.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.106.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.107.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.107.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.107.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.108.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.108.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.108.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.109.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.109.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.109.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.11.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.11.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.11.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.110.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.110.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.110.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.111.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.111.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.111.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.112.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.112.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.112.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.113.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.113.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.113.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.114.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.114.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.114.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.115.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.115.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.115.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.116.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.116.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.116.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.117.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.117.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.117.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.118.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.118.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.118.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.119.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.119.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.119.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.12.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.12.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.12.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.120.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.120.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.120.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.121.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.121.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.121.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.122.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.122.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.122.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.123.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.123.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.123.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.124.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.124.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.124.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.125.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.125.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.125.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.126.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.126.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.126.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.127.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.127.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.127.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.13.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.13.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.13.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.14.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.14.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.14.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.15.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.15.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.15.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.16.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.16.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.16.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.17.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.17.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.17.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.18.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.18.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.18.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.19.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.19.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.19.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.2.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.2.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.2.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.20.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.20.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.20.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.21.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.21.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.21.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.22.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.22.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.22.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.23.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.23.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.23.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.24.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.24.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.24.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.25.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.25.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.25.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.26.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.26.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.26.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.27.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.27.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.27.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.28.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.28.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.28.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.29.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.29.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.29.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.3.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.3.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.3.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.30.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.30.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.30.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.31.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.31.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.31.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.32.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.32.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.32.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.33.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.33.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.33.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.34.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.34.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.34.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.35.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.35.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.35.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.36.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.36.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.36.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.37.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.37.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.37.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.38.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.38.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.38.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.39.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.39.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.39.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.4.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.4.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.4.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.40.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.40.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.40.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.41.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.41.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.41.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.42.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.42.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.42.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.43.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.43.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.43.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.44.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.44.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.44.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.45.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.45.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.45.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.46.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.46.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.46.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.47.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.47.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.47.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.48.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.48.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.48.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.49.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.49.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.49.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.5.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.5.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.5.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.50.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.50.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.50.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.51.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.51.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.51.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.52.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.52.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.52.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.53.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.53.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.53.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.54.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.54.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.54.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.55.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.55.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.55.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.56.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.56.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.56.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.57.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.57.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.57.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.58.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.58.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.58.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.59.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.59.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.59.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.6.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.6.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.6.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.60.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.60.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.60.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.61.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.61.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.61.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.62.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.62.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.62.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.63.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.63.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.63.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.64.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.64.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.64.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.65.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.65.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.65.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.66.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.66.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.66.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.67.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.67.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.67.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.68.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.68.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.68.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.69.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.69.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.69.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.7.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.7.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.7.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.70.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.70.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.70.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.71.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.71.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.71.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.72.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.72.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.72.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.73.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.73.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.73.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.74.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.74.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.74.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.75.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.75.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.75.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.76.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.76.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.76.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.77.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.77.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.77.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.78.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.78.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.78.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.79.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.79.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.79.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.8.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.8.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.8.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.80.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.80.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.80.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.81.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.81.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.81.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.82.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.82.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.82.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.83.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.83.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.83.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.mlp.experts.84.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.84.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.84.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.85.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.85.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.85.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.86.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.86.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.86.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.87.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.87.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.87.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.88.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.88.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.88.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.89.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.89.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.89.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.9.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.9.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.9.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.90.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.90.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.90.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.91.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.91.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.91.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.21.mlp.experts.92.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.92.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.mlp.experts.92.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.93.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.93.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.93.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.94.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.94.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.94.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.experts.95.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.95.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.21.mlp.experts.95.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.96.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.21.mlp.experts.96.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.96.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.97.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.21.mlp.experts.97.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.97.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.mlp.experts.98.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.21.mlp.experts.98.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.21.mlp.experts.98.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.mlp.experts.99.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.21.mlp.experts.99.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.21.mlp.experts.99.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.mlp.gate.weight": "model-00013-of-00013.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.self_attn.k_norm.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.21.self_attn.q_norm.weight": "model-00005-of-00013.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.0.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.0.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.0.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.1.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.1.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.1.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.10.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.10.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.10.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.100.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.100.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.100.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.101.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.101.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.101.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.102.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.102.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.102.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.103.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.103.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.103.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.104.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.104.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.104.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.105.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.105.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.105.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.106.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.106.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.106.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.107.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.107.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.107.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.108.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.108.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.108.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.109.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.109.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.109.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.11.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.11.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.11.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.110.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.110.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.110.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.111.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.111.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.111.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.112.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.112.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.112.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.113.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.113.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.113.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.114.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.114.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.114.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.115.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.115.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.115.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.116.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.116.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.116.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.117.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.117.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.117.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.118.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.118.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.118.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.119.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.119.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.119.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.12.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.12.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.12.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.120.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.120.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.120.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.121.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.121.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.121.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.122.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.122.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.122.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.123.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.123.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.123.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.124.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.124.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.124.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.125.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.125.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.125.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.126.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.126.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.126.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.127.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.127.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.127.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.13.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.22.mlp.experts.13.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.13.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.14.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.14.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.14.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.15.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.15.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.15.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.16.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.16.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.16.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.17.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.17.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.17.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.18.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.18.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.18.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.19.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.19.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.19.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.2.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.2.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.2.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.20.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.20.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.20.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.22.mlp.experts.21.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.21.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.21.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.22.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.22.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.22.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.23.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.23.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.23.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.24.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.24.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.24.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.25.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.25.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.25.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.26.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.26.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.26.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.27.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.27.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.27.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.28.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.28.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.28.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.29.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.29.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.29.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.3.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.3.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.3.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.30.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.30.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.30.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.31.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.31.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.31.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.32.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.22.mlp.experts.32.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.32.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.33.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.33.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.33.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.34.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.34.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.34.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.35.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.35.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.35.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.36.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.36.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.36.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.37.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.37.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.37.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.38.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.38.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.38.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.39.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.39.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.39.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.4.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.4.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.4.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.40.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.40.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.40.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.41.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.22.mlp.experts.41.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.41.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.42.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.42.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.42.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.43.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.43.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.43.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.44.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.44.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.44.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.45.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.45.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.45.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.46.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.46.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.46.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.47.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.47.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.47.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.48.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.48.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.48.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.49.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.22.mlp.experts.49.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.49.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.22.mlp.experts.5.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.5.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.5.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.50.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.50.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.50.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.51.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.51.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.51.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.52.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.52.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.52.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.53.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.53.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.53.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.54.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.54.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.54.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.22.mlp.experts.55.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.55.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.55.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.56.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.56.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.56.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.57.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.57.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.57.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.58.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.58.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.58.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.59.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.59.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.59.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.6.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.6.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.6.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.60.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.60.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.60.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.22.mlp.experts.61.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.61.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.61.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.62.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.62.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.62.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.63.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.63.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.63.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.64.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.64.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.64.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.65.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.65.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.65.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.66.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.66.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.66.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.67.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.67.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.67.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.68.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.68.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.68.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.69.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.69.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.69.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.7.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.7.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.7.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.70.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.70.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.70.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.71.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.71.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.71.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.72.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.72.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.72.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.73.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.73.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.73.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.74.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.74.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.74.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.75.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.75.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.75.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.76.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.76.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.76.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.77.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.77.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.77.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.78.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.78.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.78.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.79.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.79.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.22.mlp.experts.79.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.8.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.8.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.8.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.80.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.80.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.80.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.81.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.81.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.81.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.82.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.82.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.82.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.83.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.83.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.83.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.84.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.84.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.84.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.85.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.85.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.85.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.86.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.86.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.86.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.87.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.87.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.87.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.88.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.88.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.88.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.89.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.89.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.89.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.9.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.9.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.mlp.experts.9.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.22.mlp.experts.90.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.90.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.90.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.91.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.91.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.91.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.92.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.92.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.92.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.22.mlp.experts.93.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.93.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.93.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.94.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.94.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.22.mlp.experts.94.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.95.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.95.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.95.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.mlp.experts.96.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.mlp.experts.96.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.96.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.mlp.experts.97.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.97.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.mlp.experts.97.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.22.mlp.experts.98.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.22.mlp.experts.98.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.22.mlp.experts.98.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.99.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.experts.99.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.22.mlp.experts.99.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.22.mlp.gate.weight": "model-00005-of-00013.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00013-of-00013.safetensors",
+    "model.layers.22.self_attn.k_norm.weight": "model-00003-of-00013.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.22.self_attn.q_norm.weight": "model-00011-of-00013.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.0.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.0.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.0.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.1.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.1.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.1.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.10.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.10.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.10.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.100.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.100.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.100.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.101.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.101.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.101.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.102.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.102.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.102.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.103.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.103.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.103.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.104.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.104.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.104.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.105.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.105.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.105.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.106.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.106.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.106.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.107.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.107.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.107.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.108.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.108.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.108.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.109.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.109.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.109.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.11.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.11.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.11.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.110.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.110.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.110.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.111.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.111.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.111.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.112.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.112.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.112.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.113.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.113.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.113.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.114.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.114.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.114.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.115.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.115.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.115.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.116.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.116.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.116.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.117.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.117.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.117.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.118.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.118.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.118.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.119.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.119.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.119.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.12.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.12.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.12.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.120.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.120.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.120.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.121.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.121.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.121.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.122.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.122.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.122.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.123.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.123.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.123.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.124.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.124.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.124.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.125.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.125.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.125.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.126.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.126.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.126.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.127.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.127.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.127.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.13.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.13.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.13.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.14.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.14.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.14.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.15.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.15.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.15.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.16.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.16.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.16.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.17.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.17.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.17.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.18.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.18.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.18.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.19.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.19.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.19.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.2.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.2.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.2.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.20.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.20.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.20.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.21.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.21.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.21.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.22.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.22.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.22.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.23.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.23.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.23.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.24.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.24.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.24.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.25.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.25.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.25.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.26.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.26.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.26.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.27.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.27.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.27.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.28.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.28.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.28.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.29.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.29.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.29.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.3.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.3.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.3.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.30.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.30.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.30.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.31.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.31.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.31.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.32.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.32.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.32.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.33.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.33.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.33.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.34.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.34.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.34.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.35.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.35.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.35.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.36.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.36.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.36.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.37.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.37.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.37.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.38.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.38.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.38.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.39.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.39.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.39.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.4.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.4.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.4.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.40.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.40.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.40.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.41.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.41.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.41.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.42.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.42.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.42.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.43.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.43.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.43.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.44.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.44.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.44.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.45.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.45.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.45.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.46.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.46.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.46.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.47.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.47.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.47.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.48.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.48.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.48.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.49.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.49.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.49.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.5.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.5.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.5.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.50.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.50.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.50.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.51.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.51.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.51.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.52.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.52.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.52.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.53.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.53.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.53.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.54.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.54.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.54.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.55.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.55.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.55.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.56.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.56.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.56.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.57.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.57.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.57.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.58.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.58.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.58.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.59.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.59.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.59.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.6.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.6.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.6.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.60.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.60.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.60.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.61.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.61.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.61.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.62.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.62.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.62.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.63.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.63.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.63.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.64.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.64.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.64.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.65.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.65.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.65.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.66.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.66.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.66.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.67.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.67.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.67.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.68.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.68.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.68.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.69.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.69.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.69.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.7.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.7.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.7.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.70.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.70.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.70.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.71.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.71.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.71.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.72.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.72.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.72.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.73.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.73.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.73.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.74.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.74.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.74.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.75.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.75.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.75.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.76.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.76.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.76.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.77.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.77.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.77.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.78.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.78.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.78.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.79.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.79.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.79.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.8.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.8.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.8.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.80.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.experts.80.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.80.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.81.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.81.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.81.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.82.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.82.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.82.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.83.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.83.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.83.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.84.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.84.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.84.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.85.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.85.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.85.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.86.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.86.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.86.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.87.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.87.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.87.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.88.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.88.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.88.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.89.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.89.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.89.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.9.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.9.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.9.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.90.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.mlp.experts.90.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.90.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.91.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.23.mlp.experts.91.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.91.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.92.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.92.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.92.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.93.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.93.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.93.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.23.mlp.experts.94.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.23.mlp.experts.94.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.94.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.23.mlp.experts.95.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.95.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.95.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.96.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.mlp.experts.96.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.96.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.97.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.97.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.mlp.experts.97.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.98.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.23.mlp.experts.98.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.mlp.experts.98.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.23.mlp.experts.99.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.23.mlp.experts.99.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.mlp.experts.99.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.23.mlp.gate.weight": "model-00012-of-00013.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00005-of-00013.safetensors",
+    "model.layers.23.self_attn.k_norm.weight": "model-00006-of-00013.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.self_attn.q_norm.weight": "model-00002-of-00013.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.0.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.0.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.0.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.1.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.1.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.1.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.10.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.10.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.10.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.100.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.100.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.100.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.101.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.101.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.101.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.102.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.102.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.102.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.103.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.103.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.103.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.104.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.104.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.104.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.105.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.105.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.105.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.106.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.106.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.106.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.107.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.107.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.107.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.108.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.108.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.108.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.109.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.109.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.109.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.11.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.11.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.11.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.110.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.110.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.110.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.111.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.111.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.111.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.112.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.112.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.112.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.113.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.113.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.113.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.114.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.114.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.114.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.115.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.115.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.115.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.116.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.116.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.116.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.117.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.117.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.117.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.118.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.118.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.118.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.119.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.119.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.119.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.12.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.12.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.12.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.120.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.120.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.120.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.121.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.121.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.121.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.122.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.122.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.122.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.123.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.123.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.123.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.124.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.124.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.124.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.125.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.125.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.125.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.126.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.126.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.126.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.127.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.127.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.127.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.13.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.13.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.13.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.14.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.14.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.14.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.15.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.15.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.15.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.16.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.16.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.16.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.17.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.17.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.17.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.18.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.18.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.18.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.19.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.19.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.19.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.2.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.2.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.2.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.20.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.20.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.20.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.21.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.21.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.21.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.22.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.22.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.22.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.23.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.23.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.23.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.24.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.24.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.24.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.25.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.25.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.25.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.26.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.26.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.26.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.27.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.27.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.27.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.28.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.28.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.28.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.29.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.29.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.29.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.3.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.3.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.3.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.30.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.30.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.30.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.31.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.31.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.31.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.32.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.32.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.32.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.33.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.33.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.33.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.34.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.34.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.34.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.35.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.35.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.35.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.36.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.36.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.36.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.37.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.37.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.37.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.38.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.38.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.38.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.39.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.39.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.39.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.4.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.4.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.4.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.40.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.40.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.40.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.41.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.41.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.41.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.42.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.42.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.42.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.43.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.43.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.43.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.44.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.44.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.44.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.45.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.45.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.45.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.46.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.46.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.46.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.47.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.47.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.47.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.48.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.48.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.48.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.49.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.49.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.49.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.5.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.5.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.5.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.50.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.50.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.50.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.51.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.51.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.51.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.52.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.52.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.52.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.53.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.53.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.53.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.54.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.54.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.54.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.55.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.55.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.55.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.56.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.56.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.56.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.57.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.57.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.57.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.58.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.58.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.58.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.59.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.59.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.59.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.6.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.6.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.6.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.60.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.60.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.60.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.61.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.61.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.61.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.62.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.62.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.62.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.63.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.63.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.63.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.64.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.64.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.64.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.65.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.65.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.65.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.66.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.66.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.66.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.67.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.67.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.67.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.68.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.68.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.68.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.69.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.69.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.69.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.7.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.7.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.7.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.70.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.70.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.70.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.71.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.71.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.71.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.72.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.72.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.72.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.73.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.73.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.73.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.74.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.74.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.74.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.75.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.75.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.75.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.76.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.76.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.76.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.77.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.77.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.77.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.78.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.78.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.78.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.79.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.79.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.79.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.8.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.8.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.8.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.80.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.80.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.80.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.81.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.81.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.81.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.82.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.82.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.82.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.83.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.83.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.83.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.84.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.84.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.84.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.85.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.85.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.85.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.86.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.86.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.86.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.mlp.experts.87.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.87.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.87.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.mlp.experts.88.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.88.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.88.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.89.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.89.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.89.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.9.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.9.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.mlp.experts.9.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.24.mlp.experts.90.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.90.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.90.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.91.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.91.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.91.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.92.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.92.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.92.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.93.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.93.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.mlp.experts.93.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.94.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.94.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.94.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.95.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.mlp.experts.95.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.95.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.24.mlp.experts.96.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.96.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.24.mlp.experts.96.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.mlp.experts.97.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.24.mlp.experts.97.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.97.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.experts.98.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.98.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.98.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.99.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.24.mlp.experts.99.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.24.mlp.experts.99.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.24.mlp.gate.weight": "model-00005-of-00013.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00006-of-00013.safetensors",
+    "model.layers.24.self_attn.k_norm.weight": "model-00011-of-00013.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.self_attn.q_norm.weight": "model-00001-of-00013.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.0.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.0.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.0.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.1.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.1.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.1.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.10.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.10.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.10.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.100.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.100.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.100.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.101.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.101.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.101.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.102.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.102.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.102.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.103.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.103.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.103.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.104.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.104.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.104.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.105.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.105.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.105.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.106.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.106.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.106.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.107.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.107.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.107.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.108.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.108.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.108.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.109.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.109.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.109.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.11.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.11.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.11.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.110.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.110.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.110.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.111.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.111.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.111.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.112.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.112.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.112.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.113.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.113.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.113.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.114.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.114.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.114.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.115.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.115.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.115.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.116.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.116.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.116.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.117.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.117.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.117.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.118.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.118.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.118.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.119.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.119.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.119.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.12.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.12.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.12.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.120.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.120.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.120.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.121.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.121.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.121.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.122.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.122.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.122.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.123.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.123.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.123.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.124.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.124.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.124.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.125.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.125.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.125.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.126.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.126.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.126.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.127.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.127.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.127.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.13.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.13.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.13.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.14.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.14.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.14.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.15.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.15.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.15.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.16.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.16.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.16.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.17.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.17.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.17.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.18.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.18.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.18.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.19.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.19.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.19.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.2.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.2.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.2.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.20.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.20.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.20.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.21.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.21.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.21.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.22.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.22.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.22.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.23.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.23.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.23.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.24.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.24.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.24.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.25.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.25.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.25.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.26.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.26.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.26.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.27.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.27.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.27.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.28.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.28.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.28.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.29.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.29.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.29.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.3.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.3.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.3.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.30.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.30.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.30.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.31.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.31.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.31.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.32.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.32.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.32.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.33.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.33.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.33.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.34.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.34.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.34.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.35.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.35.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.35.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.36.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.36.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.36.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.37.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.37.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.37.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.38.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.38.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.38.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.39.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.39.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.39.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.4.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.4.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.4.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.40.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.40.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.40.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.41.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.41.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.41.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.42.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.42.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.42.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.43.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.43.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.43.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.44.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.44.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.44.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.45.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.45.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.45.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.46.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.46.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.46.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.47.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.47.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.47.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.48.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.48.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.48.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.49.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.49.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.49.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.5.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.5.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.5.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.50.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.50.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.50.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.51.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.51.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.51.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.52.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.52.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.52.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.53.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.53.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.53.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.54.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.54.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.54.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.55.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.55.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.55.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.56.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.56.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.56.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.57.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.57.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.57.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.58.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.58.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.58.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.59.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.59.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.59.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.6.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.6.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.6.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.60.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.60.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.60.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.61.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.61.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.61.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.62.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.62.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.62.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.63.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.63.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.63.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.64.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.64.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.64.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.65.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.65.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.65.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.66.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.66.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.66.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.67.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.67.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.67.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.68.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.68.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.68.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.69.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.69.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.69.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.7.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.7.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.7.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.70.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.70.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.70.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.71.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.71.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.71.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.72.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.72.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.72.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.73.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.73.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.73.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.74.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.74.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.74.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.75.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.75.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.75.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.76.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.76.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.76.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.77.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.77.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.77.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.78.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.78.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.78.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.79.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.79.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.79.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.8.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.8.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.8.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.80.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.80.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.80.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.81.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.81.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.81.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.82.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.82.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.82.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.83.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.83.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.83.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.84.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.84.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.84.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.85.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.85.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.85.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.86.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.86.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.86.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.87.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.87.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.87.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.88.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.88.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.88.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.25.mlp.experts.89.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.89.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.89.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.9.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.9.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.9.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.90.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.90.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.90.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.91.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.25.mlp.experts.91.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.91.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.92.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.92.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.25.mlp.experts.92.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.mlp.experts.93.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.93.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.93.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.mlp.experts.94.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.94.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.25.mlp.experts.94.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.95.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.95.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.95.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.96.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.96.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.experts.96.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.mlp.experts.97.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.mlp.experts.97.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.97.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.98.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.25.mlp.experts.98.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.25.mlp.experts.98.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.99.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.mlp.experts.99.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.mlp.experts.99.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.25.mlp.gate.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00009-of-00013.safetensors",
+    "model.layers.25.self_attn.k_norm.weight": "model-00008-of-00013.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.25.self_attn.q_norm.weight": "model-00004-of-00013.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.0.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.0.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.0.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.1.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.1.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.1.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.10.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.10.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.10.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.100.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.100.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.100.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.101.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.101.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.101.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.102.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.102.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.102.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.103.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.103.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.103.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.104.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.104.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.104.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.105.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.105.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.105.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.106.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.106.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.106.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.107.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.107.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.107.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.108.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.108.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.108.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.109.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.109.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.109.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.11.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.11.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.11.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.110.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.110.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.110.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.111.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.111.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.111.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.112.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.112.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.112.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.113.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.113.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.113.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.114.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.114.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.114.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.115.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.115.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.115.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.116.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.116.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.116.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.117.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.117.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.117.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.118.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.118.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.118.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.119.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.119.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.119.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.12.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.12.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.12.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.120.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.120.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.120.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.121.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.121.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.121.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.122.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.122.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.122.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.123.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.123.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.123.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.124.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.124.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.124.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.125.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.125.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.125.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.126.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.126.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.126.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.127.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.127.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.127.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.13.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.13.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.13.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.14.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.14.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.14.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.15.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.15.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.15.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.16.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.16.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.16.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.17.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.17.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.17.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.18.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.18.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.18.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.19.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.19.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.19.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.2.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.2.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.2.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.20.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.20.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.20.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.21.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.21.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.21.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.22.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.22.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.22.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.23.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.23.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.23.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.24.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.24.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.24.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.25.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.25.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.25.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.26.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.26.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.26.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.27.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.27.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.27.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.28.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.28.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.28.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.29.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.29.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.29.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.3.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.3.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.3.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.30.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.30.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.30.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.31.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.31.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.31.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.32.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.32.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.32.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.33.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.33.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.33.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.34.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.34.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.34.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.35.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.35.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.35.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.36.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.36.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.36.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.37.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.37.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.37.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.38.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.38.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.38.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.39.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.39.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.39.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.4.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.4.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.4.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.40.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.40.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.40.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.41.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.41.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.41.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.42.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.42.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.42.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.43.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.43.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.43.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.44.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.44.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.44.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.45.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.45.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.45.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.46.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.46.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.46.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.47.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.47.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.47.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.48.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.48.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.48.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.49.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.49.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.49.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.5.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.5.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.5.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.50.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.50.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.50.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.51.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.51.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.51.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.52.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.52.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.52.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.53.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.53.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.53.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.54.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.54.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.54.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.55.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.55.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.55.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.56.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.56.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.56.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.57.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.57.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.57.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.58.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.58.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.58.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.59.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.59.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.59.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.6.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.6.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.6.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.60.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.60.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.60.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.61.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.61.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.61.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.62.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.62.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.62.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.63.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.63.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.63.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.64.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.64.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.64.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.65.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.65.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.65.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.66.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.66.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.66.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.67.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.67.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.67.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.68.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.68.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.68.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.69.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.69.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.69.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.7.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.7.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.7.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.70.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.70.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.70.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.71.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.71.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.71.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.72.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.72.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.72.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.73.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.73.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.73.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.74.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.74.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.74.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.75.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.75.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.75.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.76.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.76.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.76.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.77.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.77.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.77.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.78.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.78.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.78.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.79.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.79.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.79.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.8.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.8.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.8.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.80.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.80.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.80.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.81.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.81.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.81.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.82.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.82.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.82.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.83.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.83.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.83.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.84.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.84.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.84.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.85.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.85.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.85.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.86.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.86.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.86.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.87.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.87.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.87.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.88.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.88.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.88.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.89.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.26.mlp.experts.89.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.89.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.mlp.experts.9.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.9.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.26.mlp.experts.9.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.90.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.90.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.90.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.91.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.91.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.91.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.92.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.92.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.92.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.93.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.experts.93.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.93.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.94.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.94.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.26.mlp.experts.94.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.95.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.95.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.95.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.96.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.mlp.experts.96.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.mlp.experts.96.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.26.mlp.experts.97.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.26.mlp.experts.97.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.97.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.26.mlp.experts.98.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.98.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.98.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.mlp.experts.99.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.mlp.experts.99.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.26.mlp.experts.99.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.mlp.gate.weight": "model-00012-of-00013.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00006-of-00013.safetensors",
+    "model.layers.26.self_attn.k_norm.weight": "model-00004-of-00013.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.26.self_attn.q_norm.weight": "model-00007-of-00013.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.0.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.0.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.0.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.1.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.1.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.1.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.10.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.10.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.10.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.100.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.100.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.100.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.101.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.101.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.101.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.102.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.102.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.102.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.103.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.103.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.103.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.104.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.104.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.104.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.105.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.105.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.105.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.106.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.106.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.106.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.107.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.107.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.107.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.108.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.108.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.108.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.109.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.109.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.109.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.11.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.11.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.11.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.110.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.110.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.110.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.111.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.111.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.111.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.112.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.112.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.112.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.113.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.113.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.113.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.114.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.114.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.114.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.115.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.115.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.115.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.116.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.116.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.116.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.117.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.117.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.117.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.118.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.118.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.118.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.119.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.119.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.119.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.12.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.12.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.12.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.120.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.120.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.120.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.121.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.121.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.121.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.122.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.122.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.122.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.123.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.123.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.123.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.124.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.124.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.124.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.125.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.125.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.125.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.126.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.126.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.126.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.127.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.127.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.127.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.13.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.13.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.13.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.14.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.14.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.14.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.15.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.15.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.15.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.16.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.16.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.16.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.17.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.17.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.17.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.18.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.18.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.18.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.19.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.19.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.19.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.2.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.2.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.2.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.20.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.20.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.20.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.21.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.21.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.21.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.22.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.22.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.22.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.23.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.23.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.23.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.24.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.24.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.24.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.25.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.25.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.25.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.26.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.26.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.26.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.27.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.27.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.27.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.28.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.28.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.28.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.29.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.29.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.29.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.3.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.3.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.3.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.30.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.30.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.30.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.31.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.31.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.31.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.32.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.32.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.32.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.33.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.33.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.33.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.34.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.34.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.34.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.35.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.35.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.35.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.36.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.36.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.36.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.37.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.37.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.37.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.38.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.38.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.38.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.39.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.39.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.39.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.4.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.4.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.4.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.40.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.40.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.40.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.41.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.41.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.41.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.42.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.42.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.42.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.43.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.43.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.43.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.44.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.44.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.44.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.45.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.45.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.45.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.46.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.46.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.46.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.47.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.47.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.47.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.48.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.48.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.48.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.49.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.49.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.49.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.5.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.5.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.5.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.50.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.50.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.50.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.51.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.51.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.51.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.52.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.52.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.52.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.53.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.53.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.53.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.54.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.54.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.54.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.55.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.55.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.55.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.56.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.56.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.56.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.57.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.57.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.57.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.58.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.58.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.58.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.59.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.59.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.59.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.6.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.6.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.6.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.60.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.60.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.60.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.61.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.61.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.61.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.62.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.62.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.62.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.63.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.63.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.63.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.64.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.64.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.64.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.65.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.65.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.65.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.66.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.66.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.66.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.67.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.67.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.67.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.68.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.68.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.68.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.69.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.69.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.69.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.7.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.7.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.7.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.70.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.70.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.70.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.71.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.71.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.71.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.72.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.72.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.72.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.73.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.73.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.73.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.74.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.74.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.74.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.75.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.75.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.75.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.76.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.76.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.76.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.77.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.77.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.77.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.78.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.78.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.78.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.79.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.79.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.79.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.8.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.8.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.8.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.80.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.80.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.80.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.81.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.81.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.81.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.82.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.82.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.82.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.83.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.83.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.83.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.84.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.84.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.84.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.85.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.85.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.85.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.86.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.86.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.86.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.87.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.87.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.87.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.88.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.88.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.27.mlp.experts.88.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.89.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.27.mlp.experts.89.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.89.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.mlp.experts.9.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.9.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.9.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.90.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.90.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.90.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.91.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.91.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.91.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.92.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.92.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.92.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.93.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.93.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.93.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.94.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.94.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.27.mlp.experts.94.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.95.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.95.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.95.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.96.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.96.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.mlp.experts.96.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.27.mlp.experts.97.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.mlp.experts.97.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.27.mlp.experts.97.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.mlp.experts.98.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.mlp.experts.98.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.experts.98.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.99.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.mlp.experts.99.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.mlp.experts.99.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.27.mlp.gate.weight": "model-00007-of-00013.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00005-of-00013.safetensors",
+    "model.layers.27.self_attn.k_norm.weight": "model-00006-of-00013.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.27.self_attn.q_norm.weight": "model-00009-of-00013.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.0.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.0.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.0.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.1.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.1.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.1.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.10.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.10.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.10.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.100.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.100.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.100.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.101.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.101.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.101.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.102.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.102.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.102.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.103.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.103.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.103.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.104.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.104.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.104.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.105.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.105.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.105.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.106.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.106.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.106.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.107.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.107.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.107.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.108.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.108.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.108.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.109.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.109.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.109.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.11.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.11.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.11.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.110.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.110.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.110.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.111.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.111.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.111.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.112.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.112.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.112.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.113.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.113.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.113.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.114.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.114.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.114.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.115.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.115.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.115.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.116.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.116.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.116.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.117.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.117.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.117.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.118.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.118.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.118.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.119.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.119.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.119.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.12.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.12.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.12.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.120.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.120.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.120.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.121.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.121.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.121.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.122.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.122.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.122.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.123.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.123.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.123.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.124.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.124.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.124.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.125.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.125.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.125.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.126.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.126.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.126.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.127.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.127.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.127.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.13.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.13.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.13.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.14.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.14.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.14.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.15.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.15.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.15.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.16.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.16.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.16.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.17.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.17.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.17.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.18.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.18.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.18.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.19.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.19.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.19.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.2.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.2.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.2.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.20.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.20.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.20.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.21.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.21.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.21.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.22.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.22.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.22.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.23.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.23.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.23.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.24.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.24.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.24.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.25.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.25.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.25.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.26.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.26.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.26.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.27.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.27.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.27.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.28.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.28.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.28.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.29.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.29.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.29.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.3.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.3.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.3.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.30.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.30.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.30.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.31.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.31.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.31.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.32.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.32.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.32.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.33.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.33.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.33.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.34.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.34.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.34.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.35.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.35.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.35.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.36.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.36.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.36.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.37.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.37.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.37.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.38.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.38.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.38.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.39.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.39.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.39.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.4.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.4.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.4.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.40.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.40.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.40.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.41.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.41.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.41.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.42.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.42.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.42.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.43.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.43.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.43.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.44.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.44.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.44.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.45.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.45.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.45.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.46.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.46.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.46.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.47.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.47.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.47.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.48.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.48.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.48.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.49.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.49.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.49.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.5.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.5.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.5.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.50.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.50.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.50.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.51.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.51.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.51.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.52.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.52.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.52.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.53.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.53.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.53.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.54.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.54.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.54.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.55.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.55.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.55.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.56.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.56.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.56.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.57.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.57.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.57.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.58.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.58.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.58.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.59.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.59.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.59.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.6.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.6.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.6.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.60.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.60.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.60.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.61.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.61.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.61.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.62.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.62.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.62.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.63.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.63.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.63.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.64.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.64.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.64.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.65.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.65.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.65.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.66.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.66.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.66.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.67.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.67.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.67.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.68.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.68.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.68.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.69.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.69.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.69.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.7.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.7.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.7.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.70.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.70.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.70.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.71.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.71.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.71.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.72.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.72.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.72.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.73.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.73.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.73.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.74.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.74.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.74.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.75.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.75.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.75.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.76.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.76.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.76.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.77.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.77.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.77.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.78.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.78.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.78.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.79.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.79.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.79.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.8.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.8.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.8.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.80.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.80.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.80.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.81.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.81.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.81.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.82.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.82.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.82.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.83.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.83.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.83.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.84.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.84.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.84.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.85.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.85.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.85.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.86.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.86.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.86.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.87.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.87.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.87.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.88.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.88.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.88.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.89.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.89.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.89.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.28.mlp.experts.9.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.9.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.9.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.90.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.90.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.90.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.91.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.91.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.91.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.92.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.92.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.92.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.28.mlp.experts.93.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.93.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.experts.93.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.94.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.94.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.94.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.mlp.experts.95.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.28.mlp.experts.95.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.mlp.experts.95.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.96.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.96.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.96.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.97.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.97.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.mlp.experts.97.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.mlp.experts.98.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.28.mlp.experts.98.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.28.mlp.experts.98.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.mlp.experts.99.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.28.mlp.experts.99.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.28.mlp.experts.99.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.mlp.gate.weight": "model-00009-of-00013.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.self_attn.k_norm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.28.self_attn.q_norm.weight": "model-00011-of-00013.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.0.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.0.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.0.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.1.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.1.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.1.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.10.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.10.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.10.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.100.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.100.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.100.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.101.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.101.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.101.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.102.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.102.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.102.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.103.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.103.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.103.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.104.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.104.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.104.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.105.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.105.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.105.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.106.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.106.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.106.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.107.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.107.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.107.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.108.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.108.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.108.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.109.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.109.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.109.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.11.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.11.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.11.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.110.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.110.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.110.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.111.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.111.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.111.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.112.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.112.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.112.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.113.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.113.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.113.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.114.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.114.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.114.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.115.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.115.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.115.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.116.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.116.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.116.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.117.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.117.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.117.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.118.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.118.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.118.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.119.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.119.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.119.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.12.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.12.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.12.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.120.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.120.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.120.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.121.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.121.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.121.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.122.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.122.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.122.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.123.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.123.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.123.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.124.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.124.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.124.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.125.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.125.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.125.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.126.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.126.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.126.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.127.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.127.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.127.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.13.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.13.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.13.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.14.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.14.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.14.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.15.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.15.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.15.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.16.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.16.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.16.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.17.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.17.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.17.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.18.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.18.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.18.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.19.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.19.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.19.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.2.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.2.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.2.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.20.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.20.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.20.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.21.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.21.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.21.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.22.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.22.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.22.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.23.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.23.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.23.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.24.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.24.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.24.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.25.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.25.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.25.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.26.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.26.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.26.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.27.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.27.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.27.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.28.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.28.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.28.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.29.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.29.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.29.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.3.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.3.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.3.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.30.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.30.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.30.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.31.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.31.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.31.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.32.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.32.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.32.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.33.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.33.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.33.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.34.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.34.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.34.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.35.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.35.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.35.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.36.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.36.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.36.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.37.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.37.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.37.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.38.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.38.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.38.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.39.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.39.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.39.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.4.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.4.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.4.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.40.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.40.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.40.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.41.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.41.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.41.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.42.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.42.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.42.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.43.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.43.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.43.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.44.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.44.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.44.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.45.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.45.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.45.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.46.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.46.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.46.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.47.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.47.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.47.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.48.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.48.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.48.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.49.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.49.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.49.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.5.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.5.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.5.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.50.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.50.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.50.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.51.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.51.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.51.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.52.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.52.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.52.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.53.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.53.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.53.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.54.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.54.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.54.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.55.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.55.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.55.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.56.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.56.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.56.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.57.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.57.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.57.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.58.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.58.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.58.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.59.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.59.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.59.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.6.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.6.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.6.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.60.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.60.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.60.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.61.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.61.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.61.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.62.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.62.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.62.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.63.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.63.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.63.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.64.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.64.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.64.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.65.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.65.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.65.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.66.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.66.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.66.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.67.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.67.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.67.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.68.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.68.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.68.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.69.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.69.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.69.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.7.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.7.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.7.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.70.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.70.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.70.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.71.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.71.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.71.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.72.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.72.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.72.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.73.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.73.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.73.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.74.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.74.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.74.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.75.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.75.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.75.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.76.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.76.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.76.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.77.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.77.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.77.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.78.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.78.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.78.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.79.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.79.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.79.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.8.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.8.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.8.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.80.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.80.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.80.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.81.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.29.mlp.experts.81.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.81.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.82.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.82.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.82.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.83.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.83.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.83.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.84.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.84.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.84.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.85.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.85.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.85.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.86.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.86.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.86.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.87.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.87.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.87.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.88.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.88.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.88.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.89.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.89.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.89.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.9.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.9.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.9.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.90.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.90.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.90.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.91.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.91.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.91.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.29.mlp.experts.92.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.92.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.29.mlp.experts.92.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.93.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.93.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.93.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.94.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.94.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.mlp.experts.94.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.95.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.95.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.95.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.96.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.96.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.29.mlp.experts.96.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.mlp.experts.97.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.experts.97.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.29.mlp.experts.97.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.29.mlp.experts.98.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.29.mlp.experts.98.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.29.mlp.experts.98.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.99.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.mlp.experts.99.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.mlp.experts.99.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.mlp.gate.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.self_attn.k_norm.weight": "model-00012-of-00013.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.29.self_attn.q_norm.weight": "model-00009-of-00013.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.0.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.0.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.0.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.1.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.1.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.1.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.10.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.10.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.10.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.100.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.100.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.100.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.101.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.101.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.101.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.102.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.102.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.102.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.103.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.103.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.103.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.104.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.104.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.104.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.105.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.105.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.105.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.106.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.106.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.106.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.107.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.107.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.107.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.108.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.108.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.108.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.109.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.109.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.109.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.11.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.11.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.11.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.110.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.110.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.110.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.111.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.111.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.111.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.112.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.112.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.112.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.113.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.113.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.113.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.114.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.114.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.114.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.115.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.115.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.115.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.116.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.116.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.116.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.117.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.117.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.117.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.118.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.118.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.118.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.119.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.119.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.119.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.12.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.12.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.12.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.120.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.120.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.120.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.121.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.121.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.121.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.122.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.122.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.122.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.123.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.123.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.123.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.124.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.124.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.124.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.125.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.125.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.125.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.126.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.126.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.126.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.127.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.127.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.127.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.13.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.13.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.13.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.14.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.14.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.14.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.15.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.15.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.15.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.16.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.16.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.16.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.17.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.17.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.17.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.18.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.18.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.18.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.19.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.19.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.19.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.2.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.2.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.2.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.20.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.20.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.20.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.21.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.21.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.21.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.22.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.22.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.22.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.23.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.23.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.23.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.24.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.24.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.24.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.25.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.25.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.25.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.26.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.26.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.26.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.27.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.27.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.27.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.28.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.28.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.28.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.29.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.29.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.29.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.3.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.3.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.3.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.30.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.30.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.30.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.31.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.31.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.31.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.32.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.32.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.32.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.33.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.33.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.33.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.34.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.34.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.34.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.35.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.35.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.35.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.36.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.36.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.36.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.37.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.37.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.37.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.38.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.38.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.38.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.39.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.39.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.39.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.4.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.4.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.4.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.40.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.40.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.40.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.41.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.41.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.41.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.42.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.42.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.42.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.43.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.43.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.43.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.44.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.44.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.44.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.45.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.45.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.45.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.46.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.46.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.46.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.47.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.47.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.47.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.48.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.48.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.48.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.49.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.49.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.49.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.5.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.5.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.5.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.50.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.50.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.50.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.51.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.51.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.51.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.52.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.52.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.52.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.53.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.53.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.53.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.54.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.54.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.54.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.55.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.55.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.55.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.56.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.56.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.56.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.57.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.57.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.57.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.58.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.58.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.58.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.59.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.59.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.59.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.6.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.6.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.6.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.60.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.60.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.60.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.61.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.61.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.61.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.62.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.62.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.62.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.63.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.63.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.63.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.64.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.64.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.64.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.65.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.65.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.65.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.66.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.66.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.66.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.67.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.67.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.67.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.68.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.68.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.68.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.69.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.69.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.69.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.7.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.7.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.7.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.70.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.70.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.70.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.71.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.71.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.71.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.72.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.72.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.72.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.73.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.73.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.73.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.74.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.74.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.74.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.75.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.75.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.75.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.76.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.76.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.76.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.77.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.77.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.77.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.78.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.78.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.78.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.79.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.79.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.79.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.8.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.8.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.8.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.80.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.80.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.80.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.81.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.81.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.81.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.82.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.82.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.82.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.83.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.3.mlp.experts.83.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.83.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.84.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.84.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.84.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.85.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.85.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.85.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.86.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.86.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.86.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.87.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.87.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.87.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.88.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.88.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.88.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.89.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.89.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.89.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.9.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.9.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.9.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.90.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.90.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.90.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.91.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.91.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.91.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.92.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.92.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.92.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.93.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.93.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.93.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.mlp.experts.94.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.94.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.94.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.95.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.3.mlp.experts.95.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.95.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.mlp.experts.96.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.3.mlp.experts.96.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.3.mlp.experts.96.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.97.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.mlp.experts.97.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.experts.97.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.98.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.3.mlp.experts.98.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.3.mlp.experts.98.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.mlp.experts.99.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.3.mlp.experts.99.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.mlp.experts.99.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.3.mlp.gate.weight": "model-00006-of-00013.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.self_attn.k_norm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.self_attn.q_norm.weight": "model-00007-of-00013.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.0.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.0.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.0.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.1.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.1.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.1.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.10.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.10.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.10.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.100.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.100.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.100.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.101.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.101.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.101.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.102.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.102.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.102.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.103.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.103.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.103.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.104.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.104.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.104.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.105.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.105.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.105.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.106.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.106.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.106.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.107.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.107.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.107.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.108.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.108.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.108.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.109.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.109.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.109.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.11.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.11.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.11.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.110.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.110.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.110.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.111.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.111.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.111.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.112.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.112.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.112.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.113.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.113.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.113.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.114.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.114.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.114.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.115.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.115.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.115.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.116.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.116.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.116.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.117.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.117.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.117.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.118.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.118.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.118.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.119.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.119.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.119.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.12.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.12.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.12.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.120.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.120.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.120.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.121.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.121.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.121.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.122.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.122.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.122.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.123.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.123.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.123.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.124.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.124.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.124.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.125.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.125.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.125.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.126.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.126.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.126.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.127.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.127.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.127.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.13.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.13.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.13.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.14.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.14.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.14.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.15.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.15.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.15.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.16.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.16.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.16.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.17.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.17.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.17.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.18.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.18.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.18.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.19.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.19.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.19.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.2.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.2.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.2.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.20.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.20.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.20.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.21.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.21.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.21.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.22.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.22.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.22.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.23.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.23.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.23.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.24.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.24.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.24.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.25.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.25.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.25.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.26.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.26.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.26.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.27.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.27.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.27.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.28.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.28.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.28.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.29.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.29.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.29.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.3.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.3.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.3.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.30.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.30.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.30.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.31.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.31.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.31.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.32.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.32.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.32.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.33.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.33.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.33.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.34.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.34.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.34.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.35.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.35.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.35.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.36.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.36.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.36.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.37.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.37.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.37.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.38.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.38.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.38.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.39.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.39.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.39.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.4.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.4.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.4.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.40.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.40.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.40.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.41.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.41.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.41.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.42.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.42.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.42.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.43.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.43.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.43.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.44.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.44.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.44.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.45.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.45.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.45.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.46.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.46.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.46.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.47.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.47.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.47.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.48.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.48.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.48.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.49.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.49.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.49.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.5.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.5.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.5.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.50.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.50.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.50.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.51.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.51.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.51.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.52.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.52.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.52.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.53.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.53.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.53.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.54.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.54.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.54.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.55.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.55.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.55.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.56.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.56.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.56.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.57.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.57.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.57.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.58.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.58.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.58.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.59.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.59.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.59.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.6.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.6.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.6.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.60.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.60.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.60.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.61.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.61.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.61.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.62.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.62.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.62.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.63.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.63.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.63.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.64.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.64.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.64.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.65.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.65.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.65.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.66.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.66.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.66.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.67.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.67.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.67.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.68.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.68.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.68.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.69.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.69.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.69.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.7.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.7.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.7.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.70.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.70.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.70.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.71.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.71.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.71.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.72.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.72.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.72.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.73.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.73.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.73.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.74.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.74.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.74.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.75.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.75.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.75.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.76.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.76.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.76.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.77.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.77.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.77.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.78.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.78.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.78.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.79.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.79.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.79.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.8.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.8.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.8.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.80.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.80.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.80.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.81.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.81.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.81.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.82.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.82.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.82.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.83.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.83.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.83.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.84.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.84.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.84.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.85.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.85.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.85.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.86.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.86.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.86.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.87.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.87.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.87.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.88.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.88.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.88.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.89.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.89.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.89.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.9.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.9.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.9.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.90.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.90.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.90.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.91.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.91.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.91.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.92.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.92.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.92.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.93.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.93.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.93.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.94.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.30.mlp.experts.94.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.30.mlp.experts.94.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.30.mlp.experts.95.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.mlp.experts.95.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.95.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.30.mlp.experts.96.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.30.mlp.experts.96.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.96.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.30.mlp.experts.97.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.97.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.97.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.mlp.experts.98.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.mlp.experts.98.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.mlp.experts.98.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.mlp.experts.99.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.experts.99.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.30.mlp.experts.99.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.30.mlp.gate.weight": "model-00003-of-00013.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.self_attn.k_norm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.30.self_attn.q_norm.weight": "model-00008-of-00013.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.0.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.0.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.0.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.1.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.1.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.1.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.10.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.10.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.10.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.100.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.100.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.100.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.101.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.101.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.101.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.102.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.102.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.102.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.103.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.103.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.103.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.104.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.104.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.104.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.105.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.105.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.105.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.106.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.106.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.106.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.107.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.107.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.107.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.108.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.108.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.108.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.109.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.109.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.109.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.11.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.11.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.11.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.110.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.110.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.110.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.111.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.111.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.111.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.112.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.112.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.112.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.113.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.113.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.113.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.114.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.114.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.114.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.115.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.115.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.115.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.116.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.116.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.116.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.117.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.117.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.117.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.118.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.118.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.118.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.119.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.119.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.119.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.12.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.12.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.12.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.120.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.120.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.120.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.121.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.121.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.121.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.122.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.122.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.122.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.123.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.123.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.123.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.124.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.124.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.124.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.125.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.125.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.125.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.126.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.126.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.126.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.127.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.127.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.127.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.13.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.13.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.13.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.14.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.14.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.14.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.15.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.15.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.15.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.16.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.16.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.16.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.17.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.17.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.17.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.18.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.18.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.18.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.19.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.19.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.19.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.2.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.2.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.2.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.20.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.20.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.20.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.21.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.21.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.21.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.22.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.22.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.22.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.23.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.23.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.23.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.24.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.24.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.24.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.25.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.25.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.25.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.26.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.26.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.26.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.27.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.27.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.27.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.28.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.28.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.28.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.29.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.29.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.29.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.3.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.3.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.3.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.30.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.30.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.30.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.31.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.31.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.31.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.32.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.32.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.32.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.33.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.33.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.33.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.34.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.34.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.34.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.35.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.35.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.35.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.36.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.36.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.36.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.37.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.37.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.37.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.38.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.38.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.38.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.39.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.39.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.39.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.4.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.4.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.4.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.40.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.40.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.40.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.41.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.41.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.41.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.42.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.42.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.42.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.43.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.43.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.43.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.44.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.44.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.44.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.45.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.45.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.45.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.46.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.46.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.46.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.47.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.47.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.47.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.48.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.48.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.48.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.49.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.49.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.49.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.5.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.5.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.5.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.50.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.50.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.50.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.51.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.51.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.51.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.52.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.52.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.52.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.53.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.53.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.53.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.54.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.54.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.54.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.55.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.55.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.55.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.56.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.56.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.56.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.57.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.57.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.57.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.58.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.58.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.58.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.59.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.59.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.59.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.6.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.6.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.6.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.60.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.60.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.60.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.61.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.61.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.61.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.62.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.62.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.62.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.63.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.63.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.63.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.64.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.64.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.64.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.65.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.65.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.65.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.66.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.66.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.66.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.67.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.67.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.67.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.68.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.68.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.68.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.69.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.69.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.69.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.7.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.7.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.7.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.70.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.70.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.70.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.71.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.71.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.71.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.72.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.72.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.72.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.73.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.73.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.73.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.74.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.74.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.74.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.75.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.75.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.75.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.76.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.76.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.76.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.77.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.77.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.77.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.78.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.78.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.78.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.79.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.79.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.79.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.8.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.8.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.8.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.80.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.80.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.80.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.81.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.81.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.81.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.82.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.82.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.82.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.83.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.83.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.83.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.84.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.84.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.84.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.31.mlp.experts.85.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.85.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.85.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.86.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.86.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.86.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.87.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.87.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.mlp.experts.87.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.88.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.88.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.88.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.mlp.experts.89.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.89.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.89.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.9.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.9.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.9.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.90.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.90.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.90.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.91.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.91.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.91.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.92.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.mlp.experts.92.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.92.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.mlp.experts.93.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.93.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.93.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.94.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.94.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.94.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.95.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.95.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.31.mlp.experts.95.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.96.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.31.mlp.experts.96.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.31.mlp.experts.96.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.97.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.31.mlp.experts.97.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.97.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.98.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.98.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.experts.98.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.31.mlp.experts.99.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.mlp.experts.99.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.mlp.experts.99.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.mlp.gate.weight": "model-00002-of-00013.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00006-of-00013.safetensors",
+    "model.layers.31.self_attn.k_norm.weight": "model-00009-of-00013.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.31.self_attn.q_norm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.0.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.0.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.0.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.1.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.1.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.1.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.10.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.10.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.10.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.100.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.100.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.100.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.101.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.101.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.101.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.102.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.102.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.102.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.103.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.103.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.103.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.104.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.104.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.104.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.105.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.105.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.105.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.106.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.106.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.106.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.107.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.107.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.107.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.108.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.108.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.108.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.109.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.109.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.109.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.11.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.11.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.11.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.110.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.110.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.110.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.111.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.111.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.111.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.112.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.112.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.112.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.113.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.113.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.113.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.114.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.114.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.114.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.115.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.115.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.115.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.116.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.116.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.116.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.117.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.117.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.117.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.118.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.118.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.118.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.119.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.119.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.119.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.12.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.12.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.12.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.120.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.120.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.120.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.121.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.121.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.121.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.122.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.122.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.122.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.123.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.123.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.123.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.124.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.124.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.124.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.125.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.125.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.125.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.126.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.126.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.126.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.127.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.127.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.127.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.13.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.13.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.13.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.14.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.14.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.14.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.15.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.15.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.15.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.16.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.16.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.16.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.17.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.17.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.17.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.18.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.18.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.18.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.19.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.19.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.19.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.2.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.2.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.2.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.20.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.20.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.20.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.21.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.21.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.21.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.22.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.22.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.22.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.23.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.23.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.23.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.24.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.24.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.24.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.25.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.25.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.25.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.26.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.26.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.26.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.27.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.27.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.27.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.28.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.28.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.28.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.29.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.29.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.29.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.3.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.3.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.3.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.30.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.30.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.30.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.31.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.31.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.31.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.32.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.32.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.32.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.33.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.33.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.33.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.34.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.34.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.34.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.35.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.35.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.35.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.36.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.36.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.36.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.37.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.37.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.37.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.38.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.38.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.38.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.39.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.39.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.39.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.4.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.4.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.4.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.40.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.40.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.40.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.41.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.41.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.41.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.42.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.42.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.42.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.43.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.43.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.43.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.44.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.44.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.44.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.45.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.45.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.45.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.46.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.46.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.46.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.47.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.47.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.47.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.48.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.48.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.48.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.49.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.49.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.49.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.5.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.5.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.5.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.50.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.50.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.50.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.51.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.51.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.51.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.52.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.52.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.52.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.53.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.53.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.53.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.54.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.54.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.54.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.55.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.55.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.55.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.56.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.56.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.56.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.57.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.57.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.57.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.58.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.58.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.58.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.59.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.59.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.59.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.6.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.6.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.6.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.60.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.60.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.60.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.61.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.61.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.61.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.62.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.62.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.62.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.63.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.63.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.63.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.64.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.64.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.64.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.65.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.65.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.65.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.66.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.66.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.66.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.67.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.67.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.67.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.68.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.68.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.68.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.69.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.69.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.69.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.7.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.7.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.7.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.70.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.70.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.70.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.71.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.71.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.71.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.72.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.72.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.72.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.73.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.73.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.73.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.74.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.74.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.74.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.75.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.75.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.75.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.76.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.76.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.76.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.77.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.77.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.77.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.78.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.78.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.78.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.79.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.79.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.79.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.8.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.8.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.8.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.80.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.80.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.80.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.81.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.81.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.81.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.82.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.82.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.82.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.experts.83.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.83.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.83.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.84.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.84.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.84.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.85.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.85.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.85.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.86.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.86.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.86.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.87.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.87.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.87.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.88.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.88.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.88.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.89.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.4.mlp.experts.89.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.89.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.9.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.9.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.9.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.90.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.90.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.90.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.91.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.4.mlp.experts.91.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.91.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.92.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.92.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.92.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.93.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.93.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.93.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.94.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.94.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.94.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.mlp.experts.95.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.95.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.mlp.experts.95.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.96.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.4.mlp.experts.96.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.96.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.4.mlp.experts.97.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.mlp.experts.97.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.mlp.experts.97.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.98.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.4.mlp.experts.98.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.98.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.4.mlp.experts.99.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.4.mlp.experts.99.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.mlp.experts.99.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.mlp.gate.weight": "model-00011-of-00013.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00003-of-00013.safetensors",
+    "model.layers.4.self_attn.k_norm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.4.self_attn.q_norm.weight": "model-00009-of-00013.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.0.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.0.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.0.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.1.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.1.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.1.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.10.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.10.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.10.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.100.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.100.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.100.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.101.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.101.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.101.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.102.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.102.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.102.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.103.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.103.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.103.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.104.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.104.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.104.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.105.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.105.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.105.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.106.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.106.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.106.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.107.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.107.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.107.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.108.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.108.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.108.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.109.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.109.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.109.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.11.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.11.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.11.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.110.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.110.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.110.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.111.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.111.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.111.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.112.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.112.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.112.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.113.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.113.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.113.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.114.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.114.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.114.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.115.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.115.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.115.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.116.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.116.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.116.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.117.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.117.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.117.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.118.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.118.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.118.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.119.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.119.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.119.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.12.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.12.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.12.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.120.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.120.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.120.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.121.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.121.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.121.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.122.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.122.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.122.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.123.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.123.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.123.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.124.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.124.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.124.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.125.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.125.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.125.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.126.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.126.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.126.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.127.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.127.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.127.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.13.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.13.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.13.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.14.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.14.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.14.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.15.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.15.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.15.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.16.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.16.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.16.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.17.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.17.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.17.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.18.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.18.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.18.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.19.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.19.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.19.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.2.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.2.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.2.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.20.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.20.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.20.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.21.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.21.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.21.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.22.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.22.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.22.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.23.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.23.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.23.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.24.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.24.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.24.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.25.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.25.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.25.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.26.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.26.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.26.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.27.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.27.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.27.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.28.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.28.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.28.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.29.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.29.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.29.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.3.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.3.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.3.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.30.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.30.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.30.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.31.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.31.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.31.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.32.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.32.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.32.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.33.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.33.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.33.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.34.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.34.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.34.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.35.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.35.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.35.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.36.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.36.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.36.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.37.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.37.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.37.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.38.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.38.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.38.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.39.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.39.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.39.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.4.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.4.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.4.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.40.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.40.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.40.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.41.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.41.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.41.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.42.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.42.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.42.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.43.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.43.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.43.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.44.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.44.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.44.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.45.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.45.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.45.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.46.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.46.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.46.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.47.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.47.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.47.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.48.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.48.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.48.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.49.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.49.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.49.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.5.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.5.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.5.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.50.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.50.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.50.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.51.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.51.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.51.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.52.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.52.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.52.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.53.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.53.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.53.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.54.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.54.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.54.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.55.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.55.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.55.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.56.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.56.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.56.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.57.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.57.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.57.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.58.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.58.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.58.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.59.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.59.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.59.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.6.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.6.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.6.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.60.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.60.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.60.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.61.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.61.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.61.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.62.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.62.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.62.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.63.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.63.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.63.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.64.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.64.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.64.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.65.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.65.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.65.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.66.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.66.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.66.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.67.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.67.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.67.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.68.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.68.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.68.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.69.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.69.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.69.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.7.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.7.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.7.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.70.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.70.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.70.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.71.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.71.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.71.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.72.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.72.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.72.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.73.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.73.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.73.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.74.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.74.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.74.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.75.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.75.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.75.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.76.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.76.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.76.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.77.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.77.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.77.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.78.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.78.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.78.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.79.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.79.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.79.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.8.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.8.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.8.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.80.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.experts.80.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.80.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.81.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.81.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.81.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.82.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.82.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.82.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.83.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.83.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.83.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.84.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.84.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.84.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.85.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.85.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.85.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.86.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.86.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.86.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.87.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.87.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.87.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.88.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.88.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.88.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.89.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.89.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.89.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.9.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.9.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.9.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.90.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.90.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.5.mlp.experts.90.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.91.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.91.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.91.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.mlp.experts.92.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.92.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.92.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.93.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.93.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.mlp.experts.93.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.94.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.94.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.5.mlp.experts.94.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.95.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.95.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.95.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.96.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.mlp.experts.96.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.96.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.5.mlp.experts.97.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.97.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.5.mlp.experts.97.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.mlp.experts.98.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.5.mlp.experts.98.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.5.mlp.experts.98.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.99.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.mlp.experts.99.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.mlp.experts.99.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.5.mlp.gate.weight": "model-00008-of-00013.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.self_attn.k_norm.weight": "model-00009-of-00013.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.5.self_attn.q_norm.weight": "model-00001-of-00013.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.0.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.0.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.0.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.1.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.1.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.1.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.10.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.10.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.10.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.100.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.100.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.100.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.101.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.101.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.101.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.102.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.102.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.102.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.103.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.103.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.103.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.104.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.104.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.104.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.105.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.105.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.105.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.106.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.106.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.106.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.107.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.107.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.107.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.108.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.108.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.108.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.109.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.109.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.109.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.11.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.11.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.11.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.110.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.110.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.110.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.111.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.111.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.111.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.112.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.112.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.112.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.113.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.113.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.113.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.114.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.114.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.114.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.115.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.115.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.115.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.116.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.116.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.116.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.117.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.117.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.117.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.118.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.118.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.118.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.119.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.119.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.119.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.12.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.12.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.12.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.120.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.120.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.120.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.121.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.121.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.121.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.122.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.122.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.122.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.123.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.123.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.123.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.124.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.124.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.124.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.125.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.125.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.125.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.126.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.126.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.126.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.127.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.127.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.127.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.13.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.13.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.13.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.14.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.14.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.14.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.15.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.15.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.15.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.16.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.16.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.16.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.17.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.17.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.17.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.18.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.18.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.18.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.19.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.19.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.19.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.2.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.2.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.2.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.20.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.20.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.20.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.21.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.21.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.21.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.22.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.22.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.22.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.23.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.23.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.23.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.24.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.24.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.24.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.25.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.25.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.25.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.26.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.26.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.26.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.27.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.27.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.27.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.28.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.28.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.28.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.29.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.29.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.29.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.3.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.3.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.3.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.30.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.30.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.30.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.31.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.31.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.31.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.32.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.32.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.32.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.33.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.33.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.33.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.34.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.34.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.34.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.35.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.35.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.35.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.36.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.36.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.36.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.37.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.37.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.37.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.38.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.38.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.38.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.39.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.39.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.39.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.4.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.4.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.4.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.40.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.40.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.40.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.41.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.41.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.41.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.42.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.42.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.42.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.43.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.43.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.43.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.44.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.44.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.44.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.45.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.45.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.45.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.46.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.46.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.46.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.47.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.47.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.47.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.48.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.48.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.48.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.49.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.49.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.49.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.5.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.5.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.5.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.50.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.50.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.50.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.51.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.51.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.51.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.52.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.52.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.52.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.53.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.53.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.53.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.54.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.54.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.54.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.55.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.55.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.55.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.56.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.56.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.56.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.57.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.57.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.57.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.58.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.58.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.58.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.59.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.59.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.59.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.6.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.6.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.6.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.60.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.60.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.60.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.61.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.61.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.61.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.62.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.62.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.62.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.63.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.63.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.63.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.64.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.64.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.64.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.65.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.65.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.65.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.66.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.66.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.66.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.67.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.67.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.67.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.68.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.68.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.68.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.69.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.69.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.69.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.7.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.7.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.7.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.70.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.70.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.70.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.71.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.71.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.71.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.72.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.72.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.72.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.73.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.73.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.73.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.74.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.74.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.74.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.75.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.75.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.75.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.76.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.76.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.76.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.77.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.77.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.77.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.78.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.78.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.78.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.79.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.79.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.79.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.8.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.8.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.8.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.80.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.80.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.80.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.81.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.81.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.81.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.82.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.82.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.82.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.83.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.83.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.83.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.84.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.84.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.84.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.85.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.85.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.85.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.86.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.86.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.86.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.87.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.87.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.87.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.88.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.88.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.88.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.89.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.89.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.89.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.9.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.9.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.9.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.90.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.90.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.90.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.91.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.91.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.91.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.92.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.92.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.92.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.93.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.6.mlp.experts.93.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.93.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.94.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.94.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.mlp.experts.94.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.95.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.6.mlp.experts.95.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.experts.95.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.6.mlp.experts.96.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.96.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.mlp.experts.96.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.97.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.mlp.experts.97.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.6.mlp.experts.97.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.mlp.experts.98.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.6.mlp.experts.98.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.mlp.experts.98.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.99.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.mlp.experts.99.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.6.mlp.experts.99.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.6.mlp.gate.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.6.self_attn.k_norm.weight": "model-00012-of-00013.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.6.self_attn.q_norm.weight": "model-00002-of-00013.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.0.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.0.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.0.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.1.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.1.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.1.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.10.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.10.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.10.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.100.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.100.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.100.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.101.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.101.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.101.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.102.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.102.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.102.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.103.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.103.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.103.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.104.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.104.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.104.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.105.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.105.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.105.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.106.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.106.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.106.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.107.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.107.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.107.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.108.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.108.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.108.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.109.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.109.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.109.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.11.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.11.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.11.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.110.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.110.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.110.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.111.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.111.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.111.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.112.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.112.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.112.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.113.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.113.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.113.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.114.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.114.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.114.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.115.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.115.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.115.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.116.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.116.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.116.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.117.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.117.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.117.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.118.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.118.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.118.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.119.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.119.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.119.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.12.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.12.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.12.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.120.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.120.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.120.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.121.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.121.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.121.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.122.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.122.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.122.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.123.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.123.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.123.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.124.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.124.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.124.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.125.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.125.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.125.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.126.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.126.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.126.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.127.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.127.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.127.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.13.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.13.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.13.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.14.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.14.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.14.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.15.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.15.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.15.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.16.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.16.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.16.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.17.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.17.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.17.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.18.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.18.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.18.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.19.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.19.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.19.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.2.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.2.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.2.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.20.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.20.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.20.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.21.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.21.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.21.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.22.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.22.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.22.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.23.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.23.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.23.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.24.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.24.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.24.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.25.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.25.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.25.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.26.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.26.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.26.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.27.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.27.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.27.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.28.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.28.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.28.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.29.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.29.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.29.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.3.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.3.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.3.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.30.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.30.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.30.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.31.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.31.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.31.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.32.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.32.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.32.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.33.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.33.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.33.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.34.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.34.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.34.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.35.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.35.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.35.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.36.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.36.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.36.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.37.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.37.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.37.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.38.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.38.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.38.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.39.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.39.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.39.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.4.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.4.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.4.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.40.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.40.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.40.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.41.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.41.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.41.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.42.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.42.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.42.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.43.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.43.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.43.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.44.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.44.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.44.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.45.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.45.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.45.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.46.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.46.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.46.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.47.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.47.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.47.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.48.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.48.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.48.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.49.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.49.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.49.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.5.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.5.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.5.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.50.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.50.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.50.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.51.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.51.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.51.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.52.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.52.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.52.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.53.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.53.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.53.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.54.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.54.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.54.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.55.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.55.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.55.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.56.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.56.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.56.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.57.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.57.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.57.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.58.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.58.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.58.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.59.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.59.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.59.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.6.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.6.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.6.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.60.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.60.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.60.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.61.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.61.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.61.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.62.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.62.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.62.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.63.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.63.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.63.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.64.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.64.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.64.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.65.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.65.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.65.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.66.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.66.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.66.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.67.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.67.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.67.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.68.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.68.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.68.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.69.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.69.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.69.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.7.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.7.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.7.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.70.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.70.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.70.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.71.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.71.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.71.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.72.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.72.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.72.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.73.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.73.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.73.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.74.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.74.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.74.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.75.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.75.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.75.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.76.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.76.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.76.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.77.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.77.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.77.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.78.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.78.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.78.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.79.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.79.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.experts.79.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.8.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.8.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.8.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.80.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.80.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.80.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.81.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.81.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.81.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.82.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.82.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.82.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.83.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.83.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.83.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.84.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.84.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.84.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.85.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.85.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.85.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.86.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.86.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.86.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.87.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.87.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.87.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.88.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.88.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.88.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.7.mlp.experts.89.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.89.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.89.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.9.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.7.mlp.experts.9.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.9.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.90.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.90.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.90.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.91.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.91.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.91.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.92.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.92.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.92.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.93.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.93.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.93.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.94.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.94.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.mlp.experts.94.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.7.mlp.experts.95.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.95.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.95.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.mlp.experts.96.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.96.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.7.mlp.experts.96.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.mlp.experts.97.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.97.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.mlp.experts.97.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.98.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.mlp.experts.98.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.mlp.experts.98.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.99.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.mlp.experts.99.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.7.mlp.experts.99.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.7.mlp.gate.weight": "model-00004-of-00013.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.7.self_attn.k_norm.weight": "model-00009-of-00013.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.7.self_attn.q_norm.weight": "model-00003-of-00013.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.0.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.0.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.0.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.1.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.1.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.1.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.10.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.10.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.10.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.100.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.100.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.100.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.101.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.101.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.101.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.102.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.102.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.102.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.103.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.103.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.103.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.104.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.104.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.104.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.105.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.105.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.105.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.106.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.106.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.106.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.107.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.107.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.107.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.108.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.108.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.108.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.109.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.109.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.109.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.11.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.11.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.11.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.110.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.110.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.110.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.111.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.111.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.111.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.112.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.112.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.112.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.113.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.113.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.113.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.114.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.114.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.114.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.115.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.115.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.115.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.116.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.116.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.116.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.117.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.117.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.117.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.118.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.118.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.118.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.119.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.119.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.119.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.12.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.12.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.12.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.120.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.120.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.120.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.121.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.121.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.121.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.122.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.122.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.122.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.123.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.123.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.123.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.124.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.124.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.124.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.125.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.125.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.125.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.126.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.126.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.126.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.127.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.127.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.127.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.13.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.13.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.13.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.14.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.14.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.14.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.15.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.15.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.15.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.16.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.16.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.16.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.17.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.17.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.17.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.18.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.18.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.18.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.19.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.19.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.19.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.2.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.2.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.2.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.20.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.20.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.20.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.21.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.21.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.21.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.22.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.22.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.22.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.23.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.23.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.23.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.24.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.24.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.24.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.25.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.25.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.25.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.26.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.26.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.26.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.27.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.27.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.27.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.28.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.28.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.28.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.29.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.29.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.29.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.3.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.3.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.3.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.30.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.30.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.30.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.31.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.31.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.31.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.32.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.32.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.32.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.33.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.33.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.33.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.34.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.34.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.34.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.35.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.35.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.35.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.36.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.36.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.36.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.37.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.37.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.37.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.38.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.38.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.38.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.39.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.39.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.39.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.4.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.4.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.4.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.40.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.40.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.40.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.41.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.41.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.41.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.42.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.42.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.42.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.43.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.43.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.43.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.44.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.44.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.44.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.45.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.45.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.45.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.46.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.46.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.46.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.47.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.47.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.47.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.48.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.48.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.48.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.49.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.49.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.49.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.5.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.5.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.5.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.50.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.50.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.50.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.51.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.51.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.51.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.52.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.52.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.52.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.53.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.53.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.53.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.54.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.54.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.54.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.55.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.55.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.55.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.56.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.56.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.56.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.57.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.57.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.57.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.58.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.58.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.58.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.59.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.59.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.59.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.6.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.6.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.6.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.60.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.60.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.60.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.61.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.61.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.61.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.62.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.62.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.62.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.63.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.63.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.63.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.64.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.64.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.64.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.65.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.65.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.65.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.66.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.66.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.66.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.67.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.67.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.67.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.68.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.68.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.68.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.69.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.69.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.69.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.7.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.7.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.7.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.70.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.70.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.70.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.71.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.71.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.71.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.72.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.72.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.72.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.73.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.73.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.73.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.74.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.74.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.74.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.75.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.75.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.75.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.76.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.76.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.76.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.77.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.77.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.77.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.78.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.78.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.78.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.79.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.79.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.79.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.8.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.8.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.8.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.80.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.80.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.80.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.81.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.81.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.81.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.82.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.82.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.82.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.83.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.83.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.83.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.84.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.84.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.84.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.85.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.85.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.85.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.86.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.86.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.86.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.87.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.87.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.87.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.88.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.88.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.88.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.89.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.89.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.89.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.9.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.9.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.9.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.90.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.90.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.90.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.91.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.8.mlp.experts.91.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.91.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.92.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.92.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.92.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.93.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.experts.93.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.93.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.94.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.94.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.mlp.experts.94.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.8.mlp.experts.95.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.95.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.mlp.experts.95.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.96.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.mlp.experts.96.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.96.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.8.mlp.experts.97.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.mlp.experts.97.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.8.mlp.experts.97.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.98.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.98.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.8.mlp.experts.98.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.mlp.experts.99.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.8.mlp.experts.99.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.8.mlp.experts.99.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.mlp.gate.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00010-of-00013.safetensors",
+    "model.layers.8.self_attn.k_norm.weight": "model-00004-of-00013.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.8.self_attn.q_norm.weight": "model-00013-of-00013.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.0.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.0.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.0.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.1.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.1.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.1.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.10.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.10.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.10.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.100.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.100.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.100.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.101.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.101.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.101.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.102.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.102.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.102.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.103.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.103.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.103.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.104.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.104.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.104.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.105.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.105.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.105.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.106.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.106.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.106.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.107.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.107.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.107.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.108.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.108.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.108.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.109.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.109.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.109.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.11.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.11.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.11.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.110.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.110.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.110.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.111.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.111.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.111.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.112.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.112.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.112.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.113.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.113.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.113.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.114.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.114.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.114.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.115.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.115.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.115.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.116.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.116.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.116.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.117.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.117.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.117.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.118.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.118.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.118.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.119.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.119.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.119.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.12.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.12.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.12.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.120.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.120.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.120.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.121.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.121.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.121.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.122.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.122.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.122.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.123.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.123.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.123.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.124.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.124.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.124.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.125.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.125.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.125.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.126.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.126.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.126.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.127.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.127.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.127.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.13.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.13.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.13.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.14.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.14.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.14.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.15.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.15.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.15.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.16.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.16.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.16.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.17.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.17.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.17.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.18.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.18.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.18.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.19.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.19.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.19.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.2.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.2.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.2.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.20.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.20.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.20.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.21.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.21.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.21.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.22.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.22.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.22.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.23.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.23.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.23.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.24.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.24.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.24.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.25.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.25.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.25.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.26.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.26.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.26.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.27.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.27.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.27.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.28.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.28.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.28.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.29.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.29.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.29.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.3.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.3.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.3.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.30.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.30.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.30.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.31.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.31.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.31.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.32.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.32.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.32.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.33.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.33.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.33.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.34.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.34.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.34.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.35.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.35.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.35.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.36.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.36.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.36.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.37.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.37.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.37.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.38.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.38.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.38.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.39.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.39.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.39.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.4.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.4.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.4.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.40.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.40.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.40.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.41.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.41.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.41.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.42.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.42.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.42.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.43.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.43.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.43.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.44.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.44.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.44.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.45.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.45.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.45.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.46.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.46.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.46.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.47.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.47.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.47.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.48.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.48.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.48.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.49.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.49.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.49.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.5.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.5.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.5.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.50.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.50.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.50.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.51.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.51.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.51.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.52.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.52.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.52.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.53.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.53.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.53.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.54.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.54.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.54.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.55.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.55.gate_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.55.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.56.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.56.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.56.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.57.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.57.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.57.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.58.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.58.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.58.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.59.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.59.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.59.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.6.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.6.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.6.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.60.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.60.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.60.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.61.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.61.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.61.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.62.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.62.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.62.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.63.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.63.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.63.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.64.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.64.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.64.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.65.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.65.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.65.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.66.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.66.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.66.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.67.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.67.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.67.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.68.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.68.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.68.up_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.69.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.69.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.69.up_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.7.down_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.7.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.7.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.70.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.70.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.70.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.71.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.71.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.71.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.72.down_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.72.gate_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.72.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.73.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.73.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.73.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.74.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.74.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.74.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.75.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.75.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.75.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.76.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.76.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.76.up_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.77.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.77.gate_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.77.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.78.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.78.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.78.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.79.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.79.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.79.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.8.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.8.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.8.up_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.80.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.80.gate_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.80.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.81.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.81.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.81.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.82.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.82.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.82.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.83.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.83.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.83.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.84.down_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.84.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.84.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.85.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.85.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.85.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.86.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.86.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.86.up_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.87.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.87.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.87.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.88.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.88.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.88.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.89.down_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.89.gate_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.89.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.9.down_proj.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.mlp.experts.9.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.9.up_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.90.down_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.90.gate_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.90.up_proj.weight": "model-00009-of-00013.safetensors",
+    "model.layers.9.mlp.experts.91.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.91.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.91.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.92.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.92.gate_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.92.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.93.down_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.93.gate_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.93.up_proj.weight": "model-00007-of-00013.safetensors",
+    "model.layers.9.mlp.experts.94.down_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.94.gate_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.94.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.95.down_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.95.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.95.up_proj.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.mlp.experts.96.down_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.experts.96.gate_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.mlp.experts.96.up_proj.weight": "model-00011-of-00013.safetensors",
+    "model.layers.9.mlp.experts.97.down_proj.weight": "model-00012-of-00013.safetensors",
+    "model.layers.9.mlp.experts.97.gate_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.mlp.experts.97.up_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.98.down_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.98.gate_proj.weight": "model-00003-of-00013.safetensors",
+    "model.layers.9.mlp.experts.98.up_proj.weight": "model-00004-of-00013.safetensors",
+    "model.layers.9.mlp.experts.99.down_proj.weight": "model-00001-of-00013.safetensors",
+    "model.layers.9.mlp.experts.99.gate_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.mlp.experts.99.up_proj.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.mlp.gate.weight": "model-00013-of-00013.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00008-of-00013.safetensors",
+    "model.layers.9.self_attn.k_norm.weight": "model-00005-of-00013.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00010-of-00013.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.self_attn.q_norm.weight": "model-00002-of-00013.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00006-of-00013.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00009-of-00013.safetensors",
+    "model.norm.weight": "model-00012-of-00013.safetensors"
+  }
+}
diff --git a/special_tokens_map.json b/special_tokens_map.json
new file mode 100644
index 0000000..445f317
--- /dev/null
+++ b/special_tokens_map.json
@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<|startoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<|cls|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|return|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<|mask|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "<|sep|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|unk|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}
diff --git a/tokenizer.json b/tokenizer.json
new file mode 100644
index 0000000..1cbe7b6
--- /dev/null
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ac381ce8e5664fa1a196b943018b6fee4543f49ba87b1b6454cc1b53b23ece7
+size 12868535
diff --git a/tokenizer_config.json b/tokenizer_config.json
new file mode 100644
index 0000000..114941d
--- /dev/null
+++ b/tokenizer_config.json
@@ -0,0 +1,2845 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<|unk|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<|startoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<|return|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<|mask|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "<|cls|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "<|eod|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": "<|sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "8": {
+      "content": "<|constrain|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "9": {
+      "content": "<|channel|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "10": {
+      "content": "<|start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "11": {
+      "content": "<|end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "12": {
+      "content": "<|message|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "13": {
+      "content": "<|call|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "14": {
+      "content": "<|reserved_006|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "15": {
+      "content": "<|reserved_007|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "16": {
+      "content": "<|reserved_008|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "17": {
+      "content": "<|reserved_009|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "18": {
+      "content": "<|reserved_010|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "19": {
+      "content": "<|reserved_011|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "20": {
+      "content": "<|reserved_012|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "21": {
+      "content": "<|reserved_013|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "22": {
+      "content": "<|reserved_014|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "23": {
+      "content": "<|reserved_015|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "24": {
+      "content": "<|reserved_016|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "25": {
+      "content": "<|reserved_017|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "26": {
+      "content": "<|reserved_018|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "27": {
+      "content": "<|reserved_019|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "28": {
+      "content": "<|reserved_020|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "29": {
+      "content": "<|reserved_021|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "30": {
+      "content": "<|reserved_022|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "31": {
+      "content": "<|reserved_023|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32": {
+      "content": "<|reserved_024|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "33": {
+      "content": "<|reserved_025|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "34": {
+      "content": "<|reserved_026|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "35": {
+      "content": "<|reserved_027|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "36": {
+      "content": "<|reserved_028|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "37": {
+      "content": "<|reserved_029|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "38": {
+      "content": "<|reserved_030|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "39": {
+      "content": "<|reserved_031|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "40": {
+      "content": "<|reserved_032|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "41": {
+      "content": "<|reserved_033|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "42": {
+      "content": "<|reserved_034|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "43": {
+      "content": "<|reserved_035|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "44": {
+      "content": "<|reserved_036|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "45": {
+      "content": "<|reserved_037|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "46": {
+      "content": "<|reserved_038|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "47": {
+      "content": "<|reserved_039|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "48": {
+      "content": "<|reserved_040|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49": {
+      "content": "<|reserved_041|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50": {
+      "content": "<|reserved_042|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "51": {
+      "content": "<|reserved_043|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "52": {
+      "content": "<|reserved_044|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "53": {
+      "content": "<|reserved_045|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "54": {
+      "content": "<|reserved_046|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "55": {
+      "content": "<|reserved_047|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "56": {
+      "content": "<|reserved_048|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "57": {
+      "content": "<|reserved_049|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "58": {
+      "content": "<|reserved_050|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "59": {
+      "content": "<|reserved_051|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "60": {
+      "content": "<|reserved_052|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "61": {
+      "content": "<|reserved_053|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "62": {
+      "content": "<|reserved_054|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "63": {
+      "content": "<|reserved_055|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "64": {
+      "content": "<|reserved_056|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "65": {
+      "content": "<|reserved_057|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "66": {
+      "content": "<|reserved_058|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "67": {
+      "content": "<|reserved_059|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "68": {
+      "content": "<|reserved_060|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "69": {
+      "content": "<|reserved_061|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "70": {
+      "content": "<|reserved_062|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "71": {
+      "content": "<|reserved_063|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "72": {
+      "content": "<|reserved_064|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "73": {
+      "content": "<|reserved_065|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "74": {
+      "content": "<|reserved_066|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "75": {
+      "content": "<|reserved_067|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "76": {
+      "content": "<|reserved_068|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "77": {
+      "content": "<|reserved_069|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "78": {
+      "content": "<|reserved_070|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "79": {
+      "content": "<|reserved_071|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "80": {
+      "content": "<|reserved_072|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "81": {
+      "content": "<|reserved_073|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "82": {
+      "content": "<|reserved_074|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "83": {
+      "content": "<|reserved_075|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "84": {
+      "content": "<|reserved_076|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "85": {
+      "content": "<|reserved_077|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "86": {
+      "content": "<|reserved_078|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "87": {
+      "content": "<|reserved_079|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "88": {
+      "content": "<|reserved_080|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "89": {
+      "content": "<|reserved_081|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "90": {
+      "content": "<|reserved_082|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "91": {
+      "content": "<|reserved_083|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "92": {
+      "content": "<|reserved_084|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "93": {
+      "content": "<|reserved_085|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "94": {
+      "content": "<|reserved_086|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "95": {
+      "content": "<|reserved_087|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "96": {
+      "content": "<|reserved_088|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "97": {
+      "content": "<|reserved_089|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "98": {
+      "content": "<|reserved_090|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "99": {
+      "content": "<|reserved_091|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "<|reserved_092|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "<|reserved_093|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "<|reserved_094|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "<|reserved_095|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "104": {
+      "content": "<|reserved_096|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "105": {
+      "content": "<|reserved_097|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "106": {
+      "content": "<|reserved_098|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "107": {
+      "content": "<|reserved_099|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "108": {
+      "content": "<|reserved_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "109": {
+      "content": "<|reserved_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "110": {
+      "content": "<|reserved_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "111": {
+      "content": "<|reserved_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "112": {
+      "content": "<|reserved_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "113": {
+      "content": "<|reserved_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "114": {
+      "content": "<|reserved_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "115": {
+      "content": "<|reserved_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "116": {
+      "content": "<|reserved_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "117": {
+      "content": "<|reserved_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "118": {
+      "content": "<|reserved_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "119": {
+      "content": "<|reserved_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "120": {
+      "content": "<|reserved_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "121": {
+      "content": "<|reserved_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "122": {
+      "content": "<|reserved_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "123": {
+      "content": "<|reserved_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "124": {
+      "content": "<|reserved_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "125": {
+      "content": "<|reserved_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "126": {
+      "content": "<|reserved_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "127": {
+      "content": "<|reserved_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128": {
+      "content": "<|reserved_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "129": {
+      "content": "<|reserved_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "130": {
+      "content": "<|reserved_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "131": {
+      "content": "<|reserved_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "132": {
+      "content": "<|reserved_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "133": {
+      "content": "<|reserved_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "134": {
+      "content": "<|reserved_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "135": {
+      "content": "<|reserved_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "136": {
+      "content": "<|reserved_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "137": {
+      "content": "<|reserved_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "138": {
+      "content": "<|reserved_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "139": {
+      "content": "<|reserved_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "140": {
+      "content": "<|reserved_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "141": {
+      "content": "<|reserved_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "142": {
+      "content": "<|reserved_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "143": {
+      "content": "<|reserved_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "144": {
+      "content": "<|reserved_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "145": {
+      "content": "<|reserved_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "146": {
+      "content": "<|reserved_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "147": {
+      "content": "<|reserved_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "148": {
+      "content": "<|reserved_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "149": {
+      "content": "<|reserved_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "150": {
+      "content": "<|reserved_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151": {
+      "content": "<|reserved_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "152": {
+      "content": "<|reserved_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "153": {
+      "content": "<|reserved_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "154": {
+      "content": "<|reserved_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "155": {
+      "content": "<|reserved_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "156": {
+      "content": "<|reserved_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "157": {
+      "content": "<|reserved_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "158": {
+      "content": "<|reserved_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "159": {
+      "content": "<|reserved_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "160": {
+      "content": "<|reserved_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "161": {
+      "content": "<|reserved_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "162": {
+      "content": "<|reserved_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "163": {
+      "content": "<|reserved_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "164": {
+      "content": "<|reserved_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "165": {
+      "content": "<|reserved_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "166": {
+      "content": "<|reserved_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "167": {
+      "content": "<|reserved_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "168": {
+      "content": "<|reserved_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "169": {
+      "content": "<|reserved_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "170": {
+      "content": "<|reserved_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "171": {
+      "content": "<|reserved_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "172": {
+      "content": "<|reserved_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "173": {
+      "content": "<|reserved_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "174": {
+      "content": "<|reserved_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "175": {
+      "content": "<|reserved_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "176": {
+      "content": "<|reserved_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "177": {
+      "content": "<|reserved_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "178": {
+      "content": "<|reserved_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "179": {
+      "content": "<|reserved_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "180": {
+      "content": "<|reserved_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "181": {
+      "content": "<|reserved_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "182": {
+      "content": "<|reserved_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "183": {
+      "content": "<|reserved_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "184": {
+      "content": "<|reserved_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "185": {
+      "content": "<|reserved_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "186": {
+      "content": "<|reserved_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "187": {
+      "content": "<|reserved_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "188": {
+      "content": "<|reserved_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "189": {
+      "content": "<|reserved_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "190": {
+      "content": "<|reserved_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "191": {
+      "content": "<|reserved_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "192": {
+      "content": "<|reserved_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "193": {
+      "content": "<|reserved_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "194": {
+      "content": "<|reserved_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "195": {
+      "content": "<|reserved_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "196": {
+      "content": "<|reserved_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "197": {
+      "content": "<|reserved_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "198": {
+      "content": "<|reserved_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "199": {
+      "content": "<|reserved_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200": {
+      "content": "<|reserved_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "201": {
+      "content": "<|reserved_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "202": {
+      "content": "<|reserved_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "203": {
+      "content": "<|reserved_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "204": {
+      "content": "<|reserved_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "205": {
+      "content": "<|reserved_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "206": {
+      "content": "<|reserved_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "207": {
+      "content": "<|reserved_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "208": {
+      "content": "<|reserved_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "209": {
+      "content": "<|reserved_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "210": {
+      "content": "<|reserved_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "211": {
+      "content": "<|reserved_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "212": {
+      "content": "<|reserved_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "213": {
+      "content": "<|reserved_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "214": {
+      "content": "<|reserved_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "215": {
+      "content": "<|reserved_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "216": {
+      "content": "<|reserved_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "217": {
+      "content": "<|reserved_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "218": {
+      "content": "<|reserved_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "219": {
+      "content": "<|reserved_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "220": {
+      "content": "<|reserved_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "221": {
+      "content": "<|reserved_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "222": {
+      "content": "<|reserved_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "223": {
+      "content": "<|reserved_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "224": {
+      "content": "<|reserved_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "225": {
+      "content": "<|reserved_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "226": {
+      "content": "<|reserved_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "227": {
+      "content": "<|reserved_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "228": {
+      "content": "<|reserved_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "229": {
+      "content": "<|reserved_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "230": {
+      "content": "<|reserved_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "231": {
+      "content": "<|reserved_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "232": {
+      "content": "<|reserved_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "233": {
+      "content": "<|reserved_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "234": {
+      "content": "<|reserved_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "235": {
+      "content": "<|reserved_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "236": {
+      "content": "<|reserved_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "237": {
+      "content": "<|reserved_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "238": {
+      "content": "<|reserved_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "239": {
+      "content": "<|reserved_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "240": {
+      "content": "<|reserved_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "241": {
+      "content": "<|reserved_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "242": {
+      "content": "<|reserved_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "243": {
+      "content": "<|reserved_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "244": {
+      "content": "<|reserved_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "245": {
+      "content": "<|reserved_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "246": {
+      "content": "<|reserved_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "247": {
+      "content": "<|reserved_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248": {
+      "content": "<|reserved_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "249": {
+      "content": "<|reserved_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "250": {
+      "content": "<|reserved_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "251": {
+      "content": "<|reserved_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "252": {
+      "content": "<|reserved_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "253": {
+      "content": "<|reserved_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "254": {
+      "content": "<|reserved_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "255": {
+      "content": "<|reserved_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "256": {
+      "content": "<|reserved_248|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "257": {
+      "content": "<|reserved_249|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "258": {
+      "content": "<|reserved_250|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259": {
+      "content": "<|reserved_251|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "260": {
+      "content": "<|reserved_252|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "261": {
+      "content": "<|reserved_253|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "262": {
+      "content": "<|reserved_254|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "263": {
+      "content": "<|reserved_255|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "264": {
+      "content": "<|reserved_256|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "265": {
+      "content": "<|reserved_257|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "266": {
+      "content": "<|reserved_258|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "267": {
+      "content": "<|reserved_259|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "268": {
+      "content": "<|reserved_260|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "269": {
+      "content": "<|reserved_261|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "270": {
+      "content": "<|reserved_262|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "271": {
+      "content": "<|reserved_263|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "272": {
+      "content": "<|reserved_264|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "273": {
+      "content": "<|reserved_265|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "274": {
+      "content": "<|reserved_266|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "275": {
+      "content": "<|reserved_267|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "276": {
+      "content": "<|reserved_268|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "277": {
+      "content": "<|reserved_269|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "278": {
+      "content": "<|reserved_270|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "279": {
+      "content": "<|reserved_271|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "280": {
+      "content": "<|reserved_272|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "281": {
+      "content": "<|reserved_273|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "282": {
+      "content": "<|reserved_274|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "283": {
+      "content": "<|reserved_275|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "284": {
+      "content": "<|reserved_276|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "285": {
+      "content": "<|reserved_277|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "286": {
+      "content": "<|reserved_278|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "287": {
+      "content": "<|reserved_279|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "288": {
+      "content": "<|reserved_280|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "289": {
+      "content": "<|reserved_281|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "290": {
+      "content": "<|reserved_282|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "291": {
+      "content": "<|reserved_283|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "292": {
+      "content": "<|reserved_284|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "293": {
+      "content": "<|reserved_285|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "294": {
+      "content": "<|reserved_286|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "295": {
+      "content": "<|reserved_287|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "296": {
+      "content": "<|reserved_288|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "297": {
+      "content": "<|reserved_289|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "298": {
+      "content": "<|reserved_290|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "299": {
+      "content": "<|reserved_291|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "300": {
+      "content": "<|reserved_292|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "301": {
+      "content": "<|reserved_293|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "302": {
+      "content": "<|reserved_294|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "303": {
+      "content": "<|reserved_295|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "304": {
+      "content": "<|reserved_296|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "305": {
+      "content": "<|reserved_297|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "306": {
+      "content": "<|reserved_298|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "307": {
+      "content": "<|reserved_299|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "308": {
+      "content": "<|reserved_300|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "309": {
+      "content": "<|reserved_301|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "310": {
+      "content": "<|reserved_302|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "311": {
+      "content": "<|reserved_303|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "312": {
+      "content": "<|reserved_304|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "313": {
+      "content": "<|reserved_305|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "314": {
+      "content": "<|reserved_306|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "315": {
+      "content": "<|reserved_307|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "316": {
+      "content": "<|reserved_308|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "317": {
+      "content": "<|reserved_309|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "318": {
+      "content": "<|reserved_310|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "319": {
+      "content": "<|reserved_311|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "320": {
+      "content": "<|reserved_312|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "321": {
+      "content": "<|reserved_313|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "322": {
+      "content": "<|reserved_314|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "323": {
+      "content": "<|reserved_315|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "324": {
+      "content": "<|reserved_316|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "325": {
+      "content": "<|reserved_317|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "326": {
+      "content": "<|reserved_318|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "327": {
+      "content": "<|reserved_319|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "328": {
+      "content": "<|reserved_320|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "329": {
+      "content": "<|reserved_321|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "330": {
+      "content": "<|reserved_322|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "331": {
+      "content": "<|reserved_323|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "332": {
+      "content": "<|reserved_324|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "333": {
+      "content": "<|reserved_325|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "334": {
+      "content": "<|reserved_326|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "335": {
+      "content": "<|reserved_327|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "336": {
+      "content": "<|reserved_328|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "337": {
+      "content": "<|reserved_329|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "338": {
+      "content": "<|reserved_330|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "339": {
+      "content": "<|reserved_331|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "340": {
+      "content": "<|reserved_332|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "341": {
+      "content": "<|reserved_333|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "342": {
+      "content": "<|reserved_334|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "343": {
+      "content": "<|reserved_335|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "344": {
+      "content": "<|reserved_336|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "345": {
+      "content": "<|reserved_337|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "346": {
+      "content": "<|reserved_338|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "347": {
+      "content": "<|reserved_339|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "348": {
+      "content": "<|reserved_340|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "349": {
+      "content": "<|reserved_341|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "350": {
+      "content": "<|reserved_342|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "351": {
+      "content": "<|reserved_343|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "auto_map": {
+    "AutoTokenizer": [
+      "llmjp4_tokenizer.Llmjp4Tokenizer",
+      null
+    ]
+  },
+  "bos_token": "<|startoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<|cls|>",
+  "eod_token": "<|eod|>",
+  "eos_token": "<|return|>",
+  "extra_ids": 0,
+  "extra_special_tokens": {},
+  "legacy": true,
+  "mask_token": "<|mask|>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|endoftext|>",
+  "sep_token": "<|sep|>",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "Llmjp4Tokenizer",
+  "unk_token": "<|unk|>",
+  "use_default_system_prompt": false
+}