初始化项目，由ModelHub XC社区提供模型

Model: ibm-granite/granite-4.0-tiny-preview
Source: Original Platform

.gitattributes

@@ -0,0 +1,47 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
+*.tfevents* filter=lfs diff=lfs merge=lfs -text
+*.db* filter=lfs diff=lfs merge=lfs -text
+*.ark* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.gguf* filter=lfs diff=lfs merge=lfs -text
+*.ggml filter=lfs diff=lfs merge=lfs -text
+*.llamafile* filter=lfs diff=lfs merge=lfs -text
+*.pt2 filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,211 @@
+---
+pipeline_tag: text-generation
+inference: false
+license: apache-2.0
+library_name: transformers
+tags:
+- language
+- granite-4.0
+base_model:
+- ibm-granite/granite-4.0-tiny-base-preview
+---
+
+[![mof-class3-qualified](https://mot.isitopen.ai/modules/mof/assets/badge_class3_qualified.png)](https://mot.isitopen.ai/model/1168)
+
+# Granite-4.0-Tiny-Preview
+
+**Model Summary:**
+Granite-4-Tiny-Preview is a 7B parameter fine-grained hybrid mixture-of-experts (MoE) instruct model fine-tuned from Granite-4.0-Tiny-Base-Preview using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets tailored for solving long context problems. This model is developed using a diverse set of techniques with a structured chat format, including supervised fine-tuning, and model alignment using reinforcement learning.
+
+- **Developers:** Granite Team, IBM
+- **Website**: [Granite Docs](https://www.ibm.com/granite/docs/)
+- **Release Date**: May 2nd, 2025
+- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
+
+**Supported Languages:** 
+English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese. However, users may fine-tune this Granite model for languages beyond these 12 languages.
+
+**Intended Use:** 
+This model is designed to handle general instruction-following tasks and can be integrated into AI assistants across various domains, including business applications.
+
+**Capabilities**
+* Thinking
+* Summarization
+* Text classification
+* Text extraction
+* Question-answering
+* Retrieval Augmented Generation (RAG)
+* Code related tasks
+* Function-calling tasks
+* Multilingual dialog use cases
+* Long-context tasks including long document/meeting summarization, long document QA, etc.
+
+**Installation:** 
+You need to install transformer from source to use this checkpoint.
+<!-- This is a simple example of how to use Granite-4.0-Tiny-Base-Preview model. -->
+
+<!-- Usage: Install transformer from source or use transformer version v4.45 to use this checkpoint. -->
+
+HuggingFace PR: https://github.com/huggingface/transformers/pull/37658
+
+Install transformer from source: https://huggingface.co/docs/transformers/en/installation#install-from-source
+<!-- While the native support of this model in Hugging Face Transformers is pending ([PR](https://github.com/huggingface/transformers/pull/37658)), you need to install transformers from the following source to use this model:
+```shell
+git clone https://github.com/Ssukriti/transformers.git
+cd transformers
+git checkout granitemoe_hybrid_external_cleanup
+pip install -e .
+``` -->
+<!-- Install the following libraries:
+
+```shell
+pip install torch torchvision torchaudio
+pip install accelerate
+pip install transformers
+``` -->
+**Generation:**
+After installation, copy the code snippet below to run the example.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, set_seed
+import torch
+
+model_path="ibm-granite/granite-4.0-tiny-preview"
+device="cuda"
+model = AutoModelForCausalLM.from_pretrained(
+        model_path,
+        device_map=device,
+        torch_dtype=torch.bfloat16,
+    )
+tokenizer = AutoTokenizer.from_pretrained(
+        model_path
+)
+
+conv = [{"role": "user", "content":"You have 10 liters of a 30% acid solution. How many liters of a 70% acid solution must be added to achieve a 50% acid mixture?"}]
+
+input_ids = tokenizer.apply_chat_template(conv, return_tensors="pt", thinking=True, return_dict=True, add_generation_prompt=True).to(device)
+
+set_seed(42)
+output = model.generate(
+    **input_ids,
+    max_new_tokens=8192,
+)
+
+prediction = tokenizer.decode(output[0, input_ids["input_ids"].shape[1]:], skip_special_tokens=True)
+print(prediction)
+```
+
+**Evaluation Results:**
+
+<table>
+<thead>
+    <caption style="text-align:center"><b>Comparison with previous granite models<sup id="fnref1"><a href="#fn1">1</a></sup>. Scores of AlpacaEval-2.0 and Arena-Hard are calculated with thinking=True</b></caption>
+  <tr>
+    <th style="text-align:left; background-color: #001d6c; color: white;">Models</th>
+    <th style="text-align:center; background-color: #001d6c; color: white;">Arena-Hard</th>
+    <th style="text-align:center; background-color: #001d6c; color: white;">AlpacaEval-2.0</th>
+    <th style="text-align:center; background-color: #001d6c; color: white;">MMLU</th>
+    <th style="text-align:center; background-color: #001d6c; color: white;">PopQA</th>
+    <th style="text-align:center; background-color: #001d6c; color: white;">TruthfulQA</th>
+    <th style="text-align:center; background-color: #001d6c; color: white;">BigBenchHard</th>
+    <th style="text-align:center; background-color: #001d6c; color: white;">DROP</th>
+    <th style="text-align:center; background-color: #001d6c; color: white;">GSM8K</th>
+    <th style="text-align:center; background-color: #001d6c; color: white;">HumanEval</th>
+   <th style="text-align:center; background-color: #001d6c; color: white;">HumanEval+</th>
+  <th style="text-align:center; background-color: #001d6c; color: white;">IFEval</th>
+  <th style="text-align:center; background-color: #001d6c; color: white;">AttaQ</th>
+  </tr></thead>
+<tbody>
+ 
+  <tr>
+      <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;"><b>Granite-3.3-2B-Instruct</b></td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 28.86 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 43.45 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 55.88 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 18.4 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 58.97 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 52.51 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 35.98 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 72.48 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 80.51 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 75.68 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 65.8 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">87.47</td>
+      </tr>
+  <tr>
+      <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">Granite-3.3-8B-Instruct</td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 57.56 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 62.68 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 65.54 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 26.17 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 66.86 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 59.01 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 41.53 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 80.89 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 89.73 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 86.09 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 74.82 </td>
+    <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">88.5</td>
+      </tr>
+  <tr>
+      <td style="text-align:left; background-color: #DAE8FF; color: black;"><b>Granite-4.0-Tiny-Preview</b></td>
+    <td style="text-align:center; background-color: #DAE8FF; color: black;"> 26.70 </td>
+    <td style="text-align:center; background-color: #DAE8FF; color: black;"> 35.16 </td>
+    <td style="text-align:center; background-color: #DAE8FF; color: black;"> 60.40 </td>
+    <td style="text-align:center; background-color: #DAE8FF; color: black;"> 22.93 </td>
+    <td style="text-align:center; background-color: #DAE8FF; color: black;"> 58.07 </td>
+    <td style="text-align:center; background-color: #DAE8FF; color: black;"> 55.71 </td>
+    <td style="text-align:center; background-color: #DAE8FF; color: black;"> 46.22 </td>
+    <td style="text-align:center; background-color: #DAE8FF; color: black;"> 70.05  </td>
+    <td style="text-align:center; background-color: #DAE8FF; color: black;"> 82.41 </td>
+    <td style="text-align:center; background-color: #DAE8FF; color: black;"> 78.33 </td>
+    <td style="text-align:center; background-color: #DAE8FF; color: black;"> 63.03 </td>
+    <td style="text-align:center; background-color: #DAE8FF; color: black;"> 86.10 </td>
+      </tr> 
+</tbody></table>
+
+**Training Data:** 
+Overall, our training data is largely comprised of two key sources: (1) publicly available datasets with permissive license, (2) internal synthetically generated data targeted to enhance reasoning capabilities. 
+
+
+**Infrastructure:**
+We train Granite-4.0-Tiny-Preview using IBM's super computing cluster, Blue Vela, which is outfitted with NVIDIA H100 GPUs. This cluster provides a scalable and efficient infrastructure for training our models over thousands of GPUs.
+
+**Ethical Considerations and Limitations:** 
+Granite-4.0-Tiny-Preview, leverages both permissively licensed open-source and select proprietary data for enhanced performance. Since it inherits its foundation from the previous model, all ethical considerations and limitations applicable to [Granite-4.0-Tiny-Preview](https://huggingface.co/ibm-granite/granite-4.0-tiny-preview) remain relevant.
+
+**Signature verification:**
+Model signing is an experimental feature with ongoing development, which might include breaking changes. We are releasing these capabilities to improve the integrity of our models for our security-conscious users and to facilitate feedback from the community.
+
+Before trying to verify the signature, ensure that the tensor files have been downloaded with git-lfs and that no files have been added, removed, or modified in your local git checkout:
+
+```bash
+  git lfs fetch --all
+  git lfs pull
+  git lfs checkout
+```
+
+Install the model_signing (v1.0.1) library with the following command:
+
+```bash
+   pip install 'model-signing==v1.1.1'
+```
+
+
+
+Then verify the signature with the following command ensuring that the IBM identity 'granite.preview@ibm.com' was used for signing this model:
+
+```bash
+   python -m model_signing verify sigstore \
+     --signature model.sig \
+     --ignore-paths .git \
+     --ignore-paths .gitattributes \
+     --identity Granite.Preview@ibm.com \
+     --identity_provider https://sigstore.verify.ibm.com/oauth2 \
+     .
+```
+
+**Resources**
+- ⭐️ Learn about the latest updates with Granite: https://www.ibm.com/granite
+- 📄 Get started with tutorials, best practices, and prompt engineering advice: https://www.ibm.com/granite/docs/
+- 💡 Learn about the latest Granite learning resources: https://ibm.biz/granite-learning-resources

config.json

@@ -0,0 +1,90 @@
+{
+  "architectures": [
+    "GraniteMoeHybridForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attention_multiplier": 0.0078125,
+  "bos_token_id": 0,
+  "embedding_multiplier": 12,
+  "eos_token_id": 0,
+  "hidden_act": "silu",
+  "hidden_size": 1536,
+  "init_method": "mup",
+  "initializer_range": 0.1,
+  "intermediate_size": 512,
+  "layer_types": [
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "attention",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "attention",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "attention",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba",
+    "attention",
+    "mamba",
+    "mamba",
+    "mamba",
+    "mamba"
+  ],
+  "logits_scaling": 6,
+  "logits_to_keep": 1,
+  "mamba_chunk_size": 256,
+  "mamba_conv_bias": true,
+  "mamba_d_conv": 4,
+  "mamba_d_head": 64,
+  "mamba_d_state": 128,
+  "mamba_expand": 2,
+  "mamba_n_groups": 1,
+  "mamba_n_heads": 48,
+  "mamba_proj_bias": false,
+  "max_position_embeddings": 131072,
+  "model_type": "granitemoehybrid",
+  "normalization_function": "rmsnorm",
+  "num_attention_heads": 12,
+  "num_experts_per_tok": 6,
+  "num_hidden_layers": 40,
+  "num_key_value_heads": 4,
+  "num_local_experts": 62,
+  "output_router_logits": false,
+  "pad_token_id": 0,
+  "position_embedding_type": "nope",
+  "residual_multiplier": 0.22,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000,
+  "router_aux_loss_coef": 0.0,
+  "shared_intermediate_size": 1024,
+  "tie_word_embeddings": true,
+  "transformers_version": "4.51.0.dev0",
+  "use_cache": true,
+  "vocab_size": 49160
+}

configuration.json

@@ -0,0 +1 @@
+{"framework": "pytorch", "task": "text-generation", "allow_remote": true}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "eos_token_id": 0,
+  "pad_token_id": 0,
+  "transformers_version": "4.51.0.dev0"
+}

model-00001-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e14c1aa1b6ee5056336f37cb7e071388303da8393f0b46528bf2304499188396
+size 4970179576

model-00002-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae17cb9b9bed17e2667ad6cba0de12da666cd32e41f219cd92353d29f48f56f8
+size 4938695432

model-00003-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2fc801cfd12a5df7b12e1e80fe76fef64f8e722c02f34bb2fb342b89c295cb99
+size 3434272280

model.safetensors.index.json

@@ -0,0 +1,593 @@
+{
+  "metadata": {
+    "total_size": 13343079552
+  },
+  "weight_map": {
+    "model.embed_tokens.weight": "model-00001-of-00003.safetensors",
+    "model.norm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.block_sparse_moe.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.shared_mlp.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.shared_mlp.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mamba.conv1d.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mamba.conv1d.bias": "model-00001-of-00003.safetensors",
+    "model.layers.0.mamba.in_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mamba.dt_bias": "model-00001-of-00003.safetensors",
+    "model.layers.0.mamba.A_log": "model-00001-of-00003.safetensors",
+    "model.layers.0.mamba.D": "model-00001-of-00003.safetensors",
+    "model.layers.0.mamba.out_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mamba.norm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.block_sparse_moe.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.shared_mlp.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.shared_mlp.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mamba.conv1d.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mamba.conv1d.bias": "model-00001-of-00003.safetensors",
+    "model.layers.1.mamba.in_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mamba.dt_bias": "model-00001-of-00003.safetensors",
+    "model.layers.1.mamba.A_log": "model-00001-of-00003.safetensors",
+    "model.layers.1.mamba.D": "model-00001-of-00003.safetensors",
+    "model.layers.1.mamba.out_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mamba.norm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.block_sparse_moe.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.shared_mlp.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.shared_mlp.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mamba.conv1d.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mamba.conv1d.bias": "model-00001-of-00003.safetensors",
+    "model.layers.2.mamba.in_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mamba.dt_bias": "model-00001-of-00003.safetensors",
+    "model.layers.2.mamba.A_log": "model-00001-of-00003.safetensors",
+    "model.layers.2.mamba.D": "model-00001-of-00003.safetensors",
+    "model.layers.2.mamba.out_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mamba.norm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.block_sparse_moe.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.shared_mlp.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.shared_mlp.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mamba.conv1d.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mamba.conv1d.bias": "model-00001-of-00003.safetensors",
+    "model.layers.3.mamba.in_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mamba.dt_bias": "model-00001-of-00003.safetensors",
+    "model.layers.3.mamba.A_log": "model-00001-of-00003.safetensors",
+    "model.layers.3.mamba.D": "model-00001-of-00003.safetensors",
+    "model.layers.3.mamba.out_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mamba.norm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.block_sparse_moe.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.shared_mlp.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.shared_mlp.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mamba.conv1d.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mamba.conv1d.bias": "model-00001-of-00003.safetensors",
+    "model.layers.4.mamba.in_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mamba.dt_bias": "model-00001-of-00003.safetensors",
+    "model.layers.4.mamba.A_log": "model-00001-of-00003.safetensors",
+    "model.layers.4.mamba.D": "model-00001-of-00003.safetensors",
+    "model.layers.4.mamba.out_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mamba.norm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.block_sparse_moe.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.shared_mlp.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.shared_mlp.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.block_sparse_moe.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.shared_mlp.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.shared_mlp.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mamba.conv1d.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mamba.conv1d.bias": "model-00001-of-00003.safetensors",
+    "model.layers.6.mamba.in_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mamba.dt_bias": "model-00001-of-00003.safetensors",
+    "model.layers.6.mamba.A_log": "model-00001-of-00003.safetensors",
+    "model.layers.6.mamba.D": "model-00001-of-00003.safetensors",
+    "model.layers.6.mamba.out_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mamba.norm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.block_sparse_moe.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.shared_mlp.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.shared_mlp.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mamba.conv1d.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mamba.conv1d.bias": "model-00001-of-00003.safetensors",
+    "model.layers.7.mamba.in_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mamba.dt_bias": "model-00001-of-00003.safetensors",
+    "model.layers.7.mamba.A_log": "model-00001-of-00003.safetensors",
+    "model.layers.7.mamba.D": "model-00001-of-00003.safetensors",
+    "model.layers.7.mamba.out_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mamba.norm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.block_sparse_moe.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.shared_mlp.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.shared_mlp.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mamba.conv1d.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mamba.conv1d.bias": "model-00001-of-00003.safetensors",
+    "model.layers.8.mamba.in_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mamba.dt_bias": "model-00001-of-00003.safetensors",
+    "model.layers.8.mamba.A_log": "model-00001-of-00003.safetensors",
+    "model.layers.8.mamba.D": "model-00001-of-00003.safetensors",
+    "model.layers.8.mamba.out_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mamba.norm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.block_sparse_moe.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.shared_mlp.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.shared_mlp.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mamba.conv1d.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mamba.conv1d.bias": "model-00001-of-00003.safetensors",
+    "model.layers.9.mamba.in_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mamba.dt_bias": "model-00001-of-00003.safetensors",
+    "model.layers.9.mamba.A_log": "model-00001-of-00003.safetensors",
+    "model.layers.9.mamba.D": "model-00001-of-00003.safetensors",
+    "model.layers.9.mamba.out_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mamba.norm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.block_sparse_moe.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.shared_mlp.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.shared_mlp.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.mamba.conv1d.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.mamba.conv1d.bias": "model-00001-of-00003.safetensors",
+    "model.layers.10.mamba.in_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.mamba.dt_bias": "model-00001-of-00003.safetensors",
+    "model.layers.10.mamba.A_log": "model-00001-of-00003.safetensors",
+    "model.layers.10.mamba.D": "model-00001-of-00003.safetensors",
+    "model.layers.10.mamba.out_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.mamba.norm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.block_sparse_moe.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.shared_mlp.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.shared_mlp.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.mamba.conv1d.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.mamba.conv1d.bias": "model-00001-of-00003.safetensors",
+    "model.layers.11.mamba.in_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.mamba.dt_bias": "model-00001-of-00003.safetensors",
+    "model.layers.11.mamba.A_log": "model-00001-of-00003.safetensors",
+    "model.layers.11.mamba.D": "model-00001-of-00003.safetensors",
+    "model.layers.11.mamba.out_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.mamba.norm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.block_sparse_moe.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.shared_mlp.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.shared_mlp.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.mamba.conv1d.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.mamba.conv1d.bias": "model-00001-of-00003.safetensors",
+    "model.layers.12.mamba.in_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.mamba.dt_bias": "model-00001-of-00003.safetensors",
+    "model.layers.12.mamba.A_log": "model-00001-of-00003.safetensors",
+    "model.layers.12.mamba.D": "model-00001-of-00003.safetensors",
+    "model.layers.12.mamba.out_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.mamba.norm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.13.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.13.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.13.block_sparse_moe.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.13.shared_mlp.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.13.shared_mlp.output_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.13.mamba.conv1d.weight": "model-00001-of-00003.safetensors",
+    "model.layers.13.mamba.conv1d.bias": "model-00001-of-00003.safetensors",
+    "model.layers.13.mamba.in_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.13.mamba.dt_bias": "model-00001-of-00003.safetensors",
+    "model.layers.13.mamba.A_log": "model-00001-of-00003.safetensors",
+    "model.layers.13.mamba.D": "model-00001-of-00003.safetensors",
+    "model.layers.13.mamba.out_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.13.mamba.norm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.14.block_sparse_moe.router.layer.weight": "model-00001-of-00003.safetensors",
+    "model.layers.14.block_sparse_moe.input_linear.weight": "model-00001-of-00003.safetensors",
+    "model.layers.14.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mamba.conv1d.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mamba.conv1d.bias": "model-00002-of-00003.safetensors",
+    "model.layers.14.mamba.in_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mamba.dt_bias": "model-00002-of-00003.safetensors",
+    "model.layers.14.mamba.A_log": "model-00002-of-00003.safetensors",
+    "model.layers.14.mamba.D": "model-00002-of-00003.safetensors",
+    "model.layers.14.mamba.out_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mamba.norm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mamba.conv1d.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mamba.conv1d.bias": "model-00002-of-00003.safetensors",
+    "model.layers.16.mamba.in_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mamba.dt_bias": "model-00002-of-00003.safetensors",
+    "model.layers.16.mamba.A_log": "model-00002-of-00003.safetensors",
+    "model.layers.16.mamba.D": "model-00002-of-00003.safetensors",
+    "model.layers.16.mamba.out_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mamba.norm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mamba.conv1d.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mamba.conv1d.bias": "model-00002-of-00003.safetensors",
+    "model.layers.17.mamba.in_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mamba.dt_bias": "model-00002-of-00003.safetensors",
+    "model.layers.17.mamba.A_log": "model-00002-of-00003.safetensors",
+    "model.layers.17.mamba.D": "model-00002-of-00003.safetensors",
+    "model.layers.17.mamba.out_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mamba.norm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mamba.conv1d.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mamba.conv1d.bias": "model-00002-of-00003.safetensors",
+    "model.layers.18.mamba.in_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mamba.dt_bias": "model-00002-of-00003.safetensors",
+    "model.layers.18.mamba.A_log": "model-00002-of-00003.safetensors",
+    "model.layers.18.mamba.D": "model-00002-of-00003.safetensors",
+    "model.layers.18.mamba.out_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mamba.norm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mamba.conv1d.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mamba.conv1d.bias": "model-00002-of-00003.safetensors",
+    "model.layers.19.mamba.in_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mamba.dt_bias": "model-00002-of-00003.safetensors",
+    "model.layers.19.mamba.A_log": "model-00002-of-00003.safetensors",
+    "model.layers.19.mamba.D": "model-00002-of-00003.safetensors",
+    "model.layers.19.mamba.out_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mamba.norm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mamba.conv1d.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mamba.conv1d.bias": "model-00002-of-00003.safetensors",
+    "model.layers.20.mamba.in_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mamba.dt_bias": "model-00002-of-00003.safetensors",
+    "model.layers.20.mamba.A_log": "model-00002-of-00003.safetensors",
+    "model.layers.20.mamba.D": "model-00002-of-00003.safetensors",
+    "model.layers.20.mamba.out_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mamba.norm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mamba.conv1d.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mamba.conv1d.bias": "model-00002-of-00003.safetensors",
+    "model.layers.21.mamba.in_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mamba.dt_bias": "model-00002-of-00003.safetensors",
+    "model.layers.21.mamba.A_log": "model-00002-of-00003.safetensors",
+    "model.layers.21.mamba.D": "model-00002-of-00003.safetensors",
+    "model.layers.21.mamba.out_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mamba.norm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.mamba.conv1d.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.mamba.conv1d.bias": "model-00002-of-00003.safetensors",
+    "model.layers.22.mamba.in_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.mamba.dt_bias": "model-00002-of-00003.safetensors",
+    "model.layers.22.mamba.A_log": "model-00002-of-00003.safetensors",
+    "model.layers.22.mamba.D": "model-00002-of-00003.safetensors",
+    "model.layers.22.mamba.out_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.mamba.norm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.mamba.conv1d.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.mamba.conv1d.bias": "model-00002-of-00003.safetensors",
+    "model.layers.23.mamba.in_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.mamba.dt_bias": "model-00002-of-00003.safetensors",
+    "model.layers.23.mamba.A_log": "model-00002-of-00003.safetensors",
+    "model.layers.23.mamba.D": "model-00002-of-00003.safetensors",
+    "model.layers.23.mamba.out_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.mamba.norm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.mamba.conv1d.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.mamba.conv1d.bias": "model-00002-of-00003.safetensors",
+    "model.layers.24.mamba.in_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.mamba.dt_bias": "model-00002-of-00003.safetensors",
+    "model.layers.24.mamba.A_log": "model-00002-of-00003.safetensors",
+    "model.layers.24.mamba.D": "model-00002-of-00003.safetensors",
+    "model.layers.24.mamba.out_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.mamba.norm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.mamba.conv1d.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.mamba.conv1d.bias": "model-00002-of-00003.safetensors",
+    "model.layers.26.mamba.in_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.mamba.dt_bias": "model-00002-of-00003.safetensors",
+    "model.layers.26.mamba.A_log": "model-00002-of-00003.safetensors",
+    "model.layers.26.mamba.D": "model-00002-of-00003.safetensors",
+    "model.layers.26.mamba.out_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.mamba.norm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.mamba.conv1d.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.mamba.conv1d.bias": "model-00002-of-00003.safetensors",
+    "model.layers.27.mamba.in_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.mamba.dt_bias": "model-00002-of-00003.safetensors",
+    "model.layers.27.mamba.A_log": "model-00002-of-00003.safetensors",
+    "model.layers.27.mamba.D": "model-00002-of-00003.safetensors",
+    "model.layers.27.mamba.out_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.mamba.norm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.28.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.28.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.28.block_sparse_moe.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.28.shared_mlp.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.28.shared_mlp.output_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.28.mamba.conv1d.weight": "model-00002-of-00003.safetensors",
+    "model.layers.28.mamba.conv1d.bias": "model-00002-of-00003.safetensors",
+    "model.layers.28.mamba.in_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.28.mamba.dt_bias": "model-00002-of-00003.safetensors",
+    "model.layers.28.mamba.A_log": "model-00002-of-00003.safetensors",
+    "model.layers.28.mamba.D": "model-00002-of-00003.safetensors",
+    "model.layers.28.mamba.out_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.28.mamba.norm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.29.block_sparse_moe.router.layer.weight": "model-00002-of-00003.safetensors",
+    "model.layers.29.block_sparse_moe.input_linear.weight": "model-00002-of-00003.safetensors",
+    "model.layers.29.block_sparse_moe.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.shared_mlp.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.shared_mlp.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.mamba.conv1d.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.mamba.conv1d.bias": "model-00003-of-00003.safetensors",
+    "model.layers.29.mamba.in_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.mamba.dt_bias": "model-00003-of-00003.safetensors",
+    "model.layers.29.mamba.A_log": "model-00003-of-00003.safetensors",
+    "model.layers.29.mamba.D": "model-00003-of-00003.safetensors",
+    "model.layers.29.mamba.out_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.mamba.norm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.block_sparse_moe.router.layer.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.block_sparse_moe.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.block_sparse_moe.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.shared_mlp.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.shared_mlp.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mamba.conv1d.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mamba.conv1d.bias": "model-00003-of-00003.safetensors",
+    "model.layers.30.mamba.in_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mamba.dt_bias": "model-00003-of-00003.safetensors",
+    "model.layers.30.mamba.A_log": "model-00003-of-00003.safetensors",
+    "model.layers.30.mamba.D": "model-00003-of-00003.safetensors",
+    "model.layers.30.mamba.out_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mamba.norm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.block_sparse_moe.router.layer.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.block_sparse_moe.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.block_sparse_moe.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.shared_mlp.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.shared_mlp.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mamba.conv1d.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mamba.conv1d.bias": "model-00003-of-00003.safetensors",
+    "model.layers.31.mamba.in_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mamba.dt_bias": "model-00003-of-00003.safetensors",
+    "model.layers.31.mamba.A_log": "model-00003-of-00003.safetensors",
+    "model.layers.31.mamba.D": "model-00003-of-00003.safetensors",
+    "model.layers.31.mamba.out_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mamba.norm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.32.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.32.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.32.block_sparse_moe.router.layer.weight": "model-00003-of-00003.safetensors",
+    "model.layers.32.block_sparse_moe.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.32.block_sparse_moe.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.32.shared_mlp.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.32.shared_mlp.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.32.mamba.conv1d.weight": "model-00003-of-00003.safetensors",
+    "model.layers.32.mamba.conv1d.bias": "model-00003-of-00003.safetensors",
+    "model.layers.32.mamba.in_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.32.mamba.dt_bias": "model-00003-of-00003.safetensors",
+    "model.layers.32.mamba.A_log": "model-00003-of-00003.safetensors",
+    "model.layers.32.mamba.D": "model-00003-of-00003.safetensors",
+    "model.layers.32.mamba.out_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.32.mamba.norm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.33.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.33.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.33.block_sparse_moe.router.layer.weight": "model-00003-of-00003.safetensors",
+    "model.layers.33.block_sparse_moe.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.33.block_sparse_moe.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.33.shared_mlp.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.33.shared_mlp.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.33.mamba.conv1d.weight": "model-00003-of-00003.safetensors",
+    "model.layers.33.mamba.conv1d.bias": "model-00003-of-00003.safetensors",
+    "model.layers.33.mamba.in_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.33.mamba.dt_bias": "model-00003-of-00003.safetensors",
+    "model.layers.33.mamba.A_log": "model-00003-of-00003.safetensors",
+    "model.layers.33.mamba.D": "model-00003-of-00003.safetensors",
+    "model.layers.33.mamba.out_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.33.mamba.norm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.34.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.34.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.34.block_sparse_moe.router.layer.weight": "model-00003-of-00003.safetensors",
+    "model.layers.34.block_sparse_moe.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.34.block_sparse_moe.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.34.shared_mlp.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.34.shared_mlp.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.34.mamba.conv1d.weight": "model-00003-of-00003.safetensors",
+    "model.layers.34.mamba.conv1d.bias": "model-00003-of-00003.safetensors",
+    "model.layers.34.mamba.in_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.34.mamba.dt_bias": "model-00003-of-00003.safetensors",
+    "model.layers.34.mamba.A_log": "model-00003-of-00003.safetensors",
+    "model.layers.34.mamba.D": "model-00003-of-00003.safetensors",
+    "model.layers.34.mamba.out_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.34.mamba.norm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.35.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.35.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.35.block_sparse_moe.router.layer.weight": "model-00003-of-00003.safetensors",
+    "model.layers.35.block_sparse_moe.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.35.block_sparse_moe.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.35.shared_mlp.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.35.shared_mlp.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.36.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.36.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.36.block_sparse_moe.router.layer.weight": "model-00003-of-00003.safetensors",
+    "model.layers.36.block_sparse_moe.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.36.block_sparse_moe.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.36.shared_mlp.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.36.shared_mlp.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.36.mamba.conv1d.weight": "model-00003-of-00003.safetensors",
+    "model.layers.36.mamba.conv1d.bias": "model-00003-of-00003.safetensors",
+    "model.layers.36.mamba.in_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.36.mamba.dt_bias": "model-00003-of-00003.safetensors",
+    "model.layers.36.mamba.A_log": "model-00003-of-00003.safetensors",
+    "model.layers.36.mamba.D": "model-00003-of-00003.safetensors",
+    "model.layers.36.mamba.out_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.36.mamba.norm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.37.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.37.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.37.block_sparse_moe.router.layer.weight": "model-00003-of-00003.safetensors",
+    "model.layers.37.block_sparse_moe.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.37.block_sparse_moe.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.37.shared_mlp.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.37.shared_mlp.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.37.mamba.conv1d.weight": "model-00003-of-00003.safetensors",
+    "model.layers.37.mamba.conv1d.bias": "model-00003-of-00003.safetensors",
+    "model.layers.37.mamba.in_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.37.mamba.dt_bias": "model-00003-of-00003.safetensors",
+    "model.layers.37.mamba.A_log": "model-00003-of-00003.safetensors",
+    "model.layers.37.mamba.D": "model-00003-of-00003.safetensors",
+    "model.layers.37.mamba.out_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.37.mamba.norm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.38.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.38.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.38.block_sparse_moe.router.layer.weight": "model-00003-of-00003.safetensors",
+    "model.layers.38.block_sparse_moe.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.38.block_sparse_moe.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.38.shared_mlp.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.38.shared_mlp.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.38.mamba.conv1d.weight": "model-00003-of-00003.safetensors",
+    "model.layers.38.mamba.conv1d.bias": "model-00003-of-00003.safetensors",
+    "model.layers.38.mamba.in_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.38.mamba.dt_bias": "model-00003-of-00003.safetensors",
+    "model.layers.38.mamba.A_log": "model-00003-of-00003.safetensors",
+    "model.layers.38.mamba.D": "model-00003-of-00003.safetensors",
+    "model.layers.38.mamba.out_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.38.mamba.norm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.39.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.39.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.39.block_sparse_moe.router.layer.weight": "model-00003-of-00003.safetensors",
+    "model.layers.39.block_sparse_moe.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.39.block_sparse_moe.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.39.shared_mlp.input_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.39.shared_mlp.output_linear.weight": "model-00003-of-00003.safetensors",
+    "model.layers.39.mamba.conv1d.weight": "model-00003-of-00003.safetensors",
+    "model.layers.39.mamba.conv1d.bias": "model-00003-of-00003.safetensors",
+    "model.layers.39.mamba.in_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.39.mamba.dt_bias": "model-00003-of-00003.safetensors",
+    "model.layers.39.mamba.A_log": "model-00003-of-00003.safetensors",
+    "model.layers.39.mamba.D": "model-00003-of-00003.safetensors",
+    "model.layers.39.mamba.out_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.39.mamba.norm.weight": "model-00003-of-00003.safetensors"
+  }
+}

special_tokens_map.json

@@ -0,0 +1,39 @@
+{
+  "additional_special_tokens": [
+    "<|start_of_role|>",
+    "<|end_of_role|>",
+    "<|tool_call|>",
+    "<|start_of_cite|>",
+    "<|end_of_cite|>",
+    "<|start_of_plugin|>",
+    "<|end_of_plugin|>"
+  ],
+  "bos_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,241 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<fim_prefix>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<fim_middle>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<fim_suffix>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<fim_pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "<filename>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "<gh_stars>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": "<issue_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "8": {
+      "content": "<issue_comment>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "9": {
+      "content": "<issue_closed>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "10": {
+      "content": "<jupyter_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "11": {
+      "content": "<jupyter_text>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "12": {
+      "content": "<jupyter_code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "13": {
+      "content": "<jupyter_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "14": {
+      "content": "<empty_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "15": {
+      "content": "<commit_before>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "16": {
+      "content": "<commit_msg>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "17": {
+      "content": "<commit_after>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "18": {
+      "content": "<reponame>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49152": {
+      "content": "<|start_of_role|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49153": {
+      "content": "<|end_of_role|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49154": {
+      "content": "<|tool_call|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49155": {
+      "content": "<|start_of_cite|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49156": {
+      "content": "<|end_of_cite|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49157": {
+      "content": "<|start_of_plugin|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49158": {
+      "content": "<|end_of_plugin|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49159": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|start_of_role|>",
+    "<|end_of_role|>",
+    "<|tool_call|>",
+    "<|start_of_cite|>",
+    "<|end_of_cite|>",
+    "<|start_of_plugin|>",
+    "<|end_of_plugin|>"
+  ],
+  "bos_token": "<|end_of_text|>",
+  "padding_side": "left",
+  "chat_template": "{# Alias tools -> available_tools #}\n{%- if tools and not available_tools -%}\n    {%- set available_tools = tools -%}\n{%- endif -%}\n{%- if messages[0]['role'] == 'system' %}\n     {%- set system_message = messages[0]['content'] %}\n     {%- set loop_messages = messages[1:] %}\n {%- else %}\n     {%- set system_message = \"Knowledge Cutoff Date: April 2024.\nToday's Date: \" + strftime_now('%B %d, %Y') + \".\nYou are Granite, developed by IBM.\" %}\n     {%- if available_tools and documents %}\n         {%- set system_message = system_message + \" You are a helpful assistant with access to the following tools. When a tool is required to answer the user's query, respond only with <|tool_call|> followed by a JSON list of tools used. If a tool does not exist in the provided list of tools, notify the user that you do not have the ability to fulfill the request.\nWrite the response to the user's input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data.\" %}\n     {%- elif available_tools %}\n         {%- set system_message = system_message + \" You are a helpful assistant with access to the following tools. When a tool is required to answer the user's query, respond only with <|tool_call|> followed by a JSON list of tools used. If a tool does not exist in the provided list of tools, notify the user that you do not have the ability to fulfill the request.\" %}\n     {%- elif documents %}\n         {%- set system_message = system_message + \" Write the response to the user's input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data.\" %}\n    {%- elif thinking %}\n    {%- set system_message = system_message + \" You are a helpful AI assistant.\nRespond to every user query in a comprehensive and detailed way. You can write down your thoughts and reasoning process before responding. In the thought process, engage in a comprehensive cycle of analysis, summarization, exploration, reassessment, reflection, backtracing, and iteration to develop well-considered thinking process. In the response section, based on various attempts, explorations, and reflections from the thoughts section, systematically present the final solution that you deem correct. The response should summarize the thought process. Write your thoughts between <think></think> and write your response between <response></response> for each user query.\" %}\n     {%- else %}\n         {%- set system_message = system_message + \" You are a helpful AI assistant.\" %}\n     {%- endif %}\n     {%- if 'citations' in controls and documents %}\n         {%- set system_message = system_message + '\nUse the symbols <|start_of_cite|> and <|end_of_cite|> to indicate when a fact comes from a document in the search result, e.g <|start_of_cite|> {document_id: 1}my fact <|end_of_cite|> for a fact from document 1. Afterwards, list all the citations with their corresponding documents in an ordered list.' %}\n     {%- endif %}\n     {%- if 'hallucinations' in controls and documents %}\n         {%- set system_message = system_message + '\nFinally, after the response is written, include a numbered list of sentences from the response with a corresponding risk value that are hallucinated and not based in the documents.' %}\n     {%- endif %}\n     {%- set loop_messages = messages %}\n {%- endif %}\n {{- '<|start_of_role|>system<|end_of_role|>' + system_message + '<|end_of_text|>\n' }}\n {%- if available_tools %}\n     {{- '<|start_of_role|>available_tools<|end_of_role|>' }}\n     {{- available_tools | tojson(indent=4) }}\n     {{- '<|end_of_text|>\n' }}\n {%- endif %}\n {%- if documents %}\n     {%- for document in documents %}\n         {{- '<|start_of_role|>document {\"document_id\": \"' + document['doc_id'] | string + '\"}<|end_of_role|>\n' }}\n         {{- document['text'] }}\n         {{- '<|end_of_text|>\n' }}\n              {%- endfor %}\n {%- endif %}\n {%- for message in loop_messages %}\n     {{- '<|start_of_role|>' + message['role'] + '<|end_of_role|>' + message['content'] + '<|end_of_text|>\n' }}\n     {%- if loop.last and add_generation_prompt %}\n         {{- '<|start_of_role|>assistant' }}\n             {%- if controls %}\n                 {{- ' ' + controls | tojson()}}\n             {%- endif %}\n         {{- '<|end_of_role|>' }}\n     {%- endif %}\n {%- endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|end_of_text|>",
+  "extra_special_tokens": {},
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|end_of_text|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|end_of_text|>",
+  "vocab_size": 49160
+}