初始化项目，由ModelHub XC社区提供模型
Model: Zyphra/Zamba2-2.7B-instruct Source: Original Platform
This commit is contained in:
ModelHub XC
39
.gitattributes
vendored
Normal file
39
.gitattributes
vendored
Normal file
@@ -0,0 +1,39 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
model-00001-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
model-00002-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
zamba2_2p7B-instruct_original/model-00001-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
zamba2_2p7B-instruct_original/model-00002-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
112
README.md
Normal file
112
README.md
Normal file
@@ -0,0 +1,112 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
datasets:
|
||||
- HuggingFaceH4/ultrachat_200k
|
||||
- BAAI/Infinity-Instruct
|
||||
- HuggingFaceH4/ultrafeedback_binarized
|
||||
- Intel/orca_dpo_pairs
|
||||
- argilla/OpenHermesPreferences
|
||||
base_model:
|
||||
- Zyphra/Zamba2-2.7B
|
||||
library_name: transformers
|
||||
---
|
||||
# Model Card for Zamba2-2.7B-Instruct
|
||||
|
||||
Zamba2-2.7B-Instruct is obtained from [Zamba2-2.7B](https://huggingface.co/Zyphra/Zamba2-2.7B) by fine-tuning on instruction-following and chat datasets. Specifically:
|
||||
|
||||
1. SFT of the base [Zamba2-2.7B](https://huggingface.co/Zyphra/Zamba2-2.7B) model on [ultrachat_200k](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k) and [Infinity-Instruct](https://huggingface.co/datasets/BAAI/Infinity-Instruct)
|
||||
2. DPO of the SFT checkpoint on [ultrafeedback_binarized](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized), [orca_dpo_pairs](https://huggingface.co/datasets/Intel/orca_dpo_pairs), and [OpenHermesPreferences](https://huggingface.co/datasets/argilla/OpenHermesPreferences)
|
||||
|
||||
Zamba2-2.7B-Instruct is a hybrid model composed of state-space ([Mamba2](https://github.com/state-spaces/mamba)) and transformer blocks.
|
||||
|
||||
## Quick start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
To use Zamba2-1.2B-instruct, install `transformers` from source:
|
||||
|
||||
1. `git clone https://github.com/huggingface/transformers.git`
|
||||
2. `cd transformers && pip install .`
|
||||
|
||||
To install dependencies necessary to run Mamba2 kernels, install `mamba-ssm` from source (due to compatibility issues with PyTorch) as well as `causal-conv1d`:
|
||||
|
||||
1. `git clone https://github.com/state-spaces/mamba.git`
|
||||
2. `cd mamba && git checkout v2.1.0 && pip install .`
|
||||
3. `pip install causal-conv1d`
|
||||
|
||||
|
||||
You can run the model without using the optimized Mamba2 kernels, but it is **not** recommended as it will result in significantly higher latency and memory usage.
|
||||
|
||||
|
||||
### Inference
|
||||
|
||||
```python
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
import torch
|
||||
|
||||
# Instantiate model and tokenizer
|
||||
tokenizer = AutoTokenizer.from_pretrained("Zyphra/Zamba2-2.7B-instruct")
|
||||
model = AutoModelForCausalLM.from_pretrained("Zyphra/Zamba2-2.7B-instruct", device_map="cuda", torch_dtype=torch.bfloat16)
|
||||
|
||||
# Format the input as a chat template
|
||||
user_turn_1 = "In one season a flower blooms three times. In one year, there is one blooming season. How many times do two flowers bloom in two years? Please include your logic."
|
||||
assistant_turn_1 = "In one season, a flower blooms three times. In one year, there is one blooming season. Therefore, in two years, there are two blooming seasons. Since each flower blooms three times in one season, in two blooming seasons, each flower will bloom six times. Since there are two flowers, the total number of times they will bloom in two years is 12."
|
||||
user_turn_2 = "How many times do the two flowers blossom in three years?"
|
||||
sample = [{'role': 'user', 'content': user_turn_1}, {'role': 'assistant', 'content': assistant_turn_1}, {'role': 'user', 'content': user_turn_2}]
|
||||
chat_sample = tokenizer.apply_chat_template(sample, tokenize=False)
|
||||
|
||||
# Tokenize input and generate output
|
||||
input_ids = tokenizer(chat_sample, return_tensors='pt', add_special_tokens=False).to("cuda")
|
||||
outputs = model.generate(**input_ids, max_new_tokens=150, return_dict_in_generate=False, output_scores=False, use_cache=True, num_beams=1, do_sample=False)
|
||||
print((tokenizer.decode(outputs[0])))
|
||||
```
|
||||
|
||||
## Performance
|
||||
|
||||
Zamba2-2.7B-Instruct punches dramatically above its weight, achieving extremely strong instruction-following benchmark scores, significantly outperforming Gemma2-2B-Instruct of the same size and outperforming Mistral-7B-Instruct in most metrics.
|
||||
|
||||
<center>
|
||||
<img src="https://cdn-uploads.huggingface.co/production/uploads/65bc13717c6ad1994b6619e9/QnudHrMeMx_NuRc2evwRG.png" width="900"/>
|
||||
</center>
|
||||
|
||||
|
||||
| Model | Size | Aggregate MT-Bench | IFEval |
|
||||
|:---------------------------:|:-----:|:------------------:|:---------:|
|
||||
| **Zamba2-2.7B-Instruct** | 2.7B | **72.40**| **48.02**|
|
||||
| Mistral-7B-Instruct | 7B| 66.4 | 45.3 |
|
||||
| Gemma2-2B-Instruct | 2.7B | 51.69 | 42.20 |
|
||||
| H2O-Danube-4B-Chat | 4B| 52.57 | 37.96 |
|
||||
| StableLM-Zephyr-3B | 3B| 66.43 | 38.27 |
|
||||
|
||||
|
||||
Moreover, due to its unique hybrid SSM architecture, Zamba2-2.7B-Instruct achieves extremely low inference latency and rapid generation with a significantly smaller memory footprint than comparable transformer-based models.
|
||||
|
||||
<center>
|
||||
<img src="https://cdn-uploads.huggingface.co/production/uploads/65bc13717c6ad1994b6619e9/WKTcYkhDgJCHyze4TDpLa.png" width="700" alt="Zamba performance">
|
||||
</center>
|
||||
|
||||
|
||||
Time to First Token (TTFT) | Output Generation
|
||||
:-------------------------:|:-------------------------:
|
||||
 | 
|
||||
|
||||
|
||||
<center>
|
||||
<img src="https://cdn-uploads.huggingface.co/production/uploads/65bc13717c6ad1994b6619e9/nhoss41xlzfEBZzcQXI6z.png" width="700" alt="Zamba inference and memory cost">
|
||||
</center>
|
||||
|
||||
Zamba2-2.7B-Instruct's high performance, strong instruction-following and reasoning capabilities, and small inference compute and memory footprint renders it an ideal generalist model for on-device applications.
|
||||
|
||||
## Model Details
|
||||
|
||||
Zamba2-2.7B-Instruct utilizes and extends our original Zamba hybrid SSM-attention architecture. The core Zamba architecture consists of a backbone of Mamba2 layers interleaved with one or more shared attention layers (one shared attention in Zamba1, two in Zamba2). This attention has shared weights to minimize the parameter cost of the model. We find that concatenating the original model embeddings to the input to this attention block improves performance, likely due to better maintenance of information across depth. The Zamba2 architecture also applies LoRA projection matrices to the shared MLP to gain some additional expressivity in each block and allow each shared block to specialize slightly to its own unique position while keeping the additional parameter overhead small.
|
||||
|
||||
<center>
|
||||
<img src="https://cdn-uploads.huggingface.co/production/uploads/65c05e75c084467acab2f84a/XrEIEBxd0fqIgh3LyArAV.png" width="300" alt="Zamba architecture">
|
||||
</center>
|
||||
|
||||
|
||||
|
||||
Note: this is a temporary HuggingFace implementation of Zamba2-2.7B. It may not yet be fully compatible with all frameworks and tools intended to interface with HuggingFace models.
|
||||
|
||||
A standalone Pytorch implementation of Zamba2-2.7B may be found [here](https://github.com/Zyphra/Zamba2).
|
||||
115
config.json
Normal file
115
config.json
Normal file
@@ -0,0 +1,115 @@
|
||||
{
|
||||
"architectures": [
|
||||
"Zamba2ForCausalLM"
|
||||
],
|
||||
"adapter_rank": 128,
|
||||
"add_bias_linear": false,
|
||||
"attention_dropout": 0.0,
|
||||
"attention_head_dim": 160,
|
||||
"attention_hidden_size": 5120,
|
||||
"bos_token_id": 1,
|
||||
"chunk_size": 256,
|
||||
"eos_token_id": 2,
|
||||
"ffn_hidden_size": 10240,
|
||||
"hidden_act": "gelu",
|
||||
"hidden_size": 2560,
|
||||
"hybrid_layer_ids": [
|
||||
6,
|
||||
12,
|
||||
18,
|
||||
24,
|
||||
30,
|
||||
36,
|
||||
42,
|
||||
47,
|
||||
51
|
||||
],
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 10240,
|
||||
"kv_channels": 80,
|
||||
"layers_block_type": [
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"hybrid",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"hybrid",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"hybrid",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"hybrid",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"hybrid",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"hybrid",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"hybrid",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"hybrid",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"mamba",
|
||||
"hybrid",
|
||||
"mamba",
|
||||
"mamba"
|
||||
],
|
||||
"mamba_d_conv": 4,
|
||||
"mamba_d_state": 64,
|
||||
"mamba_expand": 2,
|
||||
"mamba_headdim": 64,
|
||||
"mamba_ngroups": 1,
|
||||
"max_position_embeddings": 4096,
|
||||
"model_type": "zamba2",
|
||||
"n_mamba_heads": 80,
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 54,
|
||||
"num_key_value_heads": 32,
|
||||
"num_logits_to_keep": 1,
|
||||
"num_mem_blocks": 2,
|
||||
"num_query_groups": 32,
|
||||
"pad_token_id": 0,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_theta": 10000,
|
||||
"time_step_floor": 0.0001,
|
||||
"time_step_limit": null,
|
||||
"time_step_max": 0.1,
|
||||
"time_step_min": 0.001,
|
||||
"transformers_version": "4.49.0.dev0",
|
||||
"use_cache": true,
|
||||
"use_conv_bias": true,
|
||||
"use_long_context": false,
|
||||
"use_mem_rope": false,
|
||||
"use_shared_attention_adapter": false,
|
||||
"use_shared_mlp_adapter": true,
|
||||
"vocab_size": 32000
|
||||
}
|
||||
1
configuration.json
Normal file
1
configuration.json
Normal file
@@ -0,0 +1 @@
|
||||
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}
|
||||
7
generation_config.json
Normal file
7
generation_config.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 1,
|
||||
"eos_token_id": 2,
|
||||
"pad_token_id": 0,
|
||||
"transformers_version": "4.43.0.dev0"
|
||||
}
|
||||
3
model-00001-of-00002.safetensors
Normal file
3
model-00001-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7d7c07949514ecdafa4f234339ce833afa1057ed653167e4ca2d3d80c4c85083
|
||||
size 4940965304
|
||||
3
model-00002-of-00002.safetensors
Normal file
3
model-00002-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7a64f9b73ebca806ea28ae35ddc508152179f57817c1caea26248c7d3215c495
|
||||
size 383523136
|
||||
538
model.safetensors.index.json
Normal file
538
model.safetensors.index.json
Normal file
@@ -0,0 +1,538 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 5377513280
|
||||
},
|
||||
"weight_map": {
|
||||
"model.layers.6.shared_transformer.feed_forward.gate_up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.feed_forward.gate_up_proj_adapter_list.0.0.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.feed_forward.gate_up_proj_adapter_list.2.0.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.feed_forward.gate_up_proj_adapter_list.4.0.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.feed_forward.gate_up_proj_adapter_list.6.0.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.feed_forward.gate_up_proj_adapter_list.8.0.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.feed_forward.gate_up_proj_adapter_list.0.1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.feed_forward.gate_up_proj_adapter_list.2.1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.feed_forward.gate_up_proj_adapter_list.4.1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.feed_forward.gate_up_proj_adapter_list.6.1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.feed_forward.gate_up_proj_adapter_list.8.1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.shared_transformer.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.feed_forward.gate_up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.feed_forward.gate_up_proj_adapter_list.1.0.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.feed_forward.gate_up_proj_adapter_list.3.0.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.feed_forward.gate_up_proj_adapter_list.5.0.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.feed_forward.gate_up_proj_adapter_list.7.0.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.feed_forward.gate_up_proj_adapter_list.1.1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.feed_forward.gate_up_proj_adapter_list.3.1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.feed_forward.gate_up_proj_adapter_list.5.1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.feed_forward.gate_up_proj_adapter_list.7.1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.shared_transformer.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00002.safetensors",
|
||||
"model.final_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.6.linear.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.12.linear.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.18.linear.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.linear.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.linear.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.36.linear.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.42.linear.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.47.linear.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.51.linear.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mamba_decoder.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mamba_decoder.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mamba_decoder.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mamba_decoder.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mamba_decoder.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mamba_decoder.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mamba_decoder.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mamba_decoder.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mamba_decoder.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mamba_decoder.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mamba_decoder.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mamba_decoder.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mamba_decoder.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mamba_decoder.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mamba_decoder.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mamba_decoder.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mamba_decoder.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mamba_decoder.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mamba_decoder.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mamba_decoder.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mamba_decoder.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mamba_decoder.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mamba_decoder.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mamba_decoder.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mamba_decoder.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mamba_decoder.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mamba_decoder.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.mamba_decoder.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.mamba_decoder.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.mamba_decoder.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.mamba_decoder.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.mamba_decoder.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.mamba_decoder.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.mamba_decoder.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.mamba_decoder.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.mamba_decoder.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.mamba_decoder.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.mamba_decoder.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.mamba_decoder.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.mamba_decoder.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.mamba_decoder.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.mamba_decoder.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.mamba_decoder.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.mamba_decoder.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.mamba_decoder.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.40.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.40.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.40.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.40.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.40.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.40.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.40.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.40.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.40.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.41.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.41.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.41.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.41.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.41.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.41.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.41.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.41.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.41.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.42.mamba_decoder.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.42.mamba_decoder.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.42.mamba_decoder.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.42.mamba_decoder.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.42.mamba_decoder.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.42.mamba_decoder.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.42.mamba_decoder.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.42.mamba_decoder.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.42.mamba_decoder.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.43.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.43.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.43.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.43.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.43.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.43.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.43.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.43.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.43.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.44.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.44.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.44.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.44.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.44.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.44.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.44.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.44.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.44.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.45.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.45.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.45.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.45.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.45.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.45.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.45.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.45.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.45.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.46.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.46.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.46.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.46.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.46.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.46.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.46.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.46.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.46.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.47.mamba_decoder.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.47.mamba_decoder.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.47.mamba_decoder.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.47.mamba_decoder.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.47.mamba_decoder.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.47.mamba_decoder.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.47.mamba_decoder.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.47.mamba_decoder.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.47.mamba_decoder.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.48.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.48.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.48.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.48.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.48.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.48.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.48.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.48.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.48.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.49.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.49.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.49.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.49.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.49.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.49.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.49.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.49.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.49.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.50.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.50.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.50.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.50.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.50.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.50.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.50.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.50.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.50.mamba.out_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.51.mamba_decoder.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.51.mamba_decoder.mamba.A_log": "model-00002-of-00002.safetensors",
|
||||
"model.layers.51.mamba_decoder.mamba.D": "model-00002-of-00002.safetensors",
|
||||
"model.layers.51.mamba_decoder.mamba.conv1d.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.51.mamba_decoder.mamba.conv1d.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.51.mamba_decoder.mamba.dt_bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.51.mamba_decoder.mamba.in_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.51.mamba_decoder.mamba.norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.51.mamba_decoder.mamba.out_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.52.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.52.mamba.A_log": "model-00002-of-00002.safetensors",
|
||||
"model.layers.52.mamba.D": "model-00002-of-00002.safetensors",
|
||||
"model.layers.52.mamba.conv1d.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.52.mamba.conv1d.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.52.mamba.dt_bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.52.mamba.in_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.52.mamba.norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.52.mamba.out_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.53.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.53.mamba.A_log": "model-00002-of-00002.safetensors",
|
||||
"model.layers.53.mamba.D": "model-00002-of-00002.safetensors",
|
||||
"model.layers.53.mamba.conv1d.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.53.mamba.conv1d.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.53.mamba.dt_bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.53.mamba.in_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.53.mamba.norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.53.mamba.out_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.6.mamba_decoder.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mamba_decoder.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mamba_decoder.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mamba_decoder.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mamba_decoder.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mamba_decoder.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mamba_decoder.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mamba_decoder.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mamba_decoder.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mamba.out_proj.weight": "model-00001-of-00002.safetensors"
|
||||
}
|
||||
}
|
||||
27
special_tokens_map.json
Normal file
27
special_tokens_map.json
Normal file
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>"
|
||||
],
|
||||
"bos_token": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "[PAD]",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
91123
tokenizer.json
Normal file
91123
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
51
tokenizer_config.json
Normal file
51
tokenizer_config.json
Normal file
@@ -0,0 +1,51 @@
|
||||
{
|
||||
"add_bos_token": true,
|
||||
"add_eos_token": false,
|
||||
"add_prefix_space": null,
|
||||
"added_tokens_decoder": {
|
||||
"0": {
|
||||
"content": "[PAD]",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"1": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"2": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>"
|
||||
],
|
||||
"bos_token": "<|im_start|>",
|
||||
"chat_template": "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|im_end|>",
|
||||
"legacy": true,
|
||||
"max_length": 4096,
|
||||
"model_max_length": 4096,
|
||||
"pad_token": "[PAD]",
|
||||
"sp_model_kwargs": {},
|
||||
"spaces_between_special_tokens": false,
|
||||
"stride": 0,
|
||||
"tokenizer_class": "LlamaTokenizer",
|
||||
"truncation_side": "right",
|
||||
"truncation_strategy": "longest_first",
|
||||
"unk_token": null,
|
||||
"use_default_system_prompt": false
|
||||
}
|
||||
102
zamba2_2p7B-instruct_original/config.json
Normal file
102
zamba2_2p7B-instruct_original/config.json
Normal file
@@ -0,0 +1,102 @@
|
||||
{
|
||||
"_name_or_path": "Zyphra/Zamba2-2.7B-instruct",
|
||||
"add_bias_linear": false,
|
||||
"architectures": [
|
||||
"Zamba2ForCausalLM"
|
||||
],
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 1,
|
||||
"conv_dimension": 4,
|
||||
"eos_token_id": 2,
|
||||
"expansion_factor": 2,
|
||||
"ffn_hidden_size": 10240,
|
||||
"ft_lora": false,
|
||||
"gated_linear_unit": true,
|
||||
"hidden_size": 2560,
|
||||
"initializer_range": 0.02,
|
||||
"kv_channels": 80,
|
||||
"layers_block_type": [
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"g",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"g",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"g",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"g",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"g",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"g",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"g",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"g",
|
||||
"m",
|
||||
"m",
|
||||
"m",
|
||||
"g",
|
||||
"m",
|
||||
"m"
|
||||
],
|
||||
"lora_rank": 128,
|
||||
"lora_rank_mamba": 128,
|
||||
"mamba_headdim": 64,
|
||||
"mamba_ngroups": 1,
|
||||
"max_position_embeddings": 4096,
|
||||
"model_type": "zamba2",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 54,
|
||||
"num_key_value_heads": 32,
|
||||
"num_logits_to_keep": 1,
|
||||
"num_mem_blocks": 2,
|
||||
"num_query_groups": 32,
|
||||
"pad_token_id": 0,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_theta": 10000,
|
||||
"se_shared_attention_lora": false,
|
||||
"sliding_window": null,
|
||||
"state_size": 64,
|
||||
"torch_dtype": "bfloat16",
|
||||
"transformers_version": "4.43.0.dev0",
|
||||
"use_cache": false,
|
||||
"use_long_context": false,
|
||||
"use_mamba_kernels": true,
|
||||
"use_mem_eff_path": false,
|
||||
"use_mem_rope": false,
|
||||
"use_shared_attention_lora": false,
|
||||
"use_shared_block_lora": true,
|
||||
"vocab_size": 32000
|
||||
}
|
||||
7
zamba2_2p7B-instruct_original/generation_config.json
Normal file
7
zamba2_2p7B-instruct_original/generation_config.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 1,
|
||||
"eos_token_id": 2,
|
||||
"pad_token_id": 0,
|
||||
"transformers_version": "4.43.0.dev0"
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4b2296e886390d354763bc522782582186b0e0a35da64028e8b3fe534def7350
|
||||
size 4994052944
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f120af353d405db76ab8381106f160df4c9dc17b8cc6a5911a9dd5b7789ea34a
|
||||
size 383523176
|
||||
556
zamba2_2p7B-instruct_original/model.safetensors.index.json
Normal file
556
zamba2_2p7B-instruct_original/model.safetensors.index.json
Normal file
@@ -0,0 +1,556 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 5377513280
|
||||
},
|
||||
"weight_map": {
|
||||
"model.blocks.0.feed_forward.linear_fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_A_list.0.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_A_list.1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_A_list.2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_A_list.3.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_A_list.4.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_A_list.5.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_A_list.6.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_A_list.7.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_A_list.8.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_B_list.0.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_B_list.1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_B_list.2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_B_list.3.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_B_list.4.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_B_list.5.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_B_list.6.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_B_list.7.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc1_lora_B_list.8.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.feed_forward.linear_fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_A_list.0.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_A_list.1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_A_list.2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_A_list.3.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_A_list.4.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_A_list.5.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_A_list.6.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_A_list.7.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_A_list.8.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_B_list.0.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_B_list.1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_B_list.2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_B_list.3.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_B_list.4.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_B_list.5.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_B_list.6.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_B_list.7.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc1_lora_B_list.8.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.feed_forward.linear_fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.blocks.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00002.safetensors",
|
||||
"model.final_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.linear_layers.0.weight": "model-00002-of-00002.safetensors",
|
||||
"model.linear_layers.1.weight": "model-00002-of-00002.safetensors",
|
||||
"model.linear_layers.2.weight": "model-00002-of-00002.safetensors",
|
||||
"model.linear_layers.3.weight": "model-00002-of-00002.safetensors",
|
||||
"model.linear_layers.4.weight": "model-00002-of-00002.safetensors",
|
||||
"model.linear_layers.5.weight": "model-00002-of-00002.safetensors",
|
||||
"model.linear_layers.6.weight": "model-00002-of-00002.safetensors",
|
||||
"model.linear_layers.7.weight": "model-00002-of-00002.safetensors",
|
||||
"model.linear_layers.8.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.0.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.0.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.0.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.0.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.0.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.0.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.0.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.0.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.1.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.1.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.1.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.1.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.1.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.1.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.1.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.1.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.10.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.10.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.10.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.10.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.10.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.10.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.10.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.10.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.11.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.11.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.11.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.11.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.11.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.11.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.11.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.11.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.12.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.12.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.12.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.12.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.12.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.12.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.12.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.12.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.13.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.13.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.13.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.13.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.13.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.13.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.13.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.13.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.14.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.14.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.14.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.14.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.14.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.14.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.14.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.14.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.15.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.15.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.15.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.15.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.15.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.15.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.15.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.15.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.16.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.16.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.16.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.16.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.16.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.16.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.16.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.16.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.17.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.17.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.17.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.17.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.17.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.17.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.17.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.17.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.18.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.18.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.18.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.18.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.18.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.18.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.18.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.18.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.19.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.19.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.19.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.19.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.19.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.19.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.19.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.19.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.2.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.2.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.2.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.2.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.2.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.2.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.2.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.2.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.20.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.20.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.20.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.20.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.20.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.20.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.20.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.20.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.20.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.21.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.21.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.21.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.21.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.21.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.21.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.21.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.21.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.21.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.22.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.22.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.22.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.22.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.22.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.22.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.22.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.22.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.22.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.23.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.23.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.23.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.23.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.23.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.23.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.23.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.23.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.23.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.24.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.24.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.24.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.24.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.24.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.24.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.24.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.24.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.24.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.25.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.25.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.25.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.25.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.25.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.25.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.25.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.25.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.25.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.26.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.26.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.26.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.26.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.26.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.26.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.26.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.26.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.26.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.27.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.27.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.27.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.27.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.27.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.27.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.27.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.27.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.27.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.28.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.28.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.28.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.28.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.28.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.28.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.28.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.28.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.28.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.29.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.29.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.29.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.29.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.29.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.29.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.29.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.29.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.29.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.3.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.3.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.3.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.3.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.3.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.3.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.3.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.3.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.30.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.30.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.30.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.30.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.30.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.30.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.30.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.30.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.30.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.31.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.31.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.31.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.31.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.31.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.31.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.31.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.31.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.31.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.32.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.32.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.32.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.32.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.32.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.32.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.32.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.32.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.32.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.33.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.33.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.33.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.33.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.33.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.33.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.33.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.33.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.33.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.34.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.34.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.34.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.34.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.34.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.34.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.34.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.34.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.34.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.35.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.35.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.35.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.35.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.35.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.35.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.35.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.35.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.35.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.36.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.36.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.36.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.36.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.36.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.36.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.36.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.36.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.36.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.37.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.37.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.37.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.37.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.37.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.37.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.37.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.37.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.37.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.38.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.38.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.38.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.38.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.38.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.38.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.38.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.38.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.38.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.39.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.39.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.39.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.39.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.39.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.39.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.39.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.39.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.39.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.4.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.4.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.4.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.4.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.4.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.4.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.4.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.4.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.40.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.40.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.40.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.40.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.40.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.40.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.40.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.40.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.40.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.41.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.41.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.41.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.41.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.41.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.41.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.41.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.41.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.41.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.42.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.42.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.42.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.42.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.42.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.42.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.42.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.42.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.42.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.43.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.43.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.43.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.43.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.43.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.43.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.43.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.43.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.43.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.44.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.44.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.44.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.44.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.44.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.44.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.44.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.44.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.44.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.45.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.45.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.45.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.45.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.45.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.45.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.45.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.45.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.45.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.46.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.46.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.46.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.46.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.46.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.46.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.46.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.46.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.46.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.47.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.47.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.47.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.47.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.47.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.47.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.47.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.47.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.47.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.48.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.48.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.48.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.48.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.48.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.48.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.48.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.48.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.48.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.49.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.49.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.49.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.49.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.49.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.49.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.49.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.49.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.49.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.5.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.5.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.5.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.5.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.5.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.5.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.5.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.5.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.50.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.50.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.50.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.50.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.50.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.50.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.50.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.50.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.50.mamba.out_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.51.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.51.mamba.A_log": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.51.mamba.D": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.51.mamba.conv1d.bias": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.51.mamba.conv1d.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.51.mamba.dt_bias": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.51.mamba.in_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.51.mamba.norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.51.mamba.out_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.52.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.52.mamba.A_log": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.52.mamba.D": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.52.mamba.conv1d.bias": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.52.mamba.conv1d.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.52.mamba.dt_bias": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.52.mamba.in_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.52.mamba.norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.52.mamba.out_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.53.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.53.mamba.A_log": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.53.mamba.D": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.53.mamba.conv1d.bias": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.53.mamba.conv1d.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.53.mamba.dt_bias": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.53.mamba.in_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.53.mamba.norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.53.mamba.out_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.mamba_layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.6.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.6.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.6.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.6.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.6.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.6.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.6.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.6.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.7.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.7.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.7.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.7.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.7.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.7.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.7.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.7.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.8.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.8.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.8.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.8.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.8.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.8.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.8.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.8.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.9.mamba.A_log": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.9.mamba.D": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.9.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.9.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.9.mamba.dt_bias": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.9.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.9.mamba.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.mamba_layers.9.mamba.out_proj.weight": "model-00001-of-00002.safetensors"
|
||||
}
|
||||
}
|
||||
27
zamba2_2p7B-instruct_original/special_tokens_map.json
Normal file
27
zamba2_2p7B-instruct_original/special_tokens_map.json
Normal file
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>"
|
||||
],
|
||||
"bos_token": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "[PAD]",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
91123
zamba2_2p7B-instruct_original/tokenizer.json
Normal file
91123
zamba2_2p7B-instruct_original/tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
51
zamba2_2p7B-instruct_original/tokenizer_config.json
Normal file
51
zamba2_2p7B-instruct_original/tokenizer_config.json
Normal file
@@ -0,0 +1,51 @@
|
||||
{
|
||||
"add_bos_token": true,
|
||||
"add_eos_token": false,
|
||||
"add_prefix_space": null,
|
||||
"added_tokens_decoder": {
|
||||
"0": {
|
||||
"content": "[PAD]",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"1": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"2": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>"
|
||||
],
|
||||
"bos_token": "<|im_start|>",
|
||||
"chat_template": "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|im_end|>",
|
||||
"legacy": true,
|
||||
"max_length": 4096,
|
||||
"model_max_length": 4096,
|
||||
"pad_token": "[PAD]",
|
||||
"sp_model_kwargs": {},
|
||||
"spaces_between_special_tokens": false,
|
||||
"stride": 0,
|
||||
"tokenizer_class": "LlamaTokenizer",
|
||||
"truncation_side": "right",
|
||||
"truncation_strategy": "longest_first",
|
||||
"unk_token": null,
|
||||
"use_default_system_prompt": false
|
||||
}
|
||||
Reference in New Issue
Block a user