LLM4Binary/llm4decompile-22b-v2
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: LLM4Binary/llm4decompile-22b-v2
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-05 23:00:57 +08:00
commit 97870ddf4a
20 changed files with 93133 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
.gitattributes vendored Normal file
View File

@@ -0,0 +1,35 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

0
LICENSE Normal file
View File

175
README.md Normal file
View File

@@ -0,0 +1,175 @@
---
license: mit
tags:
- decompile
- binary
---
### 1. Introduction of LLM4Decompile
LLM4Decompile aims to decompile x86 assembly instructions into C. The newly released V2 series are trained with a larger dataset (2B tokens) and a maximum token length of 4,096, with remarkable performance (up to 100% improvement) compared to the previous model.
- **Github Repository:** [LLM4Decompile](https://github.com/albertan017/LLM4Decompile)
### 2. Evaluation Results
| Metrics | Re-executability Rate | | | | | Edit Similarity | | | | |
|:-----------------------:|:---------------------:|:-------:|:-------:|:-------:|:-------:|:---------------:|:-------:|:-------:|:-------:|:-------:|
| Optimization Level | O0 | O1 | O2 | O3 | AVG | O0 | O1 | O2 | O3 | AVG |
| LLM4Decompile-End-6.7B | 0.6805 | 0.3951 | 0.3671 | 0.3720 | 0.4537 | 0.1557 | 0.1292 | 0.1293 | 0.1269 | 0.1353 |
| Ghidra | 0.3476 | 0.1646 | 0.1524 | 0.1402 | 0.2012 | 0.0699 | 0.0613 | 0.0619 | 0.0547 | 0.0620 |
| +GPT-4o | 0.4695 | 0.3415 | 0.2866 | 0.3110 | 0.3522 | 0.0660 | 0.0563 | 0.0567 | 0.0499 | 0.0572 |
| +LLM4Decompile-Ref-1.3B | 0.6890 | 0.3720 | 0.4085 | 0.3720 | 0.4604 | 0.1517 | 0.1325 | 0.1292 | 0.1267 | 0.1350 |
| +LLM4Decompile-Ref-6.7B | 0.7439 | 0.4695 | 0.4756 | 0.4207 | 0.5274 | 0.1559 | 0.1353 | 0.1342 | 0.1273 | 0.1382 |
| +LLM4Decompile-Ref-33B | 0.7073 | 0.4756 | 0.4390 | 0.4146 | 0.5091 | 0.1540 | 0.1379 | 0.1363 | 0.1307 | 0.1397 |
### 3. How to Use
Here is an example of how to use our model (Only for V2. For previous models, please check the corresponding model page at HF).
1. Install Ghidra
Download [Ghidra](https://github.com/NationalSecurityAgency/ghidra/releases/download/Ghidra_11.0.3_build/ghidra_11.0.3_PUBLIC_20240410.zip) to the current folder. You can also check the [page](https://github.com/NationalSecurityAgency/ghidra/releases) for other versions. Unzip the package to the current folder.
In bash, you can use the following:
```bash
cd LLM4Decompile/ghidra
wget https://github.com/NationalSecurityAgency/ghidra/releases/download/Ghidra_11.0.3_build/ghidra_11.0.3_PUBLIC_20240410.zip
unzip ghidra_11.0.3_PUBLIC_20240410.zip
```
2. Install Java-SDK-17
Ghidra 11 is dependent on Java-SDK-17, a simple way to install the SDK on Ubuntu:
```bash
apt-get update
apt-get upgrade
apt install openjdk-17-jdk openjdk-17-jre
```
Please check [Ghidra install guide](https://htmlpreview.github.io/?https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_11.1.1_build/GhidraDocs/InstallationGuide.html) for other platforms.
3. Use Ghidra Headless to decompile binary (demo.py)
Note: **Replace** func0 with the function name you want to decompile.
**Preprocessing:** Compile the C code into binary, and disassemble the binary into assembly instructions.
```python
import os
import subprocess
from tqdm import tqdm,trange
OPT = ["O0", "O1", "O2", "O3"]
timeout_duration = 10
ghidra_path = "./ghidra_11.0.3_PUBLIC/support/analyzeHeadless"#path to the headless analyzer, change the path accordingly
postscript = "./decompile.py"#path to the decompiler helper function, change the path accordingly
project_path = "."#path to temp folder for analysis, change the path accordingly
project_name = "tmp_ghidra_proj"
func_path = "../samples/sample.c"#path to c code for compiling and decompiling, change the path accordingly
fileName = "sample"
with tempfile.TemporaryDirectory() as temp_dir:
pid = os.getpid()
asm_all = {}
for opt in [OPT[0]]:
executable_path = os.path.join(temp_dir, f"{pid}_{opt}.o")
cmd = f'gcc -{opt} -o {executable_path} {func_path} -lm'
subprocess.run(
cmd.split(' '),
check=True,
stdout=subprocess.DEVNULL, # Suppress stdout
stderr=subprocess.DEVNULL, # Suppress stderr
timeout=timeout_duration,
)
output_path = os.path.join(temp_dir, f"{pid}_{opt}.c")
command = [
ghidra_path,
temp_dir,
project_name,
"-import", executable_path,
"-postScript", postscript, output_path,
"-deleteProject", # WARNING: This will delete the project after analysis
]
result = subprocess.run(command, text=True, capture_output=True, check=True)
with open(output_path,'r') as f:
c_decompile = f.read()
c_func = []
flag = 0
for line in c_decompile.split('\n'):
if "Function: func0" in line:#**Replace** func0 with the function name you want to decompile.
flag = 1
c_func.append(line)
continue
if flag:
if '// Function:' in line:
if len(c_func) > 1:
break
c_func.append(line)
if flag == 0:
raise ValueError('bad case no function found')
for idx_tmp in range(1,len(c_func)):##########remove the comments
if 'func0' in c_func[idx_tmp]:
break
c_func = c_func[idx_tmp:]
input_asm = '\n'.join(c_func).strip()
before = f"# This is the assembly code:\n"#prompt
after = "\n# What is the source code?\n"#prompt
input_asm_prompt = before+input_asm.strip()+after
with open(fileName +'_' + opt +'.pseudo','w',encoding='utf-8') as f:
f.write(input_asm_prompt)
```
Ghidra pseudo-code may look like this:
```c
undefined4 func0(float param_1,long param_2,int param_3)
{
int local_28;
int local_24;
local_24 = 0;
do {
local_28 = local_24;
if (param_3 <= local_24) {
return 0;
}
while (local_28 = local_28 + 1, local_28 < param_3) {
if ((double)((ulong)(double)(*(float *)(param_2 + (long)local_24 * 4) -
*(float *)(param_2 + (long)local_28 * 4)) &
SUB168(_DAT_00402010,0)) < (double)param_1) {
return 1;
}
}
local_24 = local_24 + 1;
} while( true );
}
```
4. Refine pseudo-code using LLM4Decompile (demo.py)
**Decompilation:** Use LLM4Decompile-Ref to refine the Ghidra pseudo-code into C:
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_path = 'LLM4Binary/llm4decompile-6.7b-v2' # V2 Model
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16).cuda()
with open(fileName +'_' + OPT[0] +'.pseudo','r') as f:#optimization level O0
asm_func = f.read()
inputs = tokenizer(asm_func, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(**inputs, max_new_tokens=2048)### max length to 4096, max new tokens should be below the range
c_func_decompile = tokenizer.decode(outputs[0][len(inputs[0]):-1])
with open(fileName +'_' + OPT[0] +'.pseudo','r') as f:#original file
func = f.read()
print(f'pseudo function:\n{func}')# Note we only decompile one function, where the original file may contain multiple functions
print(f'refined function:\n{c_func_decompile}')
```
### 4. License
This code repository is licensed under the MIT License.
### 5. Contact
If you have any questions, please raise an issue.

25
config.json Normal file
View File

@@ -0,0 +1,25 @@
{
"architectures": [
"MistralForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 1,
"eos_token_id": 2,
"hidden_act": "silu",
"hidden_size": 6144,
"initializer_range": 0.02,
"intermediate_size": 16384,
"max_position_embeddings": 32768,
"model_type": "mistral",
"num_attention_heads": 48,
"num_hidden_layers": 56,
"num_key_value_heads": 8,
"rms_norm_eps": 1e-05,
"rope_theta": 1000000.0,
"sliding_window": null,
"tie_word_embeddings": false,
"torch_dtype": "bfloat16",
"transformers_version": "4.40.2",
"use_cache": true,
"vocab_size": 32768
}

290
convert_mistral_weights_to_hf-22B.py Normal file
View File

@@ -0,0 +1,290 @@
# Copyright 2023 Mistral AI and The HuggingFace Inc. team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
import gc
import json
import os
import shutil
import warnings
import torch
from safetensors.torch import load_file as safe_load_file
from transformers import (
LlamaTokenizer,
MistralConfig,
MistralForCausalLM,
)
try:
from transformers import LlamaTokenizerFast
tokenizer_class = LlamaTokenizerFast
except ImportError as e:
warnings.warn(e)
warnings.warn(
"The converted tokenizer will be the `slow` tokenizer. To use the fast, update your `tokenizers` library and re-run the tokenizer conversion"
)
tokenizer_class = LlamaTokenizer
"""
Sample usage:
```
python src/transformers/models/mistral/convert_mistral_weights_to_hf.py \
--input_dir /path/to/downloaded/mistral/weights --model_size 22B --output_dir /output/path
```
Thereafter, models can be loaded via:
```py
from transformers import MistralForCausalLM, LlamaTokenizer
model = MistralForCausalLM.from_pretrained("/output/path")
tokenizer = LlamaTokenizer.from_pretrained("/output/path")
```
Important note: you need to be able to host the whole model in RAM to execute this script (even if the biggest versions
come in several checkpoints they each contain a part of each weight of the model, so we need to load them all in RAM).
"""
NUM_SHARDS = {"22B": 1}
def compute_intermediate_size(n, ffn_dim_multiplier=1, multiple_of=256):
return multiple_of * ((int(ffn_dim_multiplier * int(8 * n / 3)) + multiple_of - 1) // multiple_of)
def read_json(path):
with open(path, "r") as f:
return json.load(f)
def write_json(text, path):
with open(path, "w") as f:
json.dump(text, f)
def write_model(model_path, input_base_path, model_size, tokenizer_path=None, safe_serialization=True, is_v3=False):
# for backward compatibility, before you needed the repo to be called `my_repo/model_size`
if not os.path.isfile(os.path.join(input_base_path, "params.json")):
input_base_path = os.path.join(input_base_path, model_size)
os.makedirs(model_path, exist_ok=True)
tmp_model_path = os.path.join(model_path, "tmp")
os.makedirs(tmp_model_path, exist_ok=True)
params = read_json(os.path.join(input_base_path, "params.json"))
num_shards = NUM_SHARDS[model_size]
sliding_window = params.get("sliding_window", None)
# For some reason this is a string in the params.json
if sliding_window is not None:
sliding_window = int(sliding_window)
n_layers = params["n_layers"]
n_heads = params["n_heads"]
n_heads_per_shard = n_heads // num_shards
dim = params["dim"]
dims_per_head = dim // n_heads
base = params.get("rope_theta", 10000.0)
inv_freq = 1.0 / (base ** (torch.arange(0, dims_per_head, 2).float() / dims_per_head))
max_position_embeddings = 4096 * 8
if tokenizer_path is not None:
tokenizer = tokenizer_class(tokenizer_path + ".v3" if is_v3 else "")
tokenizer.save_pretrained(model_path)
vocab_size = tokenizer.vocab_size if tokenizer_path is not None else 32000
if "n_kv_heads" in params:
num_key_value_heads = params["n_kv_heads"] # for GQA / MQA
num_local_key_value_heads = num_key_value_heads // num_shards
key_value_dim = dims_per_head * num_local_key_value_heads
else: # compatibility with other checkpoints
num_key_value_heads = n_heads
num_local_key_value_heads = n_heads_per_shard
key_value_dim = dim
# permute for sliced rotary
def permute(w, n_heads=n_heads, dim1=dim, dim2=dim):
return w.view(n_heads, dim1 // n_heads // 2, 2, dim2).transpose(1, 2).reshape(dim1, dim2)
print(f"Fetching all parameters from the checkpoint at {input_base_path}.")
# Load weights - for v3 models the consolidated weights are in a single file format in safetensors
if is_v3:
loaded = [safe_load_file(os.path.join(input_base_path, "consolidated.safetensors"))]
else:
loaded = [
torch.load(os.path.join(input_base_path, f"consolidated.{i:02d}.pth"), map_location="cpu")
for i in range(num_shards)
]
param_count = 0
index_dict = {"weight_map": {}}
for layer_i in range(n_layers):
filename = f"pytorch_model-{layer_i + 1}-of-{n_layers + 1}.bin"
# Sharded
# Note that attention.w{q,k,v,o}, feed_fordward.w[1,2,3], attention_norm.weight and ffn_norm.weight share
# the same storage object, saving attention_norm and ffn_norm will save other weights too, which is
# redundant as other weights will be stitched from multiple shards. To avoid that, they are cloned.
state_dict = {
f"model.layers.{layer_i}.input_layernorm.weight": loaded[0][
f"layers.{layer_i}.attention_norm.weight"
].clone(),
f"model.layers.{layer_i}.post_attention_layernorm.weight": loaded[0][
f"layers.{layer_i}.ffn_norm.weight"
].clone(),
}
state_dict[f"model.layers.{layer_i}.self_attn.q_proj.weight"] = permute(
torch.cat(
[
loaded[i][f"layers.{layer_i}.attention.wq.weight"].view(n_heads_per_shard, dims_per_head, dim)
for i in range(num_shards)
],
dim=0,
).reshape(dim, dim)
)
state_dict[f"model.layers.{layer_i}.self_attn.k_proj.weight"] = permute(
torch.cat(
[
loaded[i][f"layers.{layer_i}.attention.wk.weight"].view(
num_local_key_value_heads, dims_per_head, dim
)
for i in range(num_shards)
],
dim=0,
).reshape(key_value_dim, dim),
num_key_value_heads,
key_value_dim,
dim,
)
state_dict[f"model.layers.{layer_i}.self_attn.v_proj.weight"] = torch.cat(
[
loaded[i][f"layers.{layer_i}.attention.wv.weight"].view(num_local_key_value_heads, dims_per_head, dim)
for i in range(num_shards)
],
dim=0,
).reshape(key_value_dim, dim)
state_dict[f"model.layers.{layer_i}.self_attn.o_proj.weight"] = torch.cat(
[loaded[i][f"layers.{layer_i}.attention.wo.weight"] for i in range(num_shards)], dim=1
)
state_dict[f"model.layers.{layer_i}.mlp.gate_proj.weight"] = torch.cat(
[loaded[i][f"layers.{layer_i}.feed_forward.w1.weight"] for i in range(num_shards)], dim=0
)
state_dict[f"model.layers.{layer_i}.mlp.down_proj.weight"] = torch.cat(
[loaded[i][f"layers.{layer_i}.feed_forward.w2.weight"] for i in range(num_shards)], dim=1
)
state_dict[f"model.layers.{layer_i}.mlp.up_proj.weight"] = torch.cat(
[loaded[i][f"layers.{layer_i}.feed_forward.w3.weight"] for i in range(num_shards)], dim=0
)
state_dict[f"model.layers.{layer_i}.self_attn.rotary_emb.inv_freq"] = inv_freq
for k, v in state_dict.items():
index_dict["weight_map"][k] = filename
param_count += v.numel()
torch.save(state_dict, os.path.join(tmp_model_path, filename))
filename = f"pytorch_model-{n_layers + 1}-of-{n_layers + 1}.bin"
state_dict = {
"model.norm.weight": loaded[0]["norm.weight"],
"model.embed_tokens.weight": torch.cat([loaded[i]["tok_embeddings.weight"] for i in range(num_shards)], dim=1),
"lm_head.weight": torch.cat([loaded[i]["output.weight"] for i in range(num_shards)], dim=0),
}
for k, v in state_dict.items():
index_dict["weight_map"][k] = filename
param_count += v.numel()
torch.save(state_dict, os.path.join(tmp_model_path, filename))
# Write configs
index_dict["metadata"] = {"total_size": param_count * 2}
write_json(index_dict, os.path.join(tmp_model_path, "pytorch_model.bin.index.json"))
config = MistralConfig(
hidden_size=dim,
intermediate_size=params["hidden_dim"],
num_attention_heads=params["n_heads"],
num_hidden_layers=params["n_layers"],
rms_norm_eps=params["norm_eps"],
num_key_value_heads=num_key_value_heads,
vocab_size=vocab_size,
rope_theta=base,
max_position_embeddings=max_position_embeddings,
sliding_window=sliding_window,
)
config.save_pretrained(tmp_model_path)
# Make space so we can load the model properly now.
del state_dict
del loaded
gc.collect()
print("Loading the checkpoint in a Mistral model.")
model = MistralForCausalLM.from_pretrained(tmp_model_path, torch_dtype=torch.bfloat16, low_cpu_mem_usage=True)
# Avoid saving this as part of the config.
del model.config._name_or_path
model.config.torch_dtype = torch.float16
print("Saving in the Transformers format.")
model.save_pretrained(model_path, safe_serialization=safe_serialization)
shutil.rmtree(tmp_model_path)
def write_tokenizer(tokenizer_path, input_tokenizer_path):
# Initialize the tokenizer based on the `spm` model
print(f"Saving a {tokenizer_class.__name__} to {tokenizer_path}.")
tokenizer = tokenizer_class(input_tokenizer_path)
tokenizer.save_pretrained(tokenizer_path)
def main():
parser = argparse.ArgumentParser()
parser.add_argument(
"--input_dir",
help="Location of Mistral weights, which contains tokenizer.model and model folders",
)
parser.add_argument(
"--model_size",
choices=["22B", "tokenizer_only"],
help="'f' models correspond to the finetuned versions, and are specific to the Mistral2 official release. For more details on Mistral2, checkout the original repo: https://huggingface.co/meta-mistral",
)
parser.add_argument(
"--output_dir",
help="Location to write HF model and tokenizer",
)
parser.add_argument("--safe_serialization", type=bool, help="Whether or not to save using `safetensors`.")
parser.add_argument(
"--is_v3", action="store_true", help="Whether the checkpoints correspond to the 3rd version or not."
)
args = parser.parse_args()
spm_path = os.path.join(args.input_dir, "tokenizer.model")
if args.model_size != "tokenizer_only":
write_model(
model_path=args.output_dir,
input_base_path=args.input_dir,
model_size=args.model_size,
safe_serialization=args.safe_serialization,
tokenizer_path=spm_path,
is_v3=args.is_v3,
)
else:
write_tokenizer(args.output_dir, spm_path)
if __name__ == "__main__":
main()

6
generation_config.json Normal file
View File

@@ -0,0 +1,6 @@
{
"_from_model_config": true,
"bos_token_id": 1,
"eos_token_id": 2,
"transformers_version": "4.40.2"
}

3
model-00001-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:042e004e6b96590b58ad96cf7e894b5885b901ddc9473ecad278c6a046fb2f9a
size 4882298776

3
model-00002-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7dddc146e11aece252c6e632c8d745cb6fcce391dca97999df21e555f260091a
size 4983012160

3
model-00003-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:90ef3a481570204d0269dbb057e7c114943fb7183043fedbfe82c60ea05631e2
size 4957821336

3
model-00004-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:60303704557eac02c2b6f7ad8f3696ae5e6d5dfa62ee3f578e1946e495892e82
size 4882323744

3
model-00005-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e3a955359951b07bda59d17310860c1749255bd0c4f04ff985180e0b25df33d0
size 4983012192

3
model-00006-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4ce0365da717f6b313ce5c1b84f6a43ee9ae33b93368f3a25d11f0c9ec876ea0
size 4957821336

3
model-00007-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:28fd5c9f3e2193a7cec5ceffcf205c698c02a2e59f9435099507febe878f3142
size 4882323744

3
model-00008-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5339bdc8151876346906037c5ac5ec612e12432744201694a2520f1eb4f33dc2
size 4983012192

3
model-00009-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:61d3880f6227dbcd4849152a2ed6e1774f1412e6c0f8afe42e9980ebed6ecb29
size 4982999056

514
model.safetensors.index.json Normal file
View File

@@ -0,0 +1,514 @@
{
"metadata": {
"total_size": 44494565376
},
"weight_map": {
"lm_head.weight": "model-00009-of-00009.safetensors",
"model.embed_tokens.weight": "model-00001-of-00009.safetensors",
"model.layers.0.input_layernorm.weight": "model-00001-of-00009.safetensors",
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.1.input_layernorm.weight": "model-00001-of-00009.safetensors",
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.10.input_layernorm.weight": "model-00002-of-00009.safetensors",
"model.layers.10.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.10.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.10.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.10.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
"model.layers.10.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.10.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.10.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.10.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.11.input_layernorm.weight": "model-00002-of-00009.safetensors",
"model.layers.11.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.11.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.11.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.11.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
"model.layers.11.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.11.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.11.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.11.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.12.input_layernorm.weight": "model-00003-of-00009.safetensors",
"model.layers.12.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.12.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.12.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.12.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
"model.layers.12.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.12.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.12.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.12.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.13.input_layernorm.weight": "model-00003-of-00009.safetensors",
"model.layers.13.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.13.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.13.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.13.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
"model.layers.13.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.13.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.13.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.13.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.14.input_layernorm.weight": "model-00003-of-00009.safetensors",
"model.layers.14.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.14.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.14.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.14.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
"model.layers.14.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.14.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.14.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.14.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.15.input_layernorm.weight": "model-00003-of-00009.safetensors",
"model.layers.15.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.15.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.15.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.15.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
"model.layers.15.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.15.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.15.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.15.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.16.input_layernorm.weight": "model-00003-of-00009.safetensors",
"model.layers.16.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.16.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.16.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.16.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
"model.layers.16.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.16.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.16.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.16.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.17.input_layernorm.weight": "model-00003-of-00009.safetensors",
"model.layers.17.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.17.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.17.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.17.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
"model.layers.17.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.17.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.17.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.17.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.18.input_layernorm.weight": "model-00004-of-00009.safetensors",
"model.layers.18.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.18.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.18.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.18.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
"model.layers.18.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.18.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.18.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.18.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
"model.layers.19.input_layernorm.weight": "model-00004-of-00009.safetensors",
"model.layers.19.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.19.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.19.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.19.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
"model.layers.19.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.19.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.19.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.19.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.2.input_layernorm.weight": "model-00001-of-00009.safetensors",
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.20.input_layernorm.weight": "model-00004-of-00009.safetensors",
"model.layers.20.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.20.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.20.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.20.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
"model.layers.20.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.20.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.20.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.20.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.21.input_layernorm.weight": "model-00004-of-00009.safetensors",
"model.layers.21.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.21.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.21.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.21.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
"model.layers.21.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.21.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.21.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.21.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.22.input_layernorm.weight": "model-00004-of-00009.safetensors",
"model.layers.22.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.22.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.22.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.22.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
"model.layers.22.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.22.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.22.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.22.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.23.input_layernorm.weight": "model-00004-of-00009.safetensors",
"model.layers.23.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.23.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.23.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.23.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
"model.layers.23.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.23.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.23.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.23.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.24.input_layernorm.weight": "model-00005-of-00009.safetensors",
"model.layers.24.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.24.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.24.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.24.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
"model.layers.24.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.24.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.24.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.24.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
"model.layers.25.input_layernorm.weight": "model-00005-of-00009.safetensors",
"model.layers.25.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.25.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.25.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.25.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
"model.layers.25.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.25.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.25.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.25.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.26.input_layernorm.weight": "model-00005-of-00009.safetensors",
"model.layers.26.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.26.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.26.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.26.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
"model.layers.26.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.26.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.26.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.26.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.27.input_layernorm.weight": "model-00005-of-00009.safetensors",
"model.layers.27.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.27.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.27.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.27.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
"model.layers.27.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.27.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.27.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.27.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.28.input_layernorm.weight": "model-00005-of-00009.safetensors",
"model.layers.28.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.28.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.28.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.28.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
"model.layers.28.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.28.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.28.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.28.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.29.input_layernorm.weight": "model-00005-of-00009.safetensors",
"model.layers.29.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.29.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.29.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.29.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
"model.layers.29.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.29.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.29.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.29.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.3.input_layernorm.weight": "model-00001-of-00009.safetensors",
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.30.input_layernorm.weight": "model-00005-of-00009.safetensors",
"model.layers.30.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.30.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.30.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.30.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
"model.layers.30.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.30.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.30.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.30.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.31.input_layernorm.weight": "model-00006-of-00009.safetensors",
"model.layers.31.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.31.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.31.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.31.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
"model.layers.31.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.31.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.31.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.31.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
"model.layers.32.input_layernorm.weight": "model-00006-of-00009.safetensors",
"model.layers.32.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.32.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.32.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.32.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
"model.layers.32.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.32.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.32.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.32.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.33.input_layernorm.weight": "model-00006-of-00009.safetensors",
"model.layers.33.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.33.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.33.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.33.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
"model.layers.33.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.33.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.33.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.33.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.34.input_layernorm.weight": "model-00006-of-00009.safetensors",
"model.layers.34.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.34.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.34.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.34.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
"model.layers.34.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.34.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.34.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.34.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.35.input_layernorm.weight": "model-00006-of-00009.safetensors",
"model.layers.35.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.35.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.35.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.35.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
"model.layers.35.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.35.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.35.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.35.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.36.input_layernorm.weight": "model-00006-of-00009.safetensors",
"model.layers.36.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.36.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.36.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.36.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
"model.layers.36.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.36.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.36.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.36.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.37.input_layernorm.weight": "model-00007-of-00009.safetensors",
"model.layers.37.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.37.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.37.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.37.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
"model.layers.37.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.37.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.37.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.37.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
"model.layers.38.input_layernorm.weight": "model-00007-of-00009.safetensors",
"model.layers.38.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.38.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.38.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.38.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
"model.layers.38.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.38.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.38.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.38.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.39.input_layernorm.weight": "model-00007-of-00009.safetensors",
"model.layers.39.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.39.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.39.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.39.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
"model.layers.39.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.39.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.39.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.39.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.4.input_layernorm.weight": "model-00001-of-00009.safetensors",
"model.layers.4.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.4.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.4.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.4.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.40.input_layernorm.weight": "model-00007-of-00009.safetensors",
"model.layers.40.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.40.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.40.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.40.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
"model.layers.40.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.40.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.40.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.40.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.41.input_layernorm.weight": "model-00007-of-00009.safetensors",
"model.layers.41.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.41.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.41.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.41.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
"model.layers.41.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.41.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.41.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.41.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.42.input_layernorm.weight": "model-00007-of-00009.safetensors",
"model.layers.42.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.42.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.42.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.42.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
"model.layers.42.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.42.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.42.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.42.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.43.input_layernorm.weight": "model-00008-of-00009.safetensors",
"model.layers.43.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.43.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.43.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.43.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
"model.layers.43.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.43.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.43.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.43.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
"model.layers.44.input_layernorm.weight": "model-00008-of-00009.safetensors",
"model.layers.44.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.44.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.44.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.44.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
"model.layers.44.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.44.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.44.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.44.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.45.input_layernorm.weight": "model-00008-of-00009.safetensors",
"model.layers.45.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.45.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.45.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.45.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
"model.layers.45.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.45.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.45.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.45.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.46.input_layernorm.weight": "model-00008-of-00009.safetensors",
"model.layers.46.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.46.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.46.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.46.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
"model.layers.46.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.46.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.46.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.46.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.47.input_layernorm.weight": "model-00008-of-00009.safetensors",
"model.layers.47.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.47.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.47.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.47.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
"model.layers.47.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.47.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.47.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.47.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.48.input_layernorm.weight": "model-00008-of-00009.safetensors",
"model.layers.48.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.48.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.48.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.48.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
"model.layers.48.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.48.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.48.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.48.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.49.input_layernorm.weight": "model-00008-of-00009.safetensors",
"model.layers.49.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.49.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.49.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.49.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
"model.layers.49.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.49.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.49.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.49.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.5.input_layernorm.weight": "model-00002-of-00009.safetensors",
"model.layers.5.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.5.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.5.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.5.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
"model.layers.50.input_layernorm.weight": "model-00009-of-00009.safetensors",
"model.layers.50.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.50.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.50.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.50.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
"model.layers.50.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.50.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.50.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.50.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
"model.layers.51.input_layernorm.weight": "model-00009-of-00009.safetensors",
"model.layers.51.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.51.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.51.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.51.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
"model.layers.51.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.51.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.51.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.51.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.52.input_layernorm.weight": "model-00009-of-00009.safetensors",
"model.layers.52.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.52.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.52.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.52.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
"model.layers.52.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.52.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.52.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.52.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.53.input_layernorm.weight": "model-00009-of-00009.safetensors",
"model.layers.53.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.53.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.53.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.53.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
"model.layers.53.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.53.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.53.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.53.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.54.input_layernorm.weight": "model-00009-of-00009.safetensors",
"model.layers.54.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.54.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.54.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.54.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
"model.layers.54.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.54.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.54.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.54.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.55.input_layernorm.weight": "model-00009-of-00009.safetensors",
"model.layers.55.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.55.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.55.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.55.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
"model.layers.55.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.55.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.55.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.55.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
"model.layers.6.input_layernorm.weight": "model-00002-of-00009.safetensors",
"model.layers.6.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.6.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.6.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.6.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
"model.layers.6.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.6.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.6.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.6.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.7.input_layernorm.weight": "model-00002-of-00009.safetensors",
"model.layers.7.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.7.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.7.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.7.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
"model.layers.7.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.7.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.7.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.7.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.8.input_layernorm.weight": "model-00002-of-00009.safetensors",
"model.layers.8.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.8.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.8.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.8.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
"model.layers.8.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.8.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.8.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.8.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.9.input_layernorm.weight": "model-00002-of-00009.safetensors",
"model.layers.9.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.9.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.9.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.9.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
"model.layers.9.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.9.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.9.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
"model.layers.9.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
"model.norm.weight": "model-00009-of-00009.safetensors"
}
}

23
special_tokens_map.json Normal file
View File

@@ -0,0 +1,23 @@
{
"bos_token": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

91945
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

3
tokenizer.model Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9addc8bdce5988448ae81b729336f43a81262160ae8da760674badab9d4c7d33
size 587591

90
tokenizer_config.json Normal file
View File

@@ -0,0 +1,90 @@
{
"add_bos_token": true,
"add_eos_token": false,
"add_prefix_space": true,
"added_tokens_decoder": {
"0": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"1": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"2": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"3": {
"content": "[INST]",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false,
"special": false
},
"4": {
"content": "[/INST]",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false,
"special": false
},
"10": {
"content": "[IMG]",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false,
"special": false
},
"11": {
"content": "[PREFIX]",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false,
"special": false
},
"12": {
"content": "[MIDDLE]",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false,
"special": false
},
"13": {
"content": "[SUFFIX]",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false,
"special": false
}
},
"bos_token": "<s>",
"clean_up_tokenization_spaces": false,
"eos_token": "</s>",
"legacy": true,
"model_max_length": 1000000000000000019884624838656,
"pad_token": null,
"sp_model_kwargs": {},
"spaces_between_special_tokens": false,
"tokenizer_class": "LlamaTokenizer",
"unk_token": "<unk>",
"use_default_system_prompt": false
}