openaccess-ai-collective/hippogriff-30b-chat
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: openaccess-ai-collective/hippogriff-30b-chat
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-21 23:56:31 +08:00
commit b2f9567ba7
18 changed files with 94372 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
.gitattributes vendored Normal file
View File

@@ -0,0 +1,34 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

105
README.md Normal file
View File

@@ -0,0 +1,105 @@
---
datasets:
- QingyiSi/Alpaca-CoT
- teknium/GPT4-LLM-Cleaned
- teknium/GPTeacher-General-Instruct
- metaeval/ScienceQA_text_only
- hellaswag
- openai/summarize_from_feedback
- riddle_sense
- gsm8k
- OpenAssistant/oasst1
language:
- en
library_name: transformers
pipeline_tag: text-generation
---
# Hippogriff 30B Chat
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
Hippogriff 30B Chat is an experiment that builds on Manticore with new datasets, while removing a few more instruction and chat datasets. It also includes a de-duped subset of the Pygmalion dataset. It also removes all Alpaca style prompts using `###` in favor of
chat only style prompts using `USER:`,`ASSISTANT:` as well as [pygmalion/metharme prompting](https://huggingface.co/PygmalionAI/metharme-7b#prompting) using `<|system|>, <|user|> and <|model|>` tokens.
Questions, comments, feedback, looking to donate, or want to help? Reach out on our [Discord](https://discord.gg/PugNNHAF5r) or email [wing@openaccessaicollective.org](mailto:wing@openaccessaicollective.org)
# Training Datasets
Hippogriff 30B Chat is a Llama 30B model fine-tuned on the following datasets
- OpenAssistant/oasst1 - cleaned dataset, similar to Guanaco
- synthetic jokes generation and explanation derived from reddit jokes dataset
- synthetic prose generation and rewriting self-chat
- Q&A based on provided context
- self instruct augmented logic_inference_oa
- de-duped pygmalion dataset, filtered down to RP data, cleaned, english only, 25%
- [riddle_sense](https://huggingface.co/datasets/riddle_sense) - instruct augmented
- hellaswag, updated for detailed explanations w 30K+ rows
- [gsm8k](https://huggingface.co/datasets/gsm8k) - instruct augmented
- [ewof/code-alpaca-instruct-unfiltered](https://huggingface.co/datasets/ewof/code-alpaca-instruct-unfiltered) synthetic self chat dataset derived from about 1000 rows
- [subset of QingyiSi/Alpaca-CoT for roleplay and CoT](https://huggingface.co/QingyiSi/Alpaca-CoT)
- [GPTeacher-General-Instruct](https://huggingface.co/datasets/teknium/GPTeacher-General-Instruct)
- ARC-Easy & ARC-Challenge - instruct augmented for detailed responses, derived from the `train` split
- [hellaswag](https://huggingface.co/datasets/hellaswag) - 5K row subset of instruct augmented for concise responses, derived from the `train` split
- [metaeval/ScienceQA_text_only](https://huggingface.co/datasets/metaeval/ScienceQA_text_only) - instruct for concise responses
- [openai/summarize_from_feedback](https://huggingface.co/datasets/openai/summarize_from_feedback) - instruct augmented tl;dr summarization
Hippogriff differs from Manticore as it does not use the WizardLM, WizardVicuna, Alpaca, or ShareGPT datasets.
# Initial thoughts
Hippogriff follows instructions pretty well. It still struggles with anything that has to do with math. Prose is much better than manticore. Handles in-context QA much better.
# Shoutouts
Special thanks to Nanobit for helping with Axolotl, TheBloke for quantizing these models are more accessible to all, 0x000011b for the RP dataset, and the support from everyone in our AI Discord community.
# Demo
A Spaces demo is not provided for this release due to 30B models currently not fitting in VRAM.
## Build
Hippogriff was built with [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) on 8xA100 80GB
- 1.5 epochs taking approximately 12 hours. No further epochs will be released.
## Bias, Risks, and Limitations
Hippogriff has not been aligned to human preferences with techniques like RLHF or deployed with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so).
Hippogriff was fine-tuned from the base model LlaMa 30B, please refer to its model card's Limitations Section for relevant information.
The core functionality of Llama is taking a string of text and predicting the next token. While language models are widely used for tasks other than this,
there are a lot of unknowns with this work. When prompting Hippogriff it is important to remember that the statistically most likely next token is often
not the token that produces the most "accurate" text. Never depend upon Hippogriff to produce factually accurate output.
Llama was trained on the Pile, a dataset known to contain profanity, lewd, and otherwise abrasive language. Depending upon use case Hippogriff may produce
socially unacceptable text. See Sections 5 and 6 of the Pile paper for a more detailed analysis of the biases in the Pile.
As with all language models, it is hard to predict in advance how Hippogriff will respond to particular prompts and offensive content may occur without warning.
We recommend having a human curate or filter the outputs before releasing them, both to censor undesirable content and to improve the quality of the results.
## Examples
TBD
```
USER: write an original short story of at least 300 words in the style of Zora Neale Hurston. Incorporate the following literary devices: Euphemism and Foreshadowing.
ASSISTANT: Once upon a time, there was a black girl named Rose, whose skin shined like a pearl beneath the hot sun. She lived with her mama and daddy on their farm, where they grew crops to feed the family and neighbors. On this particular day, Rose felt a tinge of excitement as she watched her parents work diligently in their fields. 'It's almost harvest season!' she thought, imagining the delicious meals that awaited them. But, soon after, it became apparent that something sinister lurked amidst the rows of fresh produce.
As her father hoed around a vine-covered cucumber plant, he unearthed a peculiar rock. It was the size of an acorn but had a strange shape, unlike anything he or his wife had ever seen. "What have we here?" He said, turning to his wife with curiosity and excitement. She examined the rock for herself and declared, "Why, it looks like a piece of coal!" But as both mother and daughter looked closer, it seemed to be far more than a mere piece of coal. A sense of unease crept up Rose's spine when she noticed that the rock resembled the dark face of a baby in the womb. As Rose tried to suppress a scream, her mother gasped, "Sweet Jesus! Let's bury this thing before anyone sees!" And so, they quickly hid the stone back underground, hoping that nobody would ever discover their secret.
```
```
USER: tell me 5 words that start with "ex" and then the sum of 33 and 77
ASSISTANT: Excite, exotic, exhibit, excitable, excessive
```
```
USER: sally (a girl) has 3 brothers. Each brother has 2 sisters. how many sisters does sally have?
ASSISTANT: Sally has 3 brothers, and each brother has 2 sisters. To find out how many sisters Sally has in total, we need to multiply the number of bro
thers by the number of sisters per brother, then add 1 to account for Sally herself. So, 3 * 2 + 1 = 6 sisters in total
```

BIN
axolotl-badge-web.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 11 KiB

24
config.json Normal file
View File

@@ -0,0 +1,24 @@
{
"_name_or_path": "/workspace/mnt/manticore-30b-chat/checkpoint-50/",
"architectures": [
"LlamaForCausalLM"
],
"bos_token_id": 1,
"eos_token_id": 2,
"hidden_act": "silu",
"hidden_size": 6656,
"initializer_range": 0.02,
"intermediate_size": 17920,
"max_position_embeddings": 2048,
"max_sequence_length": 2048,
"model_type": "llama",
"num_attention_heads": 52,
"num_hidden_layers": 60,
"pad_token_id": 0,
"rms_norm_eps": 1e-06,
"tie_word_embeddings": false,
"torch_dtype": "bfloat16",
"transformers_version": "4.30.0.dev0",
"use_cache": true,
"vocab_size": 32000
}

127
configs/hippogriff.yml Normal file
View File

@@ -0,0 +1,127 @@
base_model: huggyllama/llama-30b
base_model_config: huggyllama/llama-30b
model_type: LlamaForCausalLM
tokenizer_type: LlamaTokenizer
load_in_8bit: false
load_in_4bit: false
strict: false
push_dataset_to_hub: winglian
hf_use_auth_token: true
datasets:
- path: winglian/pygmalion-cleaned
data_files:
- v13_no_ai.cleaned.jsonl
type: pygmalion
- path: winglian/evals
data_files:
- hf/ARC-Challenge.jsonl
- hf/ARC-Easy.jsonl
- hf/riddle_sense.jsonl
type: explainchoice:chat
- path: winglian/evals
data_files:
- hf/gsm8k.jsonl
- custom/logic_inference_oa.jsonl
type: alpaca_chat.load_qa
- path: winglian/evals
data_files:
- custom/in_context_qa.jsonl
type: context_qa
- path: winglian/evals
data_files:
- custom/in_context_qa.jsonl
type: context_qa.load_404
- path: winglian/evals
data_files:
- custom/jokes_explained_500up.jsonl
type: sharegpt_jokes
- path: winglian/evals
data_files:
- custom/classify-self-chat.sharegpt.jsonl
- custom/coding-self-chat.sharegpt.jsonl
- custom/prose-gpt4.sharegpt.jsonl
- custom/prose-rewrite-gpt4.sharegpt.jsonl
type: sharegpt_simple
- path: winglian/evals
data_files:
- custom/guanaco-cleaned.en.jsonl
type: sharegpt_simple.load_guanaco
- path: winglian/evals
data_files:
- openai/tldr.jsonl
type: summarizetldr:chat
- path: winglian/evals
data_files:
- hellaswag/hellaswag.jsonl
type: explainchoice:chat
shards: 60
- path: metaeval/ScienceQA_text_only
type: concisechoice:chat
shards: 13
- path: teknium/GPTeacher-General-Instruct
data_files: gpt4-instruct-similarity-0.6-dataset.json
type: gpteacher:chat
- path: QingyiSi/Alpaca-CoT
data_files:
- Chain-of-Thought/formatted_cot_data/aqua_train.json
- Chain-of-Thought/formatted_cot_data/creak_train.json
- Chain-of-Thought/formatted_cot_data/ecqa_train.json
- Chain-of-Thought/formatted_cot_data/esnli_train.json
- Chain-of-Thought/formatted_cot_data/gsm8k_train.json
- Chain-of-Thought/formatted_cot_data/qasc_train.json
- Chain-of-Thought/formatted_cot_data/qed_train.json
- Chain-of-Thought/formatted_cot_data/sensemaking_train.json
- Chain-of-Thought/formatted_cot_data/strategyqa_train.json
- GPTeacher/Roleplay/formatted_roleplay-similarity_0.6-instruct-dataset.json
type: alpaca_chat
dataset_prepared_path: /workspace/mnt/last_run_prepared
val_set_size: 0.02
adapter:
lora_model_dir:
sequence_len: 2048
max_packed_sequence_len: 2048
lora_r: 64
lora_alpha: 32
lora_dropout: 0.000001
lora_target_modules:
lora_target_linear: true
lora_fan_in_fan_out:
wandb_project: hippogriff-30b-chat
wandb_watch:
wandb_run_id:
wandb_log_model:
output_dir: /workspace/mnt/hippogriff-30b-chat
gradient_accumulation_steps: 1
micro_batch_size: 20
num_epochs: 2
optimizer:
torchdistx_path:
lr_scheduler:
learning_rate: 0.00003
train_on_inputs: false
group_by_length: true
bf16: true
fp16: false
float16: false
tf32: true
gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention: true
flash_attention:
gptq_groupsize:
gptq_model_v1:
warmup_steps: 200
eval_steps: 20
save_steps: 50
debug:
deepspeed:
weight_decay: 0.000003
fsdp:
fsdp_config:
special_tokens:
bos_token: "<s>"
eos_token: "</s>"
unk_token: "<unk>"

7
generation_config.json Normal file
View File

@@ -0,0 +1,7 @@
{
"_from_model_config": true,
"bos_token_id": 1,
"eos_token_id": 2,
"pad_token_id": 0,
"transformers_version": "4.30.0.dev0"
}

3
pytorch_model-00001-of-00007.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:79ebb2b0f54dd52f6269e2726d9c9043cd69ddca184323fe0e0283997418f7f6
size 9818324691

3
pytorch_model-00002-of-00007.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9a068bd33a5f7be7b6bdfe4b0fe22c3f61487be5ba5af2f575da9161d9fd5331
size 9958102871

3
pytorch_model-00003-of-00007.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d0ae353a10f56293aac6810298798339f4627b1c997641b0ad397c626fd045bf
size 9896734779

3
pytorch_model-00004-of-00007.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8d2b2fbe4fb0e7aa9abc95c40164ccc834d4618121f8ddfd63abc3965a72d480
size 9869470609

3
pytorch_model-00005-of-00007.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0013e962f2f8d7da6d202e1027d28817c94f1d2c034d4424c5cdfa5d58fefd86
size 9869470573

3
pytorch_model-00006-of-00007.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cc15649979f252e0c3ebc7bd77b4ea0867ae60aee479585f9f854985774a0d60
size 9958102871

3
pytorch_model-00007-of-00007.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fe01abed55644b13476e154990438c9fa18c6b91680817be57c2110565941413
size 5687903281

610
pytorch_model.bin.index.json Normal file
View File

@@ -0,0 +1,610 @@
{
"metadata": {
"total_size": 65057894912
},
"weight_map": {
"lm_head.weight": "pytorch_model-00007-of-00007.bin",
"model.embed_tokens.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.0.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00007.bin",
"model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.1.input_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.1.mlp.down_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.1.mlp.up_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.1.post_attention_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.1.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00007.bin",
"model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.10.input_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.10.mlp.down_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.10.mlp.gate_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.10.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.10.post_attention_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.10.self_attn.k_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.10.self_attn.o_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.10.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.10.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00007.bin",
"model.layers.10.self_attn.v_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.11.input_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.11.mlp.down_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.11.mlp.gate_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.11.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.11.post_attention_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.11.self_attn.k_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.11.self_attn.o_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.11.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.11.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00007.bin",
"model.layers.11.self_attn.v_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.12.input_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.12.mlp.down_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.12.mlp.gate_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.12.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.12.post_attention_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.12.self_attn.k_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.12.self_attn.o_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.12.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.12.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00007.bin",
"model.layers.12.self_attn.v_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.13.input_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.13.mlp.down_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.13.mlp.gate_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.13.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.13.post_attention_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.13.self_attn.k_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.13.self_attn.o_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.13.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.13.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00007.bin",
"model.layers.13.self_attn.v_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.14.input_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.14.mlp.down_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.14.mlp.gate_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.14.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.14.post_attention_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.14.self_attn.k_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.14.self_attn.o_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.14.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.14.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00007.bin",
"model.layers.14.self_attn.v_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.15.input_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.15.mlp.down_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.15.mlp.gate_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.15.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.15.post_attention_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.15.self_attn.k_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.15.self_attn.o_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.15.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.15.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00007.bin",
"model.layers.15.self_attn.v_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.16.input_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.16.mlp.down_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.16.mlp.gate_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.16.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.16.post_attention_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.16.self_attn.k_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.16.self_attn.o_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.16.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.16.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00007.bin",
"model.layers.16.self_attn.v_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.17.input_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.17.mlp.down_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.17.mlp.gate_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.17.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.17.post_attention_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.17.self_attn.k_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.17.self_attn.o_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.17.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.17.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00007.bin",
"model.layers.17.self_attn.v_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.18.input_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.18.mlp.down_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.18.mlp.gate_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.18.mlp.up_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.18.post_attention_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.18.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.18.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.18.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.18.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00007.bin",
"model.layers.18.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.19.input_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.19.mlp.down_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.19.mlp.gate_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.19.mlp.up_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.19.post_attention_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.19.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.19.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.19.self_attn.q_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.19.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00007.bin",
"model.layers.19.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.2.input_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.2.mlp.down_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.2.mlp.gate_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.2.mlp.up_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.2.post_attention_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.2.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00007.bin",
"model.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.20.input_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.20.mlp.down_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.20.mlp.gate_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.20.mlp.up_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.20.post_attention_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.20.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.20.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.20.self_attn.q_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.20.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00007.bin",
"model.layers.20.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.21.input_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.21.mlp.down_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.21.mlp.gate_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.21.mlp.up_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.21.post_attention_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.21.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.21.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.21.self_attn.q_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.21.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00007.bin",
"model.layers.21.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.22.input_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.22.mlp.down_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.22.mlp.gate_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.22.mlp.up_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.22.post_attention_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.22.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.22.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.22.self_attn.q_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.22.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00007.bin",
"model.layers.22.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.23.input_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.23.mlp.down_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.23.mlp.gate_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.23.mlp.up_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.23.post_attention_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.23.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.23.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.23.self_attn.q_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.23.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00007.bin",
"model.layers.23.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.24.input_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.24.mlp.down_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.24.mlp.gate_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.24.mlp.up_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.24.post_attention_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.24.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.24.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.24.self_attn.q_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.24.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00007.bin",
"model.layers.24.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.25.input_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.25.mlp.down_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.25.mlp.gate_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.25.mlp.up_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.25.post_attention_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.25.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.25.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.25.self_attn.q_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.25.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00007.bin",
"model.layers.25.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.26.input_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.26.mlp.down_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.26.mlp.gate_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.26.mlp.up_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.26.post_attention_layernorm.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.26.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.26.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.26.self_attn.q_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.26.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00007.bin",
"model.layers.26.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.27.input_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.27.mlp.down_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.27.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.27.mlp.up_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.27.post_attention_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.27.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.27.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.27.self_attn.q_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.27.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00007.bin",
"model.layers.27.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
"model.layers.28.input_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.28.mlp.down_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.28.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.28.mlp.up_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.28.post_attention_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.28.self_attn.k_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.28.self_attn.o_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.28.self_attn.q_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.28.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00007.bin",
"model.layers.28.self_attn.v_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.29.input_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.29.mlp.down_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.29.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.29.mlp.up_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.29.post_attention_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.29.self_attn.k_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.29.self_attn.o_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.29.self_attn.q_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.29.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00007.bin",
"model.layers.29.self_attn.v_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.3.input_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.3.mlp.down_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.3.mlp.gate_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.3.mlp.up_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.3.post_attention_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.3.self_attn.k_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.3.self_attn.o_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.3.self_attn.q_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.3.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00007.bin",
"model.layers.3.self_attn.v_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.30.input_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.30.mlp.down_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.30.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.30.mlp.up_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.30.post_attention_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.30.self_attn.k_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.30.self_attn.o_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.30.self_attn.q_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.30.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00007.bin",
"model.layers.30.self_attn.v_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.31.input_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.31.mlp.down_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.31.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.31.mlp.up_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.31.post_attention_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.31.self_attn.k_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.31.self_attn.o_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.31.self_attn.q_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.31.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00007.bin",
"model.layers.31.self_attn.v_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.32.input_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.32.mlp.down_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.32.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.32.mlp.up_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.32.post_attention_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.32.self_attn.k_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.32.self_attn.o_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.32.self_attn.q_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.32.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00007.bin",
"model.layers.32.self_attn.v_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.33.input_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.33.mlp.down_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.33.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.33.mlp.up_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.33.post_attention_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.33.self_attn.k_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.33.self_attn.o_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.33.self_attn.q_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.33.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00007.bin",
"model.layers.33.self_attn.v_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.34.input_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.34.mlp.down_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.34.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.34.mlp.up_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.34.post_attention_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.34.self_attn.k_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.34.self_attn.o_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.34.self_attn.q_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.34.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00007.bin",
"model.layers.34.self_attn.v_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.35.input_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.35.mlp.down_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.35.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.35.mlp.up_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.35.post_attention_layernorm.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.35.self_attn.k_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.35.self_attn.o_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.35.self_attn.q_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.35.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00007.bin",
"model.layers.35.self_attn.v_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.36.input_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.36.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.36.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.36.mlp.up_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.36.post_attention_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.36.self_attn.k_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.36.self_attn.o_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.36.self_attn.q_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.36.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00007.bin",
"model.layers.36.self_attn.v_proj.weight": "pytorch_model-00004-of-00007.bin",
"model.layers.37.input_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.37.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.37.mlp.gate_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.37.mlp.up_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.37.post_attention_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.37.self_attn.k_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.37.self_attn.o_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.37.self_attn.q_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.37.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00007.bin",
"model.layers.37.self_attn.v_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.38.input_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.38.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.38.mlp.gate_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.38.mlp.up_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.38.post_attention_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.38.self_attn.k_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.38.self_attn.o_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.38.self_attn.q_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.38.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00007.bin",
"model.layers.38.self_attn.v_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.39.input_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.39.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.39.mlp.gate_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.39.mlp.up_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.39.post_attention_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.39.self_attn.k_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.39.self_attn.o_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.39.self_attn.q_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.39.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00007.bin",
"model.layers.39.self_attn.v_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.4.input_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.4.mlp.down_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.4.mlp.gate_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.4.mlp.up_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.4.post_attention_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.4.self_attn.k_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.4.self_attn.o_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.4.self_attn.q_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.4.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00007.bin",
"model.layers.4.self_attn.v_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.40.input_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.40.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.40.mlp.gate_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.40.mlp.up_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.40.post_attention_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.40.self_attn.k_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.40.self_attn.o_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.40.self_attn.q_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.40.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00007.bin",
"model.layers.40.self_attn.v_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.41.input_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.41.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.41.mlp.gate_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.41.mlp.up_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.41.post_attention_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.41.self_attn.k_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.41.self_attn.o_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.41.self_attn.q_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.41.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00007.bin",
"model.layers.41.self_attn.v_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.42.input_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.42.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.42.mlp.gate_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.42.mlp.up_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.42.post_attention_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.42.self_attn.k_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.42.self_attn.o_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.42.self_attn.q_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.42.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00007.bin",
"model.layers.42.self_attn.v_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.43.input_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.43.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.43.mlp.gate_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.43.mlp.up_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.43.post_attention_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.43.self_attn.k_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.43.self_attn.o_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.43.self_attn.q_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.43.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00007.bin",
"model.layers.43.self_attn.v_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.44.input_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.44.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.44.mlp.gate_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.44.mlp.up_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.44.post_attention_layernorm.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.44.self_attn.k_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.44.self_attn.o_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.44.self_attn.q_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.44.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00007.bin",
"model.layers.44.self_attn.v_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.45.input_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.45.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.45.mlp.gate_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.45.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.45.post_attention_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.45.self_attn.k_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.45.self_attn.o_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.45.self_attn.q_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.45.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00007.bin",
"model.layers.45.self_attn.v_proj.weight": "pytorch_model-00005-of-00007.bin",
"model.layers.46.input_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.46.mlp.down_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.46.mlp.gate_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.46.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.46.post_attention_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.46.self_attn.k_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.46.self_attn.o_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.46.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.46.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00007.bin",
"model.layers.46.self_attn.v_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.47.input_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.47.mlp.down_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.47.mlp.gate_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.47.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.47.post_attention_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.47.self_attn.k_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.47.self_attn.o_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.47.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.47.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00007.bin",
"model.layers.47.self_attn.v_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.48.input_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.48.mlp.down_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.48.mlp.gate_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.48.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.48.post_attention_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.48.self_attn.k_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.48.self_attn.o_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.48.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.48.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00007.bin",
"model.layers.48.self_attn.v_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.49.input_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.49.mlp.down_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.49.mlp.gate_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.49.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.49.post_attention_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.49.self_attn.k_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.49.self_attn.o_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.49.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.49.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00007.bin",
"model.layers.49.self_attn.v_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.5.input_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.5.mlp.down_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.5.mlp.gate_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.5.mlp.up_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.5.post_attention_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.5.self_attn.k_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.5.self_attn.o_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.5.self_attn.q_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.5.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00007.bin",
"model.layers.5.self_attn.v_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.50.input_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.50.mlp.down_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.50.mlp.gate_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.50.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.50.post_attention_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.50.self_attn.k_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.50.self_attn.o_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.50.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.50.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00007.bin",
"model.layers.50.self_attn.v_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.51.input_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.51.mlp.down_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.51.mlp.gate_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.51.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.51.post_attention_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.51.self_attn.k_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.51.self_attn.o_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.51.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.51.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00007.bin",
"model.layers.51.self_attn.v_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.52.input_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.52.mlp.down_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.52.mlp.gate_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.52.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.52.post_attention_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.52.self_attn.k_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.52.self_attn.o_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.52.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.52.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00007.bin",
"model.layers.52.self_attn.v_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.53.input_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.53.mlp.down_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.53.mlp.gate_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.53.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.53.post_attention_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.53.self_attn.k_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.53.self_attn.o_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.53.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.53.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00007.bin",
"model.layers.53.self_attn.v_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.54.input_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.54.mlp.down_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.54.mlp.gate_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.54.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.54.post_attention_layernorm.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.54.self_attn.k_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.54.self_attn.o_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.54.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.54.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00007.bin",
"model.layers.54.self_attn.v_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.55.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.55.mlp.down_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.55.mlp.gate_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.55.mlp.up_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.55.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.55.self_attn.k_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.55.self_attn.o_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.55.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
"model.layers.55.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00007.bin",
"model.layers.55.self_attn.v_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.56.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.56.mlp.down_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.56.mlp.gate_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.56.mlp.up_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.56.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.56.self_attn.k_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.56.self_attn.o_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.56.self_attn.q_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.56.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00007.bin",
"model.layers.56.self_attn.v_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.57.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.57.mlp.down_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.57.mlp.gate_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.57.mlp.up_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.57.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.57.self_attn.k_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.57.self_attn.o_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.57.self_attn.q_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.57.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00007.bin",
"model.layers.57.self_attn.v_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.58.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.58.mlp.down_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.58.mlp.gate_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.58.mlp.up_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.58.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.58.self_attn.k_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.58.self_attn.o_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.58.self_attn.q_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.58.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00007.bin",
"model.layers.58.self_attn.v_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.59.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.59.mlp.down_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.59.mlp.gate_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.59.mlp.up_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.59.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.59.self_attn.k_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.59.self_attn.o_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.59.self_attn.q_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.59.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00007.bin",
"model.layers.59.self_attn.v_proj.weight": "pytorch_model-00007-of-00007.bin",
"model.layers.6.input_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.6.mlp.down_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.6.mlp.gate_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.6.mlp.up_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.6.post_attention_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.6.self_attn.k_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.6.self_attn.o_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.6.self_attn.q_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.6.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00007.bin",
"model.layers.6.self_attn.v_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.7.input_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.7.mlp.down_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.7.mlp.gate_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.7.mlp.up_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.7.post_attention_layernorm.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.7.self_attn.k_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.7.self_attn.o_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.7.self_attn.q_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.7.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00007.bin",
"model.layers.7.self_attn.v_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.8.input_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.8.mlp.down_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.8.mlp.gate_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.8.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.8.post_attention_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.8.self_attn.k_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.8.self_attn.o_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.8.self_attn.q_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.8.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00007.bin",
"model.layers.8.self_attn.v_proj.weight": "pytorch_model-00001-of-00007.bin",
"model.layers.9.input_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.9.mlp.down_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.9.mlp.gate_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.9.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.9.post_attention_layernorm.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.9.self_attn.k_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.9.self_attn.o_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.9.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.layers.9.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00007.bin",
"model.layers.9.self_attn.v_proj.weight": "pytorch_model-00002-of-00007.bin",
"model.norm.weight": "pytorch_model-00007-of-00007.bin"
}
}

23
special_tokens_map.json Normal file
View File

@@ -0,0 +1,23 @@
{
"bos_token": {
"content": "<s>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "</s>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "<unk>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
}
}

93385
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

BIN
tokenizer.model (Stored with Git LFS) Normal file
View File

Binary file not shown.

33
tokenizer_config.json Normal file
View File

@@ -0,0 +1,33 @@
{
"add_bos_token": true,
"add_eos_token": false,
"bos_token": {
"__type": "AddedToken",
"content": "<s>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"clean_up_tokenization_spaces": false,
"eos_token": {
"__type": "AddedToken",
"content": "</s>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"model_max_length": 2048,
"pad_token": null,
"sp_model_kwargs": {},
"tokenizer_class": "LlamaTokenizer",
"unk_token": {
"__type": "AddedToken",
"content": "<unk>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
}
}