rtzr/ko-gemma-2-9b-it
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: rtzr/ko-gemma-2-9b-it
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-25 11:07:16 +08:00
commit d705984fbe
19 changed files with 2702 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

325
README.md Normal file
View File

@@ -0,0 +1,325 @@
---
license: gemma
library_name: transformers
pipeline_tag: text-generation
extra_gated_heading: Access Gemma on Hugging Face
extra_gated_prompt: >-
To access Gemma on Hugging Face, you’re required to review and agree to
Google’s usage license. To do this, please ensure you’re logged in to Hugging
Face and click below. Requests are processed immediately.
extra_gated_button_content: Acknowledge license
tags:
- conversational
base_model:
- google/gemma-2-9b
language:
- ko
---
## Model Details
### Ko-Gemma-2-9B-IT
**[Ko-Gemma-2-9B-IT](https://huggingface.co/rtzr/ko-gemma-2-9b-it)** is a Korean-language conversational model that is part of the Gemma family of models. It is a text-to-text, decoder-only large language model, available in Korean. We fine-tuned this model on a carefully curated high-quality dataset using Supervised Fine-Tuning (SFT). And we use [Direct Preference Optimization](https://arxiv.org/abs/2305.18290) training specifically for Human Feedback. The datasets include:
- [Orca-Math](https://huggingface.co/datasets/kuotient/orca-math-korean-dpo-pairs)
- [dpo-mix-7k](https://huggingface.co/datasets/argilla/dpo-mix-7k)
Some of these datasets were partially used and translated for training. In particular, a lot of repetition occurred during the translation process, so preprocessing was performed based on N-gram.
#### *Inputs and outputs*
- **Input:** Text string, such as a question, a prompt, or a document to be summarized.
- **Output:** Generated Korean-language text in response to the input, such as an answer to a question, or a summary of a document.
### Google Gemma 2
Gemma is a family of lightweight, state-of-the-art open models from Google,
built from the same research and technology used to create the Gemini models.
They are text-to-text, decoder-only large language models, available in English,
with open weights for both pre-trained variants and instruction-tuned variants.
Gemma models are well-suited for a variety of text generation tasks, including
question answering, summarization, and reasoning. Their relatively small size
makes it possible to deploy them in environments with limited resources such as
a laptop, desktop or your own cloud infrastructure, democratizing access to
state of the art AI models and helping foster innovation for everyone.
## Benchmark Scores
We evaluated it internally using [LogicKor](https://github.com/instructkr/LogicKor) code. While the public LogicKor code is assessed as GPT-4, our internal evaluation was conducted as GPT-4o. Public scores will be added as they are released. The scores below include only 0-shot evaluations.
| Model | Math | Reasoning | Writing | Coding | Understanding | Grammar | Single ALL | Multi ALL | Overall |
|:---------:|:-----:|:------:|:-----:|:-----:|:----:|:-----:|:-----:|:-----:|:----:|
| [rtzr/ko-gemma-2-9b-it](https://huggingface.co/rtzr/ko-gemma-2-9b-it) | 8.71 / 8.00 | 9.14 / 8.00 | 9.43 / 9.29 | 9.00 / 9.43 | 9.57 / 9.86 | 7.14 / 5.00 | 8.83 | 8.26 | 8.55 |
| [google/gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it) | 8.57 / 7.71 | 8.86 / 7.00 | 9.29 / 9.29 | 9.29 / 9.57 | 8.57 / 8.29 | 6.86 / 3.86 | 8.57 | 7.62 | 8.10 |
| [MLP-KTLim/llama-3-Korean-Bllossom-8B](https://huggingface.co/MLP-KTLim/llama-3-Korean-Bllossom-8B) | 6.43 / 5.71 | 6.86 / 5.14 | 9.14 / 8.57 | 8.29 / 8.14 | 8.43 / 9.29 | 5.71 / 5.29 | 7.48 | 7.02 | 7.25 |
| [yanolja/EEVE-Korean-Instruct-10.8B-v1.0](https://huggingface.co/yanolja/EEVE-Korean-Instruct-10.8B-v1.0) | 5.57 / 4.29 | 8.14 / 5.14 | 8.29 / 6.29 | 6.43 / 7.86 | 9.29 / 8.57 | 6.57 / 3.71 | 7.38 | 5.98 | 6.68 |
| [allganize/Llama-3-Alpha-Ko-8B-Instruct](https://huggingface.co/allganize/Llama-3-Alpha-Ko-8B-Instruct) | 4.57 / 3.00 | 6.86 / 6.43 | 7.43 / 6.71 | 8.43 / 8.43| 7.71 / 8.71 | 6.71 / 4.43 | 6.95 | 6.29 | 6.62 |
## Usage
### Install Dependencies
You must install transformers >= 4.42.3 for gemma2 models.
```bash
pip install transformers==4.42.3 accelerate
```
### Python code with Pipeline
```python
import transformers
import torch
model_id = "rtzr/ko-gemma-2-9b-it"
pipeline = transformers.pipeline(
"text-generation",
model=model_id,
model_kwargs={"torch_dtype": torch.bfloat16},
device_map="auto",
)
pipeline.model.eval()
instruction = "서울의 유명한 관광 코스를 만들어줄래?"
messages = [
{"role": "user", "content": f"{instruction}"}
]
prompt = pipeline.tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
terminators = [
pipeline.tokenizer.eos_token_id,
pipeline.tokenizer.convert_tokens_to_ids("<end_of_turn>")
]
outputs = pipeline(
prompt,
max_new_tokens=2048,
eos_token_id=terminators,
do_sample=True,
temperature=0.6,
top_p=0.9,
)
print(outputs[0]["generated_text"][len(prompt):])
```
```markdown
서울은 역사, 문화, 현대성이 조화를 이룬 매력적인 도시입니다. 여기서 즐길 수 있는 다양한 관광지와 명소를 소개합니다. 다음은 서울의 유명한 관광 코스 3가지입니다.
**1. 역사와 문화를 둘러싼 한국관광코스**
1. **경복궁**: 조선 시대의 웅장한 왕궁을 만끽할 수 있는 곳입니다. 특히 매년 봄에 열리는 '춘추연회'는 경복궁의 아름다움을 더욱 돋보이게 합니다.
2. **북촌 한옥마을**: 고풍스러운 한옥이 모여있는 곳으로, 전통 문화 체험이 가능합니다. '북촌 한옥마을 문화체험관'에서는 한복 체험부터 종이만화, 한글 쓰기 등 다양한 프로그램이 준비되어 있습니다.
3. **인사동**: 서점, 미술관, 한식당이 많은 곳입니다. 특히 '인사동 문화관'에서는 서울의 역사와 문화를 이해하는 데 도움이 되는 전시를 볼 수 있습니다.
4. **광화문** 및 **명동**: 현대적인 쇼핑과 레스토랑이 즐비한 곳입니다. 광화문은 특히 젊은이들이 많은 곳으로, 스트리트 패션을 관찰하거나 밤거리에서 활기를 느낄 수 있습니다.
**2. 도시의 모습을 바라보는 뷰투어 코스**
1. **남산타워**: 서울의 상징적인 건물로, 꼭대기에서 펼쳐지는 360도의 경치가 압니다. 특히 밤이 되면 조명이 어우러져 더욱 아름다워집니다.
2. **서울타워**: 남산타워와 비슷한 위치로, 높이가 더 높기 때문에 더 넓은 전망을 볼 수 있습니다. 서울타워 내부에는 다양한 전시관과 레스토랑도 있습니다.
3. **북악산**: 서울의 중심부에 위치한 산으로, 서울의 경치를 조금 다른 관점에서 볼 수 있습니다. 특히 북악산 정상인 북악사에서도 좋은 전망을 볼 수 있습니다.
4. **서울숲**: 녹지 공간으로, 도시의 혼잡함에서 벗어날 수 있는 곳입니다. 또한, 서울숲 내부에는 '서울숲 아트프레젠트'라는 공간이 있어 예술과 자연을 함께 체험할 수 있습니다.
**3. 현대 문화를 만나는 코스**
1. **삼성동**: 현대 미술관이 많은 곳으로, '삼성 미술관', '아모리카나스 갤러리' 등이 있습니다. 또한, '코엑스'나 '아포카로포스' 등의 명소도 가까운 곳에 있습니다.
2. **이태원**: 외국인들이 많은 곳으로, 다양한 외국 음식을 즐길 수 있는 곳입니다. 또한, '이태원 글로컬문화센터'에서는 세계 각국의 문화 체험이 가능합니다.
3. **홍대**: 젊은이들의 문화가 넘치는 곳입니다. '홍대 롤링홀'은 특히 많은 사람들이 방문하는 곳입니다. 또한, '홍대 서점거리'에서는 독서와 문화를 만날 수 있습니다.
4. **강남**: 서울의 현대적 모습을 잘 보여주는 곳입니다. '강남역'을 중심으로 많은 고급 쇼핑몰과 레스토랑이 있습니다.
이러한 코스를 통해 서울의 다양한 모습을 한 번에 만나볼 수 있을 거예요. 각자의 취향에 맞춰 코스를 조절하시면 좋겠습니다. 즐거운 여행 되세요!
```
### Python code with AutoModel
```python
import os
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "rtzr/ko-gemma-2-9b-it"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
model.eval()
instruction = "서울의 유명한 관광 코스를 만들어줄래?"
messages = [
{"role": "user", "content": f"{instruction}"}
]
input_ids = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
terminators = [
tokenizer.eos_token_id,
tokenizer.convert_tokens_to_ids("<end_of_turn>")
]
outputs = model.generate(
input_ids,
max_new_tokens=2048,
eos_token_id=terminators,
do_sample=True,
temperature=0.6,
top_p=0.9,
)
print(tokenizer.decode(outputs[0][input_ids.shape[-1]:], skip_special_tokens=True))
```
```markdown
서울 관광 코스를 제안해드릴게요. 하루 종일 즐겁게 여행할 수 있는 루트로 구성했습니다.
### 1. 서울역사관 및 북촌한옥마을(오전)
- 서울역사관: 서울의 역사와 문화를 체험할 수 있는 곳입니다. 다양한 전시물과 상설전시를 통해 서울의 변화를 살펴볼 수 있습니다.
- 북촌한옥마을: 서울의 한옥을 보존하고 관리하는 곳입니다. 조선 시대의 분위기를 느낄 수 있으며, 한옥에서 문화 콘텐츠도 제공하는 곳도 많습니다.
### 2. 북악산 입장과 북악산 등산(오전)
- 북악산은 서울의 북쪽에 위치한 산으로, 서울 한복판에서도 자연을 만날 수 있는 곳입니다. 북악산 입구에서 등산을 시작하여, 북악산 정상까지 올라가면 서울의 전경을 볼 수 있습니다.
### 3. 종로 명동 쇼핑과 맛집 투어(낮)
- 명동: 다양한 쇼핑몰과 매장이 있는 곳입니다. 명동 쇼핑타운, 미스터트위스터, 미스터리마켓 등을 방문해보세요.
- 맛집 투어: 명동에는 다양한 지역 음식을 먹을 수 있는 곳이 많습니다. 떡볶이, 순대, 닭강정 등을 맛볼 수 있는 곳을 추천드립니다.
### 4. 서울시립미술관과 덕수궁(오후)
- 서울시립미술관: 현대미술을 전시하는 곳입니다. 특별전이 열린다면 방문해 볼 수 있습니다.
- 덕수궁: 조선시대의 궁궐입니다. 특히 봄에는 벚꽃이 아름답게 만발합니다.
### 5. 남산타워와 남산공원 산책(오후)
- 남산타워: 남산에 있는 관람대입니다. 남산타워에 올라가면 서울의 360도 전경을 볼 수 있습니다.
- 남산공원: 남산에 있는 공원입니다. 다양한 테마 공원과 조경이 잘 된 곳입니다. 남산공원을 산책하며 휴식을 취할 수 있습니다.
### 6. 명동 또는 이태원에서의 저녁 식사와 문화 활동(저녁)
- 명동: 다양한 전통적인 한국 음식을 먹을 수 있는 곳입니다. 또한, 명동은 밤에도 활기차게 활발한 문화 생활을 할 수 있는 곳입니다.
- 이태원: 외국인 관광객들이 많이 찾는 곳으로, 다양한 세계 음식을 먹을 수 있으며, 클럽이나 바가 많은 문화적 활동이 가능한 곳입니다.
이 코스는 하루 종일 활발하게 여행을 할 수 있도록 계획했습니다. 각 지역에 따라 이동 시간을 고려하시고, 개장 시간과 전시 일정 등을 미리 확인하시는 것이 좋습니다. 즐거운 여행 되세요!
```
### Quantized Versions through bitsandbytes
- *Using 8-bit precision*
- *Using 4-bit precision*
```python
# pip install bitsandbytes
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
model_id = "rtzr/ko-gemma-2-9b-it"
quantization_config_8bit = BitsAndBytesConfig(load_in_8bit=True)
# quantization_config_4bit = BitsAndBytesConfig(load_in_4bit=True)
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
quantization_config=quantization_config_8bit,
# quantization_config=quantization_config_4bit,
low_cpu_mem_usage=True,
)
model.eval()
instruction = "서울의 유명한 관광 코스를 만들어줄래?"
messages = [
{"role": "user", "content": f"{instruction}"}
]
input_ids = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
terminators = [
tokenizer.eos_token_id,
tokenizer.convert_tokens_to_ids("<end_of_turn>")
]
outputs = model.generate(
input_ids,
max_new_tokens=2048,
eos_token_id=terminators,
do_sample=True,
temperature=0.6,
top_p=0.9,
)
print(tokenizer.decode(outputs[0][input_ids.shape[-1]:], skip_special_tokens=True))
```
### VLLM Usage
When we use `vllm==0.5.1`, the gemma2 model cannot be loaded yet and the following [issue](https://github.com/vllm-project/vllm/issues/6237) occurs. So it is recommended to use `vllm/vllm-openai:latest` docker or [`vllm==0.5.0.post1`](https://github.com/vllm-project/vllm/releases/tag/v0.5.0.post1).
```bash
#!/bin/bash
VLLM_ATTENTION_BACKEND=FLASHINFER
MODEL_NAME="rtzr/ko-gemma-2-9b-it"
MODEL_PATH="YOUR_PATH/${MODEL_NAME}"
docker run --rm --gpus all \
-p 8000:8000 \
--shm-size=12gb --ulimit memlock=-1 --ulimit stack=67108864 \
-e VLLM_ATTENTION_BACKEND=${VLLM_ATTENTION_BACKEND} \
-v $MODEL_PATH:/vllm-workspace/${MODEL_NAME} \
vllm/vllm-openai:latest \
--model ${MODEL_NAME} --dtype auto \
--gpu-memory-utilization 0.8
```
## License
Gemma 2 License: <https://ai.google.dev/gemma/terms>
## Citation
```none
@article{RTZR,
title={ko-gemma-2-9b-it},
author={Return Zero Team},
year={2024},
url={https://huggingface.co/rtzr/ko-gemma-2-9b-it}
}
```
```none
@article{gemma_2024,
title={Gemma},
url={https://www.kaggle.com/m/3301},
DOI={10.34740/KAGGLE/M/3301},
publisher={Kaggle},
author={Gemma Team},
year={2024}
}
```

33
config.json Normal file
View File

@@ -0,0 +1,33 @@
{
"architectures": [
"Gemma2ForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"attn_logit_softcapping": 50.0,
"bos_token_id": 2,
"cache_implementation": "hybrid",
"eos_token_id": 1,
"final_logit_softcapping": 30.0,
"head_dim": 256,
"hidden_act": "gelu_pytorch_tanh",
"hidden_activation": "gelu_pytorch_tanh",
"hidden_size": 3584,
"initializer_range": 0.02,
"intermediate_size": 14336,
"max_position_embeddings": 8192,
"model_type": "gemma2",
"num_attention_heads": 16,
"num_hidden_layers": 42,
"num_key_value_heads": 8,
"pad_token_id": 0,
"query_pre_attn_scalar": 256,
"rms_norm_eps": 1e-06,
"rope_theta": 10000.0,
"sliding_window": 4096,
"sliding_window_size": 4096,
"torch_dtype": "bfloat16",
"transformers_version": "4.42.3",
"use_cache": true,
"vocab_size": 256000
}

8
generation_config.json Normal file
View File

@@ -0,0 +1,8 @@
{
"_from_model_config": true,
"bos_token_id": 2,
"cache_implementation": "hybrid",
"eos_token_id": 1,
"pad_token_id": 0,
"transformers_version": "4.42.3"
}

3
model-00001-of-00010.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f0f6610a462c6077d8cb1c1a181dd0b63dba354ba5524b4cd7964688e6f6c1b9
size 1923088992

3
model-00002-of-00010.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:61bef8c5bdf7c60e435a584be55d03a126a74f79e301d327596f6d045c5a2499
size 1981958408

3
model-00003-of-00010.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c8e2219d492cc3026f1cda8a2af4d0117baaffa1a53347561533d7e198adf325
size 1981958400

3
model-00004-of-00010.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:036184f456da5c8b4a9e2f3d8032aef9c3c788cddcb06ded444bf348d80715c1
size 1981958464

3
model-00005-of-00010.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e6f18b81963ca65c2ae42fac51c5905b291f11ff71ac7bfa3bc6f2a826de6ea9
size 1981958464

3
model-00006-of-00010.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cd76c045f517ba3cbe875bf5e67465fac37156b9659f502320cbb9b326fd9f0a
size 1981958464

3
model-00007-of-00010.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1cc34946c0516a23803e14709e12c6bc585500e93d8e3e8afb58b7caed6872f5
size 1981958464

3
model-00008-of-00010.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:780d9d1f950d50e7012871db378a67a44e421a57790dca7e7709fae39f3d37fa
size 1981958464

3
model-00009-of-00010.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0ba9054ff9121a29459185f57ad75b5fd8690baa2f9263c1b341bbdbc4e52f4d
size 1981958464

3
model-00010-of-00010.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b49ef392f85cb689c765d8925e9ea966961c717b7c852832ff91fb64d7310e0c
size 704709784

471
model.safetensors.index.json Normal file
View File

@@ -0,0 +1,471 @@
{
"metadata": {
"total_size": 18483411968
},
"weight_map": {
"model.embed_tokens.weight": "model-00001-of-00010.safetensors",
"model.layers.0.input_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.0.mlp.down_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.0.mlp.gate_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.0.mlp.up_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.0.post_attention_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.0.post_feedforward_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.0.pre_feedforward_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00010.safetensors",
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00010.safetensors",
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00010.safetensors",
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00010.safetensors",
"model.layers.1.input_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.1.mlp.down_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.1.mlp.gate_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.1.mlp.up_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.1.post_attention_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.1.post_feedforward_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.1.pre_feedforward_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.1.self_attn.k_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.1.self_attn.o_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.1.self_attn.q_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.1.self_attn.v_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.10.input_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.10.mlp.down_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.10.mlp.gate_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.10.mlp.up_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.10.post_attention_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.10.post_feedforward_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.10.pre_feedforward_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.10.self_attn.k_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.10.self_attn.o_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.10.self_attn.q_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.10.self_attn.v_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.11.input_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.11.mlp.down_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.11.mlp.gate_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.11.mlp.up_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.11.post_attention_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.11.post_feedforward_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.11.pre_feedforward_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.11.self_attn.k_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.11.self_attn.o_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.11.self_attn.q_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.11.self_attn.v_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.12.input_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.12.mlp.down_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.12.mlp.gate_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.12.mlp.up_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.12.post_attention_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.12.post_feedforward_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.12.pre_feedforward_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.12.self_attn.k_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.12.self_attn.o_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.12.self_attn.q_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.12.self_attn.v_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.13.input_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.13.mlp.down_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.13.mlp.gate_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.13.mlp.up_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.13.post_attention_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.13.post_feedforward_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.13.pre_feedforward_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.13.self_attn.k_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.13.self_attn.o_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.13.self_attn.q_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.13.self_attn.v_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.14.input_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.14.mlp.down_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.14.mlp.gate_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.14.mlp.up_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.14.post_attention_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.14.post_feedforward_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.14.pre_feedforward_layernorm.weight": "model-00004-of-00010.safetensors",
"model.layers.14.self_attn.k_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.14.self_attn.o_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.14.self_attn.q_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.14.self_attn.v_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.15.input_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.15.mlp.down_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.15.mlp.gate_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.15.mlp.up_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.15.post_attention_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.15.post_feedforward_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.15.pre_feedforward_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.15.self_attn.k_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.15.self_attn.o_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.15.self_attn.q_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.15.self_attn.v_proj.weight": "model-00004-of-00010.safetensors",
"model.layers.16.input_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.16.mlp.down_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.16.mlp.gate_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.16.mlp.up_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.16.post_attention_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.16.post_feedforward_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.16.pre_feedforward_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.16.self_attn.k_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.16.self_attn.o_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.16.self_attn.q_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.16.self_attn.v_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.17.input_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.17.mlp.down_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.17.mlp.gate_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.17.mlp.up_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.17.post_attention_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.17.post_feedforward_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.17.pre_feedforward_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.17.self_attn.k_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.17.self_attn.o_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.17.self_attn.q_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.17.self_attn.v_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.18.input_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.18.mlp.down_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.18.mlp.gate_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.18.mlp.up_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.18.post_attention_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.18.post_feedforward_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.18.pre_feedforward_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.18.self_attn.k_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.18.self_attn.o_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.18.self_attn.q_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.18.self_attn.v_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.19.input_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.19.mlp.down_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.19.mlp.gate_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.19.mlp.up_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.19.post_attention_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.19.post_feedforward_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.19.pre_feedforward_layernorm.weight": "model-00005-of-00010.safetensors",
"model.layers.19.self_attn.k_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.19.self_attn.o_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.19.self_attn.q_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.19.self_attn.v_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.2.input_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.2.mlp.down_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.2.mlp.gate_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.2.mlp.up_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.2.post_attention_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.2.post_feedforward_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.2.pre_feedforward_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.2.self_attn.k_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.2.self_attn.o_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.2.self_attn.q_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.2.self_attn.v_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.20.input_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.20.mlp.down_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.20.mlp.gate_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.20.mlp.up_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.20.post_attention_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.20.post_feedforward_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.20.pre_feedforward_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.20.self_attn.k_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.20.self_attn.o_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.20.self_attn.q_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.20.self_attn.v_proj.weight": "model-00005-of-00010.safetensors",
"model.layers.21.input_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.21.mlp.down_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.21.mlp.gate_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.21.mlp.up_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.21.post_attention_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.21.post_feedforward_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.21.pre_feedforward_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.21.self_attn.k_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.21.self_attn.o_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.21.self_attn.q_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.21.self_attn.v_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.22.input_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.22.mlp.down_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.22.mlp.gate_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.22.mlp.up_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.22.post_attention_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.22.post_feedforward_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.22.pre_feedforward_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.22.self_attn.k_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.22.self_attn.o_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.22.self_attn.q_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.22.self_attn.v_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.23.input_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.23.mlp.down_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.23.mlp.gate_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.23.mlp.up_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.23.post_attention_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.23.post_feedforward_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.23.pre_feedforward_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.23.self_attn.k_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.23.self_attn.o_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.23.self_attn.q_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.23.self_attn.v_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.24.input_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.24.mlp.down_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.24.mlp.gate_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.24.mlp.up_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.24.post_attention_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.24.post_feedforward_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.24.pre_feedforward_layernorm.weight": "model-00006-of-00010.safetensors",
"model.layers.24.self_attn.k_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.24.self_attn.o_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.24.self_attn.q_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.24.self_attn.v_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.25.input_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.25.mlp.down_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.25.mlp.gate_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.25.mlp.up_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.25.post_attention_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.25.post_feedforward_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.25.pre_feedforward_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.25.self_attn.k_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.25.self_attn.o_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.25.self_attn.q_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.25.self_attn.v_proj.weight": "model-00006-of-00010.safetensors",
"model.layers.26.input_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.26.mlp.down_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.26.mlp.gate_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.26.mlp.up_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.26.post_attention_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.26.post_feedforward_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.26.pre_feedforward_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.26.self_attn.k_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.26.self_attn.o_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.26.self_attn.q_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.26.self_attn.v_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.27.input_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.27.mlp.down_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.27.mlp.gate_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.27.mlp.up_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.27.post_attention_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.27.post_feedforward_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.27.pre_feedforward_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.27.self_attn.k_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.27.self_attn.o_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.27.self_attn.q_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.27.self_attn.v_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.28.input_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.28.mlp.down_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.28.mlp.gate_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.28.mlp.up_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.28.post_attention_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.28.post_feedforward_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.28.pre_feedforward_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.28.self_attn.k_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.28.self_attn.o_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.28.self_attn.q_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.28.self_attn.v_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.29.input_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.29.mlp.down_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.29.mlp.gate_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.29.mlp.up_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.29.post_attention_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.29.post_feedforward_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.29.pre_feedforward_layernorm.weight": "model-00007-of-00010.safetensors",
"model.layers.29.self_attn.k_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.29.self_attn.o_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.29.self_attn.q_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.29.self_attn.v_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.3.input_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.3.mlp.down_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.3.mlp.gate_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.3.mlp.up_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.3.post_attention_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.3.post_feedforward_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.3.pre_feedforward_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.3.self_attn.k_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.3.self_attn.o_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.3.self_attn.q_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.3.self_attn.v_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.30.input_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.30.mlp.down_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.30.mlp.gate_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.30.mlp.up_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.30.post_attention_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.30.post_feedforward_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.30.pre_feedforward_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.30.self_attn.k_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.30.self_attn.o_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.30.self_attn.q_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.30.self_attn.v_proj.weight": "model-00007-of-00010.safetensors",
"model.layers.31.input_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.31.mlp.down_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.31.mlp.gate_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.31.mlp.up_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.31.post_attention_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.31.post_feedforward_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.31.pre_feedforward_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.31.self_attn.k_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.31.self_attn.o_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.31.self_attn.q_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.31.self_attn.v_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.32.input_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.32.mlp.down_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.32.mlp.gate_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.32.mlp.up_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.32.post_attention_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.32.post_feedforward_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.32.pre_feedforward_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.32.self_attn.k_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.32.self_attn.o_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.32.self_attn.q_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.32.self_attn.v_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.33.input_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.33.mlp.down_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.33.mlp.gate_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.33.mlp.up_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.33.post_attention_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.33.post_feedforward_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.33.pre_feedforward_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.33.self_attn.k_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.33.self_attn.o_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.33.self_attn.q_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.33.self_attn.v_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.34.input_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.34.mlp.down_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.34.mlp.gate_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.34.mlp.up_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.34.post_attention_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.34.post_feedforward_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.34.pre_feedforward_layernorm.weight": "model-00008-of-00010.safetensors",
"model.layers.34.self_attn.k_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.34.self_attn.o_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.34.self_attn.q_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.34.self_attn.v_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.35.input_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.35.mlp.down_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.35.mlp.gate_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.35.mlp.up_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.35.post_attention_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.35.post_feedforward_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.35.pre_feedforward_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.35.self_attn.k_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.35.self_attn.o_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.35.self_attn.q_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.35.self_attn.v_proj.weight": "model-00008-of-00010.safetensors",
"model.layers.36.input_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.36.mlp.down_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.36.mlp.gate_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.36.mlp.up_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.36.post_attention_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.36.post_feedforward_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.36.pre_feedforward_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.36.self_attn.k_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.36.self_attn.o_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.36.self_attn.q_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.36.self_attn.v_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.37.input_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.37.mlp.down_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.37.mlp.gate_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.37.mlp.up_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.37.post_attention_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.37.post_feedforward_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.37.pre_feedforward_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.37.self_attn.k_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.37.self_attn.o_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.37.self_attn.q_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.37.self_attn.v_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.38.input_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.38.mlp.down_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.38.mlp.gate_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.38.mlp.up_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.38.post_attention_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.38.post_feedforward_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.38.pre_feedforward_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.38.self_attn.k_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.38.self_attn.o_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.38.self_attn.q_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.38.self_attn.v_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.39.input_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.39.mlp.down_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.39.mlp.gate_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.39.mlp.up_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.39.post_attention_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.39.post_feedforward_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.39.pre_feedforward_layernorm.weight": "model-00009-of-00010.safetensors",
"model.layers.39.self_attn.k_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.39.self_attn.o_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.39.self_attn.q_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.39.self_attn.v_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.4.input_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.4.mlp.down_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.4.mlp.gate_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.4.mlp.up_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.4.post_attention_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.4.post_feedforward_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.4.pre_feedforward_layernorm.weight": "model-00002-of-00010.safetensors",
"model.layers.4.self_attn.k_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.4.self_attn.o_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.4.self_attn.q_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.4.self_attn.v_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.40.input_layernorm.weight": "model-00010-of-00010.safetensors",
"model.layers.40.mlp.down_proj.weight": "model-00010-of-00010.safetensors",
"model.layers.40.mlp.gate_proj.weight": "model-00010-of-00010.safetensors",
"model.layers.40.mlp.up_proj.weight": "model-00010-of-00010.safetensors",
"model.layers.40.post_attention_layernorm.weight": "model-00010-of-00010.safetensors",
"model.layers.40.post_feedforward_layernorm.weight": "model-00010-of-00010.safetensors",
"model.layers.40.pre_feedforward_layernorm.weight": "model-00010-of-00010.safetensors",
"model.layers.40.self_attn.k_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.40.self_attn.o_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.40.self_attn.q_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.40.self_attn.v_proj.weight": "model-00009-of-00010.safetensors",
"model.layers.41.input_layernorm.weight": "model-00010-of-00010.safetensors",
"model.layers.41.mlp.down_proj.weight": "model-00010-of-00010.safetensors",
"model.layers.41.mlp.gate_proj.weight": "model-00010-of-00010.safetensors",
"model.layers.41.mlp.up_proj.weight": "model-00010-of-00010.safetensors",
"model.layers.41.post_attention_layernorm.weight": "model-00010-of-00010.safetensors",
"model.layers.41.post_feedforward_layernorm.weight": "model-00010-of-00010.safetensors",
"model.layers.41.pre_feedforward_layernorm.weight": "model-00010-of-00010.safetensors",
"model.layers.41.self_attn.k_proj.weight": "model-00010-of-00010.safetensors",
"model.layers.41.self_attn.o_proj.weight": "model-00010-of-00010.safetensors",
"model.layers.41.self_attn.q_proj.weight": "model-00010-of-00010.safetensors",
"model.layers.41.self_attn.v_proj.weight": "model-00010-of-00010.safetensors",
"model.layers.5.input_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.5.mlp.down_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.5.mlp.gate_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.5.mlp.up_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.5.post_attention_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.5.post_feedforward_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.5.pre_feedforward_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.5.self_attn.k_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.5.self_attn.o_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.5.self_attn.q_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.5.self_attn.v_proj.weight": "model-00002-of-00010.safetensors",
"model.layers.6.input_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.6.mlp.down_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.6.mlp.gate_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.6.mlp.up_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.6.post_attention_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.6.post_feedforward_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.6.pre_feedforward_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.6.self_attn.k_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.6.self_attn.o_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.6.self_attn.q_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.6.self_attn.v_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.7.input_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.7.mlp.down_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.7.mlp.gate_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.7.mlp.up_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.7.post_attention_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.7.post_feedforward_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.7.pre_feedforward_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.7.self_attn.k_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.7.self_attn.o_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.7.self_attn.q_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.7.self_attn.v_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.8.input_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.8.mlp.down_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.8.mlp.gate_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.8.mlp.up_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.8.post_attention_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.8.post_feedforward_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.8.pre_feedforward_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.8.self_attn.k_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.8.self_attn.o_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.8.self_attn.q_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.8.self_attn.v_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.9.input_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.9.mlp.down_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.9.mlp.gate_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.9.mlp.up_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.9.post_attention_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.9.post_feedforward_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.9.pre_feedforward_layernorm.weight": "model-00003-of-00010.safetensors",
"model.layers.9.self_attn.k_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.9.self_attn.o_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.9.self_attn.q_proj.weight": "model-00003-of-00010.safetensors",
"model.layers.9.self_attn.v_proj.weight": "model-00003-of-00010.safetensors",
"model.norm.weight": "model-00010-of-00010.safetensors"
}
}

34
special_tokens_map.json Normal file
View File

@@ -0,0 +1,34 @@
{
"additional_special_tokens": [
"<start_of_turn>",
"<end_of_turn>"
],
"bos_token": {
"content": "<bos>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "<eos>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "<pad>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

3
tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7da53ca29fb16f6b2489482fc0bc6a394162cdab14d12764a1755ebc583fea79
size 17518525

3
tokenizer.model Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:61a7b147390c64585d6c3543dd6fc636906c9af3865a5548f27f31aee1d4c8e2
size 4241003

1759
tokenizer_config.json Normal file
View File

File diff suppressed because it is too large Load Diff