初始化项目，由ModelHub XC社区提供模型
Model: windprak/open_steuerllm Source: Original Platform
This commit is contained in:
ModelHub XC
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
60
README.md
Normal file
60
README.md
Normal file
@@ -0,0 +1,60 @@
|
||||
---
|
||||
language:
|
||||
- de
|
||||
license: cc-by-nc-4.0
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
datasets:
|
||||
- windprak/steuerllm_pretraining_dataset
|
||||
- windprak/steuerllm_instruct_dataset
|
||||
- windprak/SteuerEx
|
||||
---
|
||||
|
||||
# SteuerLLM: Local specialized large language model for German tax law analysis
|
||||
|
||||
SteuerLLM is a domain-adapted Large Language Model (LLM) consisting of 28 billion parameters, specifically designed for German tax law analysis. It was introduced in the paper [SteuerLLM: Local specialized large language model for German tax law analysis](https://huggingface.co/papers/2602.11081).
|
||||
|
||||
The model excels in domains governed by strict formal rules, precise terminology, and legally binding structures, such as tax law, where correct answers require exact statutory citation, structured legal argumentation, and numerical accuracy.
|
||||
|
||||
- **Paper:** [SteuerLLM: Local specialized large language model for German tax law analysis](https://huggingface.co/papers/2602.11081)
|
||||
- **GitHub Repository:** [https://github.com/windprak/steuerllm](https://github.com/windprak/steuerllm)
|
||||
- **Demo:** [https://steuerllm.i5.ai.fau.de/](https://steuerllm.i5.ai.fau.de/)
|
||||
|
||||
## Model Description
|
||||
|
||||
SteuerLLM is based on an expanded Mistral Small architecture (extended from 24B to 28B parameters through a block expansion method). It was trained on a large-scale synthetic dataset generated from seed questions out of examination material using an automated retrieval-augmented pipeline (websearch).
|
||||
|
||||
The training procedure follows a two-stage approach:
|
||||
1. **Continual Pretraining:** The base model's representations are adapted to tax-specific terminology and concepts by pretraining on domain-filtered web data.
|
||||
2. **Instruction Fine-tuning:** The model is then fine-tuned on synthetically generated question-answer pairs derived from primary German legal sources (e.g., EStG, AO, KStG) using the "Water Fountain Algorithm." This algorithm employs retrieval-augmented generation with semantic ranking to ensure factual grounding and contextual relevance.
|
||||
|
||||
SteuerLLM consistently outperforms general-purpose instruction-tuned models of comparable size and, in several cases, substantially larger systems, demonstrating the critical role of domain-specific data and architectural adaptation for performance on realistic legal reasoning tasks.
|
||||
|
||||
## Evaluation
|
||||
|
||||
The model's performance was evaluated using **SteuerEx**, the first open benchmark derived from authentic German university tax law examinations. SteuerEx comprises 115 expert-validated examination questions spanning six core tax law domains and multiple academic levels, utilizing a statement-level, partial-credit evaluation framework.
|
||||
|
||||
## Usage
|
||||
|
||||
SteuerLLM can be served via various frameworks, including **Transformers**, **vLLM**, and **SGLang**, as it is based on the Mistral architecture.
|
||||
|
||||
**Recommended Inference Parameters:**
|
||||
- **Temperature:** 0.3
|
||||
|
||||
## Citation
|
||||
|
||||
If you use this work, please cite:
|
||||
|
||||
```bibtex
|
||||
@article{steuerllm,
|
||||
author = {Wind, Sebastian and Sopa, Jeta and Schmid, Laurin and Jackl, Quirin and Kiefer, Sebastian and Wu, Fei and Mayr, Martin and Köstler, Harald and Wellein, Gerhard and Maier, Andreas and Tayebi Arasteh, Soroosh},
|
||||
title = {SteuerLLM: Local specialized large language model for German tax law analysis},
|
||||
year = {2026},
|
||||
journal = {arXiv preprint arXiv:2602.11081},
|
||||
url = {https://arxiv.org/abs/2602.11081}
|
||||
}
|
||||
```
|
||||
|
||||
## License
|
||||
CC BY-NC 4.0
|
||||
Research and academic use only.
|
||||
26
config.json
Normal file
26
config.json
Normal file
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"architectures": [
|
||||
"MistralForCausalLM"
|
||||
],
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 1,
|
||||
"eos_token_id": 2,
|
||||
"head_dim": 128,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 5120,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 32768,
|
||||
"max_position_embeddings": 32768,
|
||||
"model_type": "mistral",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 48,
|
||||
"num_key_value_heads": 8,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_theta": 100000000.0,
|
||||
"sliding_window": null,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "bfloat16",
|
||||
"transformers_version": "4.51.3",
|
||||
"use_cache": false,
|
||||
"vocab_size": 131072
|
||||
}
|
||||
8
generation_config.json
Normal file
8
generation_config.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 1,
|
||||
"do_sample": true,
|
||||
"eos_token_id": 2,
|
||||
"temperature": 0.15,
|
||||
"transformers_version": "4.51.3"
|
||||
}
|
||||
3
model-00001-of-00012.safetensors
Normal file
3
model-00001-of-00012.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:77587f5409fe5d5b3e4dc2cfb01dc8a61bf1cb6429052e9d3f557cb89a9e6437
|
||||
size 4781571736
|
||||
3
model-00002-of-00012.safetensors
Normal file
3
model-00002-of-00012.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:caee292b2f250702ddae8a0fa5b2c8a75f0285cf277b1cf0548f76d4683c0b67
|
||||
size 4781592784
|
||||
3
model-00003-of-00012.safetensors
Normal file
3
model-00003-of-00012.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3a213ad0c4c4de19216f9cc9a127c8f3414b91ba435d5cf9c1844758d0a0268b
|
||||
size 4781592800
|
||||
3
model-00004-of-00012.safetensors
Normal file
3
model-00004-of-00012.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:140705893acc3e7174a2f8a244520bdda44ba5b9bb4e147584ae3e1d5220df51
|
||||
size 4886471600
|
||||
3
model-00005-of-00012.safetensors
Normal file
3
model-00005-of-00012.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:80249c5d488f8081d40dacc82dad29bb7203c88ffad13c76274d2c7130a47db6
|
||||
size 4781592824
|
||||
3
model-00006-of-00012.safetensors
Normal file
3
model-00006-of-00012.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9a3c6db206847bcc083310fa3114b1ecc537b88f7740d2ad3056677a04c66802
|
||||
size 4781592816
|
||||
3
model-00007-of-00012.safetensors
Normal file
3
model-00007-of-00012.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:857a227dbf192476402a069587edb8f020a86e478d277f120340dc23f73b5e43
|
||||
size 4886471600
|
||||
3
model-00008-of-00012.safetensors
Normal file
3
model-00008-of-00012.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:323081e7dd952324e5b3d12cd7db0dd2297fd8ab8d24d630d21007fb7e4bb167
|
||||
size 4781592824
|
||||
3
model-00009-of-00012.safetensors
Normal file
3
model-00009-of-00012.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a5765368905e3fc7eaf185e92552a5bb0fa33b41fc2df369bd98804ac3861d89
|
||||
size 4781592816
|
||||
3
model-00010-of-00012.safetensors
Normal file
3
model-00010-of-00012.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:2124acef8ba494a2f454a62632e1813ae8e65674dea3345eb8a4f4a6943ee714
|
||||
size 4886471600
|
||||
3
model-00011-of-00012.safetensors
Normal file
3
model-00011-of-00012.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:06f50d5506d56dc4f46b76a1184913a79b5ba974df3834ec00c446553b0105db
|
||||
size 4781592824
|
||||
3
model-00012-of-00012.safetensors
Normal file
3
model-00012-of-00012.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e08ef2dffe4c8e3eeaa888af62943da344fcdd6a434f45a853db6ff572366324
|
||||
size 3124809416
|
||||
442
model.safetensors.index.json
Normal file
442
model.safetensors.index.json
Normal file
@@ -0,0 +1,442 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 56036894720
|
||||
},
|
||||
"weight_map": {
|
||||
"lm_head.weight": "model-00012-of-00012.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00004-of-00012.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00005-of-00012.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00006-of-00012.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.28.mlp.down_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.28.mlp.gate_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.28.mlp.up_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.28.post_attention_layernorm.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.28.self_attn.o_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.29.mlp.down_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.29.mlp.gate_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.29.mlp.up_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.29.post_attention_layernorm.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.29.self_attn.o_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00007-of-00012.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00012.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.30.mlp.down_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.30.mlp.gate_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.30.mlp.up_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.30.post_attention_layernorm.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.30.self_attn.o_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.31.mlp.down_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.31.mlp.gate_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.31.mlp.up_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.31.post_attention_layernorm.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.31.self_attn.o_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.32.input_layernorm.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.32.mlp.down_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.32.mlp.gate_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.32.mlp.up_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.32.post_attention_layernorm.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.32.self_attn.k_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.32.self_attn.o_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.32.self_attn.q_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.32.self_attn.v_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.33.input_layernorm.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.33.mlp.down_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.33.mlp.gate_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.33.mlp.up_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.33.post_attention_layernorm.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.33.self_attn.k_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.33.self_attn.o_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.33.self_attn.q_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.33.self_attn.v_proj.weight": "model-00008-of-00012.safetensors",
|
||||
"model.layers.34.input_layernorm.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.34.mlp.down_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.34.mlp.gate_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.34.mlp.up_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.34.post_attention_layernorm.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.34.self_attn.k_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.34.self_attn.o_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.34.self_attn.q_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.34.self_attn.v_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.35.input_layernorm.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.35.mlp.down_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.35.mlp.gate_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.35.mlp.up_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.35.post_attention_layernorm.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.35.self_attn.k_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.35.self_attn.o_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.35.self_attn.q_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.35.self_attn.v_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.36.input_layernorm.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.36.mlp.down_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.36.mlp.gate_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.36.mlp.up_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.36.post_attention_layernorm.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.36.self_attn.k_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.36.self_attn.o_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.36.self_attn.q_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.36.self_attn.v_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.37.input_layernorm.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.37.mlp.down_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.37.mlp.gate_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.37.mlp.up_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.37.post_attention_layernorm.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.37.self_attn.k_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.37.self_attn.o_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.37.self_attn.q_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.37.self_attn.v_proj.weight": "model-00009-of-00012.safetensors",
|
||||
"model.layers.38.input_layernorm.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.38.mlp.down_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.38.mlp.gate_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.38.mlp.up_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.38.post_attention_layernorm.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.38.self_attn.k_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.38.self_attn.o_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.38.self_attn.q_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.38.self_attn.v_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.39.input_layernorm.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.39.mlp.down_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.39.mlp.gate_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.39.mlp.up_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.39.post_attention_layernorm.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.39.self_attn.k_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.39.self_attn.o_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.39.self_attn.q_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.39.self_attn.v_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.40.input_layernorm.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.40.mlp.down_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.40.mlp.gate_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.40.mlp.up_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.40.post_attention_layernorm.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.40.self_attn.k_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.40.self_attn.o_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.40.self_attn.q_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.40.self_attn.v_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.41.input_layernorm.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.41.mlp.down_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.41.mlp.gate_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.41.mlp.up_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.41.post_attention_layernorm.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.41.self_attn.k_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.41.self_attn.o_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.41.self_attn.q_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.41.self_attn.v_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.42.input_layernorm.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.42.mlp.down_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.42.mlp.gate_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.42.mlp.up_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.42.post_attention_layernorm.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.42.self_attn.k_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.42.self_attn.o_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.42.self_attn.q_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.42.self_attn.v_proj.weight": "model-00010-of-00012.safetensors",
|
||||
"model.layers.43.input_layernorm.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.43.mlp.down_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.43.mlp.gate_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.43.mlp.up_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.43.post_attention_layernorm.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.43.self_attn.k_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.43.self_attn.o_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.43.self_attn.q_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.43.self_attn.v_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.44.input_layernorm.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.44.mlp.down_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.44.mlp.gate_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.44.mlp.up_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.44.post_attention_layernorm.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.44.self_attn.k_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.44.self_attn.o_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.44.self_attn.q_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.44.self_attn.v_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.45.input_layernorm.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.45.mlp.down_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.45.mlp.gate_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.45.mlp.up_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.45.post_attention_layernorm.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.45.self_attn.k_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.45.self_attn.o_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.45.self_attn.q_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.45.self_attn.v_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.46.input_layernorm.weight": "model-00012-of-00012.safetensors",
|
||||
"model.layers.46.mlp.down_proj.weight": "model-00012-of-00012.safetensors",
|
||||
"model.layers.46.mlp.gate_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.46.mlp.up_proj.weight": "model-00012-of-00012.safetensors",
|
||||
"model.layers.46.post_attention_layernorm.weight": "model-00012-of-00012.safetensors",
|
||||
"model.layers.46.self_attn.k_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.46.self_attn.o_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.46.self_attn.q_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.46.self_attn.v_proj.weight": "model-00011-of-00012.safetensors",
|
||||
"model.layers.47.input_layernorm.weight": "model-00012-of-00012.safetensors",
|
||||
"model.layers.47.mlp.down_proj.weight": "model-00012-of-00012.safetensors",
|
||||
"model.layers.47.mlp.gate_proj.weight": "model-00012-of-00012.safetensors",
|
||||
"model.layers.47.mlp.up_proj.weight": "model-00012-of-00012.safetensors",
|
||||
"model.layers.47.post_attention_layernorm.weight": "model-00012-of-00012.safetensors",
|
||||
"model.layers.47.self_attn.k_proj.weight": "model-00012-of-00012.safetensors",
|
||||
"model.layers.47.self_attn.o_proj.weight": "model-00012-of-00012.safetensors",
|
||||
"model.layers.47.self_attn.q_proj.weight": "model-00012-of-00012.safetensors",
|
||||
"model.layers.47.self_attn.v_proj.weight": "model-00012-of-00012.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00002-of-00012.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00003-of-00012.safetensors",
|
||||
"model.norm.weight": "model-00012-of-00012.safetensors"
|
||||
}
|
||||
}
|
||||
1032
special_tokens_map.json
Normal file
1032
special_tokens_map.json
Normal file
File diff suppressed because it is too large
Load Diff
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b76085f9923309d873994d444989f7eb6ec074b06f25b58f1e8d7b7741070949
|
||||
size 17078037
|
||||
9020
tokenizer_config.json
Normal file
9020
tokenizer_config.json
Normal file
File diff suppressed because it is too large
Load Diff
3
training_args.bin
Normal file
3
training_args.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e328df5f719913a7342f74d188e1705ceda30e2c29901cf9a5efc7a57e8dd1e5
|
||||
size 8977
|
||||
Reference in New Issue
Block a user