初始化项目，由ModelHub XC社区提供模型
Model: NucleusAI/nucleus-22B-token-500B Source: Original Platform
This commit is contained in:
ModelHub XC
59
.gitattributes
vendored
Normal file
59
.gitattributes
vendored
Normal file
@@ -0,0 +1,59 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zstandard filter=lfs diff=lfs merge=lfs -text
|
||||
*.tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
*.db* filter=lfs diff=lfs merge=lfs -text
|
||||
*.ark* filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.gguf* filter=lfs diff=lfs merge=lfs -text
|
||||
*.ggml filter=lfs diff=lfs merge=lfs -text
|
||||
*.llamafile* filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
model-00009-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
model-00002-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
model-00001-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
model-00004-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
model-00006-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
model-00003-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
model-00007-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.model filter=lfs diff=lfs merge=lfs -text
|
||||
model-00008-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
model-00005-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
93
README.md
Normal file
93
README.md
Normal file
@@ -0,0 +1,93 @@
|
||||
---
|
||||
license: mit
|
||||
language:
|
||||
- en
|
||||
---
|
||||
|
||||
# 🚀 Nucleus-22B-token-500B
|
||||
|
||||
**Nucleus-22B-token-500B is a 22B parameters causal decoder-only model built by Nucleus.AI and trained on 500B tokens of [RefinedWeb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb) along with curated corpora. It is made available under the MIT license.**
|
||||
|
||||
*1T-token model coming soon* 😊.
|
||||
|
||||
|
||||
## What about Nucleus-22B-token-500B?
|
||||
|
||||
* **It performs well compared to similar-size open-source models** (e.g., [MPT-7B](https://huggingface.co/mosaicml/mpt-7b), [StableLM](https://github.com/Stability-AI/StableLM), [RedPajama](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-7B-v0.1) etc.), thanks to being trained on 500B tokens of [RefinedWeb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb) enhanced with curated corpora. See the [OpenLLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
|
||||
* **It is made available under an MIT license**.
|
||||
* **It is trained by a small team of four passionate for Open Source**
|
||||
|
||||
⚠️ **This is a raw, pretrained model, which should be further finetuned for most usecases.**
|
||||
|
||||
# Model Card for Nucleus-22B-token-500B
|
||||
|
||||
## Model Details
|
||||
|
||||
### Model Description
|
||||
|
||||
- **Developed by:** NucleusAI;
|
||||
- **Model type:** Causal decoder-only;
|
||||
- **Language(s) (NLP):** English;
|
||||
- **License:** MIT.
|
||||
|
||||
### Model Source
|
||||
|
||||
- **Paper:** *coming soon*.
|
||||
|
||||
## Uses
|
||||
|
||||
### Direct Use
|
||||
|
||||
Research on large language models; as a foundation for further specialization and finetuning for specific usecases (e.g., summarization, text generation, chatbot, etc.)
|
||||
|
||||
### Out-of-Scope Use
|
||||
|
||||
Production use without adequate assessment of risks and mitigation; any use cases which may be considered irresponsible or harmful.
|
||||
|
||||
## Bias, Risks, and Limitations
|
||||
|
||||
Nucleus-22B-token-500B is trained on English data only, and will not generalize appropriately to other languages. Furthermore, as it is trained on a large-scale corpora representative of the web, it will carry the stereotypes and biases commonly encountered online.
|
||||
|
||||
### Recommendations
|
||||
|
||||
We recommend users of Nucleus-22B-token-500B to consider finetuning it for the specific set of tasks of interest, and for guardrails and appropriate precautions to be taken for any production use.
|
||||
|
||||
## How to Get Started with the Mode
|
||||
|
||||
|
||||
## Training Details
|
||||
|
||||
### Training Data
|
||||
|
||||
Nucleus-22B-token-500B was trained on 500B tokens of [RefinedWeb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb), along with other corpora.
|
||||
|
||||
| **Data source** | **Fraction** | **Tokens** | **Sources** |
|
||||
|--------------------|--------------|------------|-----------------------------------|
|
||||
| [RefinedWeb-English](https://huggingface.co/datasets/tiiuae/falcon-refinedweb) | 75% | 200B | massive web crawl |
|
||||
| Books | 7% | 21B | |
|
||||
| Code | 7% | 21B | Big Code, CodeNet |
|
||||
| Technical | 6% | 19B | arXiv |
|
||||
| Math | 5% | 17B | Mathematica, Khan Academy |
|
||||
|
||||
|
||||
The data was tokenized with the tokenizer similar to Llama-[7B](https://huggingface.co/meta-llama/Llama-2-7b).
|
||||
|
||||
### Training Procedure
|
||||
|
||||
Nucleus-22B-token-500B was trained on 256 A100 80GB GPUs, using a FSDP
|
||||
|
||||
#### Training Hyperparameters
|
||||
|
||||
| **Hyperparameter** | **Value** | **Comment** |
|
||||
|--------------------|------------|-------------------------------------------|
|
||||
| Precision | `bfloat16` | |
|
||||
| Optimizer | AdamW | |
|
||||
| Learning rate | 2e-4 | 8B tokens warm-up, cosine decay to 1.e-5 |
|
||||
| Weight decay | 1e-1 | |
|
||||
| Batch size | 2048 | constant |
|
||||
| Context length | 2048 | constant |
|
||||
|
||||
|
||||
#### Speeds, Sizes, Times
|
||||
|
||||
Training happened in early August 2023 and took about two weeks.
|
||||
26
config.json
Normal file
26
config.json
Normal file
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"bos_token_id": 1,
|
||||
"eos_token_id": 2,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 6656,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 17920,
|
||||
"max_position_embeddings": 2048,
|
||||
"max_sequence_length": 2048,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 52,
|
||||
"num_hidden_layers": 40,
|
||||
"num_key_value_heads": 52,
|
||||
"pad_token_id": 0,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-06,
|
||||
"rope_scaling": null,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "float32",
|
||||
"transformers_version": "4.30.2",
|
||||
"use_cache": true,
|
||||
"vocab_size": 32000
|
||||
}
|
||||
1
configuration.json
Normal file
1
configuration.json
Normal file
@@ -0,0 +1 @@
|
||||
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}
|
||||
7
generation_config.json
Normal file
7
generation_config.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 1,
|
||||
"eos_token_id": 2,
|
||||
"pad_token_id": 0,
|
||||
"transformers_version": "4.30.2"
|
||||
}
|
||||
3
model-00001-of-00009.safetensors
Normal file
3
model-00001-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:00e7a41342638aa1bca3fa879ef1cee17ff74ab7ff6f41a31bfcd900ee0f400d
|
||||
size 9944389584
|
||||
3
model-00002-of-00009.safetensors
Normal file
3
model-00002-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:58ad98b7b2f8a9446338199c68782766f884ead49238e34824e7ae36ff5e2ee1
|
||||
size 9692207312
|
||||
3
model-00003-of-00009.safetensors
Normal file
3
model-00003-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c3fe2a1144b2fcf7c0288829cd65e14d038d7a5d6d19adac0077fb5232b09fad
|
||||
size 9746787008
|
||||
3
model-00004-of-00009.safetensors
Normal file
3
model-00004-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:eecd1103c8de5728d54c7d5711a3cefd3c1ab9388f830160d9506041f08d927e
|
||||
size 9992153192
|
||||
3
model-00005-of-00009.safetensors
Normal file
3
model-00005-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:513b3505ef2015ba5964d7481a5ac3f48491bae6c281b78083745a9f4757d34d
|
||||
size 9746733544
|
||||
3
model-00006-of-00009.safetensors
Normal file
3
model-00006-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4d51bea26310158cdf92cfcdcd22b012d8286178f168c93d718a9768fb53b182
|
||||
size 9869469920
|
||||
3
model-00007-of-00009.safetensors
Normal file
3
model-00007-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4fa74c99e8c5825be472e9c0eedb49d9ec5ace76bb976222e03eef3b5570f9e6
|
||||
size 9869416816
|
||||
3
model-00008-of-00009.safetensors
Normal file
3
model-00008-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0511a669a8efbdca08c351b40ebbd0d822ee17d067b5bae10941d29e02aee753
|
||||
size 9746787016
|
||||
3
model-00009-of-00009.safetensors
Normal file
3
model-00009-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:18e2cb5ab08ae126edeb389c635fa61d35065c0d60a0ed0e8955022f8f917a0a
|
||||
size 8703949736
|
||||
410
model.safetensors.index.json
Normal file
410
model.safetensors.index.json
Normal file
@@ -0,0 +1,410 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 87311847424
|
||||
},
|
||||
"weight_map": {
|
||||
"lm_head.weight": "model-00009-of-00009.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.rotary_emb.inv_freq": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.rotary_emb.inv_freq": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.rotary_emb.inv_freq": "model-00003-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.rotary_emb.inv_freq": "model-00003-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.rotary_emb.inv_freq": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.rotary_emb.inv_freq": "model-00003-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.rotary_emb.inv_freq": "model-00004-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.rotary_emb.inv_freq": "model-00004-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.rotary_emb.inv_freq": "model-00004-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.rotary_emb.inv_freq": "model-00004-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.rotary_emb.inv_freq": "model-00005-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.rotary_emb.inv_freq": "model-00005-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.rotary_emb.inv_freq": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.rotary_emb.inv_freq": "model-00005-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.rotary_emb.inv_freq": "model-00005-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.rotary_emb.inv_freq": "model-00005-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.rotary_emb.inv_freq": "model-00006-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.24.self_attn.rotary_emb.inv_freq": "model-00006-of-00009.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.25.self_attn.rotary_emb.inv_freq": "model-00006-of-00009.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.26.self_attn.rotary_emb.inv_freq": "model-00006-of-00009.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.27.self_attn.rotary_emb.inv_freq": "model-00007-of-00009.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.28.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.28.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.28.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.28.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.28.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.28.self_attn.rotary_emb.inv_freq": "model-00007-of-00009.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.29.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.29.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.29.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.29.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.29.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.29.self_attn.rotary_emb.inv_freq": "model-00007-of-00009.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.rotary_emb.inv_freq": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.30.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.30.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.30.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.30.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.30.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.30.self_attn.rotary_emb.inv_freq": "model-00007-of-00009.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.31.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.31.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.31.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.31.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.31.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.31.self_attn.rotary_emb.inv_freq": "model-00007-of-00009.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.32.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.32.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.32.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.32.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.32.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.32.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.32.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.32.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.32.self_attn.rotary_emb.inv_freq": "model-00008-of-00009.safetensors",
|
||||
"model.layers.32.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.33.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.33.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.33.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.33.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.33.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.33.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.33.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.33.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.33.self_attn.rotary_emb.inv_freq": "model-00008-of-00009.safetensors",
|
||||
"model.layers.33.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.34.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.34.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.34.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.34.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.34.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.34.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.34.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.34.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.34.self_attn.rotary_emb.inv_freq": "model-00008-of-00009.safetensors",
|
||||
"model.layers.34.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.35.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.35.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.35.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.35.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.35.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.35.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.35.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.35.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.35.self_attn.rotary_emb.inv_freq": "model-00008-of-00009.safetensors",
|
||||
"model.layers.35.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.36.input_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.36.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.36.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.36.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.36.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.36.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.36.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.36.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.36.self_attn.rotary_emb.inv_freq": "model-00008-of-00009.safetensors",
|
||||
"model.layers.36.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.37.input_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.37.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.37.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.37.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.37.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.37.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.37.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.37.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.37.self_attn.rotary_emb.inv_freq": "model-00009-of-00009.safetensors",
|
||||
"model.layers.37.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.38.input_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.38.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.38.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.38.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.38.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.38.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.38.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.38.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.38.self_attn.rotary_emb.inv_freq": "model-00009-of-00009.safetensors",
|
||||
"model.layers.38.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.39.input_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.39.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.39.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.39.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.39.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.39.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.39.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.39.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.39.self_attn.rotary_emb.inv_freq": "model-00009-of-00009.safetensors",
|
||||
"model.layers.39.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.rotary_emb.inv_freq": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.rotary_emb.inv_freq": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.rotary_emb.inv_freq": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.rotary_emb.inv_freq": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.rotary_emb.inv_freq": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.rotary_emb.inv_freq": "model-00003-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.norm.weight": "model-00009-of-00009.safetensors"
|
||||
}
|
||||
}
|
||||
23
special_tokens_map.json
Normal file
23
special_tokens_map.json
Normal file
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "</s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"unk_token": {
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
Binary file not shown.
BIN
tokenizer.model
(Stored with Git LFS)
Normal file
BIN
tokenizer.model
(Stored with Git LFS)
Normal file
Binary file not shown.
33
tokenizer_config.json
Normal file
33
tokenizer_config.json
Normal file
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"__type": "AddedToken",
|
||||
"content": "<s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": {
|
||||
"__type": "AddedToken",
|
||||
"content": "</s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"legacy": false,
|
||||
"model_max_length": 1000000000000000019884624838656,
|
||||
"pad_token": null,
|
||||
"padding_side": "right",
|
||||
"sp_model_kwargs": {},
|
||||
"tokenizer_class": "LlamaTokenizer",
|
||||
"unk_token": {
|
||||
"__type": "AddedToken",
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user