初始化项目，由ModelHub XC社区提供模型
Model: BramVanroy/fietje-2-instruct Source: Original Platform
This commit is contained in:
ModelHub XC
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
109
README.md
Normal file
109
README.md
Normal file
@@ -0,0 +1,109 @@
|
||||
---
|
||||
language:
|
||||
- nl
|
||||
license: mit
|
||||
tags:
|
||||
- trl
|
||||
- fietje
|
||||
- alignment-handbook
|
||||
- sft
|
||||
base_model: BramVanroy/fietje-2
|
||||
datasets:
|
||||
- BramVanroy/ultrachat_200k_dutch
|
||||
- BramVanroy/no_robots_dutch
|
||||
- BramVanroy/belebele_dutch
|
||||
pipeline_tag: text-generation
|
||||
inference: false
|
||||
model-index:
|
||||
- name: fietje-2-instruct
|
||||
results: []
|
||||
---
|
||||
|
||||
<p align="center" style="margin:0;padding:0">
|
||||
<img src="https://huggingface.co/BramVanroy/fietje-2-instruct/resolve/main/img/fietje-2b-banner-rounded.png" alt="Fietje banner" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
|
||||
</p>
|
||||
|
||||
<div style="margin:auto; text-align:center">
|
||||
<h1 style="margin-bottom: 0">Fietje 2 Instruct</h1>
|
||||
<em>An open and efficient LLM for Dutch</em>
|
||||
</div>
|
||||
|
||||
<blockquote class="tip" style="padding: 1.5em; border: 0">
|
||||
<p align="center" style="text-align: center; margin: 0">
|
||||
<a rel="nofollow" href="https://huggingface.co/BramVanroy/fietje-2">👱♀️ Base version</a> -
|
||||
<a rel="nofollow" href="https://huggingface.co/BramVanroy/fietje-2-instruct">🤖 Instruct version</a> (this one) -
|
||||
<a rel="nofollow" href="https://huggingface.co/BramVanroy/fietje-2-chat">💬 Chat version</a> -
|
||||
<a rel="nofollow" href="https://huggingface.co/BramVanroy/fietje-2-chat-GGUF">🚀 GGUF of Instruct</a>
|
||||
</p>
|
||||
</blockquote>
|
||||
|
||||
This is the instruct version of Fietje, an SFT-tuned (instruction-tuned) variant of [the base model](https://huggingface.co/BramVanroy/fietje-2). Fietje is an adapated version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2), tailored to Dutch text generation by training on 28B tokens. It is small and efficient with a size of 2.7 billion parameters while performing almost on par with more powerful Dutch LLMs of twice its size like [GEITje 7B Ultra](https://huggingface.co/BramVanroy/GEITje-7B-ultra).
|
||||
|
||||
A thorough description of the creation and evaluation of Fietje as well as usage examples are available in [this Github repository](https://github.com/BramVanroy/fietje).
|
||||
|
||||
## Citation
|
||||
|
||||
If you use Fietje or the [CulturaX + Wikipedia filtered subset](https://huggingface.co/datasets/BramVanroy/wikipedia_culturax_dutch) in your work, please cite to the following paper:
|
||||
|
||||
```bibtex
|
||||
@article{vanroy2024fietje,
|
||||
author="Vanroy, Bram",
|
||||
title="Fietje: An open, efficient LLM for Dutch",
|
||||
url="https://www.clinjournal.org/clinj/article/view/213",
|
||||
journal="Computational Linguistics in the Netherlands Journal",
|
||||
volume="14",
|
||||
year="2025",
|
||||
pages="473--504"
|
||||
}
|
||||
```
|
||||
|
||||
## Intended uses & limitations
|
||||
|
||||
The same limitations as [phi-2](https://huggingface.co/microsoft/phi-2#limitations-of-phi-2), and LLMs in general, apply here. LLMs hallucinate, make mistakes, and should not be trusted. Use at your own risk!
|
||||
|
||||
## Training and evaluation data
|
||||
|
||||
Fietje 2 instruct was finetuned from [the base model](https://huggingface.co/BramVanroy/fietje-2) on the following datasets. Number of training samples per dataset given in brackets, totalling 201,579 samples.
|
||||
|
||||
- [BramVanroy/ultrachat_200k_dutch](https://huggingface.co/datasets/BramVanroy/ultrachat_200k_dutch): gpt-4-1106-preview; multi-turn; fully generated (192,598)
|
||||
- [BramVanroy/no_robots_dutch](https://huggingface.co/datasets/BramVanroy/no_robots_dutch): gpt-4-1106-preview; prompt translate, answer generated; some items have system messages (8181)
|
||||
- [BramVanroy/belebele_dutch](https://huggingface.co/datasets/BramVanroy/belebele_dutch): Dutch portion of [belebele](https://huggingface.co/datasets/facebook/belebele), formatted into SFT format (800)
|
||||
|
||||
## Training procedure
|
||||
|
||||
I am thankful to the [Flemish Supercomputer Center](https://www.vscentrum.be/) (VSC) for providing the computational power to accomplish this project. Accounting for waiting for jobs, training took around a day on four nodes of 4x A100 80GB each (16 total). I cannot find the exact time anymore and I do not think that the runtime in `all_results.json` accounts for interrupted-and-continued jobs.
|
||||
|
||||
Training was done with the wonderful [alignment-handbook](https://github.com/huggingface/alignment-handbook), using DeepSpeed as a back-end. Exact training recipes and SLURM script are given in the [Github repository](https://github.com/BramVanroy/fietje).
|
||||
|
||||
|
||||
### Training hyperparameters
|
||||
|
||||
The following hyperparameters were used during training:
|
||||
- learning_rate: 6e-05
|
||||
- train_batch_size: 42
|
||||
- eval_batch_size: 42
|
||||
- seed: 42
|
||||
- distributed_type: multi-GPU
|
||||
- num_devices: 16
|
||||
- total_train_batch_size: 672
|
||||
- total_eval_batch_size: 672
|
||||
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-07
|
||||
- lr_scheduler_type: cosine
|
||||
- lr_scheduler_warmup_ratio: 0.1
|
||||
- num_epochs: 3.0
|
||||
|
||||
### Training results
|
||||
|
||||
| Training Loss | Epoch | Step | Validation Loss |
|
||||
|:-------------:|:-----:|:----:|:---------------:|
|
||||
| 0.9325 | 1.0 | 178 | 0.9060 |
|
||||
| 0.8687 | 2.0 | 356 | 0.8850 |
|
||||
| 0.8385 | 3.0 | 534 | 0.8818 |
|
||||
|
||||
|
||||
### Framework versions
|
||||
|
||||
- Transformers 4.39.1
|
||||
- Pytorch 2.1.2+cu121
|
||||
- Datasets 2.18.0
|
||||
- Tokenizers 0.15.2
|
||||
42
added_tokens.json
Normal file
42
added_tokens.json
Normal file
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"\t\t": 50294,
|
||||
"\t\t\t": 50293,
|
||||
"\t\t\t\t": 50292,
|
||||
"\t\t\t\t\t": 50291,
|
||||
"\t\t\t\t\t\t": 50290,
|
||||
"\t\t\t\t\t\t\t": 50289,
|
||||
"\t\t\t\t\t\t\t\t": 50288,
|
||||
"\t\t\t\t\t\t\t\t\t": 50287,
|
||||
" ": 50286,
|
||||
" ": 50285,
|
||||
" ": 50284,
|
||||
" ": 50283,
|
||||
" ": 50282,
|
||||
" ": 50281,
|
||||
" ": 50280,
|
||||
" ": 50279,
|
||||
" ": 50278,
|
||||
" ": 50277,
|
||||
" ": 50276,
|
||||
" ": 50275,
|
||||
" ": 50274,
|
||||
" ": 50273,
|
||||
" ": 50272,
|
||||
" ": 50271,
|
||||
" ": 50270,
|
||||
" ": 50269,
|
||||
" ": 50268,
|
||||
" ": 50267,
|
||||
" ": 50266,
|
||||
" ": 50265,
|
||||
" ": 50264,
|
||||
" ": 50263,
|
||||
" ": 50262,
|
||||
" ": 50261,
|
||||
" ": 50260,
|
||||
" ": 50259,
|
||||
" ": 50258,
|
||||
" ": 50257,
|
||||
"<|im_end|>": 50296,
|
||||
"<|im_start|>": 50295
|
||||
}
|
||||
8
all_results.json
Normal file
8
all_results.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"epoch": 3.0,
|
||||
"train_loss": 0.9222131907270196,
|
||||
"train_runtime": 33956.1835,
|
||||
"train_samples": 201571,
|
||||
"train_samples_per_second": 10.545,
|
||||
"train_steps_per_second": 0.016
|
||||
}
|
||||
30
config.json
Normal file
30
config.json
Normal file
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"_name_or_path": "BramVanroy/fietje-2b-instruct",
|
||||
"architectures": [
|
||||
"PhiForCausalLM"
|
||||
],
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 50256,
|
||||
"embd_pdrop": 0.0,
|
||||
"eos_token_id": 50256,
|
||||
"hidden_act": "gelu_new",
|
||||
"hidden_size": 2560,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 10240,
|
||||
"layer_norm_eps": 1e-05,
|
||||
"max_position_embeddings": 2048,
|
||||
"model_type": "phi",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 32,
|
||||
"num_key_value_heads": 32,
|
||||
"partial_rotary_factor": 0.4,
|
||||
"qk_layernorm": false,
|
||||
"resid_pdrop": 0.1,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 10000.0,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "bfloat16",
|
||||
"transformers_version": "4.39.1",
|
||||
"use_cache": false,
|
||||
"vocab_size": 50297
|
||||
}
|
||||
7
generation_config.json
Normal file
7
generation_config.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 50295,
|
||||
"eos_token_id": 50296,
|
||||
"pad_token_id": 50296,
|
||||
"transformers_version": "4.39.1"
|
||||
}
|
||||
BIN
img/fietje-2b-avatar.png
Normal file
BIN
img/fietje-2b-avatar.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 137 KiB |
BIN
img/fietje-2b-banner-coming-soon.png
Normal file
BIN
img/fietje-2b-banner-coming-soon.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 224 KiB |
BIN
img/fietje-2b-banner-rounded.png
Normal file
BIN
img/fietje-2b-banner-rounded.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 252 KiB |
BIN
img/fietje-2b-banner.png
Normal file
BIN
img/fietje-2b-banner.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 221 KiB |
BIN
img/fietje-2b.png
Normal file
BIN
img/fietje-2b.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 973 KiB |
50001
merges.txt
Normal file
50001
merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
model-00001-of-00002.safetensors
Normal file
3
model-00001-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:fa5fdc0699ada9a30d5743b6cd7fe8fb8d47fa11d8aab9ad200274f4506d5377
|
||||
size 4990961488
|
||||
3
model-00002-of-00002.safetensors
Normal file
3
model-00002-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d51beda028780b6ce995b6aaf725a35a176df8cc2acec3649e095dfb5f5fc660
|
||||
size 559207842
|
||||
460
model.safetensors.index.json
Normal file
460
model.safetensors.index.json
Normal file
@@ -0,0 +1,460 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 5550119154
|
||||
},
|
||||
"weight_map": {
|
||||
"lm_head.bias": "model-00002-of-00002.safetensors",
|
||||
"lm_head.weight": "model-00002-of-00002.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00002.safetensors",
|
||||
"model.final_layernorm.bias": "model-00002-of-00002.safetensors",
|
||||
"model.final_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.0.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.input_layernorm.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.mlp.fc1.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.mlp.fc1.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.mlp.fc2.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.mlp.fc2.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.dense.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.dense.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.input_layernorm.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.fc1.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.fc1.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.fc2.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.fc2.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.dense.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.dense.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.4.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.input_layernorm.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.fc1.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.fc1.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.fc2.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.fc2.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.dense.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.dense.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors"
|
||||
}
|
||||
}
|
||||
28
special_tokens_map.json
Normal file
28
special_tokens_map.json
Normal file
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"additional_special_tokens": [
|
||||
{
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
{
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
],
|
||||
"bos_token": "<|im_start|>",
|
||||
"eos_token": "<|im_end|>",
|
||||
"pad_token": "<|im_end|>",
|
||||
"unk_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
100665
tokenizer.json
Normal file
100665
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
345
tokenizer_config.json
Normal file
345
tokenizer_config.json
Normal file
@@ -0,0 +1,345 @@
|
||||
{
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"50256": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50257": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50258": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50259": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50260": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50261": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50262": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50263": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50264": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50265": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50266": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50267": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50268": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50269": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50270": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50271": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50272": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50273": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50274": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50275": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50276": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50277": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50278": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50279": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50280": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50281": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50282": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50283": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50284": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50285": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50286": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50287": {
|
||||
"content": "\t\t\t\t\t\t\t\t\t",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50288": {
|
||||
"content": "\t\t\t\t\t\t\t\t",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50289": {
|
||||
"content": "\t\t\t\t\t\t\t",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50290": {
|
||||
"content": "\t\t\t\t\t\t",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50291": {
|
||||
"content": "\t\t\t\t\t",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50292": {
|
||||
"content": "\t\t\t\t",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50293": {
|
||||
"content": "\t\t\t",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50294": {
|
||||
"content": "\t\t",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50295": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50296": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>"
|
||||
],
|
||||
"bos_token": "<|im_start|>",
|
||||
"chat_template": "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
|
||||
"clean_up_tokenization_spaces": true,
|
||||
"eos_token": "<|im_end|>",
|
||||
"model_max_length": 2048,
|
||||
"pad_token": "<|im_end|>",
|
||||
"tokenizer_class": "CodeGenTokenizer",
|
||||
"unk_token": "<|endoftext|>"
|
||||
}
|
||||
8
train_results.json
Normal file
8
train_results.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"epoch": 3.0,
|
||||
"train_loss": 0.9222131907270196,
|
||||
"train_runtime": 33956.1835,
|
||||
"train_samples": 201571,
|
||||
"train_samples_per_second": 10.545,
|
||||
"train_steps_per_second": 0.016
|
||||
}
|
||||
3792
trainer_state.json
Normal file
3792
trainer_state.json
Normal file
File diff suppressed because it is too large
Load Diff
1
vocab.json
Normal file
1
vocab.json
Normal file
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user