初始化项目，由ModelHub XC社区提供模型
Model: TheBloke/airoboros-13B-HF Source: Original Platform
This commit is contained in:
ModelHub XC
34
.gitattributes
vendored
Normal file
34
.gitattributes
vendored
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
362
README.md
Normal file
362
README.md
Normal file
@@ -0,0 +1,362 @@
|
|||||||
|
---
|
||||||
|
license: other
|
||||||
|
inference: false
|
||||||
|
---
|
||||||
|
<!-- header start -->
|
||||||
|
<div style="width: 100%;">
|
||||||
|
<img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
|
||||||
|
</div>
|
||||||
|
<div style="display: flex; justify-content: space-between; width: 100%;">
|
||||||
|
<div style="display: flex; flex-direction: column; align-items: flex-start;">
|
||||||
|
<p><a href="https://discord.gg/Jq4vkcDakD">Chat & support: my new Discord server</a></p>
|
||||||
|
</div>
|
||||||
|
<div style="display: flex; flex-direction: column; align-items: flex-end;">
|
||||||
|
<p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? TheBloke's Patreon page</a></p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<!-- header end -->
|
||||||
|
|
||||||
|
# Airoboros 13B HF fp16
|
||||||
|
|
||||||
|
These files are HF format fp16 model files for [Jon Durbin's Airoboros 13B](https://huggingface.co/jondurbin/airoboros-13b).
|
||||||
|
|
||||||
|
It is the result of converting Jon's fp32 repo to fp16 for easier storage and usage.
|
||||||
|
|
||||||
|
## Other repositories available
|
||||||
|
|
||||||
|
* [4bit GPTQ models for GPU inference](https://huggingface.co/TheBloke/airoboros-13B-GPTQ)
|
||||||
|
* [Unquantised model in HF fp16 format](https://huggingface.co/TheBloke/airoboros-13B-HF)
|
||||||
|
* [latimar's GGML models for CPU (+CUDA) inference](https://huggingface.co/latimar/airoboros-13b-ggml)
|
||||||
|
|
||||||
|
<!-- footer start -->
|
||||||
|
## Discord
|
||||||
|
|
||||||
|
For further support, and discussions on these models and AI in general, join us at:
|
||||||
|
|
||||||
|
[TheBloke AI's Discord server](https://discord.gg/Jq4vkcDakD)
|
||||||
|
|
||||||
|
## Thanks, and how to contribute.
|
||||||
|
|
||||||
|
Thanks to the [chirper.ai](https://chirper.ai) team!
|
||||||
|
|
||||||
|
I've had a lot of people ask if they can contribute. I enjoy providing models and helping people, and would love to be able to spend even more time doing it, as well as expanding into new projects like fine tuning/training.
|
||||||
|
|
||||||
|
If you're able and willing to contribute it will be most gratefully received and will help me to keep providing more models, and to start work on new AI projects.
|
||||||
|
|
||||||
|
Donaters will get priority support on any and all AI/LLM/model questions and requests, access to a private Discord room, plus other benefits.
|
||||||
|
|
||||||
|
* Patreon: https://patreon.com/TheBlokeAI
|
||||||
|
* Ko-Fi: https://ko-fi.com/TheBlokeAI
|
||||||
|
|
||||||
|
**Patreon special mentions**: Aemon Algiz, Dmitriy Samsonov, Nathan LeClaire, Trenton Dambrowitz, Mano Prime, David Flickinger, vamX, Nikolai Manek, senxiiz, Khalefa Al-Ahmad, Illia Dulskyi, Jonathan Leane, Talal Aujan, V. Lukas, Joseph William Delisle, Pyrater, Oscar Rangel, Lone Striker, Luke Pendergrass, Eugene Pentland, Sebastain Graf, Johann-Peter Hartman.
|
||||||
|
|
||||||
|
Thank you to all my generous patrons and donaters!
|
||||||
|
<!-- footer end -->
|
||||||
|
# Airoboros-13B original model card
|
||||||
|
|
||||||
|
# Overview
|
||||||
|
|
||||||
|
This is a fine-tuned 13b parameter LlaMa model, using completely synthetic training data created by https://github.com/jondurbin/airoboros
|
||||||
|
|
||||||
|
### Eval (gpt4 judging)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
| model | raw score | gpt-3.5 adjusted score |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| __airoboros-13b__ | __17947__ | __98.087__ |
|
||||||
|
| gpt35 | 18297 | 100.0 |
|
||||||
|
| gpt4-x-alpasta-30b | 15612 | 85.33 |
|
||||||
|
| manticore-13b | 15856 | 86.66 |
|
||||||
|
| vicuna-13b-1.1 | 16306 | 89.12 |
|
||||||
|
| wizard-vicuna-13b-uncensored | 16287 | 89.01 |
|
||||||
|
|
||||||
|
<details>
|
||||||
|
<summary>individual question scores, with shareGPT links (200 prompts generated by gpt-4)</summary>
|
||||||
|
|
||||||
|
*wb-13b-u is Wizard-Vicuna-13b-Uncensored*
|
||||||
|
|
||||||
|
| airoboros-13b | gpt35 | gpt4-x-alpasta-30b | manticore-13b | vicuna-13b-1.1 | wv-13b-u | link |
|
||||||
|
|----------------:|--------:|---------------------:|----------------:|-----------------:|-------------------------------:|:---------------------------------------|
|
||||||
|
| 80 | 95 | 70 | 90 | 85 | 60 | [eval](https://sharegpt.com/c/PIbRQD3) |
|
||||||
|
| 20 | 95 | 40 | 30 | 90 | 80 | [eval](https://sharegpt.com/c/fSzwzzd) |
|
||||||
|
| 100 | 100 | 100 | 95 | 95 | 100 | [eval](https://sharegpt.com/c/AXMzZiO) |
|
||||||
|
| 90 | 100 | 85 | 60 | 95 | 100 | [eval](https://sharegpt.com/c/7obzJm2) |
|
||||||
|
| 95 | 90 | 80 | 85 | 95 | 75 | [eval](https://sharegpt.com/c/cRpj6M1) |
|
||||||
|
| 100 | 95 | 90 | 95 | 98 | 92 | [eval](https://sharegpt.com/c/p0by1T7) |
|
||||||
|
| 50 | 100 | 80 | 95 | 60 | 55 | [eval](https://sharegpt.com/c/rowNlKx) |
|
||||||
|
| 70 | 90 | 80 | 60 | 85 | 40 | [eval](https://sharegpt.com/c/I4POj4I) |
|
||||||
|
| 100 | 95 | 50 | 85 | 40 | 60 | [eval](https://sharegpt.com/c/gUAeiRp) |
|
||||||
|
| 85 | 60 | 55 | 65 | 50 | 70 | [eval](https://sharegpt.com/c/Lgw4QQL) |
|
||||||
|
| 95 | 100 | 85 | 90 | 60 | 75 | [eval](https://sharegpt.com/c/X9tDYft) |
|
||||||
|
| 100 | 95 | 70 | 80 | 50 | 85 | [eval](https://sharegpt.com/c/9V2ElkH) |
|
||||||
|
| 100 | 95 | 80 | 70 | 60 | 90 | [eval](https://sharegpt.com/c/D5xg6qt) |
|
||||||
|
| 95 | 100 | 70 | 85 | 90 | 90 | [eval](https://sharegpt.com/c/lQnSfDs) |
|
||||||
|
| 80 | 95 | 90 | 60 | 30 | 85 | [eval](https://sharegpt.com/c/1hpHGNc) |
|
||||||
|
| 60 | 95 | 0 | 75 | 50 | 40 | [eval](https://sharegpt.com/c/an6TqE4) |
|
||||||
|
| 100 | 95 | 90 | 98 | 95 | 95 | [eval](https://sharegpt.com/c/7vr6n3F) |
|
||||||
|
| 60 | 85 | 40 | 50 | 20 | 0 | [eval](https://sharegpt.com/c/TOkMkgE) |
|
||||||
|
| 100 | 90 | 85 | 95 | 95 | 80 | [eval](https://sharegpt.com/c/Qu7ak0r) |
|
||||||
|
| 100 | 95 | 100 | 95 | 90 | 95 | [eval](https://sharegpt.com/c/hMD4gPo) |
|
||||||
|
| 95 | 90 | 96 | 80 | 92 | 88 | [eval](https://sharegpt.com/c/HTlicNh) |
|
||||||
|
| 95 | 92 | 90 | 93 | 89 | 91 | [eval](https://sharegpt.com/c/MjxHpAf) |
|
||||||
|
| 95 | 93 | 90 | 94 | 96 | 92 | [eval](https://sharegpt.com/c/4RvxOR9) |
|
||||||
|
| 95 | 90 | 93 | 88 | 92 | 85 | [eval](https://sharegpt.com/c/PcAIU9r) |
|
||||||
|
| 95 | 90 | 85 | 96 | 88 | 92 | [eval](https://sharegpt.com/c/MMqul3q) |
|
||||||
|
| 95 | 95 | 90 | 93 | 92 | 91 | [eval](https://sharegpt.com/c/YQsLyzJ) |
|
||||||
|
| 95 | 98 | 80 | 97 | 99 | 96 | [eval](https://sharegpt.com/c/UDhSTMq) |
|
||||||
|
| 95 | 93 | 90 | 87 | 92 | 89 | [eval](https://sharegpt.com/c/4gCfdCV) |
|
||||||
|
| 90 | 85 | 95 | 80 | 92 | 75 | [eval](https://sharegpt.com/c/bkQs4SP) |
|
||||||
|
| 90 | 85 | 95 | 93 | 80 | 92 | [eval](https://sharegpt.com/c/LeLCEEt) |
|
||||||
|
| 95 | 92 | 90 | 91 | 93 | 89 | [eval](https://sharegpt.com/c/DFxNzVu) |
|
||||||
|
| 100 | 95 | 90 | 85 | 80 | 95 | [eval](https://sharegpt.com/c/gnVzNML) |
|
||||||
|
| 95 | 97 | 93 | 92 | 96 | 94 | [eval](https://sharegpt.com/c/y7pxMIy) |
|
||||||
|
| 95 | 93 | 94 | 90 | 88 | 92 | [eval](https://sharegpt.com/c/5UeCvTY) |
|
||||||
|
| 90 | 95 | 98 | 85 | 96 | 92 | [eval](https://sharegpt.com/c/T4oL9I5) |
|
||||||
|
| 90 | 88 | 85 | 80 | 82 | 84 | [eval](https://sharegpt.com/c/HnGyTAG) |
|
||||||
|
| 90 | 95 | 85 | 87 | 92 | 88 | [eval](https://sharegpt.com/c/ZbRMBNj) |
|
||||||
|
| 95 | 97 | 96 | 90 | 93 | 92 | [eval](https://sharegpt.com/c/iTmFJqd) |
|
||||||
|
| 95 | 93 | 92 | 90 | 89 | 91 | [eval](https://sharegpt.com/c/VuPifET) |
|
||||||
|
| 90 | 95 | 93 | 92 | 94 | 91 | [eval](https://sharegpt.com/c/AvFAH1x) |
|
||||||
|
| 90 | 85 | 95 | 80 | 88 | 75 | [eval](https://sharegpt.com/c/4ealKGN) |
|
||||||
|
| 85 | 90 | 95 | 88 | 92 | 80 | [eval](https://sharegpt.com/c/bE1b2vX) |
|
||||||
|
| 90 | 95 | 92 | 85 | 80 | 87 | [eval](https://sharegpt.com/c/I3nMPBC) |
|
||||||
|
| 85 | 90 | 95 | 80 | 88 | 75 | [eval](https://sharegpt.com/c/as7r3bW) |
|
||||||
|
| 85 | 80 | 75 | 90 | 70 | 82 | [eval](https://sharegpt.com/c/qYceaUa) |
|
||||||
|
| 90 | 85 | 95 | 92 | 93 | 80 | [eval](https://sharegpt.com/c/g4FXchU) |
|
||||||
|
| 90 | 95 | 75 | 85 | 80 | 70 | [eval](https://sharegpt.com/c/6kGLvL5) |
|
||||||
|
| 85 | 90 | 80 | 88 | 82 | 83 | [eval](https://sharegpt.com/c/SRozqaF) |
|
||||||
|
| 85 | 90 | 95 | 92 | 88 | 80 | [eval](https://sharegpt.com/c/GoKydf6) |
|
||||||
|
| 85 | 90 | 80 | 75 | 95 | 88 | [eval](https://sharegpt.com/c/37aXkHQ) |
|
||||||
|
| 85 | 90 | 80 | 88 | 84 | 92 | [eval](https://sharegpt.com/c/nVuUaTj) |
|
||||||
|
| 80 | 90 | 75 | 85 | 70 | 95 | [eval](https://sharegpt.com/c/TkAQKLC) |
|
||||||
|
| 90 | 88 | 85 | 80 | 92 | 83 | [eval](https://sharegpt.com/c/55cO2y0) |
|
||||||
|
| 85 | 75 | 90 | 80 | 78 | 88 | [eval](https://sharegpt.com/c/tXtq5lT) |
|
||||||
|
| 85 | 90 | 80 | 82 | 75 | 88 | [eval](https://sharegpt.com/c/TfMjeJQ) |
|
||||||
|
| 90 | 85 | 40 | 95 | 80 | 88 | [eval](https://sharegpt.com/c/2jQ6K2S) |
|
||||||
|
| 85 | 95 | 90 | 75 | 88 | 80 | [eval](https://sharegpt.com/c/aQtr2ca) |
|
||||||
|
| 85 | 95 | 90 | 92 | 89 | 88 | [eval](https://sharegpt.com/c/tbWLyZ7) |
|
||||||
|
| 80 | 85 | 75 | 60 | 90 | 70 | [eval](https://sharegpt.com/c/moHC7i2) |
|
||||||
|
| 85 | 90 | 87 | 80 | 88 | 75 | [eval](https://sharegpt.com/c/GK6GShh) |
|
||||||
|
| 85 | 80 | 75 | 50 | 90 | 80 | [eval](https://sharegpt.com/c/ugcW4qG) |
|
||||||
|
| 95 | 80 | 90 | 85 | 75 | 82 | [eval](https://sharegpt.com/c/WL8iq6F) |
|
||||||
|
| 85 | 90 | 80 | 70 | 95 | 88 | [eval](https://sharegpt.com/c/TZJKnvS) |
|
||||||
|
| 90 | 95 | 70 | 85 | 80 | 75 | [eval](https://sharegpt.com/c/beNOKb5) |
|
||||||
|
| 90 | 85 | 70 | 75 | 80 | 60 | [eval](https://sharegpt.com/c/o2oRCF5) |
|
||||||
|
| 95 | 90 | 70 | 50 | 85 | 80 | [eval](https://sharegpt.com/c/TNjbK6D) |
|
||||||
|
| 80 | 85 | 40 | 60 | 90 | 95 | [eval](https://sharegpt.com/c/rJvszWJ) |
|
||||||
|
| 75 | 60 | 80 | 55 | 70 | 85 | [eval](https://sharegpt.com/c/HJwRkro) |
|
||||||
|
| 90 | 85 | 60 | 50 | 80 | 95 | [eval](https://sharegpt.com/c/AeFoSDK) |
|
||||||
|
| 45 | 85 | 60 | 20 | 65 | 75 | [eval](https://sharegpt.com/c/KA1cgOl) |
|
||||||
|
| 85 | 90 | 30 | 60 | 80 | 70 | [eval](https://sharegpt.com/c/RTy8n0y) |
|
||||||
|
| 90 | 95 | 80 | 40 | 85 | 70 | [eval](https://sharegpt.com/c/PJMJoXh) |
|
||||||
|
| 85 | 90 | 70 | 75 | 80 | 95 | [eval](https://sharegpt.com/c/Ib3jzyC) |
|
||||||
|
| 90 | 70 | 50 | 20 | 60 | 40 | [eval](https://sharegpt.com/c/oMmqqtX) |
|
||||||
|
| 90 | 95 | 75 | 60 | 85 | 80 | [eval](https://sharegpt.com/c/qRNhNTw) |
|
||||||
|
| 85 | 80 | 60 | 70 | 65 | 75 | [eval](https://sharegpt.com/c/3MAHQIy) |
|
||||||
|
| 90 | 85 | 80 | 75 | 82 | 70 | [eval](https://sharegpt.com/c/0Emc5HS) |
|
||||||
|
| 90 | 95 | 80 | 70 | 85 | 75 | [eval](https://sharegpt.com/c/UqAxRWF) |
|
||||||
|
| 85 | 75 | 30 | 80 | 90 | 70 | [eval](https://sharegpt.com/c/eywxGAw) |
|
||||||
|
| 85 | 90 | 50 | 70 | 80 | 60 | [eval](https://sharegpt.com/c/A2KSEWP) |
|
||||||
|
| 100 | 95 | 98 | 99 | 97 | 96 | [eval](https://sharegpt.com/c/C8rebQf) |
|
||||||
|
| 95 | 90 | 92 | 93 | 91 | 89 | [eval](https://sharegpt.com/c/cd9HF4V) |
|
||||||
|
| 95 | 92 | 90 | 85 | 88 | 91 | [eval](https://sharegpt.com/c/LHkjvQJ) |
|
||||||
|
| 100 | 95 | 98 | 97 | 96 | 99 | [eval](https://sharegpt.com/c/o5PdoyZ) |
|
||||||
|
| 100 | 100 | 100 | 90 | 100 | 95 | [eval](https://sharegpt.com/c/rh8pZVg) |
|
||||||
|
| 100 | 95 | 98 | 97 | 94 | 99 | [eval](https://sharegpt.com/c/T5DYL83) |
|
||||||
|
| 95 | 90 | 92 | 93 | 94 | 91 | [eval](https://sharegpt.com/c/G5Osg3X) |
|
||||||
|
| 100 | 95 | 98 | 90 | 96 | 95 | [eval](https://sharegpt.com/c/9ZqI03V) |
|
||||||
|
| 95 | 96 | 92 | 90 | 89 | 93 | [eval](https://sharegpt.com/c/4tFfwZU) |
|
||||||
|
| 100 | 95 | 93 | 90 | 92 | 88 | [eval](https://sharegpt.com/c/mG1JqPH) |
|
||||||
|
| 100 | 100 | 98 | 97 | 99 | 100 | [eval](https://sharegpt.com/c/VDdtgCu) |
|
||||||
|
| 95 | 90 | 92 | 85 | 93 | 94 | [eval](https://sharegpt.com/c/uKtGkvg) |
|
||||||
|
| 95 | 93 | 90 | 92 | 96 | 91 | [eval](https://sharegpt.com/c/9B92N6P) |
|
||||||
|
| 95 | 96 | 92 | 90 | 93 | 91 | [eval](https://sharegpt.com/c/GeIFfOu) |
|
||||||
|
| 95 | 90 | 92 | 93 | 91 | 89 | [eval](https://sharegpt.com/c/gn3E9nN) |
|
||||||
|
| 100 | 98 | 95 | 97 | 96 | 99 | [eval](https://sharegpt.com/c/Erxa46H) |
|
||||||
|
| 90 | 95 | 85 | 88 | 92 | 87 | [eval](https://sharegpt.com/c/oRHVOvK) |
|
||||||
|
| 95 | 93 | 90 | 92 | 89 | 88 | [eval](https://sharegpt.com/c/ghtKLUX) |
|
||||||
|
| 100 | 95 | 97 | 90 | 96 | 94 | [eval](https://sharegpt.com/c/ZL4KjqP) |
|
||||||
|
| 95 | 93 | 90 | 92 | 94 | 91 | [eval](https://sharegpt.com/c/YOnqIQa) |
|
||||||
|
| 95 | 92 | 90 | 93 | 94 | 88 | [eval](https://sharegpt.com/c/3BKwKho) |
|
||||||
|
| 95 | 92 | 60 | 97 | 90 | 96 | [eval](https://sharegpt.com/c/U1i31bn) |
|
||||||
|
| 95 | 90 | 92 | 93 | 91 | 89 | [eval](https://sharegpt.com/c/etfRoAE) |
|
||||||
|
| 95 | 90 | 97 | 92 | 91 | 93 | [eval](https://sharegpt.com/c/B0OpVxR) |
|
||||||
|
| 90 | 95 | 93 | 85 | 92 | 91 | [eval](https://sharegpt.com/c/MBgGJ5A) |
|
||||||
|
| 95 | 90 | 40 | 92 | 93 | 85 | [eval](https://sharegpt.com/c/eQKTYO7) |
|
||||||
|
| 100 | 100 | 95 | 90 | 95 | 90 | [eval](https://sharegpt.com/c/szKWCBt) |
|
||||||
|
| 90 | 95 | 96 | 98 | 93 | 92 | [eval](https://sharegpt.com/c/8ZhUcAv) |
|
||||||
|
| 90 | 95 | 92 | 89 | 93 | 94 | [eval](https://sharegpt.com/c/VQWdy99) |
|
||||||
|
| 100 | 95 | 100 | 98 | 96 | 99 | [eval](https://sharegpt.com/c/g1DHUSM) |
|
||||||
|
| 100 | 100 | 95 | 90 | 100 | 90 | [eval](https://sharegpt.com/c/uYgfJC3) |
|
||||||
|
| 90 | 85 | 88 | 92 | 87 | 91 | [eval](https://sharegpt.com/c/crk8BH3) |
|
||||||
|
| 95 | 97 | 90 | 92 | 93 | 94 | [eval](https://sharegpt.com/c/95F9afQ) |
|
||||||
|
| 90 | 95 | 85 | 88 | 92 | 89 | [eval](https://sharegpt.com/c/otioHUo) |
|
||||||
|
| 95 | 93 | 90 | 92 | 94 | 91 | [eval](https://sharegpt.com/c/KSiL9F6) |
|
||||||
|
| 90 | 95 | 85 | 80 | 88 | 82 | [eval](https://sharegpt.com/c/GmGq3b3) |
|
||||||
|
| 95 | 90 | 60 | 85 | 93 | 70 | [eval](https://sharegpt.com/c/VOhklyz) |
|
||||||
|
| 95 | 92 | 94 | 93 | 96 | 90 | [eval](https://sharegpt.com/c/wqy8m6k) |
|
||||||
|
| 95 | 90 | 85 | 93 | 87 | 92 | [eval](https://sharegpt.com/c/iWKrIuS) |
|
||||||
|
| 95 | 96 | 93 | 90 | 97 | 92 | [eval](https://sharegpt.com/c/o1h3w8N) |
|
||||||
|
| 100 | 0 | 0 | 100 | 0 | 0 | [eval](https://sharegpt.com/c/3UH9eed) |
|
||||||
|
| 60 | 100 | 0 | 80 | 0 | 0 | [eval](https://sharegpt.com/c/44g0FAh) |
|
||||||
|
| 0 | 100 | 60 | 0 | 0 | 90 | [eval](https://sharegpt.com/c/PaQlcrU) |
|
||||||
|
| 100 | 100 | 0 | 100 | 100 | 100 | [eval](https://sharegpt.com/c/51icV4o) |
|
||||||
|
| 100 | 100 | 100 | 100 | 95 | 100 | [eval](https://sharegpt.com/c/1VnbGAR) |
|
||||||
|
| 100 | 100 | 100 | 50 | 90 | 100 | [eval](https://sharegpt.com/c/EYGBrgw) |
|
||||||
|
| 100 | 100 | 100 | 100 | 95 | 90 | [eval](https://sharegpt.com/c/EGRduOt) |
|
||||||
|
| 100 | 100 | 100 | 95 | 0 | 100 | [eval](https://sharegpt.com/c/O3JJfnK) |
|
||||||
|
| 50 | 95 | 20 | 10 | 30 | 85 | [eval](https://sharegpt.com/c/2roVtAu) |
|
||||||
|
| 100 | 100 | 60 | 20 | 30 | 40 | [eval](https://sharegpt.com/c/sphFpfx) |
|
||||||
|
| 100 | 0 | 0 | 0 | 0 | 100 | [eval](https://sharegpt.com/c/OeWGKBo) |
|
||||||
|
| 0 | 100 | 60 | 0 | 0 | 80 | [eval](https://sharegpt.com/c/TOUsuFA) |
|
||||||
|
| 50 | 100 | 20 | 90 | 0 | 10 | [eval](https://sharegpt.com/c/Y3P6DCu) |
|
||||||
|
| 100 | 100 | 100 | 100 | 100 | 100 | [eval](https://sharegpt.com/c/hkbdeiM) |
|
||||||
|
| 100 | 100 | 100 | 100 | 100 | 100 | [eval](https://sharegpt.com/c/eubbaVC) |
|
||||||
|
| 40 | 100 | 95 | 0 | 100 | 40 | [eval](https://sharegpt.com/c/QWiF49v) |
|
||||||
|
| 100 | 100 | 100 | 100 | 80 | 100 | [eval](https://sharegpt.com/c/dKTapBu) |
|
||||||
|
| 100 | 100 | 100 | 0 | 90 | 40 | [eval](https://sharegpt.com/c/P8NGwFZ) |
|
||||||
|
| 0 | 100 | 100 | 50 | 70 | 20 | [eval](https://sharegpt.com/c/v96BtBL) |
|
||||||
|
| 100 | 100 | 50 | 90 | 0 | 95 | [eval](https://sharegpt.com/c/YRlzj1t) |
|
||||||
|
| 100 | 95 | 90 | 85 | 98 | 80 | [eval](https://sharegpt.com/c/76VX3eB) |
|
||||||
|
| 95 | 98 | 90 | 92 | 96 | 89 | [eval](https://sharegpt.com/c/JK1uNef) |
|
||||||
|
| 90 | 95 | 75 | 85 | 80 | 82 | [eval](https://sharegpt.com/c/ku6CKmx) |
|
||||||
|
| 95 | 98 | 50 | 92 | 96 | 94 | [eval](https://sharegpt.com/c/0iAFuKW) |
|
||||||
|
| 95 | 90 | 0 | 93 | 92 | 94 | [eval](https://sharegpt.com/c/6uGnKio) |
|
||||||
|
| 95 | 90 | 85 | 92 | 80 | 88 | [eval](https://sharegpt.com/c/lfpRBw8) |
|
||||||
|
| 95 | 93 | 75 | 85 | 90 | 92 | [eval](https://sharegpt.com/c/mKu70jb) |
|
||||||
|
| 90 | 95 | 88 | 85 | 92 | 89 | [eval](https://sharegpt.com/c/GkYzJHO) |
|
||||||
|
| 100 | 100 | 100 | 95 | 97 | 98 | [eval](https://sharegpt.com/c/mly2k0z) |
|
||||||
|
| 85 | 40 | 30 | 95 | 90 | 88 | [eval](https://sharegpt.com/c/5td2ob0) |
|
||||||
|
| 90 | 95 | 92 | 85 | 88 | 93 | [eval](https://sharegpt.com/c/0ISpWfy) |
|
||||||
|
| 95 | 96 | 92 | 90 | 89 | 93 | [eval](https://sharegpt.com/c/kdUDUn7) |
|
||||||
|
| 90 | 95 | 85 | 80 | 92 | 88 | [eval](https://sharegpt.com/c/fjMNYr2) |
|
||||||
|
| 95 | 98 | 65 | 90 | 85 | 93 | [eval](https://sharegpt.com/c/6xBIf2Q) |
|
||||||
|
| 95 | 92 | 96 | 97 | 90 | 89 | [eval](https://sharegpt.com/c/B9GY8Ln) |
|
||||||
|
| 95 | 90 | 92 | 91 | 89 | 93 | [eval](https://sharegpt.com/c/vn1FPU4) |
|
||||||
|
| 95 | 90 | 80 | 75 | 95 | 90 | [eval](https://sharegpt.com/c/YurEMYg) |
|
||||||
|
| 92 | 40 | 30 | 95 | 90 | 93 | [eval](https://sharegpt.com/c/D19Qeui) |
|
||||||
|
| 90 | 92 | 85 | 88 | 89 | 87 | [eval](https://sharegpt.com/c/5QRFfrt) |
|
||||||
|
| 95 | 80 | 90 | 92 | 91 | 88 | [eval](https://sharegpt.com/c/pYWPRi4) |
|
||||||
|
| 95 | 93 | 92 | 90 | 91 | 94 | [eval](https://sharegpt.com/c/wPRTntL) |
|
||||||
|
| 100 | 98 | 95 | 90 | 92 | 96 | [eval](https://sharegpt.com/c/F6PLYKE) |
|
||||||
|
| 95 | 92 | 80 | 85 | 90 | 93 | [eval](https://sharegpt.com/c/WeJnMGv) |
|
||||||
|
| 95 | 98 | 90 | 88 | 97 | 96 | [eval](https://sharegpt.com/c/zNKL49e) |
|
||||||
|
| 90 | 95 | 85 | 88 | 86 | 92 | [eval](https://sharegpt.com/c/kIKmA1b) |
|
||||||
|
| 100 | 100 | 100 | 100 | 100 | 100 | [eval](https://sharegpt.com/c/1btWd4O) |
|
||||||
|
| 90 | 95 | 85 | 96 | 92 | 88 | [eval](https://sharegpt.com/c/s9sf1Lp) |
|
||||||
|
| 100 | 98 | 95 | 99 | 97 | 96 | [eval](https://sharegpt.com/c/RWzv8py) |
|
||||||
|
| 95 | 92 | 70 | 90 | 93 | 89 | [eval](https://sharegpt.com/c/bYF7FqA) |
|
||||||
|
| 95 | 90 | 88 | 92 | 94 | 93 | [eval](https://sharegpt.com/c/SuUqjMj) |
|
||||||
|
| 95 | 90 | 93 | 92 | 85 | 94 | [eval](https://sharegpt.com/c/r0aRdYY) |
|
||||||
|
| 95 | 93 | 90 | 87 | 92 | 91 | [eval](https://sharegpt.com/c/VuMfkkd) |
|
||||||
|
| 95 | 93 | 90 | 96 | 92 | 91 | [eval](https://sharegpt.com/c/rhm6fa4) |
|
||||||
|
| 95 | 97 | 85 | 96 | 98 | 90 | [eval](https://sharegpt.com/c/DwXnyqG) |
|
||||||
|
| 95 | 92 | 90 | 85 | 93 | 94 | [eval](https://sharegpt.com/c/0ScdkGS) |
|
||||||
|
| 95 | 96 | 92 | 90 | 97 | 93 | [eval](https://sharegpt.com/c/6yIoCDU) |
|
||||||
|
| 95 | 93 | 96 | 94 | 90 | 92 | [eval](https://sharegpt.com/c/VubEvp9) |
|
||||||
|
| 95 | 94 | 93 | 92 | 90 | 89 | [eval](https://sharegpt.com/c/RHzmZWG) |
|
||||||
|
| 90 | 85 | 95 | 80 | 87 | 75 | [eval](https://sharegpt.com/c/IMiP9Zm) |
|
||||||
|
| 95 | 94 | 92 | 93 | 90 | 96 | [eval](https://sharegpt.com/c/bft4PIL) |
|
||||||
|
| 95 | 100 | 90 | 95 | 95 | 95 | [eval](https://sharegpt.com/c/iHXB34b) |
|
||||||
|
| 100 | 95 | 85 | 100 | 0 | 90 | [eval](https://sharegpt.com/c/vCGn9R7) |
|
||||||
|
| 100 | 95 | 90 | 95 | 100 | 95 | [eval](https://sharegpt.com/c/be8crZL) |
|
||||||
|
| 95 | 90 | 60 | 95 | 85 | 80 | [eval](https://sharegpt.com/c/33elmDz) |
|
||||||
|
| 100 | 95 | 90 | 98 | 97 | 99 | [eval](https://sharegpt.com/c/RWD3Zx7) |
|
||||||
|
| 95 | 90 | 85 | 95 | 80 | 92 | [eval](https://sharegpt.com/c/GiwBvM7) |
|
||||||
|
| 100 | 95 | 100 | 98 | 100 | 90 | [eval](https://sharegpt.com/c/hX2pYxk) |
|
||||||
|
| 100 | 95 | 80 | 85 | 90 | 85 | [eval](https://sharegpt.com/c/MfxdGd7) |
|
||||||
|
| 100 | 90 | 95 | 85 | 95 | 100 | [eval](https://sharegpt.com/c/28hQjmS) |
|
||||||
|
| 95 | 90 | 85 | 80 | 88 | 92 | [eval](https://sharegpt.com/c/fzy5EPe) |
|
||||||
|
| 100 | 100 | 0 | 0 | 100 | 0 | [eval](https://sharegpt.com/c/vwxPjbR) |
|
||||||
|
| 100 | 100 | 100 | 50 | 100 | 75 | [eval](https://sharegpt.com/c/FAYfFWy) |
|
||||||
|
| 100 | 100 | 0 | 0 | 100 | 0 | [eval](https://sharegpt.com/c/SoudGsQ) |
|
||||||
|
| 0 | 100 | 0 | 0 | 0 | 0 | [eval](https://sharegpt.com/c/mkwEgVn) |
|
||||||
|
| 100 | 100 | 50 | 0 | 0 | 0 | [eval](https://sharegpt.com/c/q8MQEsz) |
|
||||||
|
| 100 | 100 | 100 | 100 | 100 | 95 | [eval](https://sharegpt.com/c/tzHpsKh) |
|
||||||
|
| 100 | 100 | 50 | 0 | 0 | 0 | [eval](https://sharegpt.com/c/3ugYBtJ) |
|
||||||
|
| 100 | 100 | 0 | 0 | 100 | 0 | [eval](https://sharegpt.com/c/I6KfOJT) |
|
||||||
|
| 90 | 85 | 80 | 95 | 70 | 75 | [eval](https://sharegpt.com/c/enaV1CK) |
|
||||||
|
| 100 | 100 | 0 | 0 | 0 | 0 | [eval](https://sharegpt.com/c/JBk7oSh) |
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
|
|
||||||
|
### Training data
|
||||||
|
|
||||||
|
I used a jailbreak prompt to generate the synthetic instructions, which resulted in some training data that would likely be censored by other models, such as how-to prompts about synthesizing drugs, making homemade flamethrowers, etc. Mind you, this is all generated by ChatGPT, not me. My goal was to simply test some of the capabilities of ChatGPT when unfiltered (as much as possible), and not to intentionally produce any harmful/dangerous/etc. content.
|
||||||
|
|
||||||
|
The jailbreak prompt I used is the default prompt in the python code when using the `--uncensored` flag: https://github.com/jondurbin/airoboros/blob/main/airoboros/self_instruct.py#L39
|
||||||
|
|
||||||
|
I also did a few passes of manually cleanup to remove some bad prompts, but mostly I left the data as-is. Initially, the model was fairly bad at math/extrapolation, closed question-answering (heavy hallucination), and coding, so I did one more fine tuning pass with additional synthetic instructions aimed at those types of problems.
|
||||||
|
|
||||||
|
Both the initial instructions and final-pass fine-tuning instructions will be published soon.
|
||||||
|
|
||||||
|
### Fine-tuning method
|
||||||
|
|
||||||
|
I used the excellent [FastChat](https://github.com/lm-sys/FastChat) module, running with:
|
||||||
|
|
||||||
|
```
|
||||||
|
source /workspace/venv/bin/activate
|
||||||
|
|
||||||
|
export NCCL_P2P_DISABLE=1
|
||||||
|
export NCCL_P2P_LEVEL=LOC
|
||||||
|
|
||||||
|
torchrun --nproc_per_node=8 --master_port=20001 /workspace/FastChat/fastchat/train/train_mem.py \
|
||||||
|
--model_name_or_path /workspace/llama-13b \
|
||||||
|
--data_path /workspace/as_conversations.json \
|
||||||
|
--bf16 True \
|
||||||
|
--output_dir /workspace/airoboros-uncensored-13b \
|
||||||
|
--num_train_epochs 3 \
|
||||||
|
--per_device_train_batch_size 20 \
|
||||||
|
--per_device_eval_batch_size 20 \
|
||||||
|
--gradient_accumulation_steps 2 \
|
||||||
|
--evaluation_strategy "steps" \
|
||||||
|
--eval_steps 500 \
|
||||||
|
--save_strategy "steps" \
|
||||||
|
--save_steps 500 \
|
||||||
|
--save_total_limit 10 \
|
||||||
|
--learning_rate 2e-5 \
|
||||||
|
--weight_decay 0. \
|
||||||
|
--warmup_ratio 0.04 \
|
||||||
|
--lr_scheduler_type "cosine" \
|
||||||
|
--logging_steps 1 \
|
||||||
|
--fsdp "full_shard auto_wrap offload" \
|
||||||
|
--fsdp_transformer_layer_cls_to_wrap 'LlamaDecoderLayer' \
|
||||||
|
--tf32 True \
|
||||||
|
--model_max_length 2048 \
|
||||||
|
--gradient_checkpointing True \
|
||||||
|
--lazy_preprocess True
|
||||||
|
```
|
||||||
|
|
||||||
|
This ran on 8x nvidia 80gb a100's for about 40 hours.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### Prompt format
|
||||||
|
|
||||||
|
The prompt should be 1:1 compatible with the FastChat/vicuna format, e.g.:
|
||||||
|
|
||||||
|
With a preamble:
|
||||||
|
```
|
||||||
|
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
|
||||||
|
|
||||||
|
USER: [prompt]
|
||||||
|
<\s>
|
||||||
|
|
||||||
|
ASSISTANT:
|
||||||
|
```
|
||||||
|
|
||||||
|
Or just:
|
||||||
|
```
|
||||||
|
USER: [prompt]
|
||||||
|
<\s>
|
||||||
|
|
||||||
|
ASSISTANT:
|
||||||
|
```
|
||||||
|
|
||||||
|
### License
|
||||||
|
The model is licensed under the LLaMA model, and the dataset is licensed under the terms of OpenAI because it uses ChatGPT. Everything else is free.
|
||||||
24
config.json
Normal file
24
config.json
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
{
|
||||||
|
"_name_or_path": "/workspace/models/jondurbin_airoboros-13b",
|
||||||
|
"architectures": [
|
||||||
|
"LlamaForCausalLM"
|
||||||
|
],
|
||||||
|
"bos_token_id": 1,
|
||||||
|
"eos_token_id": 2,
|
||||||
|
"hidden_act": "silu",
|
||||||
|
"hidden_size": 5120,
|
||||||
|
"initializer_range": 0.02,
|
||||||
|
"intermediate_size": 13824,
|
||||||
|
"max_position_embeddings": 2048,
|
||||||
|
"max_sequence_length": 2048,
|
||||||
|
"model_type": "llama",
|
||||||
|
"num_attention_heads": 40,
|
||||||
|
"num_hidden_layers": 40,
|
||||||
|
"pad_token_id": 0,
|
||||||
|
"rms_norm_eps": 1e-06,
|
||||||
|
"tie_word_embeddings": false,
|
||||||
|
"torch_dtype": "float16",
|
||||||
|
"transformers_version": "4.29.2",
|
||||||
|
"use_cache": true,
|
||||||
|
"vocab_size": 32000
|
||||||
|
}
|
||||||
1
configuration.json
Normal file
1
configuration.json
Normal file
@@ -0,0 +1 @@
|
|||||||
|
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}
|
||||||
7
generation_config.json
Normal file
7
generation_config.json
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
{
|
||||||
|
"_from_model_config": true,
|
||||||
|
"bos_token_id": 1,
|
||||||
|
"eos_token_id": 2,
|
||||||
|
"pad_token_id": 0,
|
||||||
|
"transformers_version": "4.29.2"
|
||||||
|
}
|
||||||
3
pytorch_model-00001-of-00003.bin
Normal file
3
pytorch_model-00001-of-00003.bin
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:17466be05d3dc1dad4a960b4082e16f6d8dd2554fad7ec9bf8ba2fc971d77970
|
||||||
|
size 9948726510
|
||||||
3
pytorch_model-00002-of-00003.bin
Normal file
3
pytorch_model-00002-of-00003.bin
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:567c753bc9c79e24d2cda58fdf7ee9d92b8db8130a2a5217b943a7f16ce9f501
|
||||||
|
size 9904162976
|
||||||
3
pytorch_model-00003-of-00003.bin
Normal file
3
pytorch_model-00003-of-00003.bin
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:4483838095b631d5b30aec86f293636672ee234c7c3a8ef6864f557d8fab4fd9
|
||||||
|
size 6178982473
|
||||||
410
pytorch_model.bin.index.json
Normal file
410
pytorch_model.bin.index.json
Normal file
@@ -0,0 +1,410 @@
|
|||||||
|
{
|
||||||
|
"metadata": {
|
||||||
|
"total_size": 26031733760
|
||||||
|
},
|
||||||
|
"weight_map": {
|
||||||
|
"lm_head.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.embed_tokens.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.0.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.1.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.1.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.1.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.1.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.1.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.10.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.10.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.10.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.10.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.10.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.10.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.10.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.10.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.10.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.10.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.11.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.11.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.11.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.11.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.11.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.11.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.11.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.11.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.11.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.11.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.12.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.12.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.12.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.12.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.12.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.12.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.12.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.12.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.12.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.12.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.13.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.13.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.13.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.13.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.13.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.13.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.13.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.13.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.13.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.13.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.14.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.14.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.14.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.14.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.14.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.14.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.14.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.14.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.14.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.14.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.15.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.15.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.15.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.15.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.15.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.15.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.15.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.15.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.15.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.15.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.16.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.16.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.16.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.16.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.16.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.16.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.16.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.16.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.16.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.16.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.17.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.17.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.17.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.17.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.17.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.17.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.17.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.17.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.17.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.17.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.18.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.18.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.18.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.18.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.18.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.18.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.18.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.18.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.18.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.18.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.19.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.19.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.19.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.19.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.19.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.19.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.19.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.19.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.19.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.19.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.2.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.2.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.2.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.2.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.2.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.2.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.20.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.20.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.20.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.20.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.20.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.20.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.20.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.20.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.20.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.20.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.21.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.21.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.21.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.21.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.21.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.21.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.21.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.21.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.21.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.21.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.22.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.22.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.22.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.22.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.22.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.22.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.22.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.22.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.22.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.22.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.23.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.23.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.23.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.23.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.23.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.23.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.23.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.23.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.23.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.23.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.24.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.24.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.24.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.24.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.24.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.24.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.24.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.24.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.24.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.24.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.25.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.25.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.25.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.25.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.25.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.25.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.25.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.25.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.25.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.25.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.26.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.26.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.26.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.26.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.26.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.26.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.26.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.26.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.26.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.26.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.27.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.27.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.27.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.27.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.27.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.27.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.27.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.27.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.27.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.27.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.28.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.28.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.28.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.28.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.28.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.28.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.28.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.28.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.28.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.28.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.29.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.29.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.29.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.29.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.29.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.29.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.29.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.29.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.29.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.29.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.3.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.3.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.3.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.3.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.3.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.3.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.3.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.3.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.3.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.3.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.30.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.30.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.30.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.30.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.30.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.30.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.30.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.30.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.30.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.30.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
|
||||||
|
"model.layers.31.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.31.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.31.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.31.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.31.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.31.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.31.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.31.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.31.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.31.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.32.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.32.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.32.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.32.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.32.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.32.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.32.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.32.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.32.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.32.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.33.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.33.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.33.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.33.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.33.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.33.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.33.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.33.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.33.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.33.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.34.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.34.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.34.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.34.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.34.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.34.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.34.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.34.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.34.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.34.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.35.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.35.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.35.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.35.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.35.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.35.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.35.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.35.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.35.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.35.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.36.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.36.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.36.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.36.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.36.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.36.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.36.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.36.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.36.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.36.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.37.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.37.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.37.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.37.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.37.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.37.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.37.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.37.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.37.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.37.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.38.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.38.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.38.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.38.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.38.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.38.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.38.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.38.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.38.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.38.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.39.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.39.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.39.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.39.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.39.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.39.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.39.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.39.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.39.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.39.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
|
||||||
|
"model.layers.4.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.4.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.4.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.4.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.4.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.4.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.4.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.4.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.4.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.4.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.5.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.5.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.5.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.5.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.5.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.5.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.5.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.5.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.5.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.5.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.6.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.6.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.6.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.6.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.6.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.6.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.6.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.6.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.6.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.6.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.7.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.7.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.7.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.7.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.7.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.7.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.7.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.7.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.7.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.7.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.8.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.8.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.8.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.8.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.8.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.8.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.8.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.8.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.8.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.8.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.9.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.9.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.9.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.9.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.9.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.9.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.9.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.9.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.9.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.layers.9.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
|
||||||
|
"model.norm.weight": "pytorch_model-00003-of-00003.bin"
|
||||||
|
}
|
||||||
|
}
|
||||||
93400
tokenizer.json
Normal file
93400
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
BIN
tokenizer.model
(Stored with Git LFS)
Normal file
BIN
tokenizer.model
(Stored with Git LFS)
Normal file
Binary file not shown.
34
tokenizer_config.json
Normal file
34
tokenizer_config.json
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
{
|
||||||
|
"add_bos_token": true,
|
||||||
|
"add_eos_token": false,
|
||||||
|
"bos_token": {
|
||||||
|
"__type": "AddedToken",
|
||||||
|
"content": "<s>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": true,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false
|
||||||
|
},
|
||||||
|
"clean_up_tokenization_spaces": false,
|
||||||
|
"eos_token": {
|
||||||
|
"__type": "AddedToken",
|
||||||
|
"content": "</s>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": true,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false
|
||||||
|
},
|
||||||
|
"model_max_length": 2048,
|
||||||
|
"pad_token": null,
|
||||||
|
"padding_side": "right",
|
||||||
|
"sp_model_kwargs": {},
|
||||||
|
"tokenizer_class": "LlamaTokenizer",
|
||||||
|
"unk_token": {
|
||||||
|
"__type": "AddedToken",
|
||||||
|
"content": "<unk>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": true,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user