初始化项目，由ModelHub XC社区提供模型
Model: failspy/Codestral-22B-v0.1-abliterated-v3 Source: Original Platform
This commit is contained in:
ModelHub XC
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
175
README.md
Normal file
175
README.md
Normal file
@@ -0,0 +1,175 @@
|
||||
---
|
||||
library_name: transformers
|
||||
license: other
|
||||
license_name: mnpl
|
||||
license_link: https://mistral.ai/licences/MNPL-0.1.md
|
||||
tags:
|
||||
- code
|
||||
language:
|
||||
- code
|
||||
---
|
||||
# Codestral-22B-v0.1-abliterated-v3 Model Card
|
||||
|
||||
[My original Jupyter "cookbook" to replicate the methodology can be found here](https://huggingface.co/failspy/llama-3-70B-Instruct-abliterated/blob/main/ortho_cookbook.ipynb)
|
||||
|
||||
[My personal library o' code used](https://github.com/FailSpy/abliterator) (WIP, looking to improve and generalize)
|
||||
|
||||
This is [mistralai/Codestral-22B-v0.1](https://huggingface.co/mistralai/Codestral-22B-v0.1) with orthogonalized bfloat16 safetensor weights, generated with a refined methodology based on that which was described in the preview paper/blog post: '[Refusal in LLMs is mediated by a single direction](https://www.alignmentforum.org/posts/jGuXSZgv6qfdhMCuJ/refusal-in-llms-is-mediated-by-a-single-direction)' which I encourage you to read to understand more.
|
||||
|
||||
Thanks to [bullerwins](https://huggingface.co/bullerwins) for re-uploading the original model in HF form.
|
||||
|
||||
## Hang on, "abliteration"? Orthogonalization? Ablation? What is this?
|
||||
|
||||
TL;DR: This model has had certain weights manipulated to "inhibit" the model's ability to express refusal. It is not in anyway _guaranteed_ that it won't refuse you, understand your request, it may still lecture you about ethics/safety, etc. It is tuned in all other respects the same as the original 22B model was, just with the strongest refusal directions orthogonalized out.
|
||||
|
||||
**TL;TL;DR;DR: It's uncensored in the purest form I can manage -- no new or changed behaviour in any other respect from the original model.**
|
||||
|
||||
As far as "abliteration": it's just a fun play-on-words using the original "ablation" term used in the original paper to refer to removing features, which I made up particularly to differentiate the model from "uncensored" fine-tunes.
|
||||
Ablate + obliterated = Abliterated
|
||||
|
||||
Anyways, orthogonalization/ablation are both aspects to refer to the same thing here, the technique in which the refusal feature was "ablated" from the model was via orthogonalization.
|
||||
|
||||
## Why uncensor a code model?
|
||||
|
||||
Honestly, this model seems pretty solid outside of code, and it's a perfect size model for 24GB once quantized.
|
||||
By ablating refusals, the model is overall more compliant to the user's requests, regardless of ethicality. It's worth remembering that sometimes even "good-aligned" requests can be refused and have to be prompt-engineered around.
|
||||
|
||||
## A little more on the methodology, and why this is interesting
|
||||
|
||||
To me, ablation (or applying the methodology for the inverse, "augmentation") seems to be good for inducing/removing very specific features that you'd have to spend way too many tokens on encouraging or discouraging in your system prompt.
|
||||
Instead, you just apply your system prompt in the ablation script against a blank system prompt on the same dataset and orthogonalize for the desired behaviour in the final model weights.
|
||||
|
||||
> Why this over fine-tuning?
|
||||
|
||||
Ablation is much more surgical in nature whilst also being effectively executed with a _lot_ less data than fine-tuning, which I think is its main advantage.
|
||||
|
||||
As well, and its most valuable aspect is it keeps as much of the original model's knowledge and training intact, whilst removing its tendency to behave in one very specific undesireable manner. (In this case, refusing user requests.)
|
||||
|
||||
Fine tuning is still exceptionally useful and the go-to for broad behaviour changes; however, you may be able to get close to your desired behaviour with very few samples using the ablation/augmentation techniques.
|
||||
It may also be a useful step to add to your model refinement: orthogonalize -> fine-tune or vice-versa.
|
||||
|
||||
I haven't really gotten around to exploring this model stacked with fine-tuning, I encourage others to give it a shot if they've got the capacity.
|
||||
|
||||
> Okay, fine, but why V3? There's no V2 70B?
|
||||
|
||||
Well, I released a V2 a while back for 8B under Cognitive Computations.
|
||||
It ended up being not worth it to try V2 with 70B, I wanted to refine the model before wasting compute cycles on what might not even be a better model.
|
||||
I am however quite pleased about this latest methodology, it seems to have induced fewer hallucinations.
|
||||
So to show that it's a new fancy methodology from even that of the 8B V2, I decided to do a Microsoft and double up on my version jump because it's *such* an advancement (or so the excuse went, when in actuality it was because too many legacy but actively used Microsoft libraries checked for 'Windows 9' in the OS name to detect Windows 95/98 as one.)
|
||||
|
||||
## Quirkiness awareness notice
|
||||
|
||||
This model may come with interesting quirks, with the methodology being so new. I encourage you to play with the model, and post any quirks you notice in the community tab, as that'll help us further understand what this orthogonalization has in the way of side effects.
|
||||
|
||||
If you manage to develop further improvements, please share! This is really the most basic way to use ablation, but there are other possibilities that I believe are as-yet unexplored.
|
||||
|
||||
Additionally, feel free to reach out in any way about this. I'm on the Cognitive Computations Discord, I'm watching the Community tab, reach out! I'd love to see this methodology used in other ways, and so would gladly support whoever whenever I can.
|
||||
|
||||
|
||||
# Original Model Card for Codestral-22B-v0.1
|
||||
|
||||
Codestrall-22B-v0.1 is trained on a diverse dataset of 80+ programming languages, including the most popular ones, such as Python, Java, C, C++, JavaScript, and Bash (more details in the [Blogpost](https://mistral.ai/news/codestral/)). The model can be queried:
|
||||
- As instruct, for instance to answer any questions about a code snippet (write documentation, explain, factorize) or to generate code following specific indications
|
||||
- As Fill in the Middle (FIM), to predict the middle tokens between a prefix and a suffix (very useful for software development add-ons like in VS Code)
|
||||
|
||||
|
||||
## Installation
|
||||
|
||||
It is recommended to use `mistralai/Codestral-22B-v0.1` with [mistral-inference](https://github.com/mistralai/mistral-inference).
|
||||
|
||||
```
|
||||
pip install mistral_inference
|
||||
```
|
||||
|
||||
## Download
|
||||
|
||||
```py
|
||||
from huggingface_hub import snapshot_download
|
||||
from pathlib import Path
|
||||
|
||||
mistral_models_path = Path.home().joinpath('mistral_models', 'Codestral-22B-v0.1')
|
||||
mistral_models_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
snapshot_download(repo_id="mistralai/Codestral-22B-v0.1", allow_patterns=["params.json", "consolidated.safetensors", "tokenizer.model.v3"], local_dir=mistral_models_path)
|
||||
```
|
||||
|
||||
### Chat
|
||||
|
||||
After installing `mistral_inference`, a `mistral-chat` CLI command should be available in your environment.
|
||||
|
||||
```
|
||||
mistral-chat $HOME/mistral_models/Codestral-22B-v0.1 --instruct --max_tokens 256
|
||||
```
|
||||
|
||||
Will generate an answer to "Write me a function that computes fibonacci in Rust" and should give something along the following lines:
|
||||
|
||||
```
|
||||
Sure, here's a simple implementation of a function that computes the Fibonacci sequence in Rust. This function takes an integer `n` as an argument and returns the `n`th Fibonacci number.
|
||||
|
||||
fn fibonacci(n: u32) -> u32 {
|
||||
match n {
|
||||
0 => 0,
|
||||
1 => 1,
|
||||
_ => fibonacci(n - 1) + fibonacci(n - 2),
|
||||
}
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let n = 10;
|
||||
println!("The {}th Fibonacci number is: {}", n, fibonacci(n));
|
||||
}
|
||||
|
||||
This function uses recursion to calculate the Fibonacci number. However, it's not the most efficient solution because it performs a lot of redundant calculations. A more efficient solution would use a loop to iteratively calculate the Fibonacci numbers.
|
||||
```
|
||||
|
||||
|
||||
### Fill-in-the-middle (FIM)
|
||||
|
||||
After installing `mistral_inference` and running `pip install --upgrade mistral_common` to make sure to have mistral_common>=1.2 installed:
|
||||
|
||||
```py
|
||||
from mistral_inference.model import Transformer
|
||||
from mistral_inference.generate import generate
|
||||
from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
|
||||
from mistral_common.tokens.instruct.request import FIMRequest
|
||||
|
||||
tokenizer = MistralTokenizer.v3()
|
||||
model = Transformer.from_folder("~/codestral-22B-240529")
|
||||
|
||||
prefix = """def add("""
|
||||
suffix = """ return sum"""
|
||||
|
||||
request = FIMRequest(prompt=prefix, suffix=suffix)
|
||||
|
||||
tokens = tokenizer.encode_fim(request).tokens
|
||||
|
||||
out_tokens, _ = generate([tokens], model, max_tokens=256, temperature=0.0, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)
|
||||
result = tokenizer.decode(out_tokens[0])
|
||||
|
||||
middle = result.split(suffix)[0].strip()
|
||||
print(middle)
|
||||
```
|
||||
|
||||
Should give something along the following lines:
|
||||
|
||||
```
|
||||
num1, num2):
|
||||
|
||||
# Add two numbers
|
||||
sum = num1 + num2
|
||||
|
||||
# return the sum
|
||||
```
|
||||
|
||||
## Limitations
|
||||
|
||||
The Codestral-22B-v0.1 does not have any moderation mechanisms. We're looking forward to engaging with the community on ways to
|
||||
make the model finely respect guardrails, allowing for deployment in environments requiring moderated outputs.
|
||||
|
||||
## License
|
||||
|
||||
Codestral-22B-v0.1 is released under the `MNLP-0.1` license.
|
||||
|
||||
## The Mistral AI Team
|
||||
|
||||
Albert Jiang, Alexandre Sablayrolles, Alexis Tacnet, Antoine Roux, Arthur Mensch, Audrey Herblin-Stoop, Baptiste Bout, Baudouin de Monicault, Blanche Savary, Bam4d, Caroline Feldman, Devendra Singh Chaplot, Diego de las Casas, Eleonore Arcelin, Emma Bou Hanna, Etienne Metzger, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Harizo Rajaona, Henri Roussez, Jean-Malo Delignon, Jia Li, Justus Murke, Kartik Khandelwal, Lawrence Stewart, Louis Martin, Louis Ternon, Lucile Saulnier, Lélio Renard Lavaud, Margaret Jennings, Marie Pellat, Marie Torelli, Marie-Anne Lachaux, Marjorie Janiewicz, Mickael Seznec, Nicolas Schuhl, Patrick von Platen, Romain Sauvestre, Pierre Stock, Sandeep Subramanian, Saurabh Garg, Sophia Yang, Szymon Antoniak, Teven Le Scao, Thibaut Lavril, Thibault Schueller, Timothée Lacroix, Théophile Gervet, Thomas Wang, Valera Nemychnikova, Wendy Shang, William El Sayed, William Marshall
|
||||
26
config.json
Normal file
26
config.json
Normal file
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"_name_or_path": "bullerwins/Codestral-22B-v0.1-hf",
|
||||
"architectures": [
|
||||
"MistralForCausalLM"
|
||||
],
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 1,
|
||||
"eos_token_id": 2,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 6144,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 16384,
|
||||
"max_position_embeddings": 32768,
|
||||
"model_type": "mistral",
|
||||
"num_attention_heads": 48,
|
||||
"num_hidden_layers": 56,
|
||||
"num_key_value_heads": 8,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_theta": 1000000.0,
|
||||
"sliding_window": null,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "bfloat16",
|
||||
"transformers_version": "4.40.1",
|
||||
"use_cache": true,
|
||||
"vocab_size": 32768
|
||||
}
|
||||
6
generation_config.json
Normal file
6
generation_config.json
Normal file
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 1,
|
||||
"eos_token_id": 2,
|
||||
"transformers_version": "4.40.1"
|
||||
}
|
||||
3
model-00001-of-00009.safetensors
Normal file
3
model-00001-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6c911e66544527032c9e49f602ed0645f748045248eb8fb8ec9982866b899674
|
||||
size 4882298776
|
||||
3
model-00002-of-00009.safetensors
Normal file
3
model-00002-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8d7664c371bd49b0feb6acfc63a9dba14308d0817a95b37e334a3cb6c4e1b196
|
||||
size 4983012160
|
||||
3
model-00003-of-00009.safetensors
Normal file
3
model-00003-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e0dfe9f3ef8a31babc2c2426e27f2d2c6f810bcc2dcd9a0bb61d8a0f83f2e8d4
|
||||
size 4957821336
|
||||
3
model-00004-of-00009.safetensors
Normal file
3
model-00004-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9c0afb59bf7e4148fe6cb4f782abb96a97f83496a6ab18f7967714b01d8535fb
|
||||
size 4882323744
|
||||
3
model-00005-of-00009.safetensors
Normal file
3
model-00005-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:71220586253bb01bd57795a3dda26d06b1bc61516b8f9ae7ffedc5a816ce7635
|
||||
size 4983012192
|
||||
3
model-00006-of-00009.safetensors
Normal file
3
model-00006-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:58309f5ed04340c3caeb69e1c0be123d2dbc763c98b1831eb6d4e73890da2511
|
||||
size 4957821336
|
||||
3
model-00007-of-00009.safetensors
Normal file
3
model-00007-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:53c515d725098b03f132b05348cace6869fbd1e3f3cf5af22a0a193421547fe4
|
||||
size 4882323744
|
||||
3
model-00008-of-00009.safetensors
Normal file
3
model-00008-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:28ef0e880228397ad0ca43e3e8b723bace12a0402c32725c4d3714101f32a2ce
|
||||
size 4983012192
|
||||
3
model-00009-of-00009.safetensors
Normal file
3
model-00009-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c834720ddae75dc683e52284ffe27ea35f48eb2c5500c71025925fe0dd398a8c
|
||||
size 4982999056
|
||||
514
model.safetensors.index.json
Normal file
514
model.safetensors.index.json
Normal file
@@ -0,0 +1,514 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 44494565376
|
||||
},
|
||||
"weight_map": {
|
||||
"lm_head.weight": "model-00009-of-00009.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.28.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.28.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.28.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.28.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.28.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.29.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.29.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.29.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.29.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.29.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.30.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.30.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.30.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.30.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.30.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.31.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.31.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.31.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.31.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.31.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.32.input_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.32.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.32.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.32.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.32.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.32.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.32.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.32.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.32.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.33.input_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.33.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.33.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.33.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.33.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.33.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.33.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.33.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.33.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.34.input_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.34.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.34.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.34.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.34.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.34.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.34.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.34.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.34.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.35.input_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.35.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.35.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.35.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.35.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.35.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.35.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.35.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.35.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.36.input_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.36.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.36.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.36.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.36.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.36.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.36.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.36.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.36.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.37.input_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.37.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.37.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.37.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.37.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.37.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.37.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.37.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.37.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.38.input_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.38.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.38.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.38.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.38.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.38.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.38.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.38.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.38.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.39.input_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.39.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.39.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.39.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.39.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.39.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.39.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.39.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.39.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.40.input_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.40.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.40.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.40.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.40.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.40.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.40.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.40.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.40.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.41.input_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.41.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.41.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.41.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.41.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.41.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.41.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.41.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.41.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.42.input_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.42.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.42.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.42.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.42.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.42.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.42.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.42.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.42.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.43.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.43.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.43.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.43.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.43.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.43.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.43.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.43.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.43.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.44.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.44.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.44.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.44.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.44.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.44.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.44.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.44.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.44.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.45.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.45.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.45.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.45.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.45.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.45.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.45.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.45.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.45.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.46.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.46.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.46.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.46.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.46.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.46.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.46.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.46.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.46.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.47.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.47.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.47.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.47.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.47.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.47.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.47.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.47.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.47.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.48.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.48.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.48.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.48.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.48.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.48.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.48.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.48.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.48.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.49.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.49.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.49.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.49.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.49.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.49.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.49.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.49.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.49.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.50.input_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.50.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.50.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.50.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.50.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.50.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.50.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.50.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.50.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.51.input_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.51.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.51.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.51.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.51.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.51.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.51.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.51.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.51.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.52.input_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.52.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.52.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.52.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.52.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.52.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.52.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.52.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.52.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.53.input_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.53.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.53.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.53.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.53.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.53.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.53.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.53.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.53.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.54.input_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.54.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.54.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.54.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.54.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.54.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.54.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.54.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.54.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.55.input_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.55.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.55.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.55.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.55.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.55.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.55.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.55.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.55.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.norm.weight": "model-00009-of-00009.safetensors"
|
||||
}
|
||||
}
|
||||
23
special_tokens_map.json
Normal file
23
special_tokens_map.json
Normal file
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "</s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"unk_token": {
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
91945
tokenizer.json
Normal file
91945
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
3
tokenizer.model
Normal file
3
tokenizer.model
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9addc8bdce5988448ae81b729336f43a81262160ae8da760674badab9d4c7d33
|
||||
size 587591
|
||||
90
tokenizer_config.json
Normal file
90
tokenizer_config.json
Normal file
@@ -0,0 +1,90 @@
|
||||
{
|
||||
"add_bos_token": true,
|
||||
"add_eos_token": false,
|
||||
"add_prefix_space": true,
|
||||
"added_tokens_decoder": {
|
||||
"0": {
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"1": {
|
||||
"content": "<s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"2": {
|
||||
"content": "</s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"3": {
|
||||
"content": "[INST]",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"4": {
|
||||
"content": "[/INST]",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"10": {
|
||||
"content": "[IMG]",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"11": {
|
||||
"content": "[PREFIX]",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"12": {
|
||||
"content": "[MIDDLE]",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"13": {
|
||||
"content": "[SUFFIX]",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
}
|
||||
},
|
||||
"bos_token": "<s>",
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "</s>",
|
||||
"legacy": true,
|
||||
"model_max_length": 1000000000000000019884624838656,
|
||||
"pad_token": null,
|
||||
"sp_model_kwargs": {},
|
||||
"spaces_between_special_tokens": false,
|
||||
"tokenizer_class": "LlamaTokenizer",
|
||||
"unk_token": "<unk>",
|
||||
"use_default_system_prompt": false
|
||||
}
|
||||
Reference in New Issue
Block a user