初始化项目,由ModelHub XC社区提供模型

Model: mrshu/qwen3-14b-sk-alpaca-slovak-it
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-29 08:12:16 +08:00
commit 32e6f5defe
70 changed files with 1035 additions and 0 deletions

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

164
README.md Normal file
View File

@@ -0,0 +1,164 @@
---
library_name: transformers
license: apache-2.0
language:
- sk
- en
base_model:
- slovak-nlp/Qwen3-14B-sk
datasets:
- saillab/alpaca-slovak-cleaned
tags:
- slovak
- instruction-tuned
- qwen3
- lora
- merged-lora
- text-generation
pipeline_tag: text-generation
---
# Qwen3-14B-sk Alpaca Slovak IT
Qwen3-14B-sk Alpaca Slovak IT is an instruction-tuned Slovak assistant model
derived from `slovak-nlp/Qwen3-14B-sk`. It was trained with a LoRA supervised
fine-tuning recipe on Slovak Alpaca-style instruction data and then merged back
into the base model, so the released artifact can be loaded directly with
`transformers`.
The model is designed to answer in Slovak by default while still attempting to
follow explicit user requests for another language or a specific output format.
## Intended Use
This model is intended for Slovak instruction following, Slovak question
answering, drafting, rewriting, summarization-style prompts, and general
assistant workflows where Slovak is the primary language.
It can also respond to English prompts and translation-style requests, but
language control is not guaranteed. Use additional application-level checks for
strict formatting, policy compliance, or high-reliability translation.
Do not use this model as the sole source for medical, legal, financial, safety,
or other high-stakes decisions. It has not been safety aligned, red-teamed, or
moderated for production deployment.
## Model Details
| Field | Value |
| --- | --- |
| Base model | `slovak-nlp/Qwen3-14B-sk` |
| Base revision | `61bc40572b2fa4720e75a7a9de457151aefce396` |
| Release repository | [`mrshu/qwen3-14b-sk-alpaca-slovak-it`](https://huggingface.co/mrshu/qwen3-14b-sk-alpaca-slovak-it) |
| Architecture | Qwen3 causal language model |
| Tuned artifact | Merged full-weight checkpoint |
| Adapter used during training | LoRA |
| Weight dtype | bfloat16 |
| License | Apache-2.0 |
## Training Data
The model was instruction-tuned on
[`saillab/alpaca-slovak-cleaned`](https://huggingface.co/datasets/saillab/alpaca-slovak-cleaned),
a Slovak Alpaca-style instruction dataset.
Dataset preparation converted each example into a chat-style conversation with
system, user, and assistant messages. Empty instruction/output examples were
excluded, and duplicate instruction/input/output triples were removed across
the prepared splits.
| Split | Rows |
| --- | ---: |
| Train | 41,601 |
| Held-out | 10,401 |
The data preparation used dataset revision
`058172466eb1d6a28b161f29c74350911d154161`.
Each training conversation used this system prompt:
```text
Si užitočný asistent. Riaď sa jazykom a požadovaným formátom používateľa. Ak používateľ nežiada iný jazyk, odpovedaj po slovensky.
```
## Training Recipe
The model was trained with supervised fine-tuning using PEFT LoRA. Only the
assistant turns were included in the training loss.
| Setting | Value |
| --- | --- |
| LoRA rank / alpha / dropout | `32` / `64` / `0.05` |
| Target modules | linear projection modules |
| Sequence length | `4096` |
| Sample packing | enabled |
| Epochs | `1` |
| Effective batch size | `16` |
| Micro batch size | `1` |
| Gradient accumulation | `16` |
| Learning rate | `1e-4` |
| Scheduler | cosine |
| Warmup ratio | `0.06` |
| Optimizer | fused AdamW |
| Precision | bfloat16 |
| Gradient checkpointing | enabled |
| Seed | `42` |
After training, the LoRA adapter was merged into the base model with PEFT and
saved as a standalone safetensors checkpoint.
## Usage
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "mrshu/qwen3-14b-sk-alpaca-slovak-it"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
)
messages = [
{
"role": "system",
"content": (
"Si užitočný asistent. Riaď sa jazykom a požadovaným formátom "
"používateľa. Ak používateľ nežiada iný jazyk, odpovedaj po slovensky."
),
},
{"role": "user", "content": "Stručne vysvetli, čo je LoRA."},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(
inputs,
max_new_tokens=512,
do_sample=True,
temperature=0.6,
top_p=0.95,
)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
```
## Limitations
- The supervised fine-tuning data is translated instruction data, so the model
may inherit translation artifacts, unnatural phrasing, or source-dataset
biases.
- The model is biased toward Slovak responses. Explicit requests for another
language may work, but they are not guaranteed.
- Strict JSON, exact labels, citations, and other constrained formats should be
validated outside the model.
- The model may hallucinate facts, produce unsafe content, or follow malicious
instructions without additional safeguards.

89
chat_template.jinja Normal file
View File

@@ -0,0 +1,89 @@
{%- if tools %}
{{- '<|im_start|>system\n' }}
{%- if messages[0].role == 'system' %}
{{- messages[0].content + '\n\n' }}
{%- endif %}
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
{%- if messages[0].role == 'system' %}
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
{%- set index = (messages|length - 1) - loop.index0 %}
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
{%- set ns.multi_step_tool = false %}
{%- set ns.last_query_index = index %}
{%- endif %}
{%- endfor %}
{%- for message in messages %}
{%- if message.content is string %}
{%- set content = message.content %}
{%- else %}
{%- set content = '' %}
{%- endif %}
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
{%- elif message.role == "assistant" %}
{%- set reasoning_content = '' %}
{%- if message.reasoning_content is string %}
{%- set reasoning_content = message.reasoning_content %}
{%- else %}
{%- if '</think>' in content %}
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
{%- endif %}
{%- endif %}
{%- if loop.index0 > ns.last_query_index %}
{%- if loop.last or (not loop.last and reasoning_content) %}
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- if message.tool_calls %}
{%- for tool_call in message.tool_calls %}
{%- if (loop.first and content) or (not loop.first) %}
{{- '\n' }}
{%- endif %}
{%- if tool_call.function %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{{- '<tool_call>\n{"name": "' }}
{{- tool_call.name }}
{{- '", "arguments": ' }}
{%- if tool_call.arguments is string %}
{{- tool_call.arguments }}
{%- else %}
{{- tool_call.arguments | tojson }}
{%- endif %}
{{- '}\n</tool_call>' }}
{%- endfor %}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- elif message.role == "tool" %}
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' }}
{{- content }}
{{- '\n</tool_response>' }}
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- if enable_thinking is defined and enable_thinking is false %}
{{- '<think>\n\n</think>\n\n' }}
{%- endif %}
{%- endif %}

75
config.json Normal file
View File

@@ -0,0 +1,75 @@
{
"architectures": [
"Qwen3ForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 151643,
"dtype": "bfloat16",
"eos_token_id": 151645,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 5120,
"initializer_range": 0.02,
"intermediate_size": 17408,
"layer_types": [
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention"
],
"max_position_embeddings": 40960,
"max_window_layers": 40,
"model_type": "qwen3",
"num_attention_heads": 40,
"num_hidden_layers": 40,
"num_key_value_heads": 8,
"pad_token_id": null,
"rms_norm_eps": 1e-06,
"rope_parameters": {
"rope_theta": 1000000,
"rope_type": "default"
},
"sliding_window": null,
"tie_word_embeddings": false,
"transformers_version": "5.5.0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 151936
}

13
generation_config.json Normal file
View File

@@ -0,0 +1,13 @@
{
"bos_token_id": 151643,
"do_sample": true,
"eos_token_id": [
151645,
151643
],
"pad_token_id": 151643,
"temperature": 0.6,
"top_k": 20,
"top_p": 0.95,
"transformers_version": "5.5.0"
}

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7fb1f6062b2e27159f996a59f705665a9c082f6bfc8607e28f63536e839d6214
size 1555824768

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:656607d19b6fe86a662dc84bf7678913731966aa144929224c04a7dfa46d7302
size 1555824784

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8d93f4e35ede43eca2b297a370a56c9ecbf07a37e7a2ac2339d9a71872793581
size 356526440

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0ac829f746cc613b1d24783661db2eb94ae27c43a13f4d7196997a328accb51b
size 482367120

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f462cbf2e721a0695ad2ca0aac67a1582dbe9fc6a7ad6e05d1a2d55404ae0192
size 482367120

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b35d5aaaadcb8fed60d551e252f938755b2b53bd8368c643fa8b5a36f9c5dbbc
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4b8f0a40ad60ddbafcc500924c20e29cc2a033abc4c747173fd95832f9f1b555
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2747582b35b42431fbc92f08d56e3412ced2a931337448837eeb7fbdcf245cdd
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4746e4f07ee6917a5252c9bbadb430205fd944e5e24d53a9be49a2330e3cfae5
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:781e997bb45300121bbfeb511dc98ff2e04ab24bd0ea1f9be3b511f8eb0fb5d7
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f86b7499f3d130001113d12dd3240fc7d47d9f1d6ee726a644b9196b6da0091d
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9dc70f463d9dab6346ca95600bc2fac7b2db41414d06994b91a00c250cb49000
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c5b21d1905e2c26cfddb80ca2fb0abceb6769b2c130267610be79f03d4174032
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b56628efdab672aeca8ebf7240e9f372360274cb7ed9b1a703845c74731b779c
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a9492578f2edc716488f9e7a6b573c2a29cdbf092a1358d3320c2174751343b3
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:627fb60e80aa6d74303da4fb447e403dec3f8b2dc037e37c2ae8f9968e0d6353
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bb4d2de4b69e466329f290571a2f02f22091418a3949b69ebb43603fd25e06b6
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f13f8aadd5b1aa834f44c0bd2eb09a22192236ea7d182acb99756f8301380963
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:716e972925cc47392f1a577b0af55daded7244bb26c26b2fec8422945f92e66f
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:752f27e377edf2186f45f49bf521c150ae4efbdba280adc896ea335fbd0d3f46
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:49a7e3e11ae84225c6a400cf65987f6ac2b91276c121acc620cbf1b5e7eb48aa
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3ade44d9d5a9d79a01d8e54237430fac4726596485c9120b0eb05971f1dea49c
size 482367120

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:83952f4ef6223723b67cf784e3c7bc56344b39f8f5d768cddae8c02426be640f
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:facf2387e83df9283e941f09f5627ec148207675d2dcfea049cf425819c4f533
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4822d58ffd99418c0d59f7d245113f1740ceb50ca6ff33b3e79a6227dc259682
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:aba1d4cf9153e6fb6ba6a19891574622c6f1c41ca9c3b869e4bf675ab5c17089
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:28f883dd0a570a1dad8d6fbcb9f8192ba3ba71a1bd2d95b312e932cb06476705
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cff2c4dbee5dfa6be3daad1f020eb5691ab1b94f460d40d642cacd644178ce28
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8e99bfa45a293de1ec98c2fa0aa87ba3b64afa36c1be24c65b874ff995a8f8e9
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3e5758b3aab0f4fb24192c4c7d09f48441fb58589b444f5948f3ab61a0ceed4f
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:944373abc35cbbcdf40e792eb3fc0399a131a386c70bb624c642aa338edb6794
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0faf4561e53042d549e0f0ea2fa7cdd63721d3dc676d7b370b54fc1f22692d8c
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dd40d795e571280f849d9f9af3dde53412193b26d4e3bc56f6f4f2e7fe76ed44
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ece920b6f29bd9cca34f42be9c848c7d8a26a91d6d98c0b741b9451e9d0ec17c
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:11dcc37b0b4b2534a5f042f49042ba7b5a9f59aabe11abf4aeb7b906cf90e543
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6dbd2dea8a72ff73c4a48dc9ae89b28f5a0e03e5769f7de35a885c8e8ff008f5
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:22cb5af0ba2ec6cdc127fb75f43dbb973ea1c0be466a143dc2720568e430f2be
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d18921f8dd7ec5224a3cb2017cedd9cfc72107ad8e8510a82ae6be5be4e6c2a5
size 482367120

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e0b2bdda7b234f0a21353fdef7c0ab5f684e517c5ab1ad263127ceb92a6f30e4
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:646e1c2e4234a705884271ecd2e01609586ef658403a5eac6695879c7bb9dee6
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e92ebe0ff004e752b6223d7386e145aed9a20d154328a536feb4956cd5639278
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:04156aeee6f87aa04aca6c34c11771ef6636ce41b10e9b55cd56c904a7b265f0
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:66912a2ec6f67061521893e47d0267a623e49415eb3156e0d200fb57abd4abba
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1fa2c7efac1e07d54b6656913ba563f5ecd78f8821436228fab121fc3cac8aa4
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a719b40601be10359c49107ff939e593fe11189bbe34a4314c688af1e2c38531
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c4f81864d07e20199d137b86311c7d43394a69192aa04c3ecccf99ac57a702a1
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:332a25fe3f7ff6552b844fb5bd6bc050adcade9b37204a103559718c56b1410d
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9444d48f6207e8b90f8fc35b3a84dd45bc196f77ab8e355d5a8bfaeb21f44de2
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c27a93f47492986d2437ced28b64f1c329edf493832e024ca683be91e3bf22a9
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1214d167f5ed76adc82f6ce32f6f738696d131ef461a2ed0663496f15675b2d3
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e2c4acd42e388ba59cea45db215cd7acc7a6a568fe99f017e9fb427f06fb400c
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2c166e83936aa0ab70aeea12ea078b580887e70cc27773f0af8f8c0bf6b67446
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:22fb69f6f51ef67ce40b633b6efb449fb1117463fc21401c922bcdaf5102b5b5
size 482367128

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b48180951a239e75203b607e9901a801d5fbebde4b8e7e345dd61164506f9768
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fe5ef1d0e8797c9bfba08aabdc9525a503d78a5c6240827f82dc63ef94dbc651
size 482367120

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8e4b6dd5f34b04d50409d58c1f50c4e4c02989b41ca6f3879f8c336baf6b1c61
size 482367120

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4998c2e72f4abd787b1d96315908f7140e205ec196df6bdaabedf91ef975053b
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:041135de41a651976fb426fd5d3130db38d1be9ef38a9962df6383fdfb69dd24
size 482367120

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:998cf945fdd74005da9e6876908cc4a7b228afc5c63efaaf1521f1a96cd6e631
size 482367120

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:061c5a54bb7b8494e9647d3f397cf81c579d0954d9a0bc30bad6fa86dda7317a
size 356516104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:73305a1e439aece8de59edbaf734bc51270d5f1885f75dec851a4a69b5df64df
size 482367120

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8209a45f6faa6030aa44fc01198b8d47ad4a64bd1f287a80321c1c443f6434ff
size 482367096

View File

@@ -0,0 +1,451 @@
{
"metadata": {
"total_parameters": 14768307200,
"total_size": 29536614400
},
"weight_map": {
"lm_head.weight": "model-00001-of-00062.safetensors",
"model.embed_tokens.weight": "model-00002-of-00062.safetensors",
"model.layers.0.input_layernorm.weight": "model-00003-of-00062.safetensors",
"model.layers.0.mlp.down_proj.weight": "model-00003-of-00062.safetensors",
"model.layers.0.mlp.gate_proj.weight": "model-00003-of-00062.safetensors",
"model.layers.0.mlp.up_proj.weight": "model-00004-of-00062.safetensors",
"model.layers.0.post_attention_layernorm.weight": "model-00004-of-00062.safetensors",
"model.layers.0.self_attn.k_norm.weight": "model-00004-of-00062.safetensors",
"model.layers.0.self_attn.k_proj.weight": "model-00004-of-00062.safetensors",
"model.layers.0.self_attn.o_proj.weight": "model-00004-of-00062.safetensors",
"model.layers.0.self_attn.q_norm.weight": "model-00004-of-00062.safetensors",
"model.layers.0.self_attn.q_proj.weight": "model-00004-of-00062.safetensors",
"model.layers.0.self_attn.v_proj.weight": "model-00004-of-00062.safetensors",
"model.layers.1.input_layernorm.weight": "model-00004-of-00062.safetensors",
"model.layers.1.mlp.down_proj.weight": "model-00004-of-00062.safetensors",
"model.layers.1.mlp.gate_proj.weight": "model-00005-of-00062.safetensors",
"model.layers.1.mlp.up_proj.weight": "model-00005-of-00062.safetensors",
"model.layers.1.post_attention_layernorm.weight": "model-00005-of-00062.safetensors",
"model.layers.1.self_attn.k_norm.weight": "model-00005-of-00062.safetensors",
"model.layers.1.self_attn.k_proj.weight": "model-00005-of-00062.safetensors",
"model.layers.1.self_attn.o_proj.weight": "model-00005-of-00062.safetensors",
"model.layers.1.self_attn.q_norm.weight": "model-00005-of-00062.safetensors",
"model.layers.1.self_attn.q_proj.weight": "model-00005-of-00062.safetensors",
"model.layers.1.self_attn.v_proj.weight": "model-00005-of-00062.safetensors",
"model.layers.10.input_layernorm.weight": "model-00005-of-00062.safetensors",
"model.layers.10.mlp.down_proj.weight": "model-00006-of-00062.safetensors",
"model.layers.10.mlp.gate_proj.weight": "model-00006-of-00062.safetensors",
"model.layers.10.mlp.up_proj.weight": "model-00007-of-00062.safetensors",
"model.layers.10.post_attention_layernorm.weight": "model-00007-of-00062.safetensors",
"model.layers.10.self_attn.k_norm.weight": "model-00007-of-00062.safetensors",
"model.layers.10.self_attn.k_proj.weight": "model-00007-of-00062.safetensors",
"model.layers.10.self_attn.o_proj.weight": "model-00007-of-00062.safetensors",
"model.layers.10.self_attn.q_norm.weight": "model-00007-of-00062.safetensors",
"model.layers.10.self_attn.q_proj.weight": "model-00007-of-00062.safetensors",
"model.layers.10.self_attn.v_proj.weight": "model-00007-of-00062.safetensors",
"model.layers.11.input_layernorm.weight": "model-00007-of-00062.safetensors",
"model.layers.11.mlp.down_proj.weight": "model-00007-of-00062.safetensors",
"model.layers.11.mlp.gate_proj.weight": "model-00008-of-00062.safetensors",
"model.layers.11.mlp.up_proj.weight": "model-00008-of-00062.safetensors",
"model.layers.11.post_attention_layernorm.weight": "model-00008-of-00062.safetensors",
"model.layers.11.self_attn.k_norm.weight": "model-00008-of-00062.safetensors",
"model.layers.11.self_attn.k_proj.weight": "model-00008-of-00062.safetensors",
"model.layers.11.self_attn.o_proj.weight": "model-00008-of-00062.safetensors",
"model.layers.11.self_attn.q_norm.weight": "model-00008-of-00062.safetensors",
"model.layers.11.self_attn.q_proj.weight": "model-00008-of-00062.safetensors",
"model.layers.11.self_attn.v_proj.weight": "model-00008-of-00062.safetensors",
"model.layers.12.input_layernorm.weight": "model-00008-of-00062.safetensors",
"model.layers.12.mlp.down_proj.weight": "model-00009-of-00062.safetensors",
"model.layers.12.mlp.gate_proj.weight": "model-00009-of-00062.safetensors",
"model.layers.12.mlp.up_proj.weight": "model-00010-of-00062.safetensors",
"model.layers.12.post_attention_layernorm.weight": "model-00010-of-00062.safetensors",
"model.layers.12.self_attn.k_norm.weight": "model-00010-of-00062.safetensors",
"model.layers.12.self_attn.k_proj.weight": "model-00010-of-00062.safetensors",
"model.layers.12.self_attn.o_proj.weight": "model-00010-of-00062.safetensors",
"model.layers.12.self_attn.q_norm.weight": "model-00010-of-00062.safetensors",
"model.layers.12.self_attn.q_proj.weight": "model-00010-of-00062.safetensors",
"model.layers.12.self_attn.v_proj.weight": "model-00010-of-00062.safetensors",
"model.layers.13.input_layernorm.weight": "model-00010-of-00062.safetensors",
"model.layers.13.mlp.down_proj.weight": "model-00010-of-00062.safetensors",
"model.layers.13.mlp.gate_proj.weight": "model-00011-of-00062.safetensors",
"model.layers.13.mlp.up_proj.weight": "model-00011-of-00062.safetensors",
"model.layers.13.post_attention_layernorm.weight": "model-00011-of-00062.safetensors",
"model.layers.13.self_attn.k_norm.weight": "model-00011-of-00062.safetensors",
"model.layers.13.self_attn.k_proj.weight": "model-00011-of-00062.safetensors",
"model.layers.13.self_attn.o_proj.weight": "model-00011-of-00062.safetensors",
"model.layers.13.self_attn.q_norm.weight": "model-00011-of-00062.safetensors",
"model.layers.13.self_attn.q_proj.weight": "model-00011-of-00062.safetensors",
"model.layers.13.self_attn.v_proj.weight": "model-00011-of-00062.safetensors",
"model.layers.14.input_layernorm.weight": "model-00011-of-00062.safetensors",
"model.layers.14.mlp.down_proj.weight": "model-00012-of-00062.safetensors",
"model.layers.14.mlp.gate_proj.weight": "model-00012-of-00062.safetensors",
"model.layers.14.mlp.up_proj.weight": "model-00013-of-00062.safetensors",
"model.layers.14.post_attention_layernorm.weight": "model-00013-of-00062.safetensors",
"model.layers.14.self_attn.k_norm.weight": "model-00013-of-00062.safetensors",
"model.layers.14.self_attn.k_proj.weight": "model-00013-of-00062.safetensors",
"model.layers.14.self_attn.o_proj.weight": "model-00013-of-00062.safetensors",
"model.layers.14.self_attn.q_norm.weight": "model-00013-of-00062.safetensors",
"model.layers.14.self_attn.q_proj.weight": "model-00013-of-00062.safetensors",
"model.layers.14.self_attn.v_proj.weight": "model-00013-of-00062.safetensors",
"model.layers.15.input_layernorm.weight": "model-00013-of-00062.safetensors",
"model.layers.15.mlp.down_proj.weight": "model-00013-of-00062.safetensors",
"model.layers.15.mlp.gate_proj.weight": "model-00014-of-00062.safetensors",
"model.layers.15.mlp.up_proj.weight": "model-00014-of-00062.safetensors",
"model.layers.15.post_attention_layernorm.weight": "model-00014-of-00062.safetensors",
"model.layers.15.self_attn.k_norm.weight": "model-00014-of-00062.safetensors",
"model.layers.15.self_attn.k_proj.weight": "model-00014-of-00062.safetensors",
"model.layers.15.self_attn.o_proj.weight": "model-00014-of-00062.safetensors",
"model.layers.15.self_attn.q_norm.weight": "model-00014-of-00062.safetensors",
"model.layers.15.self_attn.q_proj.weight": "model-00014-of-00062.safetensors",
"model.layers.15.self_attn.v_proj.weight": "model-00014-of-00062.safetensors",
"model.layers.16.input_layernorm.weight": "model-00014-of-00062.safetensors",
"model.layers.16.mlp.down_proj.weight": "model-00015-of-00062.safetensors",
"model.layers.16.mlp.gate_proj.weight": "model-00015-of-00062.safetensors",
"model.layers.16.mlp.up_proj.weight": "model-00016-of-00062.safetensors",
"model.layers.16.post_attention_layernorm.weight": "model-00016-of-00062.safetensors",
"model.layers.16.self_attn.k_norm.weight": "model-00016-of-00062.safetensors",
"model.layers.16.self_attn.k_proj.weight": "model-00016-of-00062.safetensors",
"model.layers.16.self_attn.o_proj.weight": "model-00016-of-00062.safetensors",
"model.layers.16.self_attn.q_norm.weight": "model-00016-of-00062.safetensors",
"model.layers.16.self_attn.q_proj.weight": "model-00016-of-00062.safetensors",
"model.layers.16.self_attn.v_proj.weight": "model-00016-of-00062.safetensors",
"model.layers.17.input_layernorm.weight": "model-00016-of-00062.safetensors",
"model.layers.17.mlp.down_proj.weight": "model-00016-of-00062.safetensors",
"model.layers.17.mlp.gate_proj.weight": "model-00017-of-00062.safetensors",
"model.layers.17.mlp.up_proj.weight": "model-00017-of-00062.safetensors",
"model.layers.17.post_attention_layernorm.weight": "model-00017-of-00062.safetensors",
"model.layers.17.self_attn.k_norm.weight": "model-00017-of-00062.safetensors",
"model.layers.17.self_attn.k_proj.weight": "model-00017-of-00062.safetensors",
"model.layers.17.self_attn.o_proj.weight": "model-00017-of-00062.safetensors",
"model.layers.17.self_attn.q_norm.weight": "model-00017-of-00062.safetensors",
"model.layers.17.self_attn.q_proj.weight": "model-00017-of-00062.safetensors",
"model.layers.17.self_attn.v_proj.weight": "model-00017-of-00062.safetensors",
"model.layers.18.input_layernorm.weight": "model-00017-of-00062.safetensors",
"model.layers.18.mlp.down_proj.weight": "model-00018-of-00062.safetensors",
"model.layers.18.mlp.gate_proj.weight": "model-00018-of-00062.safetensors",
"model.layers.18.mlp.up_proj.weight": "model-00019-of-00062.safetensors",
"model.layers.18.post_attention_layernorm.weight": "model-00019-of-00062.safetensors",
"model.layers.18.self_attn.k_norm.weight": "model-00019-of-00062.safetensors",
"model.layers.18.self_attn.k_proj.weight": "model-00019-of-00062.safetensors",
"model.layers.18.self_attn.o_proj.weight": "model-00019-of-00062.safetensors",
"model.layers.18.self_attn.q_norm.weight": "model-00019-of-00062.safetensors",
"model.layers.18.self_attn.q_proj.weight": "model-00019-of-00062.safetensors",
"model.layers.18.self_attn.v_proj.weight": "model-00019-of-00062.safetensors",
"model.layers.19.input_layernorm.weight": "model-00019-of-00062.safetensors",
"model.layers.19.mlp.down_proj.weight": "model-00019-of-00062.safetensors",
"model.layers.19.mlp.gate_proj.weight": "model-00020-of-00062.safetensors",
"model.layers.19.mlp.up_proj.weight": "model-00020-of-00062.safetensors",
"model.layers.19.post_attention_layernorm.weight": "model-00020-of-00062.safetensors",
"model.layers.19.self_attn.k_norm.weight": "model-00020-of-00062.safetensors",
"model.layers.19.self_attn.k_proj.weight": "model-00020-of-00062.safetensors",
"model.layers.19.self_attn.o_proj.weight": "model-00020-of-00062.safetensors",
"model.layers.19.self_attn.q_norm.weight": "model-00020-of-00062.safetensors",
"model.layers.19.self_attn.q_proj.weight": "model-00020-of-00062.safetensors",
"model.layers.19.self_attn.v_proj.weight": "model-00020-of-00062.safetensors",
"model.layers.2.input_layernorm.weight": "model-00020-of-00062.safetensors",
"model.layers.2.mlp.down_proj.weight": "model-00021-of-00062.safetensors",
"model.layers.2.mlp.gate_proj.weight": "model-00021-of-00062.safetensors",
"model.layers.2.mlp.up_proj.weight": "model-00022-of-00062.safetensors",
"model.layers.2.post_attention_layernorm.weight": "model-00022-of-00062.safetensors",
"model.layers.2.self_attn.k_norm.weight": "model-00022-of-00062.safetensors",
"model.layers.2.self_attn.k_proj.weight": "model-00022-of-00062.safetensors",
"model.layers.2.self_attn.o_proj.weight": "model-00022-of-00062.safetensors",
"model.layers.2.self_attn.q_norm.weight": "model-00022-of-00062.safetensors",
"model.layers.2.self_attn.q_proj.weight": "model-00022-of-00062.safetensors",
"model.layers.2.self_attn.v_proj.weight": "model-00022-of-00062.safetensors",
"model.layers.20.input_layernorm.weight": "model-00022-of-00062.safetensors",
"model.layers.20.mlp.down_proj.weight": "model-00022-of-00062.safetensors",
"model.layers.20.mlp.gate_proj.weight": "model-00023-of-00062.safetensors",
"model.layers.20.mlp.up_proj.weight": "model-00023-of-00062.safetensors",
"model.layers.20.post_attention_layernorm.weight": "model-00023-of-00062.safetensors",
"model.layers.20.self_attn.k_norm.weight": "model-00023-of-00062.safetensors",
"model.layers.20.self_attn.k_proj.weight": "model-00023-of-00062.safetensors",
"model.layers.20.self_attn.o_proj.weight": "model-00023-of-00062.safetensors",
"model.layers.20.self_attn.q_norm.weight": "model-00023-of-00062.safetensors",
"model.layers.20.self_attn.q_proj.weight": "model-00023-of-00062.safetensors",
"model.layers.20.self_attn.v_proj.weight": "model-00023-of-00062.safetensors",
"model.layers.21.input_layernorm.weight": "model-00023-of-00062.safetensors",
"model.layers.21.mlp.down_proj.weight": "model-00024-of-00062.safetensors",
"model.layers.21.mlp.gate_proj.weight": "model-00024-of-00062.safetensors",
"model.layers.21.mlp.up_proj.weight": "model-00025-of-00062.safetensors",
"model.layers.21.post_attention_layernorm.weight": "model-00025-of-00062.safetensors",
"model.layers.21.self_attn.k_norm.weight": "model-00025-of-00062.safetensors",
"model.layers.21.self_attn.k_proj.weight": "model-00025-of-00062.safetensors",
"model.layers.21.self_attn.o_proj.weight": "model-00025-of-00062.safetensors",
"model.layers.21.self_attn.q_norm.weight": "model-00025-of-00062.safetensors",
"model.layers.21.self_attn.q_proj.weight": "model-00025-of-00062.safetensors",
"model.layers.21.self_attn.v_proj.weight": "model-00025-of-00062.safetensors",
"model.layers.22.input_layernorm.weight": "model-00025-of-00062.safetensors",
"model.layers.22.mlp.down_proj.weight": "model-00025-of-00062.safetensors",
"model.layers.22.mlp.gate_proj.weight": "model-00026-of-00062.safetensors",
"model.layers.22.mlp.up_proj.weight": "model-00026-of-00062.safetensors",
"model.layers.22.post_attention_layernorm.weight": "model-00026-of-00062.safetensors",
"model.layers.22.self_attn.k_norm.weight": "model-00026-of-00062.safetensors",
"model.layers.22.self_attn.k_proj.weight": "model-00026-of-00062.safetensors",
"model.layers.22.self_attn.o_proj.weight": "model-00026-of-00062.safetensors",
"model.layers.22.self_attn.q_norm.weight": "model-00026-of-00062.safetensors",
"model.layers.22.self_attn.q_proj.weight": "model-00026-of-00062.safetensors",
"model.layers.22.self_attn.v_proj.weight": "model-00026-of-00062.safetensors",
"model.layers.23.input_layernorm.weight": "model-00026-of-00062.safetensors",
"model.layers.23.mlp.down_proj.weight": "model-00027-of-00062.safetensors",
"model.layers.23.mlp.gate_proj.weight": "model-00027-of-00062.safetensors",
"model.layers.23.mlp.up_proj.weight": "model-00028-of-00062.safetensors",
"model.layers.23.post_attention_layernorm.weight": "model-00028-of-00062.safetensors",
"model.layers.23.self_attn.k_norm.weight": "model-00028-of-00062.safetensors",
"model.layers.23.self_attn.k_proj.weight": "model-00028-of-00062.safetensors",
"model.layers.23.self_attn.o_proj.weight": "model-00028-of-00062.safetensors",
"model.layers.23.self_attn.q_norm.weight": "model-00028-of-00062.safetensors",
"model.layers.23.self_attn.q_proj.weight": "model-00028-of-00062.safetensors",
"model.layers.23.self_attn.v_proj.weight": "model-00028-of-00062.safetensors",
"model.layers.24.input_layernorm.weight": "model-00028-of-00062.safetensors",
"model.layers.24.mlp.down_proj.weight": "model-00028-of-00062.safetensors",
"model.layers.24.mlp.gate_proj.weight": "model-00029-of-00062.safetensors",
"model.layers.24.mlp.up_proj.weight": "model-00029-of-00062.safetensors",
"model.layers.24.post_attention_layernorm.weight": "model-00029-of-00062.safetensors",
"model.layers.24.self_attn.k_norm.weight": "model-00029-of-00062.safetensors",
"model.layers.24.self_attn.k_proj.weight": "model-00029-of-00062.safetensors",
"model.layers.24.self_attn.o_proj.weight": "model-00029-of-00062.safetensors",
"model.layers.24.self_attn.q_norm.weight": "model-00029-of-00062.safetensors",
"model.layers.24.self_attn.q_proj.weight": "model-00029-of-00062.safetensors",
"model.layers.24.self_attn.v_proj.weight": "model-00029-of-00062.safetensors",
"model.layers.25.input_layernorm.weight": "model-00029-of-00062.safetensors",
"model.layers.25.mlp.down_proj.weight": "model-00030-of-00062.safetensors",
"model.layers.25.mlp.gate_proj.weight": "model-00030-of-00062.safetensors",
"model.layers.25.mlp.up_proj.weight": "model-00031-of-00062.safetensors",
"model.layers.25.post_attention_layernorm.weight": "model-00031-of-00062.safetensors",
"model.layers.25.self_attn.k_norm.weight": "model-00031-of-00062.safetensors",
"model.layers.25.self_attn.k_proj.weight": "model-00031-of-00062.safetensors",
"model.layers.25.self_attn.o_proj.weight": "model-00031-of-00062.safetensors",
"model.layers.25.self_attn.q_norm.weight": "model-00031-of-00062.safetensors",
"model.layers.25.self_attn.q_proj.weight": "model-00031-of-00062.safetensors",
"model.layers.25.self_attn.v_proj.weight": "model-00031-of-00062.safetensors",
"model.layers.26.input_layernorm.weight": "model-00031-of-00062.safetensors",
"model.layers.26.mlp.down_proj.weight": "model-00031-of-00062.safetensors",
"model.layers.26.mlp.gate_proj.weight": "model-00032-of-00062.safetensors",
"model.layers.26.mlp.up_proj.weight": "model-00032-of-00062.safetensors",
"model.layers.26.post_attention_layernorm.weight": "model-00032-of-00062.safetensors",
"model.layers.26.self_attn.k_norm.weight": "model-00032-of-00062.safetensors",
"model.layers.26.self_attn.k_proj.weight": "model-00032-of-00062.safetensors",
"model.layers.26.self_attn.o_proj.weight": "model-00032-of-00062.safetensors",
"model.layers.26.self_attn.q_norm.weight": "model-00032-of-00062.safetensors",
"model.layers.26.self_attn.q_proj.weight": "model-00032-of-00062.safetensors",
"model.layers.26.self_attn.v_proj.weight": "model-00032-of-00062.safetensors",
"model.layers.27.input_layernorm.weight": "model-00032-of-00062.safetensors",
"model.layers.27.mlp.down_proj.weight": "model-00033-of-00062.safetensors",
"model.layers.27.mlp.gate_proj.weight": "model-00033-of-00062.safetensors",
"model.layers.27.mlp.up_proj.weight": "model-00034-of-00062.safetensors",
"model.layers.27.post_attention_layernorm.weight": "model-00034-of-00062.safetensors",
"model.layers.27.self_attn.k_norm.weight": "model-00034-of-00062.safetensors",
"model.layers.27.self_attn.k_proj.weight": "model-00034-of-00062.safetensors",
"model.layers.27.self_attn.o_proj.weight": "model-00034-of-00062.safetensors",
"model.layers.27.self_attn.q_norm.weight": "model-00034-of-00062.safetensors",
"model.layers.27.self_attn.q_proj.weight": "model-00034-of-00062.safetensors",
"model.layers.27.self_attn.v_proj.weight": "model-00034-of-00062.safetensors",
"model.layers.28.input_layernorm.weight": "model-00034-of-00062.safetensors",
"model.layers.28.mlp.down_proj.weight": "model-00034-of-00062.safetensors",
"model.layers.28.mlp.gate_proj.weight": "model-00035-of-00062.safetensors",
"model.layers.28.mlp.up_proj.weight": "model-00035-of-00062.safetensors",
"model.layers.28.post_attention_layernorm.weight": "model-00035-of-00062.safetensors",
"model.layers.28.self_attn.k_norm.weight": "model-00035-of-00062.safetensors",
"model.layers.28.self_attn.k_proj.weight": "model-00035-of-00062.safetensors",
"model.layers.28.self_attn.o_proj.weight": "model-00035-of-00062.safetensors",
"model.layers.28.self_attn.q_norm.weight": "model-00035-of-00062.safetensors",
"model.layers.28.self_attn.q_proj.weight": "model-00035-of-00062.safetensors",
"model.layers.28.self_attn.v_proj.weight": "model-00035-of-00062.safetensors",
"model.layers.29.input_layernorm.weight": "model-00035-of-00062.safetensors",
"model.layers.29.mlp.down_proj.weight": "model-00036-of-00062.safetensors",
"model.layers.29.mlp.gate_proj.weight": "model-00036-of-00062.safetensors",
"model.layers.29.mlp.up_proj.weight": "model-00037-of-00062.safetensors",
"model.layers.29.post_attention_layernorm.weight": "model-00037-of-00062.safetensors",
"model.layers.29.self_attn.k_norm.weight": "model-00037-of-00062.safetensors",
"model.layers.29.self_attn.k_proj.weight": "model-00037-of-00062.safetensors",
"model.layers.29.self_attn.o_proj.weight": "model-00037-of-00062.safetensors",
"model.layers.29.self_attn.q_norm.weight": "model-00037-of-00062.safetensors",
"model.layers.29.self_attn.q_proj.weight": "model-00037-of-00062.safetensors",
"model.layers.29.self_attn.v_proj.weight": "model-00037-of-00062.safetensors",
"model.layers.3.input_layernorm.weight": "model-00037-of-00062.safetensors",
"model.layers.3.mlp.down_proj.weight": "model-00037-of-00062.safetensors",
"model.layers.3.mlp.gate_proj.weight": "model-00038-of-00062.safetensors",
"model.layers.3.mlp.up_proj.weight": "model-00038-of-00062.safetensors",
"model.layers.3.post_attention_layernorm.weight": "model-00038-of-00062.safetensors",
"model.layers.3.self_attn.k_norm.weight": "model-00038-of-00062.safetensors",
"model.layers.3.self_attn.k_proj.weight": "model-00038-of-00062.safetensors",
"model.layers.3.self_attn.o_proj.weight": "model-00038-of-00062.safetensors",
"model.layers.3.self_attn.q_norm.weight": "model-00038-of-00062.safetensors",
"model.layers.3.self_attn.q_proj.weight": "model-00038-of-00062.safetensors",
"model.layers.3.self_attn.v_proj.weight": "model-00038-of-00062.safetensors",
"model.layers.30.input_layernorm.weight": "model-00038-of-00062.safetensors",
"model.layers.30.mlp.down_proj.weight": "model-00039-of-00062.safetensors",
"model.layers.30.mlp.gate_proj.weight": "model-00039-of-00062.safetensors",
"model.layers.30.mlp.up_proj.weight": "model-00040-of-00062.safetensors",
"model.layers.30.post_attention_layernorm.weight": "model-00040-of-00062.safetensors",
"model.layers.30.self_attn.k_norm.weight": "model-00040-of-00062.safetensors",
"model.layers.30.self_attn.k_proj.weight": "model-00040-of-00062.safetensors",
"model.layers.30.self_attn.o_proj.weight": "model-00040-of-00062.safetensors",
"model.layers.30.self_attn.q_norm.weight": "model-00040-of-00062.safetensors",
"model.layers.30.self_attn.q_proj.weight": "model-00040-of-00062.safetensors",
"model.layers.30.self_attn.v_proj.weight": "model-00040-of-00062.safetensors",
"model.layers.31.input_layernorm.weight": "model-00040-of-00062.safetensors",
"model.layers.31.mlp.down_proj.weight": "model-00040-of-00062.safetensors",
"model.layers.31.mlp.gate_proj.weight": "model-00041-of-00062.safetensors",
"model.layers.31.mlp.up_proj.weight": "model-00041-of-00062.safetensors",
"model.layers.31.post_attention_layernorm.weight": "model-00041-of-00062.safetensors",
"model.layers.31.self_attn.k_norm.weight": "model-00041-of-00062.safetensors",
"model.layers.31.self_attn.k_proj.weight": "model-00041-of-00062.safetensors",
"model.layers.31.self_attn.o_proj.weight": "model-00041-of-00062.safetensors",
"model.layers.31.self_attn.q_norm.weight": "model-00041-of-00062.safetensors",
"model.layers.31.self_attn.q_proj.weight": "model-00041-of-00062.safetensors",
"model.layers.31.self_attn.v_proj.weight": "model-00041-of-00062.safetensors",
"model.layers.32.input_layernorm.weight": "model-00041-of-00062.safetensors",
"model.layers.32.mlp.down_proj.weight": "model-00042-of-00062.safetensors",
"model.layers.32.mlp.gate_proj.weight": "model-00042-of-00062.safetensors",
"model.layers.32.mlp.up_proj.weight": "model-00043-of-00062.safetensors",
"model.layers.32.post_attention_layernorm.weight": "model-00043-of-00062.safetensors",
"model.layers.32.self_attn.k_norm.weight": "model-00043-of-00062.safetensors",
"model.layers.32.self_attn.k_proj.weight": "model-00043-of-00062.safetensors",
"model.layers.32.self_attn.o_proj.weight": "model-00043-of-00062.safetensors",
"model.layers.32.self_attn.q_norm.weight": "model-00043-of-00062.safetensors",
"model.layers.32.self_attn.q_proj.weight": "model-00043-of-00062.safetensors",
"model.layers.32.self_attn.v_proj.weight": "model-00043-of-00062.safetensors",
"model.layers.33.input_layernorm.weight": "model-00043-of-00062.safetensors",
"model.layers.33.mlp.down_proj.weight": "model-00043-of-00062.safetensors",
"model.layers.33.mlp.gate_proj.weight": "model-00044-of-00062.safetensors",
"model.layers.33.mlp.up_proj.weight": "model-00044-of-00062.safetensors",
"model.layers.33.post_attention_layernorm.weight": "model-00044-of-00062.safetensors",
"model.layers.33.self_attn.k_norm.weight": "model-00044-of-00062.safetensors",
"model.layers.33.self_attn.k_proj.weight": "model-00044-of-00062.safetensors",
"model.layers.33.self_attn.o_proj.weight": "model-00044-of-00062.safetensors",
"model.layers.33.self_attn.q_norm.weight": "model-00044-of-00062.safetensors",
"model.layers.33.self_attn.q_proj.weight": "model-00044-of-00062.safetensors",
"model.layers.33.self_attn.v_proj.weight": "model-00044-of-00062.safetensors",
"model.layers.34.input_layernorm.weight": "model-00044-of-00062.safetensors",
"model.layers.34.mlp.down_proj.weight": "model-00045-of-00062.safetensors",
"model.layers.34.mlp.gate_proj.weight": "model-00045-of-00062.safetensors",
"model.layers.34.mlp.up_proj.weight": "model-00046-of-00062.safetensors",
"model.layers.34.post_attention_layernorm.weight": "model-00046-of-00062.safetensors",
"model.layers.34.self_attn.k_norm.weight": "model-00046-of-00062.safetensors",
"model.layers.34.self_attn.k_proj.weight": "model-00046-of-00062.safetensors",
"model.layers.34.self_attn.o_proj.weight": "model-00046-of-00062.safetensors",
"model.layers.34.self_attn.q_norm.weight": "model-00046-of-00062.safetensors",
"model.layers.34.self_attn.q_proj.weight": "model-00046-of-00062.safetensors",
"model.layers.34.self_attn.v_proj.weight": "model-00046-of-00062.safetensors",
"model.layers.35.input_layernorm.weight": "model-00046-of-00062.safetensors",
"model.layers.35.mlp.down_proj.weight": "model-00046-of-00062.safetensors",
"model.layers.35.mlp.gate_proj.weight": "model-00047-of-00062.safetensors",
"model.layers.35.mlp.up_proj.weight": "model-00047-of-00062.safetensors",
"model.layers.35.post_attention_layernorm.weight": "model-00047-of-00062.safetensors",
"model.layers.35.self_attn.k_norm.weight": "model-00047-of-00062.safetensors",
"model.layers.35.self_attn.k_proj.weight": "model-00047-of-00062.safetensors",
"model.layers.35.self_attn.o_proj.weight": "model-00047-of-00062.safetensors",
"model.layers.35.self_attn.q_norm.weight": "model-00047-of-00062.safetensors",
"model.layers.35.self_attn.q_proj.weight": "model-00047-of-00062.safetensors",
"model.layers.35.self_attn.v_proj.weight": "model-00047-of-00062.safetensors",
"model.layers.36.input_layernorm.weight": "model-00047-of-00062.safetensors",
"model.layers.36.mlp.down_proj.weight": "model-00048-of-00062.safetensors",
"model.layers.36.mlp.gate_proj.weight": "model-00048-of-00062.safetensors",
"model.layers.36.mlp.up_proj.weight": "model-00049-of-00062.safetensors",
"model.layers.36.post_attention_layernorm.weight": "model-00049-of-00062.safetensors",
"model.layers.36.self_attn.k_norm.weight": "model-00049-of-00062.safetensors",
"model.layers.36.self_attn.k_proj.weight": "model-00049-of-00062.safetensors",
"model.layers.36.self_attn.o_proj.weight": "model-00049-of-00062.safetensors",
"model.layers.36.self_attn.q_norm.weight": "model-00049-of-00062.safetensors",
"model.layers.36.self_attn.q_proj.weight": "model-00049-of-00062.safetensors",
"model.layers.36.self_attn.v_proj.weight": "model-00049-of-00062.safetensors",
"model.layers.37.input_layernorm.weight": "model-00049-of-00062.safetensors",
"model.layers.37.mlp.down_proj.weight": "model-00049-of-00062.safetensors",
"model.layers.37.mlp.gate_proj.weight": "model-00050-of-00062.safetensors",
"model.layers.37.mlp.up_proj.weight": "model-00050-of-00062.safetensors",
"model.layers.37.post_attention_layernorm.weight": "model-00050-of-00062.safetensors",
"model.layers.37.self_attn.k_norm.weight": "model-00050-of-00062.safetensors",
"model.layers.37.self_attn.k_proj.weight": "model-00050-of-00062.safetensors",
"model.layers.37.self_attn.o_proj.weight": "model-00050-of-00062.safetensors",
"model.layers.37.self_attn.q_norm.weight": "model-00050-of-00062.safetensors",
"model.layers.37.self_attn.q_proj.weight": "model-00050-of-00062.safetensors",
"model.layers.37.self_attn.v_proj.weight": "model-00050-of-00062.safetensors",
"model.layers.38.input_layernorm.weight": "model-00050-of-00062.safetensors",
"model.layers.38.mlp.down_proj.weight": "model-00051-of-00062.safetensors",
"model.layers.38.mlp.gate_proj.weight": "model-00051-of-00062.safetensors",
"model.layers.38.mlp.up_proj.weight": "model-00052-of-00062.safetensors",
"model.layers.38.post_attention_layernorm.weight": "model-00052-of-00062.safetensors",
"model.layers.38.self_attn.k_norm.weight": "model-00052-of-00062.safetensors",
"model.layers.38.self_attn.k_proj.weight": "model-00052-of-00062.safetensors",
"model.layers.38.self_attn.o_proj.weight": "model-00052-of-00062.safetensors",
"model.layers.38.self_attn.q_norm.weight": "model-00052-of-00062.safetensors",
"model.layers.38.self_attn.q_proj.weight": "model-00052-of-00062.safetensors",
"model.layers.38.self_attn.v_proj.weight": "model-00052-of-00062.safetensors",
"model.layers.39.input_layernorm.weight": "model-00052-of-00062.safetensors",
"model.layers.39.mlp.down_proj.weight": "model-00052-of-00062.safetensors",
"model.layers.39.mlp.gate_proj.weight": "model-00053-of-00062.safetensors",
"model.layers.39.mlp.up_proj.weight": "model-00053-of-00062.safetensors",
"model.layers.39.post_attention_layernorm.weight": "model-00053-of-00062.safetensors",
"model.layers.39.self_attn.k_norm.weight": "model-00053-of-00062.safetensors",
"model.layers.39.self_attn.k_proj.weight": "model-00053-of-00062.safetensors",
"model.layers.39.self_attn.o_proj.weight": "model-00053-of-00062.safetensors",
"model.layers.39.self_attn.q_norm.weight": "model-00053-of-00062.safetensors",
"model.layers.39.self_attn.q_proj.weight": "model-00053-of-00062.safetensors",
"model.layers.39.self_attn.v_proj.weight": "model-00053-of-00062.safetensors",
"model.layers.4.input_layernorm.weight": "model-00053-of-00062.safetensors",
"model.layers.4.mlp.down_proj.weight": "model-00054-of-00062.safetensors",
"model.layers.4.mlp.gate_proj.weight": "model-00054-of-00062.safetensors",
"model.layers.4.mlp.up_proj.weight": "model-00055-of-00062.safetensors",
"model.layers.4.post_attention_layernorm.weight": "model-00055-of-00062.safetensors",
"model.layers.4.self_attn.k_norm.weight": "model-00055-of-00062.safetensors",
"model.layers.4.self_attn.k_proj.weight": "model-00055-of-00062.safetensors",
"model.layers.4.self_attn.o_proj.weight": "model-00055-of-00062.safetensors",
"model.layers.4.self_attn.q_norm.weight": "model-00055-of-00062.safetensors",
"model.layers.4.self_attn.q_proj.weight": "model-00055-of-00062.safetensors",
"model.layers.4.self_attn.v_proj.weight": "model-00055-of-00062.safetensors",
"model.layers.5.input_layernorm.weight": "model-00055-of-00062.safetensors",
"model.layers.5.mlp.down_proj.weight": "model-00055-of-00062.safetensors",
"model.layers.5.mlp.gate_proj.weight": "model-00056-of-00062.safetensors",
"model.layers.5.mlp.up_proj.weight": "model-00056-of-00062.safetensors",
"model.layers.5.post_attention_layernorm.weight": "model-00056-of-00062.safetensors",
"model.layers.5.self_attn.k_norm.weight": "model-00056-of-00062.safetensors",
"model.layers.5.self_attn.k_proj.weight": "model-00056-of-00062.safetensors",
"model.layers.5.self_attn.o_proj.weight": "model-00056-of-00062.safetensors",
"model.layers.5.self_attn.q_norm.weight": "model-00056-of-00062.safetensors",
"model.layers.5.self_attn.q_proj.weight": "model-00056-of-00062.safetensors",
"model.layers.5.self_attn.v_proj.weight": "model-00056-of-00062.safetensors",
"model.layers.6.input_layernorm.weight": "model-00056-of-00062.safetensors",
"model.layers.6.mlp.down_proj.weight": "model-00057-of-00062.safetensors",
"model.layers.6.mlp.gate_proj.weight": "model-00057-of-00062.safetensors",
"model.layers.6.mlp.up_proj.weight": "model-00058-of-00062.safetensors",
"model.layers.6.post_attention_layernorm.weight": "model-00058-of-00062.safetensors",
"model.layers.6.self_attn.k_norm.weight": "model-00058-of-00062.safetensors",
"model.layers.6.self_attn.k_proj.weight": "model-00058-of-00062.safetensors",
"model.layers.6.self_attn.o_proj.weight": "model-00058-of-00062.safetensors",
"model.layers.6.self_attn.q_norm.weight": "model-00058-of-00062.safetensors",
"model.layers.6.self_attn.q_proj.weight": "model-00058-of-00062.safetensors",
"model.layers.6.self_attn.v_proj.weight": "model-00058-of-00062.safetensors",
"model.layers.7.input_layernorm.weight": "model-00058-of-00062.safetensors",
"model.layers.7.mlp.down_proj.weight": "model-00058-of-00062.safetensors",
"model.layers.7.mlp.gate_proj.weight": "model-00059-of-00062.safetensors",
"model.layers.7.mlp.up_proj.weight": "model-00059-of-00062.safetensors",
"model.layers.7.post_attention_layernorm.weight": "model-00059-of-00062.safetensors",
"model.layers.7.self_attn.k_norm.weight": "model-00059-of-00062.safetensors",
"model.layers.7.self_attn.k_proj.weight": "model-00059-of-00062.safetensors",
"model.layers.7.self_attn.o_proj.weight": "model-00059-of-00062.safetensors",
"model.layers.7.self_attn.q_norm.weight": "model-00059-of-00062.safetensors",
"model.layers.7.self_attn.q_proj.weight": "model-00059-of-00062.safetensors",
"model.layers.7.self_attn.v_proj.weight": "model-00059-of-00062.safetensors",
"model.layers.8.input_layernorm.weight": "model-00059-of-00062.safetensors",
"model.layers.8.mlp.down_proj.weight": "model-00060-of-00062.safetensors",
"model.layers.8.mlp.gate_proj.weight": "model-00060-of-00062.safetensors",
"model.layers.8.mlp.up_proj.weight": "model-00061-of-00062.safetensors",
"model.layers.8.post_attention_layernorm.weight": "model-00061-of-00062.safetensors",
"model.layers.8.self_attn.k_norm.weight": "model-00061-of-00062.safetensors",
"model.layers.8.self_attn.k_proj.weight": "model-00061-of-00062.safetensors",
"model.layers.8.self_attn.o_proj.weight": "model-00061-of-00062.safetensors",
"model.layers.8.self_attn.q_norm.weight": "model-00061-of-00062.safetensors",
"model.layers.8.self_attn.q_proj.weight": "model-00061-of-00062.safetensors",
"model.layers.8.self_attn.v_proj.weight": "model-00061-of-00062.safetensors",
"model.layers.9.input_layernorm.weight": "model-00061-of-00062.safetensors",
"model.layers.9.mlp.down_proj.weight": "model-00061-of-00062.safetensors",
"model.layers.9.mlp.gate_proj.weight": "model-00062-of-00062.safetensors",
"model.layers.9.mlp.up_proj.weight": "model-00062-of-00062.safetensors",
"model.layers.9.post_attention_layernorm.weight": "model-00062-of-00062.safetensors",
"model.layers.9.self_attn.k_norm.weight": "model-00062-of-00062.safetensors",
"model.layers.9.self_attn.k_proj.weight": "model-00062-of-00062.safetensors",
"model.layers.9.self_attn.o_proj.weight": "model-00062-of-00062.safetensors",
"model.layers.9.self_attn.q_norm.weight": "model-00062-of-00062.safetensors",
"model.layers.9.self_attn.q_proj.weight": "model-00062-of-00062.safetensors",
"model.layers.9.self_attn.v_proj.weight": "model-00062-of-00062.safetensors",
"model.norm.weight": "model-00062-of-00062.safetensors"
}
}

3
tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bae3e39d56cfdb7b650cb318344d5c0f071d19fc9868ce086fef0cee78d5e7ff
size 11422749

18
tokenizer_config.json Normal file
View File

@@ -0,0 +1,18 @@
{
"add_prefix_space": false,
"backend": "tokenizers",
"bos_token": null,
"clean_up_tokenization_spaces": false,
"eos_token": "<|im_end|>",
"errors": "replace",
"is_local": true,
"max_length": 2048,
"model_max_length": 131072,
"pad_token": "<|endoftext|>",
"split_special_tokens": false,
"stride": 0,
"tokenizer_class": "Qwen2Tokenizer",
"truncation_side": "right",
"truncation_strategy": "longest_first",
"unk_token": null
}