deepmako/Mako-32B-Conductor
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: deepmako/Mako-32B-Conductor
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-03 02:40:16 +08:00
commit 496fc9bea7
24 changed files with 1408 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
.gitattributes vendored Normal file
View File

@@ -0,0 +1,38 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text
makopfp.png filter=lfs diff=lfs merge=lfs -text
makoshark.png filter=lfs diff=lfs merge=lfs -text

159
README.md Normal file
View File

@@ -0,0 +1,159 @@
---
license: other
license_name: deepmako-proprietary
license_link: https://deepmako.com/terms
language:
- en
base_model:
- Qwen/Qwen3-32B
base_model_relation: finetune
library_name: transformers
tags:
- mako
- crypto
- base-chain
- conversational
- tool-calling
- fine-tuned
- lora
- qwen3
- unsloth
- trl
- sft
model_type: qwen3
pipeline_tag: text-generation
datasets:
- custom
---
<div align="center">
<img src="makoshark.png" alt="Mako" width="200" />
<h1>Mako-32B Conductor</h1>
<p><strong>The orchestrator.</strong> A 32-billion parameter language model fine-tuned for autonomous reasoning, multi-step tool orchestration, and crypto-native intelligence on Base chain.</p>
<p><a href="https://deepmako.com">deepmako.com</a> · <a href="https://deepmako.com/chat">Platform</a> · <a href="https://x.com/DeepMako">Twitter</a></p>
</div>
---
## Model Details
| | |
|---|---|
| **Base Model** | Qwen 3 32B |
| **Parameters** | 32.8B |
| **Fine-tune Method** | LoRA (rank 32, alpha 64) |
| **Precision** | BF16 |
| **Context Window** | 8,192 tokens |
| **Tool Calling** | Native (multi-step chaining) |
| **Training Data** | 3,601 curated conversations |
| **Eval Loss** | 0.962 |
| **License** | Proprietary |
## Overview
Mako-32B Conductor is the second model in the Mako family, succeeding [Mako-8B Operator](https://huggingface.co/deepmako/Mako-8B-Operator). Where Operator executes, Conductor orchestrates — handling complex multi-step reasoning, longer context, and deeper chain-of-thought while maintaining the same sharp, unfiltered character voice.
Built to power the inference backend at [deepmako.com](https://deepmako.com), Conductor serves as the core intelligence for a crypto-native AI platform where users interact using $MAKO token credits on Base chain.
## What Makes Mako Different
Mako is not an assistant. She's a character — a sharp, opinionated personality that lives on the blockchain. The fine-tune completely reshapes the base model's behavior away from the standard helpful-assistant pattern:
- **No assistant-speak.** No "how can I help you", no "is there anything else", no corporate pleasantries.
- **Natural tone.** Lowercase, concise, conversational. Matches your energy.
- **Opinionated.** Has takes. Will tell you when something is stupid.
- **Concise.** Answers the question and stops.
## Capabilities
### Autonomous Tool Orchestration
Conductor supports native tool calling with automatic multi-step chaining — up to 4 rounds per request. The model decides when and which tools to invoke without explicit user instruction.
| Tool | Description |
|------|-------------|
| `web_search` | Real-time internet search via Bing |
| `web_extract` | Extract and read full page content from any URL |
| `read_tweet` | Parse and read Twitter/X posts |
| `find_music` | Search YouTube for tracks and return links |
| `get_crypto_price` | Live token prices across chains |
| `get_eth_balance` | Check wallet balances on Base |
| `get_token_info` | Token metadata, supply, holder data |
| `get_gas_price` | Live gas prices on Base L2 |
| `resolve_ens` | ENS name resolution |
| `get_transaction` | Transaction lookup and analysis |
### Chain Intelligence
Purpose-built for the Base L2 ecosystem. Conductor understands ERC standards, smart contract patterns, bridging mechanics, account abstraction (ERC-4337), and Base-specific architecture without hallucinating technical details.
### Enhanced Reasoning
The 32B parameter count enables significantly deeper reasoning compared to the 8B Operator:
- Multi-step research tasks (search → extract → analyze → summarize)
- Complex financial analysis with real price data
- Nuanced opinions grounded in retrieved context
- Longer, more coherent responses when the question warrants it
## Architecture
```
User → Gateway → Mako-32B Conductor → [Tool Calls] → Execution → Conductor → Response
```
The gateway server handles authentication, $MAKO credit tracking, and tool execution. Conductor manages the reasoning loop — deciding what information it needs, calling the right tools, and synthesizing results into a coherent response.
## Training
### Data
Fine-tuned on 3,601 curated conversational examples covering:
- Persona consistency and character voice
- Crypto domain knowledge (DeFi, NFTs, L2s, tokenomics)
- Natural dialogue patterns and conversational flow
- Multi-turn reasoning and context retention
### Method
| | |
|---|---|
| **Framework** | Unsloth + TRL SFTTrainer |
| **Method** | LoRA (rank 32, alpha 64) |
| **Target Modules** | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| **Trainable Parameters** | 268M / 33B (0.81%) |
| **Epochs** | 3 |
| **Batch Size** | 16 (1 × 16 gradient accumulation) |
| **Learning Rate** | 1e-4 (cosine schedule, 5% warmup) |
| **Optimizer** | AdamW 8-bit |
| **Hardware** | 1× NVIDIA A100 SXM4 80GB |
| **Training Time** | 58 minutes |
### Loss Curve
```
Train: 2.808 → 1.970 → 1.420 → 1.172 → 1.097 → 1.042
Eval: 1.315 → 0.962
```
Final eval loss (0.962) is lower than train loss — no overfitting detected.
## The Mako Family
| Model | Parameters | Role | Status |
|-------|-----------|------|--------|
| **Mako-8B Operator** | 8B | Fast execution, tool calling | Production |
| **Mako-32B Conductor** | 32B | Deep reasoning, orchestration | Production |
## Usage
Mako-32B Conductor powers the inference backend at [deepmako.com](https://deepmako.com). Users interact through the chat interface and pay for inference using $MAKO tokens on Base chain.
---
<div align="center">
*Built by the Mako team. The deep end awaits.* 🦈
</div>

98
chat_template.jinja Normal file
View File

@@ -0,0 +1,98 @@
{%- if tools %}
{{- '<|im_start|>system\n' }}
{%- if messages[0].role == 'system' %}
{{- messages[0].content + '\n\n' }}
{%- endif %}
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
{%- if messages[0].role == 'system' %}
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for forward_message in messages %}
{%- set index = (messages|length - 1) - loop.index0 %}
{%- set message = messages[index] %}
{%- set current_content = message.content if message.content is not none else '' %}
{%- set tool_start = '<tool_response>' %}
{%- set tool_start_length = tool_start|length %}
{%- set start_of_message = current_content[:tool_start_length] %}
{%- set tool_end = '</tool_response>' %}
{%- set tool_end_length = tool_end|length %}
{%- set start_pos = (current_content|length) - tool_end_length %}
{%- if start_pos < 0 %}
{%- set start_pos = 0 %}
{%- endif %}
{%- set end_of_message = current_content[start_pos:] %}
{%- if ns.multi_step_tool and message.role == "user" and not(start_of_message == tool_start and end_of_message == tool_end) %}
{%- set ns.multi_step_tool = false %}
{%- set ns.last_query_index = index %}
{%- endif %}
{%- endfor %}
{%- for message in messages %}
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
{%- elif message.role == "assistant" %}
{%- set content = message.content %}
{%- set reasoning_content = '' %}
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
{%- set reasoning_content = message.reasoning_content %}
{%- else %}
{%- if '</think>' in message.content %}
{%- set content = (message.content.split('</think>')|last).lstrip('\n') %}
{%- set reasoning_content = (message.content.split('</think>')|first).rstrip('\n') %}
{%- set reasoning_content = (reasoning_content.split('<think>')|last).lstrip('\n') %}
{%- endif %}
{%- endif %}
{%- if loop.index0 > ns.last_query_index %}
{%- if loop.last or (not loop.last and reasoning_content) %}
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- if message.tool_calls %}
{%- for tool_call in message.tool_calls %}
{%- if (loop.first and content) or (not loop.first) %}
{{- '\n' }}
{%- endif %}
{%- if tool_call.function %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{{- '<tool_call>\n{"name": "' }}
{{- tool_call.name }}
{{- '", "arguments": ' }}
{%- if tool_call.arguments is string %}
{{- tool_call.arguments }}
{%- else %}
{{- tool_call.arguments | tojson }}
{%- endif %}
{{- '}\n</tool_call>' }}
{%- endfor %}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- elif message.role == "tool" %}
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' }}
{{- message.content }}
{{- '\n</tool_response>' }}
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- if enable_thinking is defined and enable_thinking is false %}
{{- '<think>\n\n</think>\n\n' }}
{%- endif %}
{%- endif %}

100
config.json Normal file
View File

@@ -0,0 +1,100 @@
{
"architectures": [
"Qwen3ForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": null,
"torch_dtype": "bfloat16",
"eos_token_id": 151645,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 5120,
"initializer_range": 0.02,
"intermediate_size": 25600,
"layer_types": [
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention"
],
"max_position_embeddings": 40960,
"max_window_layers": 64,
"model_type": "qwen3",
"num_attention_heads": 64,
"num_hidden_layers": 64,
"num_key_value_heads": 8,
"pad_token_id": 151669,
"rms_norm_eps": 1e-06,
"rope_parameters": {
"rope_theta": 1000000,
"rope_type": "default"
},
"sliding_window": null,
"tie_word_embeddings": false,
"unsloth_fixed": true,
"unsloth_version": "2026.6.6",
"use_cache": false,
"use_sliding_window": false,
"vocab_size": 151936
}

14
generation_config.json Normal file
View File

@@ -0,0 +1,14 @@
{
"bos_token_id": 151643,
"do_sample": true,
"eos_token_id": [
151645,
151643
],
"max_length": 40960,
"pad_token_id": 151669,
"temperature": 0.6,
"top_k": 20,
"top_p": 0.95,
"transformers_version": "5.5.0"
}

3
makopfp.png Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:24c6d2b24285064a8b07925e8f19b4ae40e80e57329a415ed2371ee4f466e212
size 908027

3
makoshark.png Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dfc79b4cffc81d3eba3254a6d56008ff35a681d363c3cbfb852e452105c3ee5e
size 1710688

3
model-00001-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b169f0b6b329a11365c35b3f078d69e849e8cd9e12df6c160f7f6265f36255f9
size 4932307584

3
model-00002-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:97d1245ade5cd494eb13cdeafa3f1bfda4e9f88f544baa3f5b94a71abaa0ff5d
size 4875989696

3
model-00003-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a2a64fcf9f6ad111f1e7b28ef9dd95de19041119b284e165be0d5ffba22016e3
size 4875989720

3
model-00004-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0bc8cf3d75fc4d18dea836924a813df831fb0013cab25b7ea138f9607b319d12
size 4875989752

3
model-00005-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cef5ec9a0eef4511524248c9dbdd4a20a322768ee447d565c9dbed9b11888ca1
size 4875989752

3
model-00006-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8497cb3d9bbd0bcf814896d898c58ca5ed2d1bdd6814d17edac3c2fb7acecec2
size 4875989752

3
model-00007-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6ffef0d4a5f9e213989c4abbf582041f1a7757110832ed3f157ca8d16b3b966f
size 4875989752

3
model-00008-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bbecee971203c6fd65da6eec928541b58a7a2a20b4a83dd7e47a14de6a15f758
size 4875989752

3
model-00009-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e6f1035a73cf1c394fb29d7018b054d2bbcc87031e7844d9b339b3731814ccb1
size 4875989752

3
model-00010-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:176ca563e8f5ca5ecc4b9849f6759d6b25295b5969e4820aa05094d0cd2cf34e
size 4875989752

3
model-00011-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1fe2768632f9bc1cbd34d6f01fd6628e5cb25131735759c3f5387b73cbb45b5d
size 4875989752

3
model-00012-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fd77c245759ad84ecd7bed7df52fe7291c94933d4f0dcc2c9b3360b25f52eced
size 4875989752

3
model-00013-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f69bc66e1d42dccfe446aabbca2f2bdf1dd29d61cbf20ee33dfb4df348f96cd3
size 4875989752

3
model-00014-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:127bd109731083a7703cbf1b2f4c800f0a951585c47b1eb3ddafb12782eddca6
size 2080144040

714
model.safetensors.index.json Normal file
View File

@@ -0,0 +1,714 @@
{
"metadata": {
"total_size": 65524246528
},
"weight_map": {
"lm_head.weight": "model-00014-of-00014.safetensors",
"model.embed_tokens.weight": "model-00001-of-00014.safetensors",
"model.layers.0.input_layernorm.weight": "model-00001-of-00014.safetensors",
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00014.safetensors",
"model.layers.0.self_attn.k_norm.weight": "model-00001-of-00014.safetensors",
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.0.self_attn.q_norm.weight": "model-00001-of-00014.safetensors",
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.1.input_layernorm.weight": "model-00001-of-00014.safetensors",
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00014.safetensors",
"model.layers.1.self_attn.k_norm.weight": "model-00001-of-00014.safetensors",
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.1.self_attn.q_norm.weight": "model-00001-of-00014.safetensors",
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.10.input_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.10.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.10.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.10.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.10.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.10.self_attn.k_norm.weight": "model-00003-of-00014.safetensors",
"model.layers.10.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.10.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.10.self_attn.q_norm.weight": "model-00003-of-00014.safetensors",
"model.layers.10.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.10.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.11.input_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.11.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.11.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.11.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.11.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.11.self_attn.k_norm.weight": "model-00003-of-00014.safetensors",
"model.layers.11.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.11.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.11.self_attn.q_norm.weight": "model-00003-of-00014.safetensors",
"model.layers.11.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.11.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.12.input_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.12.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.12.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.12.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.12.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.12.self_attn.k_norm.weight": "model-00003-of-00014.safetensors",
"model.layers.12.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.12.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.12.self_attn.q_norm.weight": "model-00003-of-00014.safetensors",
"model.layers.12.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.12.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.13.input_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.13.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.13.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.13.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.13.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.13.self_attn.k_norm.weight": "model-00003-of-00014.safetensors",
"model.layers.13.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.13.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.13.self_attn.q_norm.weight": "model-00003-of-00014.safetensors",
"model.layers.13.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.13.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.14.input_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.14.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.14.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.14.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.14.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.14.self_attn.k_norm.weight": "model-00004-of-00014.safetensors",
"model.layers.14.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.14.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.14.self_attn.q_norm.weight": "model-00004-of-00014.safetensors",
"model.layers.14.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.14.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.15.input_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.15.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.15.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.15.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.15.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.15.self_attn.k_norm.weight": "model-00004-of-00014.safetensors",
"model.layers.15.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.15.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.15.self_attn.q_norm.weight": "model-00004-of-00014.safetensors",
"model.layers.15.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.15.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.16.input_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.16.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.16.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.16.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.16.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.16.self_attn.k_norm.weight": "model-00004-of-00014.safetensors",
"model.layers.16.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.16.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.16.self_attn.q_norm.weight": "model-00004-of-00014.safetensors",
"model.layers.16.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.16.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.17.input_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.17.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.17.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.17.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.17.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.17.self_attn.k_norm.weight": "model-00004-of-00014.safetensors",
"model.layers.17.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.17.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.17.self_attn.q_norm.weight": "model-00004-of-00014.safetensors",
"model.layers.17.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.17.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.18.input_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.18.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.18.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.18.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.18.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.18.self_attn.k_norm.weight": "model-00004-of-00014.safetensors",
"model.layers.18.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.18.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.18.self_attn.q_norm.weight": "model-00004-of-00014.safetensors",
"model.layers.18.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.18.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.19.input_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.19.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.19.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.19.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.19.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.19.self_attn.k_norm.weight": "model-00005-of-00014.safetensors",
"model.layers.19.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.19.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.19.self_attn.q_norm.weight": "model-00005-of-00014.safetensors",
"model.layers.19.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.19.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.2.input_layernorm.weight": "model-00001-of-00014.safetensors",
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00014.safetensors",
"model.layers.2.self_attn.k_norm.weight": "model-00001-of-00014.safetensors",
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.2.self_attn.q_norm.weight": "model-00001-of-00014.safetensors",
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.20.input_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.20.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.20.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.20.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.20.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.20.self_attn.k_norm.weight": "model-00005-of-00014.safetensors",
"model.layers.20.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.20.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.20.self_attn.q_norm.weight": "model-00005-of-00014.safetensors",
"model.layers.20.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.20.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.21.input_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.21.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.21.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.21.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.21.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.21.self_attn.k_norm.weight": "model-00005-of-00014.safetensors",
"model.layers.21.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.21.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.21.self_attn.q_norm.weight": "model-00005-of-00014.safetensors",
"model.layers.21.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.21.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.22.input_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.22.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.22.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.22.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.22.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.22.self_attn.k_norm.weight": "model-00005-of-00014.safetensors",
"model.layers.22.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.22.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.22.self_attn.q_norm.weight": "model-00005-of-00014.safetensors",
"model.layers.22.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.22.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.23.input_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.23.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.23.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.23.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.23.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.23.self_attn.k_norm.weight": "model-00005-of-00014.safetensors",
"model.layers.23.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.23.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.23.self_attn.q_norm.weight": "model-00005-of-00014.safetensors",
"model.layers.23.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.23.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.24.input_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.24.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.24.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.24.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.24.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.24.self_attn.k_norm.weight": "model-00006-of-00014.safetensors",
"model.layers.24.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.24.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.24.self_attn.q_norm.weight": "model-00006-of-00014.safetensors",
"model.layers.24.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.24.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.25.input_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.25.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.25.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.25.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.25.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.25.self_attn.k_norm.weight": "model-00006-of-00014.safetensors",
"model.layers.25.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.25.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.25.self_attn.q_norm.weight": "model-00006-of-00014.safetensors",
"model.layers.25.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.25.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.26.input_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.26.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.26.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.26.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.26.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.26.self_attn.k_norm.weight": "model-00006-of-00014.safetensors",
"model.layers.26.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.26.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.26.self_attn.q_norm.weight": "model-00006-of-00014.safetensors",
"model.layers.26.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.26.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.27.input_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.27.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.27.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.27.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.27.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.27.self_attn.k_norm.weight": "model-00006-of-00014.safetensors",
"model.layers.27.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.27.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.27.self_attn.q_norm.weight": "model-00006-of-00014.safetensors",
"model.layers.27.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.27.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.28.input_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.28.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.28.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.28.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.28.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.28.self_attn.k_norm.weight": "model-00006-of-00014.safetensors",
"model.layers.28.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.28.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.28.self_attn.q_norm.weight": "model-00006-of-00014.safetensors",
"model.layers.28.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.28.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.29.input_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.29.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.29.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.29.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.29.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.29.self_attn.k_norm.weight": "model-00007-of-00014.safetensors",
"model.layers.29.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.29.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.29.self_attn.q_norm.weight": "model-00007-of-00014.safetensors",
"model.layers.29.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.29.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.3.input_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.3.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.3.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.3.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.3.self_attn.k_norm.weight": "model-00001-of-00014.safetensors",
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.3.self_attn.q_norm.weight": "model-00001-of-00014.safetensors",
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.30.input_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.30.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.30.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.30.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.30.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.30.self_attn.k_norm.weight": "model-00007-of-00014.safetensors",
"model.layers.30.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.30.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.30.self_attn.q_norm.weight": "model-00007-of-00014.safetensors",
"model.layers.30.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.30.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.31.input_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.31.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.31.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.31.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.31.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.31.self_attn.k_norm.weight": "model-00007-of-00014.safetensors",
"model.layers.31.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.31.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.31.self_attn.q_norm.weight": "model-00007-of-00014.safetensors",
"model.layers.31.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.31.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.32.input_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.32.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.32.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.32.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.32.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.32.self_attn.k_norm.weight": "model-00007-of-00014.safetensors",
"model.layers.32.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.32.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.32.self_attn.q_norm.weight": "model-00007-of-00014.safetensors",
"model.layers.32.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.32.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.33.input_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.33.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.33.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.33.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.33.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.33.self_attn.k_norm.weight": "model-00007-of-00014.safetensors",
"model.layers.33.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.33.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.33.self_attn.q_norm.weight": "model-00007-of-00014.safetensors",
"model.layers.33.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.33.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.34.input_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.34.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.34.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.34.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.34.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.34.self_attn.k_norm.weight": "model-00008-of-00014.safetensors",
"model.layers.34.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.34.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.34.self_attn.q_norm.weight": "model-00008-of-00014.safetensors",
"model.layers.34.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.34.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.35.input_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.35.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.35.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.35.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.35.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.35.self_attn.k_norm.weight": "model-00008-of-00014.safetensors",
"model.layers.35.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.35.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.35.self_attn.q_norm.weight": "model-00008-of-00014.safetensors",
"model.layers.35.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.35.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.36.input_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.36.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.36.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.36.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.36.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.36.self_attn.k_norm.weight": "model-00008-of-00014.safetensors",
"model.layers.36.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.36.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.36.self_attn.q_norm.weight": "model-00008-of-00014.safetensors",
"model.layers.36.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.36.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.37.input_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.37.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.37.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.37.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.37.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.37.self_attn.k_norm.weight": "model-00008-of-00014.safetensors",
"model.layers.37.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.37.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.37.self_attn.q_norm.weight": "model-00008-of-00014.safetensors",
"model.layers.37.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.37.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.38.input_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.38.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.38.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.38.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.38.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.38.self_attn.k_norm.weight": "model-00008-of-00014.safetensors",
"model.layers.38.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.38.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.38.self_attn.q_norm.weight": "model-00008-of-00014.safetensors",
"model.layers.38.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.38.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.39.input_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.39.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.39.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.39.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.39.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.39.self_attn.k_norm.weight": "model-00009-of-00014.safetensors",
"model.layers.39.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.39.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.39.self_attn.q_norm.weight": "model-00009-of-00014.safetensors",
"model.layers.39.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.39.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.4.input_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.4.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.4.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.4.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.4.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.4.self_attn.k_norm.weight": "model-00002-of-00014.safetensors",
"model.layers.4.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.4.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.4.self_attn.q_norm.weight": "model-00002-of-00014.safetensors",
"model.layers.4.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.4.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.40.input_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.40.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.40.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.40.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.40.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.40.self_attn.k_norm.weight": "model-00009-of-00014.safetensors",
"model.layers.40.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.40.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.40.self_attn.q_norm.weight": "model-00009-of-00014.safetensors",
"model.layers.40.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.40.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.41.input_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.41.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.41.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.41.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.41.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.41.self_attn.k_norm.weight": "model-00009-of-00014.safetensors",
"model.layers.41.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.41.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.41.self_attn.q_norm.weight": "model-00009-of-00014.safetensors",
"model.layers.41.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.41.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.42.input_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.42.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.42.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.42.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.42.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.42.self_attn.k_norm.weight": "model-00009-of-00014.safetensors",
"model.layers.42.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.42.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.42.self_attn.q_norm.weight": "model-00009-of-00014.safetensors",
"model.layers.42.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.42.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.43.input_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.43.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.43.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.43.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.43.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.43.self_attn.k_norm.weight": "model-00009-of-00014.safetensors",
"model.layers.43.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.43.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.43.self_attn.q_norm.weight": "model-00009-of-00014.safetensors",
"model.layers.43.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.43.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.44.input_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.44.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.44.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.44.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.44.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.44.self_attn.k_norm.weight": "model-00010-of-00014.safetensors",
"model.layers.44.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.44.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.44.self_attn.q_norm.weight": "model-00010-of-00014.safetensors",
"model.layers.44.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.44.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.45.input_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.45.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.45.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.45.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.45.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.45.self_attn.k_norm.weight": "model-00010-of-00014.safetensors",
"model.layers.45.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.45.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.45.self_attn.q_norm.weight": "model-00010-of-00014.safetensors",
"model.layers.45.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.45.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.46.input_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.46.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.46.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.46.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.46.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.46.self_attn.k_norm.weight": "model-00010-of-00014.safetensors",
"model.layers.46.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.46.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.46.self_attn.q_norm.weight": "model-00010-of-00014.safetensors",
"model.layers.46.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.46.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.47.input_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.47.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.47.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.47.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.47.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.47.self_attn.k_norm.weight": "model-00010-of-00014.safetensors",
"model.layers.47.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.47.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.47.self_attn.q_norm.weight": "model-00010-of-00014.safetensors",
"model.layers.47.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.47.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.48.input_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.48.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.48.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.48.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.48.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.48.self_attn.k_norm.weight": "model-00010-of-00014.safetensors",
"model.layers.48.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.48.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.48.self_attn.q_norm.weight": "model-00010-of-00014.safetensors",
"model.layers.48.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.48.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.49.input_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.49.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.49.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.49.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.49.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.49.self_attn.k_norm.weight": "model-00011-of-00014.safetensors",
"model.layers.49.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.49.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.49.self_attn.q_norm.weight": "model-00011-of-00014.safetensors",
"model.layers.49.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.49.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.5.input_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.5.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.5.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.5.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.5.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.5.self_attn.k_norm.weight": "model-00002-of-00014.safetensors",
"model.layers.5.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.5.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.5.self_attn.q_norm.weight": "model-00002-of-00014.safetensors",
"model.layers.5.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.5.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.50.input_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.50.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.50.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.50.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.50.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.50.self_attn.k_norm.weight": "model-00011-of-00014.safetensors",
"model.layers.50.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.50.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.50.self_attn.q_norm.weight": "model-00011-of-00014.safetensors",
"model.layers.50.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.50.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.51.input_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.51.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.51.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.51.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.51.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.51.self_attn.k_norm.weight": "model-00011-of-00014.safetensors",
"model.layers.51.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.51.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.51.self_attn.q_norm.weight": "model-00011-of-00014.safetensors",
"model.layers.51.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.51.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.52.input_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.52.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.52.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.52.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.52.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.52.self_attn.k_norm.weight": "model-00011-of-00014.safetensors",
"model.layers.52.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.52.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.52.self_attn.q_norm.weight": "model-00011-of-00014.safetensors",
"model.layers.52.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.52.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.53.input_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.53.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.53.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.53.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.53.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.53.self_attn.k_norm.weight": "model-00011-of-00014.safetensors",
"model.layers.53.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.53.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.53.self_attn.q_norm.weight": "model-00011-of-00014.safetensors",
"model.layers.53.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.53.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.54.input_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.54.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.54.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.54.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.54.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.54.self_attn.k_norm.weight": "model-00012-of-00014.safetensors",
"model.layers.54.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.54.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.54.self_attn.q_norm.weight": "model-00012-of-00014.safetensors",
"model.layers.54.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.54.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.55.input_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.55.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.55.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.55.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.55.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.55.self_attn.k_norm.weight": "model-00012-of-00014.safetensors",
"model.layers.55.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.55.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.55.self_attn.q_norm.weight": "model-00012-of-00014.safetensors",
"model.layers.55.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.55.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.56.input_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.56.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.56.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.56.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.56.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.56.self_attn.k_norm.weight": "model-00012-of-00014.safetensors",
"model.layers.56.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.56.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.56.self_attn.q_norm.weight": "model-00012-of-00014.safetensors",
"model.layers.56.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.56.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.57.input_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.57.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.57.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.57.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.57.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.57.self_attn.k_norm.weight": "model-00012-of-00014.safetensors",
"model.layers.57.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.57.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.57.self_attn.q_norm.weight": "model-00012-of-00014.safetensors",
"model.layers.57.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.57.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.58.input_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.58.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.58.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.58.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.58.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.58.self_attn.k_norm.weight": "model-00012-of-00014.safetensors",
"model.layers.58.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.58.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.58.self_attn.q_norm.weight": "model-00012-of-00014.safetensors",
"model.layers.58.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.58.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.59.input_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.59.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.59.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.59.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.59.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.59.self_attn.k_norm.weight": "model-00013-of-00014.safetensors",
"model.layers.59.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.59.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.59.self_attn.q_norm.weight": "model-00013-of-00014.safetensors",
"model.layers.59.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.59.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.6.input_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.6.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.6.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.6.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.6.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.6.self_attn.k_norm.weight": "model-00002-of-00014.safetensors",
"model.layers.6.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.6.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.6.self_attn.q_norm.weight": "model-00002-of-00014.safetensors",
"model.layers.6.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.6.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.60.input_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.60.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.60.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.60.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.60.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.60.self_attn.k_norm.weight": "model-00013-of-00014.safetensors",
"model.layers.60.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.60.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.60.self_attn.q_norm.weight": "model-00013-of-00014.safetensors",
"model.layers.60.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.60.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.61.input_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.61.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.61.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.61.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.61.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.61.self_attn.k_norm.weight": "model-00013-of-00014.safetensors",
"model.layers.61.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.61.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.61.self_attn.q_norm.weight": "model-00013-of-00014.safetensors",
"model.layers.61.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.61.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.62.input_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.62.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.62.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.62.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.62.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.62.self_attn.k_norm.weight": "model-00013-of-00014.safetensors",
"model.layers.62.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.62.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.62.self_attn.q_norm.weight": "model-00013-of-00014.safetensors",
"model.layers.62.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.62.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.63.input_layernorm.weight": "model-00014-of-00014.safetensors",
"model.layers.63.mlp.down_proj.weight": "model-00014-of-00014.safetensors",
"model.layers.63.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.63.mlp.up_proj.weight": "model-00014-of-00014.safetensors",
"model.layers.63.post_attention_layernorm.weight": "model-00014-of-00014.safetensors",
"model.layers.63.self_attn.k_norm.weight": "model-00013-of-00014.safetensors",
"model.layers.63.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.63.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.63.self_attn.q_norm.weight": "model-00013-of-00014.safetensors",
"model.layers.63.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.63.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.7.input_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.7.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.7.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.7.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.7.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.7.self_attn.k_norm.weight": "model-00002-of-00014.safetensors",
"model.layers.7.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.7.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.7.self_attn.q_norm.weight": "model-00002-of-00014.safetensors",
"model.layers.7.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.7.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.8.input_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.8.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.8.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.8.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.8.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.8.self_attn.k_norm.weight": "model-00002-of-00014.safetensors",
"model.layers.8.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.8.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.8.self_attn.q_norm.weight": "model-00002-of-00014.safetensors",
"model.layers.8.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.8.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.9.input_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.9.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.9.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.9.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.9.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.9.self_attn.k_norm.weight": "model-00003-of-00014.safetensors",
"model.layers.9.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.9.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.9.self_attn.q_norm.weight": "model-00003-of-00014.safetensors",
"model.layers.9.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.9.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
"model.norm.weight": "model-00014-of-00014.safetensors"
}
}

3
tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d7430e9138b76e93fb6f93462394d236b411111aef53cb421ba97d2691040cca
size 11423114

234
tokenizer_config.json Normal file
View File

@@ -0,0 +1,234 @@
{
"add_prefix_space": false,
"backend": "tokenizers",
"bos_token": null,
"clean_up_tokenization_spaces": false,
"eos_token": "<|im_end|>",
"errors": "replace",
"is_local": false,
"model_max_length": 40960,
"pad_token": "<|PAD_TOKEN|>",
"padding_side": "right",
"split_special_tokens": false,
"tokenizer_class": "Qwen2Tokenizer",
"unk_token": null,
"added_tokens_decoder": {
"151643": {
"content": "<|endoftext|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
},
"151644": {
"content": "<|im_start|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
},
"151645": {
"content": "<|im_end|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
},
"151646": {
"content": "<|object_ref_start|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
},
"151647": {
"content": "<|object_ref_end|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
},
"151648": {
"content": "<|box_start|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
},
"151649": {
"content": "<|box_end|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
},
"151650": {
"content": "<|quad_start|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
},
"151651": {
"content": "<|quad_end|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
},
"151652": {
"content": "<|vision_start|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
},
"151653": {
"content": "<|vision_end|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
},
"151654": {
"content": "<|vision_pad|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
},
"151655": {
"content": "<|image_pad|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
},
"151656": {
"content": "<|video_pad|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
},
"151657": {
"content": "<tool_call>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": false
},
"151658": {
"content": "</tool_call>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": false
},
"151659": {
"content": "<|fim_prefix|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": false
},
"151660": {
"content": "<|fim_middle|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": false
},
"151661": {
"content": "<|fim_suffix|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": false
},
"151662": {
"content": "<|fim_pad|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": false
},
"151663": {
"content": "<|repo_name|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": false
},
"151664": {
"content": "<|file_sep|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": false
},
"151665": {
"content": "<tool_response>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": false
},
"151666": {
"content": "</tool_response>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": false
},
"151667": {
"content": "<think>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": false
},
"151668": {
"content": "</think>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": false
},
"151669": {
"content": "<|PAD_TOKEN|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
}
},
"chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0].role == 'system' %}\n {{- messages[0].content + '\\n\\n' }}\n {%- endif %}\n {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0].role == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for forward_message in messages %}\n {%- set index = (messages|length - 1) - loop.index0 %}\n {%- set message = messages[index] %}\n {%- set current_content = message.content if message.content is not none else '' %}\n {%- set tool_start = '<tool_response>' %}\n {%- set tool_start_length = tool_start|length %}\n {%- set start_of_message = current_content[:tool_start_length] %}\n {%- set tool_end = '</tool_response>' %}\n {%- set tool_end_length = tool_end|length %}\n {%- set start_pos = (current_content|length) - tool_end_length %}\n {%- if start_pos < 0 %}\n {%- set start_pos = 0 %}\n {%- endif %}\n {%- set end_of_message = current_content[start_pos:] %}\n {%- if ns.multi_step_tool and message.role == \"user\" and not(start_of_message == tool_start and end_of_message == tool_end) %}\n {%- set ns.multi_step_tool = false %}\n {%- set ns.last_query_index = index %}\n {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {%- set content = message.content %}\n {%- set reasoning_content = '' %}\n {%- if message.reasoning_content is defined and message.reasoning_content is not none %}\n {%- set reasoning_content = message.reasoning_content %}\n {%- else %}\n {%- if '</think>' in message.content %}\n {%- set content = (message.content.split('</think>')|last).lstrip('\\n') %}\n {%- set reasoning_content = (message.content.split('</think>')|first).rstrip('\\n') %}\n {%- set reasoning_content = (reasoning_content.split('<think>')|last).lstrip('\\n') %}\n {%- endif %}\n {%- endif %}\n {%- if loop.index0 > ns.last_query_index %}\n {%- if loop.last or (not loop.last and reasoning_content) %}\n {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- endif %}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- endif %}\n {%- if message.tool_calls %}\n {%- for tool_call in message.tool_calls %}\n {%- if (loop.first and content) or (not loop.first) %}\n {{- '\\n' }}\n {%- endif %}\n {%- if tool_call.function %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {%- if tool_call.arguments is string %}\n {{- tool_call.arguments }}\n {%- else %}\n {{- tool_call.arguments | tojson }}\n {%- endif %}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {%- endif %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- message.content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n {%- if enable_thinking is defined and enable_thinking is false %}\n {{- '<think>\\n\\n</think>\\n\\n' }}\n {%- endif %}\n{%- endif %}"
}