初始化项目，由ModelHub XC社区提供模型
Model: DataPilot/ArrowCanaria-Llama-8B-SFT-v0.1 Source: Original Platform
This commit is contained in:
ModelHub XC
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
246
README.md
Normal file
246
README.md
Normal file
@@ -0,0 +1,246 @@
|
||||
---
|
||||
language:
|
||||
- ja
|
||||
- en
|
||||
license:
|
||||
- llama3.3
|
||||
- gemma
|
||||
library_name: transformers
|
||||
base_model:
|
||||
- tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.5
|
||||
tags:
|
||||
- llama3
|
||||
- sft
|
||||
- japanese
|
||||
- aituber
|
||||
- roleplay
|
||||
- chat
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# ArrowCanaria-Llama-8B-SFT-v0.1
|
||||
|
||||
## モデル概要
|
||||
|
||||
**ArrowCanaria-Llama-8B-SFT-v0.1** は、AItuber（AI VTuber）向けに雑談性能を重視して設計された日本語特化の8Bパラメータ言語モデルです。
|
||||
|
||||
多くの日本語LLMは翻訳調の硬い文体や、定型的なテンプレート応答に陥りがちです。本モデルは「AItuberとして視聴者と自然に雑談できるモデル」を目指し、**雑談・ロールプレイ（RP）・推論・日本語の自然さ**を高い水準で獲得することを目標に開発されました。テンプレート的・機械的な応答ではなく、**自然な日本語で人間らしく回答できること**が最大の特徴です。
|
||||
|
||||
本モデルは `tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.5` をベースに、高品質な日本語合成小説データセットによる継続事前学習（CPT）でドメイン知識を拡張した後、Chat Vectorマージで対話能力を復元し、さらに17.5万件超の合成データセットによる3フェーズのカリキュラムSFTを実施して構築されています。独自のデータ生成フレームワーク **SDG_LOOM** によって作成された高品質なデータと、EQ（感情知性）を優先する段階的な学習率設計により、従来のモデルよりも効率的かつ効果的なトレーニングを実現しています。
|
||||
|
||||
### 想定ユースケース
|
||||
|
||||
- **AItuber / AI VTuber**: 配信中のリスナーとの雑談・コメント対応
|
||||
- **チャットボット**: 自然な日本語での日常会話・悩み相談
|
||||
- **ロールプレイ**: キャラクターを演じた対話・創作支援
|
||||
- **汎用アシスタント**: 知識応答・推論・ツール呼び出しを含む幅広いタスク
|
||||
|
||||
### 主な特徴
|
||||
|
||||
- 🗣️ **自然な日本語応答** — 定型文や翻訳調を排した、人間らしい対話。相談・共感・傾聴といった感情的な応答にも対応し、会話相手に寄り添った回答を生成する
|
||||
- 💬 **高い雑談性能** — 日常会話・相談・共感的応答に強く、AItuberとしてリスナーと自然にコミュニケーションできる対話能力を持つ
|
||||
- 🎭 **RP・キャラクター対話** — 一貫した人格・感情表現を持ったロールプレイ対話が可能。キャラクターの感情の揺れや対話の駆け引きを表現できる
|
||||
- 🧠 **推論力** — 論理的思考・数学的推論の基礎を保持。高難度推論データで学習しており、複雑な質問にも対応可能
|
||||
- 🔧 **Tool Use / RAG** — Function Calling・検索拡張生成にも対応。日本語・英語両方でのツール呼び出しや、与えられたコンテキストに基づく正確な回答が可能
|
||||
- ✍️ **クリエイティブ表現** — 文学的な比喩・暗喩・情景描写など、豊かな日本語表現力を持つ
|
||||
|
||||
### モデル仕様
|
||||
|
||||
| 項目 | 詳細 |
|
||||
|---|---|
|
||||
| **モデル名** | `DataPilot/ArrowCanaria-Llama-8B-SFT-v0.1` |
|
||||
| **ベースモデル** | `tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.5` |
|
||||
| **アーキテクチャ** | Llama 3.1 (Transformer decoder-only) |
|
||||
| **パラメータ数** | 8B |
|
||||
| **学習データ量** | 175,000件超（JP 70.9% / EN 29.1%） |
|
||||
| **コンテキスト長** | 4,096 tokens |
|
||||
| **精度** | BF16 |
|
||||
| **ライセンス** | Llama 3.1 Community License + Gemma Terms of Use License |
|
||||
|
||||
---
|
||||
|
||||
## 推論方法
|
||||
|
||||
### 🤗 Transformers
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model_name = "DataPilot/ArrowCanaria-Llama-8B-SFT-v0.1"
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
model_name,
|
||||
torch_dtype="bfloat16",
|
||||
device_map="auto",
|
||||
)
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": "あなたは親しみやすいAIアシスタントです。自然な日本語で会話してください。"},
|
||||
{"role": "user", "content": "最近ちょっと疲れてるんだよね。何かリフレッシュできる方法ない？"},
|
||||
]
|
||||
|
||||
input_ids = tokenizer.apply_chat_template(
|
||||
messages,
|
||||
add_generation_prompt=True,
|
||||
return_tensors="pt",
|
||||
).to(model.device)
|
||||
|
||||
outputs = model.generate(
|
||||
input_ids,
|
||||
max_new_tokens=512,
|
||||
temperature=0.7,
|
||||
top_p=0.9,
|
||||
repetition_penalty=1.05,
|
||||
do_sample=True,
|
||||
)
|
||||
|
||||
response = tokenizer.decode(outputs[0][input_ids.shape[-1]:], skip_special_tokens=True)
|
||||
print(response)
|
||||
```
|
||||
|
||||
### ⚡ vLLM
|
||||
|
||||
```bash
|
||||
# vLLM サーバーの起動
|
||||
vllm serve DataPilot/ArrowCanaria-Llama-8B-SFT-v0.1 \
|
||||
--dtype bfloat16 \
|
||||
--max-model-len 4096 \
|
||||
--host 0.0.0.0 \
|
||||
--port 8000
|
||||
```
|
||||
|
||||
```bash
|
||||
# リクエスト例（curl）
|
||||
curl http://localhost:8000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "DataPilot/ArrowCanaria-Llama-8B-SFT-v0.1",
|
||||
"messages": [
|
||||
{"role": "system", "content": "あなたは親しみやすいAIアシスタントです。自然な日本語で会話してください。"},
|
||||
{"role": "user", "content": "最近ちょっと疲れてるんだよね。何かリフレッシュできる方法ない？"}
|
||||
],
|
||||
"temperature": 0.7,
|
||||
"max_tokens": 512
|
||||
}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## データ概要
|
||||
|
||||
### SDG_LOOM による高品質データ生成
|
||||
|
||||
本モデルの学習データは、独自の合成データ生成フレームワーク **[SDG_LOOM](https://github.com/foxn2000/sdg_loom)** を活用して作成されています。SDG_LOOM は、高品質な合成データを効率的に生成するためのパイプラインであり、データの品質管理・フィルタリングを体系的に行うことで、学習に最適なデータセットを構築します。
|
||||
|
||||
### データ構成
|
||||
|
||||
合計 **175,000件超** の合成データセットで学習を行いました。
|
||||
|
||||
| 言語 | 件数 | 比率 |
|
||||
|---|---|---|
|
||||
| 日本語 | 124,000件 | 70.9% |
|
||||
| 英語 | 51,000件 | 29.1% |
|
||||
| **合計** | **175,000件超** | **100%** |
|
||||
|
||||
### タスク別内訳
|
||||
|
||||
| カテゴリ | 内容 | 言語 |
|
||||
|---|---|---|
|
||||
| **知識応答** | シングルターン・マルチターンの知識QA | 🇯🇵 JP |
|
||||
| **雑談・相談** | 日常会話・悩み相談・共感的応答（EQの中核） | 🇯🇵 JP |
|
||||
| **Tool Use** | Function Calling（日本語/英語）・エージェント行動 | 🇯🇵🇬🇧 JP/EN |
|
||||
| **RAG** | 検索拡張生成（コンテキスト参照→質問応答） | 🇬🇧 EN |
|
||||
| **推論** | 高難度推論・数学的思考 | 🇬🇧 EN |
|
||||
| **RP** | ロールプレイ対話・キャラクター感情表現 | 🇯🇵 JP |
|
||||
| **AItuber RP** | 配信者↔リスナー対話・即興的なキャラクター応答 | 🇯🇵 JP |
|
||||
| **クリエイティブ** | 文学的表現力（比喩・暗喩・描写） | 🇯🇵 JP |
|
||||
|
||||
---
|
||||
|
||||
## 学習概要
|
||||
|
||||
### モデル作成パイプライン
|
||||
|
||||
本モデルは以下の3段階で構築されています：
|
||||
|
||||
```
|
||||
1. CPT（Continual Pre-Training）
|
||||
tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.5
|
||||
+ 10k フィルタリング済み高品質日本語合成小説データセット
|
||||
↓
|
||||
2. Chat Vector マージ（Mergekit）
|
||||
CPTモデルに元モデルの Chat Vector をマージし、
|
||||
対話能力を復元
|
||||
↓
|
||||
3. SFT（Supervised Fine-Tuning）
|
||||
175,000件超の合成データセットで LoRA による
|
||||
3フェーズカリキュラム学習
|
||||
```
|
||||
|
||||
### SFT 学習設計
|
||||
|
||||
#### カリキュラム学習（3フェーズ構成）
|
||||
|
||||
SFT は「声→能力→人格」の順序で段階的に学習する **カリキュラム学習** を採用しています。
|
||||
|
||||
```
|
||||
Phase 1: 基盤構築 ──▶ Phase 2: 能力拡張 ──▶ Phase 3: 個性統合
|
||||
65,000件 63,000件 47,000件
|
||||
```
|
||||
|
||||
| フェーズ | データ件数 | 目的 | 学習率 |
|
||||
|---|---|---|---|
|
||||
| **Phase 1: 基盤構築** | 65,000件 | 日本語 EQ・知識応答・推論力の原型を確立 | `1e-4` |
|
||||
| **Phase 2: 能力拡張** | 63,000件 | Tool Use / RAG / エージェント能力の獲得 | `3e-5` |
|
||||
| **Phase 3: 個性統合** | 47,000件 | EQ・キャラクター性・対話人格の統合 | `5e-5` |
|
||||
|
||||
#### EQ優先の学習率設計
|
||||
|
||||
```
|
||||
P1 (1e-4) >> P3 (5e-5) > P2 (3e-5)
|
||||
```
|
||||
|
||||
- **Phase 1 (`1e-4`)**: 日本語EQ・知識応答の基盤を強く刻み込む
|
||||
- **Phase 2 (`3e-5`)**: Tool/RAG は補助的能力のため控えめに学習し、Phase 1 の文体を最大限保護
|
||||
- **Phase 3 (`5e-5`)**: 個性統合は EQ の中核目標のため Phase 2 より積極的に学習。ただし Phase 1 の基盤を上書きしない範囲に留める
|
||||
|
||||
#### 壊滅的忘却の防止
|
||||
|
||||
各フェーズでは **リプレイバッファ** を導入し、前フェーズのデータの一部を再混合することで壊滅的忘却を防止しています。
|
||||
|
||||
#### 主な学習パラメータ
|
||||
|
||||
| パラメータ | 値 |
|
||||
|---|---|
|
||||
| **LoRA rank (r)** | 32 |
|
||||
| **LoRA alpha** | 64 |
|
||||
| **LoRA target modules** | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
|
||||
| **LoRA dropout** | 0.0（Unsloth 高速パッチ適用のため） |
|
||||
| **最大シーケンス長** | 4,096 tokens |
|
||||
| **実効バッチサイズ** | 16（batch=2 × grad_accum=8） |
|
||||
| **Optimizer** | AdamW (torch fused) |
|
||||
| **精度** | BF16 |
|
||||
| **Packing** | 有効（GPU 利用率最大化） |
|
||||
| **Loss** | NLL（標準 Negative Log-Likelihood） |
|
||||
| **Warmup ratio** | 0.05（Phase 1/2）/ 0.10（Phase 3） |
|
||||
| **Epochs** | 1.0（全フェーズ共通、packing により 1 エポックで十分） |
|
||||
| **学習フレームワーク** | Unsloth + TRL `SFTTrainer` |
|
||||
|
||||
#### Phase 3 の工夫
|
||||
|
||||
Phase 3 では RP・クリエイティブという **スタイル転移力の強いデータ** を扱うため、以下の工夫を施しています：
|
||||
|
||||
- **Warmup ratio を 0.10 に倍増**（Phase 1/2 は 0.05）：学習初期の急激なパラメータ変動を抑制し、Phase 1/2 で蓄積した能力を保護
|
||||
- **学習率を Phase 1 の半分（5e-5）に設定**：スタイル転移の強度を適度に抑制しつつ、Phase 2 より積極的に個性を学習
|
||||
|
||||
---
|
||||
|
||||
## 謝辞
|
||||
|
||||
- 質の高い日本語モデルを作成してくれたSwallowチームに感謝します。 - [tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.5](https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.5)
|
||||
- 高品質なベースモデルを作成してくれたMetaのLlamaチームに感謝します。 - [Meta Llama](https://www.llama.com)
|
||||
- 高効率な学習フレームワークを作成してくださったUnslothに感謝します。 - [Unsloth](https://github.com/unslothai/unsloth)
|
||||
- そのほか、高品質なデータを作成する際にDeepSeek-V3.2とkimi K2.5を使用しました。[DeepSeekチーム](https://www.deepseek.com)と[Kimiチーム](https://www.moonshot.ai)に感謝します。
|
||||
|
||||
Built by Kimi
|
||||
1
chat_template.jinja
Normal file
1
chat_template.jinja
Normal file
@@ -0,0 +1 @@
|
||||
{{- bos_token }}{% for message in messages %}{{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n' }}{{- message['content'] | trim + '<|eot_id|>' }}{% endfor %}{% if add_generation_prompt %}{{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}
|
||||
39
config.json
Normal file
39
config.json
Normal file
@@ -0,0 +1,39 @@
|
||||
{
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 128000,
|
||||
"dtype": "bfloat16",
|
||||
"eos_token_id": 128009,
|
||||
"head_dim": 128,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 4096,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 14336,
|
||||
"max_position_embeddings": 131072,
|
||||
"mlp_bias": false,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 32,
|
||||
"num_key_value_heads": 8,
|
||||
"pad_token_id": 128004,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_parameters": {
|
||||
"factor": 8.0,
|
||||
"high_freq_factor": 4.0,
|
||||
"low_freq_factor": 1.0,
|
||||
"original_max_position_embeddings": 8192,
|
||||
"rope_theta": 500000.0,
|
||||
"rope_type": "llama3"
|
||||
},
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 10000.0,
|
||||
"tie_word_embeddings": false,
|
||||
"transformers_version": "4.57.1",
|
||||
"unsloth_version": "2026.2.1",
|
||||
"use_cache": true,
|
||||
"vocab_size": 128256
|
||||
}
|
||||
3
model-00001-of-00004.safetensors
Normal file
3
model-00001-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8cd7d2c7eb9ff74e2d1c7f83d98bb29fdf696171671d64110553d63b31fc01ca
|
||||
size 4953586384
|
||||
3
model-00002-of-00004.safetensors
Normal file
3
model-00002-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:bc291d5bf600c3def12f4ccb4584548091f12ac3dd027aadeed2414cffc99d0c
|
||||
size 4999819336
|
||||
3
model-00003-of-00004.safetensors
Normal file
3
model-00003-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9f6665249e491a7f00522fa595c2873802e8ef0189a9d1caffeb8fc34dc7fe0c
|
||||
size 4915916144
|
||||
3
model-00004-of-00004.safetensors
Normal file
3
model-00004-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0915ba0f4e5ea5391df340f12405779912bdc201c95e953bb8f4a9c3ac87f65e
|
||||
size 1191234472
|
||||
299
model.safetensors.index.json
Normal file
299
model.safetensors.index.json
Normal file
@@ -0,0 +1,299 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 16060522496,
|
||||
"mergekit_version": "0.1.4"
|
||||
},
|
||||
"weight_map": {
|
||||
"lm_head.weight": "model-00001-of-00004.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.28.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.28.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.28.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.28.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.28.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.29.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.29.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.29.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.29.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.29.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.30.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.30.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.30.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.30.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.30.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.31.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.31.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.31.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.31.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.31.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.norm.weight": "model-00004-of-00004.safetensors"
|
||||
}
|
||||
}
|
||||
17
special_tokens_map.json
Normal file
17
special_tokens_map.json
Normal file
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<|begin_of_text|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": "<|eot_id|>",
|
||||
"pad_token": {
|
||||
"content": "<|finetune_right_pad_id|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
Binary file not shown.
2065
tokenizer_config.json
Normal file
2065
tokenizer_config.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user