WisdomShell/GRIP-Llama-3-8B
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: WisdomShell/GRIP-Llama-3-8B
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-04-27 16:43:37 +08:00
commit 2b1e28e63d
14 changed files with 3034 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

283
README.md Normal file
View File

@@ -0,0 +1,283 @@
---
license: apache-2.0
tags:
- rlhf
- llama
- GRIP
pipeline_tag: text-generation
base_model:
- meta-llama/Meta-Llama-3-8B
language:
- en
- zh
---
<div align="center">
<h1> Retrieval as Generation: A Unified Framework with Self-Triggered Information Planning </h1>
<p>
<strong>English</strong> | <a href="https://huggingface.co/WisdomShell/GRIP-Llama-3-8B/blob/main/README_zh.md">简体中文</a>
</p>
<p>
<a href="https://arxiv.org/abs/2604.11407"><img src="https://img.shields.io/badge/Paper-arXiv-b31b1b?logo=arxiv&logoColor=white" /></a>
<a href="https://wisdomshell.github.io/GRIP/"><img src="https://img.shields.io/badge/Project-Homepage-2ea44f?logo=githubpages&logoColor=white" /></a>
<a href="#overview"><img src="https://img.shields.io/badge/Task-Agentic%20RAG-purple.svg" /></a>
<a href="https://github.com/WisdomShell/GRIP"><img src="https://img.shields.io/badge/GitHub-Repository-181717?logo=github&logoColor=white" /></a>
<a href="https://2026.aclweb.org/"><img src="https://img.shields.io/badge/Venue-ACL%202026-blue" /></a>
<a href="#installation"><img src="https://img.shields.io/badge/Python-3.9%2B-3776AB?logo=python&logoColor=white" /></a>
</p>
<h2>[ACL'26 Main Conference]</h2>
<a href="https://deepblue666.github.io/">Bo Li</a>&emsp;
<a>Mingda Wang</a>&emsp;
<a>GeXiang Fang</a>&emsp;
<a>Shikun Zhang</a>&emsp;
<a>Wei Ye</a>&emsp;
<div>
</div>
</div>
Traditional RAG (Retrieval-Augmented Generation) systems treat retrieval as an external, one-shot intervention, rigidly fetching documents before generation begins, which often fails when information needs emerge gradually during complex reasoning. Even dynamic search methods heavily rely on disconnected external controllers or heuristic rules.
We believe that, much like human cognitive processes, retrieval should be an intrinsic, generative capability. LLMs must be able to autonomously evaluate their knowledge, trigger searches, and formulate contextual follow-up queries tightly coupled with their evolving reasoning states.
GRIP (Generation-guided Retrieval with Information Planning) embodies this new paradigm. Under the framework of Retrieval as Generation, our model internalizes retrieval decisions directly into token-level decoding using specific control tokens. This approach shifts from relying on auxiliary multi-stage search modules to achieving end-to-end, self-triggered information planning within a single autoregressive trajectory.
## 🌟 Key Features
- 🎯 **Token-Driven Control**: Embeds retrieval behaviors directly into the model's generative policy via explicit control tokens (e.g., [RETRIEVE], [ANSWER], [INTERMEDIARY]) without external classifiers.
- 🔄 **Self-Triggered Planning**: Autonomously decides when to fall back to internal knowledge, how to reformulate targeted queries based on partial reasoning, and when to terminate the search.
- ⚖️ **Adaptive Retrieval Depth**: Dynamically adjusts the number of retrieval rounds based on question complexity, successfully avoiding redundant searches while extrapolating beyond strict training budgets.
- 🚀 **State-of-the-Art Performance**: Surpasses strong open-source RAG baselines (e.g., GainRAG, R1-Searcher) and achieves performance competitive with GPT-4o across five QA benchmarks using a much smaller backbone (LLaMA3-8B).
- 🧩 **Unified Decoding Trajectory**: Tightly couples multi-step reasoning and on-the-fly evidence integration into a single, continuous generation flow.
- 🛠️ **Optimized Training Recipe**: Employs a structured supervised fine-tuning (SFT) over four distinct behavioral patterns, further refined by rule-based Reinforcement Learning (DAPO) to ensure accurate and balanced retrieval control.
## 🚀 Quick Start
### Installation
```bash
git clone https://github.com/WisdomShell/GRIP
cd GRIP
conda create -n GRIP python=3.9
conda activate GRIP
cd GRIP/model/Train
pip install -e .
cd ../
pip install -r requirements.txt
```
## Preparation
### Build Wikipedia index
Download the Wikipedia dump.
```python
mkdir wiki_data
cd wiki_data
wget https://dl.fbaipublicfiles.com/dpr/wikipedia_split/psgs_w100.tsv.gz
gzip -d psgs_w100.tsv.gz
```
Use Elasticsearch to index the Wikipedia dump
```python
mkdir ret
cd ret
wget -O elasticsearch-7.17.9.tar.gz https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.9-linux-x86_64.tar.gz
tar zxvf elasticsearch-7.17.9.tar.gz
rm elasticsearch-7.17.9.tar.gz
cd elasticsearch-7.17.9
nohup bin/elasticsearch
python data_generation/index.py --data_path path/to/your/psgs_w100.tsv --index_name wiki
```
## Checkpoints and Datasets
Below are the datasets used for SFT and RL training in our work, and the weights of the already trained GRIP model.
| Dataset | HF Dataset Repo |
|------------------------------|-----------------------------------------------------------------------------------------------------------|
| GRIP_SFT_Train_Data | [WisdomShell/GRIP_SFT_Data](https://huggingface.co/datasets/WisdomShell/GRIP_SFT_Data) |
| GRIP_RL_Train_Data | [WisdomShell/GRIP_RL_Data](https://huggingface.co/datasets/WisdomShell/GRIP_RL_Data) |
| Model | HF Model Repo |
|------------------------------|-----------------------------------------------------------------------------------------------------------|
| Meta-LLaMa-3-8b-GRIP | [WisdomShell/LLaMa-3-8b-GRIP](https://huggingface.co/WisdomShell/GRIP-Llama-3-8B) |
## Generation SFT and RL Training Data
Before that, you need to download the `NaturalQuestion-open` training set, `WebQuestions` training set and `TriviaQA` training set, extract their Questions and answers and merge them into a jsonl file,Convert them into the following format:
```json
{
"question": "",
"answer":["Answer", ...]
}
```
Use the `Meta-LLaMa-3-8B-Instruct` model to execute the following code
```python
bash data_generation/first.sh
```
Write your *OpenAI token* into the use_gpt_for_data.py file,and configure `C.jsonl` file path.
After the generation is completed, it will automatically overwrite the original file.
```
python generation_train_data/use_gpt_for_data.py
```
Write the directory where A, B, C, and D are located into the merge_dataset.py file.
The output path will save SFT_Train_data and RL_Train_data.
```
python generation_train_data/merge_dataset.py
```
## Train
### SFT
1. Data Process
- Script: `Train/examples/data_preprocess/grip/sft.py`
- You need to specify the `data_path` parameter, indicating the path of the data synthesized by GRIP
```python
parser.add_argument('--data_path', default='<PATH_TO_RAW_DATASET_ROOT>/SFT_data.jsonl')
```
- You should specify the name of the `dataset` for use during subsequent training.
```python
# The data path is stored in the "datasets" folder by default.
parser.add_argument('--save_dir', default='datasets/GRIPSFT')
```
2. Train Script
- Script: `Train/examples/sft/run_sft_llama.sh`
- Train using the **Base version** of the model.
```bash
set -x
NAME=GRIPSFT # Here to specify the names of the processed training data from the previous step
torchrun --standalone --nnodes=1 --nproc_per_node=8 -m verl.trainer.fsdp_sft_trainer \
data.train_files=datasets/$NAME/train.parquet \
data.val_files=datasets/$NAME/test.parquet \
data.prompt_key=extra_info \
data.response_key=extra_info \
optim.lr=1e-6 \
data.prompt_dict_keys=['question'] \
+data.response_dict_keys=['answer'] \
data.micro_batch_size=4 \
model.partial_pretrain=meta-llama/Meta-Llama-3-8B-Base \ #Use Base to Train
trainer.default_local_dir=/path/to/your/SFT_model \ # Finetuned Model Save Path
trainer.project_name=GRIPSFT \
trainer.experiment_name=$NAME \
trainer.logger=['console'] \ # Report `console` or `wandb`
trainer.total_epochs=8 \ # Training Epoches
trainer.default_hdfs_dir=null $@ \
ulysses_sequence_parallel_size=2 \
use_remove_padding=true
```
### RL
1. Data Process
- Script: `Train/examples/data_preprocess/grip/rl.py`
- You need to specify the `data_path` parameter, indicating the path of the data synthesized by GRIP
```python
parser.add_argument('--data_path', default='<PATH_TO_RAW_DATASET_ROOT>/RL_data.jsonl')
```
- You should specify the name of the `dataset` for use during subsequent training.
```python
# The data path is stored in the "datasets" folder by default.
parser.add_argument('--save_dir', default='datasets/GRIPRL')
```
- You should specify the name of the `data_source` for use during subsequent training to select reward model.
```python
parser.add_argument('--data_source', default='GRIPRL') # Necessary
```
2. Train Script using `DAPO`
- Script: `Train/recipe/dapo/dapo_4w_continue_rl_ep3_llama.sh`
- You should modify these parameters to suit RL training.
```bash
...
# Paths
MODEL_PATH=<PATH_TO_SAVE>/GRIPSFT_LLaMa/global_step_xxx # SFT Checkpoint
CKPTS_DIR=<PATH_TO_SAVE>/RL_model # RL Model Save Path
TRAIN_FILE=datasets/GRIPRL/train.parquet # RL Datasets
TEST_FILE=datasets/GRIPRL/test.parquet # RL Datasets
...
```
3. The specific implementation of the Reward Model is in the file `Train/verl/utils/reward_score/grip.py`.
4. After training, you should merge the slices saved from the model into Hugging Face format by script `Train/scripts/merge.sh`.
### Local Inference using GRIP
#### Test data format alignment
```json
{
"question": "Test Query",
"answer": ["Answer List", ...]
}
```
#### Mutil-Turn GRIP Inference
- Main Script: `inference/inference.sh`
```python
# Model Saved Path
parser.add_argument('--model_path', type=str, default="/path/to/your/RL_model/step_xxx")
# Predicted file output path
parser.add_argument('--output_file', type=str, default="output/rl_step_xxx_hotpot.jsonl")
# File to be predicted
parser.add_argument('--input_file', type=str, default="test_data/hotpotQA.jsonl")
```
- This script will generate predicitons by format:
```json
{
"Question": "String",
"prediction": ["String",......]
}
```
## Eval
```python
python eval/eval.py \
--references_path test_dataset.jsonl \
--predictions_path prediction.jsonl
```
## 🤝 Contributing
We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
## 📄 Citation
```bibtex
@article{li2026retrieval,
title={Retrieval as Generation: A Unified Framework with Self-Triggered Information Planning},
author={Li, Bo and Wang, Mingda and Fang, Gexiang and Zhang, Shikun and Ye, Wei},
journal={arXiv preprint arXiv:2604.11407},
year={2026}
}
```
## 📝 License
This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details.
## 🙏 Acknowledgments
Special thanks to the open-source community and all contributors who made this project possible.

272
README_zh.md Normal file
View File

@@ -0,0 +1,272 @@
<div align="center">
<h1> Retrieval as Generation: A Unified Framework with Self-Triggered Information Planning </h1>
<p>
<a href="README.md">English</a> | <strong>简体中文</strong>
</p>
<p>
<a href="https://arxiv.org/abs/2604.11407"><img src="https://img.shields.io/badge/Paper-arXiv-b31b1b?logo=arxiv&logoColor=white" /></a>
<a href="https://wisdomshell.github.io/GRIP/"><img src="https://img.shields.io/badge/Project-Homepage-2ea44f?logo=githubpages&logoColor=white" /></a>
<a href="#overview"><img src="https://img.shields.io/badge/Task-Agentic%20RAG-purple.svg" /></a>
<a href="https://github.com/WisdomShell/GRIP"><img src="https://img.shields.io/badge/GitHub-Repository-181717?logo=github&logoColor=white" /></a>
<a href="https://2026.aclweb.org/"><img src="https://img.shields.io/badge/Venue-ACL%202026-blue" /></a>
<a href="#installation"><img src="https://img.shields.io/badge/Python-3.9%2B-3776AB?logo=python&logoColor=white" /></a>
</p>
<h2>[ACL'26 Main Conference]</h2>
<a href="https://deepblue666.github.io/">Bo Li</a>&emsp;
<a>Mingda Wang</a>&emsp;
<a>GeXiang Fang</a>&emsp;
<a>Shikun Zhang</a>&emsp;
<a>Wei Ye</a>&emsp;
<div>
</div>
</div>
传统的 RAG（检索增强生成）系统将检索视为一种外部的、一次性的干预行为——在生成开始前僵硬地预取文档。这种方式在复杂推理过程中信息需求逐步涌现时往往表现不佳。即便是动态搜索方法，也高度依赖于孤立的外部控制器或启发式规则。
我们认为，正如人类的认知过程一样，检索应当是一种内在的、生成式的能力。大语言模型必须能够自主地评估自身知识状态、触发搜索，并根据不断演进的推理状态构建具有上下文关联的后续查询。
GRIP（Generation-guided Retrieval with Information Planning，生成引导的信息规划检索）正是这一新范式的具体体现。在"检索即生成"框架下，我们的模型通过特定控制词元（control tokens），将检索决策直接内化于词元级的解码过程中。这一方法将模型从依赖辅助性多阶段搜索模块中解放出来，在单一自回归轨迹内实现端到端、自触发的信息规划。
## 🌟 核心特性
- 🎯 **词元驱动控制**：通过显式控制词元（如 `[RETRIEVE]`、`[ANSWER]`、`[INTERMEDIARY]`），将检索行为直接嵌入模型的生成策略，无需外部分类器。
- 🔄 **自触发规划**：自主决定何时回退到内部知识、如何根据部分推理重新构建针对性查询，以及何时终止搜索。
- ⚖️ **自适应检索深度**：根据问题复杂度动态调整检索轮次，在成功避免冗余搜索的同时，还能突破严格的训练预算限制进行外推。
- 🚀 **最先进的性能**：在五个问答基准测试上，以更小的骨干模型（LLaMA3-8B）超越了强力的开源 RAG 基线（如 GainRAG、R1-Searcher），并达到了与 GPT-4o 相当的竞争性水平。
- 🧩 **统一解码轨迹**：将多步推理与即时证据整合紧密耦合于单一、连续的生成流程中。
- 🛠️ **优化的训练方案**：采用针对四种不同行为模式的结构化有监督微调（SFT），并通过基于规则的强化学习（DAPO）进一步精炼，以确保准确且均衡的检索控制。
## 🚀 快速开始
### 安装
```bash
git clone https://github.com/WisdomShell/GRIP
cd GRIP
conda create -n GRIP python=3.9
conda activate GRIP
cd GRIP/model/Train
pip install -e .
cd ../
pip install -r requirements.txt
```
## 准备工作
### 构建 Wikipedia 索引
下载 Wikipedia 数据转储文件。
```python
mkdir wiki_data
cd wiki_data
wget https://dl.fbaipublicfiles.com/dpr/wikipedia_split/psgs_w100.tsv.gz
gzip -d psgs_w100.tsv.gz
```
使用 Elasticsearch 对 Wikipedia 数据建立索引。
```python
mkdir ret
cd ret
wget -O elasticsearch-7.17.9.tar.gz https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.9-linux-x86_64.tar.gz
tar zxvf elasticsearch-7.17.9.tar.gz
rm elasticsearch-7.17.9.tar.gz
cd elasticsearch-7.17.9
nohup bin/elasticsearch
python data_generation/index.py --data_path path/to/your/psgs_w100.tsv --index_name wiki
```
## 检查点与数据集
以下是本工作中用于 SFT 和 RL 训练的数据集，以及已训练完成的 GRIP 模型权重。
| 数据集 | HF 数据集仓库 |
|------------------------------|-----------------------------------------------------------------------------------------------------------|
| GRIP_SFT_Train_Data | [WisdomShell/GRIP_SFT_Data](https://huggingface.co/datasets/WisdomShell/GRIP_SFT_Data) |
| GRIP_RL_Train_Data | [WisdomShell/GRIP_RL_Data](https://huggingface.co/datasets/WisdomShell/GRIP_RL_Data) |
| 模型 | HF 模型仓库 |
|------------------------------|-----------------------------------------------------------------------------------------------------------|
| Meta-LLaMa-3-8b-GRIP | [WisdomShell/LLaMa-3-8b-GRIP](https://huggingface.co/WisdomShell/GRIP-Llama-3-8B) |
## 生成 SFT 和 RL 训练数据
在此之前，您需要下载 `NaturalQuestion-open` 训练集、`WebQuestions` 训练集和 `TriviaQA` 训练集，提取其中的问题与答案，将它们合并为一个 jsonl 文件，并转换为以下格式：
```json
{
"question": "",
"answer":["Answer", ...]
}
```
使用 `Meta-LLaMa-3-8B-Instruct` 模型执行以下代码：
```python
bash data_generation/first.sh
```
将您的 *OpenAI token* 写入 `use_gpt_for_data.py` 文件，并配置 `C.jsonl` 文件路径。
生成完成后，将自动覆盖原始文件。
```
python generation_train_data/use_gpt_for_data.py
```
将 A、B、C、D 所在的目录写入 `merge_dataset.py` 文件。
输出路径将保存 SFT_Train_data 和 RL_Train_data。
```
python generation_train_data/merge_dataset.py
```
## 训练
### SFT
1. 数据处理
- 脚本：`Train/examples/data_preprocess/grip/sft.py`
- 您需要指定 `data_path` 参数，即 GRIP 合成数据的路径：
```python
parser.add_argument('--data_path', default='<PATH_TO_RAW_DATASET_ROOT>/SFT_data.jsonl')
```
- 您需要指定 `dataset` 的名称，以便在后续训练中使用：
```python
# 数据路径默认保存在 "datasets" 文件夹中
parser.add_argument('--save_dir', default='datasets/GRIPSFT')
```
2. 训练脚本
- 脚本：`Train/examples/sft/run_sft_llama.sh`
- 使用模型的 **Base 版本** 进行训练：
```bash
set -x
NAME=GRIPSFT # 在此指定上一步处理后的训练数据名称
torchrun --standalone --nnodes=1 --nproc_per_node=8 -m verl.trainer.fsdp_sft_trainer \
data.train_files=datasets/$NAME/train.parquet \
data.val_files=datasets/$NAME/test.parquet \
data.prompt_key=extra_info \
data.response_key=extra_info \
optim.lr=1e-6 \
data.prompt_dict_keys=['question'] \
+data.response_dict_keys=['answer'] \
data.micro_batch_size=4 \
model.partial_pretrain=meta-llama/Meta-Llama-3-8B-Base \ #使用 Base 版本训练
trainer.default_local_dir=/path/to/your/SFT_model \ # 微调模型保存路径
trainer.project_name=GRIPSFT \
trainer.experiment_name=$NAME \
trainer.logger=['console'] \ # 上报到 `console` 或 `wandb`
trainer.total_epochs=8 \ # 训练轮次
trainer.default_hdfs_dir=null $@ \
ulysses_sequence_parallel_size=2 \
use_remove_padding=true
```
### RL
1. 数据处理
- 脚本：`Train/examples/data_preprocess/grip/rl.py`
- 您需要指定 `data_path` 参数，即 GRIP 合成数据的路径：
```python
parser.add_argument('--data_path', default='<PATH_TO_RAW_DATASET_ROOT>/RL_data.jsonl')
```
- 您需要指定 `dataset` 的名称，以便在后续训练中使用：
```python
# 数据路径默认保存在 "datasets" 文件夹中
parser.add_argument('--save_dir', default='datasets/GRIPRL')
```
- 您需要指定 `data_source` 的名称，以便在后续训练中选择奖励模型：
```python
parser.add_argument('--data_source', default='GRIPRL') # 必填
```
2. 使用 `DAPO` 训练脚本
- 脚本：`Train/recipe/dapo/dapo_4w_continue_rl_ep3_llama.sh`
- 您需要修改以下参数以适配 RL 训练：
```bash
...
# 路径配置
MODEL_PATH=<PATH_TO_SAVE>/GRIPSFT_LLaMa/global_step_xxx # SFT 检查点
CKPTS_DIR=<PATH_TO_SAVE>/RL_model # RL 模型保存路径
TRAIN_FILE=datasets/GRIPRL/train.parquet # RL 数据集
TEST_FILE=datasets/GRIPRL/test.parquet # RL 数据集
...
```
3. 奖励模型的具体实现位于文件 `Train/verl/utils/reward_score/grip.py` 中。
4. 训练完成后，您需要通过脚本 `Train/scripts/merge.sh` 将模型保存的分片合并为 Hugging Face 格式。
### 使用 GRIP 进行本地推理
#### 测试数据格式对齐
```json
{
"question": "Test Query",
"answer": ["Answer List", ...]
}
```
#### 多轮 GRIP 推理
- 主脚本：`inference/inference.sh`
```python
# 模型保存路径
parser.add_argument('--model_path', type=str, default="/path/to/your/RL_model/step_xxx")
# 预测文件输出路径
parser.add_argument('--output_file', type=str, default="output/rl_step_xxx_hotpot.jsonl")
# 待预测文件
parser.add_argument('--input_file', type=str, default="test_data/hotpotQA.jsonl")
```
- 该脚本将按以下格式生成预测结果：
```json
{
"Question": "String",
"prediction": ["String",......]
}
```
## 评估
```python
python eval/eval.py \
--references_path test_dataset.jsonl \
--predictions_path prediction.jsonl
```
## 🤝 贡献
欢迎贡献！请参阅 [CONTRIBUTING.md](CONTRIBUTING.md) 了解相关指引。
## 📄 引用
```bibtex
@article{li2026retrieval,
title={Retrieval as Generation: A Unified Framework with Self-Triggered Information Planning},
author={Li, Bo and Wang, Mingda and Fang, Gexiang and Zhang, Shikun and Ye, Wei},
journal={arXiv preprint arXiv:2604.11407},
year={2026}
}
```
## 📝 许可证
本项目采用 Apache 2.0 许可证——详情请参阅 [LICENSE](LICENSE) 文件。
## 🙏 致谢
特别感谢开源社区及所有使本项目成为可能的贡献者。

5
chat_template.jinja Normal file
View File

@@ -0,0 +1,5 @@
{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>
'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>
' }}{% endif %}

29
config.json Normal file
View File

@@ -0,0 +1,29 @@
{
"architectures": [
"LlamaForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 128000,
"dtype": "float32",
"eos_token_id": 128001,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 14336,
"max_position_embeddings": 8192,
"mlp_bias": false,
"model_type": "llama",
"num_attention_heads": 32,
"num_hidden_layers": 32,
"num_key_value_heads": 8,
"pretraining_tp": 1,
"rms_norm_eps": 1e-05,
"rope_scaling": null,
"rope_theta": 500000.0,
"tie_word_embeddings": false,
"transformers_version": "4.57.6",
"use_cache": true,
"vocab_size": 128256
}

9
generation_config.json Normal file
View File

@@ -0,0 +1,9 @@
{
"bos_token_id": 128000,
"do_sample": true,
"eos_token_id": 128001,
"max_length": 4096,
"temperature": 0.6,
"top_p": 0.9,
"transformers_version": "4.57.6"
}

3
model-00001-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:50650870b1c5f1a5cfd54e64488ae81071d70b37ddd28a0845e8f180577a9595
size 4926424856

3
model-00002-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3a6136f66ef9810f1c6107175dd34b9aac253de3934261e402bc1e6f3ad12982
size 4974636512

3
model-00003-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ed28b0a50ccb998cb797386c28507f09e374b8b2f0d8ad423ed80bdd961b10c2
size 4892779768

3
model-00004-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3a17969d14d3f33f01605bcd92a1164363a8c0385abebc8e60d179740a882798
size 1266715232

299
model.safetensors.index.json Normal file
View File

@@ -0,0 +1,299 @@
{
"metadata": {
"total_parameters": 8030261248,
"total_size": 16060522496
},
"weight_map": {
"lm_head.weight": "model-00001-of-00004.safetensors",
"model.embed_tokens.weight": "model-00003-of-00004.safetensors",
"model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.0.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.0.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.0.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.1.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.1.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.1.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.1.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.1.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.10.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.10.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.10.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.11.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.11.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.11.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.11.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.11.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.12.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.12.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.12.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.12.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.12.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.12.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.12.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.13.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.13.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.13.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.13.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.13.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.13.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.13.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.13.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.13.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.14.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.14.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.14.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.14.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.14.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.15.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.15.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.15.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.16.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.16.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.16.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.16.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.16.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.16.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.17.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.17.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.17.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.17.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.17.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.17.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.18.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.18.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.18.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.18.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.18.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.18.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.19.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.19.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.19.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.19.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.19.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.19.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.19.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.19.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.2.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.2.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.2.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.2.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.2.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.2.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.2.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.20.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.20.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.20.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.20.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.20.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.20.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.20.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.20.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.21.input_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.21.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.21.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.21.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.21.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.21.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.21.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.22.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.22.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.22.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.23.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.23.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.23.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.input_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.24.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.24.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.24.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.24.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.24.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.24.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.25.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.25.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.25.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.25.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.25.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.25.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.25.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.26.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.26.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.26.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.26.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.26.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.26.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.26.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.27.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.27.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.27.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.27.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.27.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.27.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.27.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.27.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.27.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.28.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.28.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.28.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.28.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.28.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.28.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.28.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.28.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.28.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.29.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.29.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.29.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.29.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.29.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.29.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.29.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.29.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.29.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.3.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.3.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.3.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.30.input_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.30.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.30.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.30.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.30.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.30.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.30.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.30.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.30.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.31.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.31.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.31.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.31.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.31.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.31.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.31.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.31.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.31.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.4.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.4.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.4.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.4.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.5.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.5.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.5.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.5.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.6.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.6.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.6.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.6.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.6.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.6.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.7.input_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.7.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.7.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.7.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.7.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.7.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.7.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.8.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.8.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.8.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.8.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.8.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.8.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.8.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.8.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.8.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.9.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.9.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.9.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.9.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.norm.weight": "model-00002-of-00004.safetensors"
}
}

23
special_tokens_map.json Normal file
View File

@@ -0,0 +1,23 @@
{
"bos_token": {
"content": "<|begin_of_text|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "<|end_of_text|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "<|end_of_text|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

3
tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3c5cf44023714fb39b05e71e425f8d7b92805ff73f7988b083b8c87f0bf87393
size 17209961

2063
tokenizer_config.json Normal file
View File

File diff suppressed because it is too large Load Diff