初始化项目，由ModelHub XC社区提供模型
Model: zhantz/Qwen3-0.6B-cuda Source: Original Platform
This commit is contained in:
ModelHub XC
52
.gitattributes
vendored
Normal file
52
.gitattributes
vendored
Normal file
@@ -0,0 +1,52 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zstandard filter=lfs diff=lfs merge=lfs -text
|
||||
*.tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
*.db* filter=lfs diff=lfs merge=lfs -text
|
||||
*.ark* filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.gguf* filter=lfs diff=lfs merge=lfs -text
|
||||
*.ggml filter=lfs diff=lfs merge=lfs -text
|
||||
*.llamafile* filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
merges.txt filter=lfs diff=lfs merge=lfs -text
|
||||
vocab.json filter=lfs diff=lfs merge=lfs -text
|
||||
model.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
11
Modelfile
Normal file
11
Modelfile
Normal file
@@ -0,0 +1,11 @@
|
||||
# ollama modelfile auto-generated by llamafactory
|
||||
|
||||
FROM .
|
||||
|
||||
TEMPLATE """{{ if .System }}System: {{ .System }}<|im_end|>
|
||||
{{ end }}{{ range .Messages }}{{ if eq .Role "user" }}Human: {{ .Content }}<|im_end|>
|
||||
Assistant:{{ else if eq .Role "assistant" }}{{ .Content }}<|im_end|>
|
||||
{{ end }}{{ end }}"""
|
||||
|
||||
PARAMETER stop "<|im_end|>"
|
||||
PARAMETER num_ctx 4096
|
||||
189
README.md
Normal file
189
README.md
Normal file
@@ -0,0 +1,189 @@
|
||||
# Qwen3-0.6B-Code-Expert 模型 LoRA 微调作业说明
|
||||
## 项目简介
|
||||
本项目是基于 **LLaMA Factory** 工具对阿里云通义千问的 **Qwen3-0.6B-Code-Expert** 小型代码大模型进行 **LoRA（Low-Rank Adaptation）轻量化微调** 的实践作业。核心目标是掌握大语言模型（LLM）的轻量化微调流程、训练检查点的恢复与管理、微调后模型的导出，以及通过 SSH 远程连接训练服务器完成模型下载、训练和文件传输的全流程操作。
|
||||
|
||||
项目针对 **CUDA 编程相关知识** 数据集（`cuda_train`）进行微调，最终得到适配 CUDA 知识问答的轻量化模型，同时实现了本地与远程训练服务器的协同操作。
|
||||
|
||||
### 核心技术栈
|
||||
- 模型基座：Qwen3-0.6B-Code-Expert（代码领域轻量级大模型）
|
||||
- 微调框架：LLaMA Factory（一站式大模型微调工具）
|
||||
- 微调方法：LoRA（低秩适配，减少训练参数量）
|
||||
- 远程操作：SSH/SCP/Paramiko（服务器登录与文件传输）
|
||||
- 依赖库：Transformers、PyTorch、Safetensors、Paramiko 等
|
||||
|
||||
## 环境配置
|
||||
### 1. 本地环境（开发机）
|
||||
适用于 Windows/macOS/Linux，需安装 Python 3.8+，执行以下命令安装依赖：
|
||||
```bash
|
||||
# 基础依赖（模型加载与微调）
|
||||
pip install torch==2.6.0 transformers==4.54.0 safetensors llama-factory
|
||||
# 远程操作依赖（SSH 连接服务器）
|
||||
pip install paramiko scp
|
||||
```
|
||||
|
||||
### 2. 远程训练服务器（Seetacloud）
|
||||
通过 SSH 登录服务器后，执行相同的依赖安装命令。若服务器无 GPU，需指定 CPU 版本依赖：
|
||||
```bash
|
||||
# 服务器无 GPU 时安装 CPU 版 PyTorch
|
||||
pip install torch==2.6.0+cpu transformers==4.54.0 safetensors llama-factory --index-url https://download.pytorch.org/whl/cpu
|
||||
```
|
||||
|
||||
## 快速开始
|
||||
### 1. 模型下载
|
||||
通过 `download_model.py` 脚本从 Hugging Face 下载 Qwen3-0.6B-Code-Expert 基座模型（或 Qwen2.5-0.5B 轻量化版本），支持本地/远程服务器下载。
|
||||
|
||||
#### 脚本使用
|
||||
```python
|
||||
# download_model.py
|
||||
import os
|
||||
os.environ["HF_ENDPOINT"] = "https://hf-mirror.com" # 国内镜像加速
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model_name = "Qwen/Qwen3-0.6B-Code-Expert" # 基座模型名称
|
||||
local_model_path = "./local-model" # 本地保存路径（服务器可改为 /root/local-model）
|
||||
|
||||
# 加载并保存模型
|
||||
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
|
||||
model.save_pretrained(local_model_path)
|
||||
tokenizer.save_pretrained(local_model_path)
|
||||
print("模型下载完成！")
|
||||
```
|
||||
|
||||
**运行脚本**：
|
||||
- 本地：`python download_model.py`
|
||||
- 远程服务器：先通过 SCP 上传脚本，再执行 `python /root/download_model.py`
|
||||
|
||||
### 2. 模型微调（LoRA）
|
||||
使用 LLaMA Factory 的命令行工具进行 LoRA 微调，核心命令如下（可保存为 `train.sh` 脚本）：
|
||||
```bash
|
||||
#!/bin/bash
|
||||
llamafactory-cli train \
|
||||
--stage sft \
|
||||
--do_train True \
|
||||
--model_name_or_path /root/autodl-tmp/models/modelscope_cache/models/suayptalha/Qwen3-0.6B-Code-Expert \
|
||||
--preprocessing_num_workers 16 \
|
||||
--finetuning_type lora \
|
||||
--template default \
|
||||
--flash_attn auto \
|
||||
--dataset_dir data \
|
||||
--dataset cuda_train \
|
||||
--cutoff_len 2048 \
|
||||
--learning_rate 5e-05 \
|
||||
--num_train_epochs 20.0 \
|
||||
--max_samples 100000 \
|
||||
--per_device_train_batch_size 2 \
|
||||
--gradient_accumulation_steps 8 \
|
||||
--lr_scheduler_type cosine \
|
||||
--max_grad_norm 1.0 \
|
||||
--logging_steps 5 \
|
||||
--save_steps 100 \
|
||||
--warmup_steps 0 \
|
||||
--packing False \
|
||||
--enable_thinking True \
|
||||
--report_to none \
|
||||
--output_dir saves/Qwen3-0.6B-Base/lora/train_2025-12-21-07-31-02 \
|
||||
--bf16 True \
|
||||
--plot_loss True \
|
||||
--trust_remote_code True \
|
||||
--ddp_timeout 180000000 \
|
||||
--include_num_input_tokens_seen True \
|
||||
--optim adamw_torch \
|
||||
--lora_rank 8 \
|
||||
--lora_alpha 16 \
|
||||
--lora_dropout 0 \
|
||||
--lora_target all \
|
||||
--save_safetensors True
|
||||
```
|
||||
|
||||
**关键参数说明**：
|
||||
- `--finetuning_type lora`：指定 LoRA 轻量化微调；
|
||||
- `--dataset cuda_train`：使用 CUDA 编程知识数据集；
|
||||
- `--output_dir`：训练检查点保存路径；
|
||||
- `--save_safetensors True`：使用 Safetensors 格式保存权重（规避 PyTorch 安全漏洞）。
|
||||
|
||||
**运行微调**：
|
||||
```bash
|
||||
# 赋予脚本执行权限
|
||||
chmod +x train.sh
|
||||
# 执行微调
|
||||
./train.sh
|
||||
```
|
||||
|
||||
### 3. 检查点恢复训练
|
||||
若训练中断，可通过 `--resume_from_checkpoint` 参数恢复训练：
|
||||
```bash
|
||||
llamafactory-cli train \
|
||||
# 保留原微调参数...
|
||||
--resume_from_checkpoint saves/Qwen3-0.6B-Base/lora/train_2025-12-21-07-13-16 \
|
||||
--save_safetensors True
|
||||
```
|
||||
|
||||
> **注意**：恢复训练需保证 PyTorch 版本 ≥2.6（规避 `torch.load` 安全漏洞），若版本过低需执行 `pip install torch==2.6.0` 升级。
|
||||
|
||||
### 4. 模型导出
|
||||
将 LoRA 微调权重与基座模型合并，导出为 Hugging Face 标准格式：
|
||||
```bash
|
||||
llamafactory-cli export \
|
||||
--model_name_or_path /root/autodl-tmp/models/modelscope_cache/models/suayptalha/Qwen3-0.6B-Code-Expert \
|
||||
--adapter_name_or_path saves/Qwen3-0.6B-Base/lora/train_2025-12-21-07-31-02 \
|
||||
--template default \
|
||||
--export_dir ./exported_qwen3_model \
|
||||
--export_format huggingface \
|
||||
--trust_remote_code True
|
||||
```
|
||||
|
||||
导出后，`./exported_qwen3_model` 目录下为完整的可部署模型，可直接通过 `transformers` 加载使用。
|
||||
|
||||
## 远程服务器操作（Seetacloud）
|
||||
本项目需通过 SSH 连接学校的 Seetacloud 训练服务器完成模型训练与文件传输，核心操作如下：
|
||||
|
||||
### 1. 手动 SSH 登录
|
||||
```bash
|
||||
# 连接服务器（端口 14788，输入密码即可登录）
|
||||
ssh -p 14788 root@connect.sha1.seetacloud.com
|
||||
```
|
||||
|
||||
### 2. 文件传输（SCP）
|
||||
- **上传本地文件到服务器**：
|
||||
```bash
|
||||
scp -P 14788 download_model.py root@connect.sha1.seetacloud.com:/root/
|
||||
```
|
||||
- **下载服务器模型到本地**：
|
||||
```bash
|
||||
scp -P 14788 -r root@connect.sha1.seetacloud.com:/root/exported_qwen3_model ./
|
||||
```
|
||||
|
||||
### 3. 自动 SSH 操作脚本
|
||||
通过 `paramiko` 库实现自动化登录、执行命令与文件传输，脚本见 `ssh_model_operate.py`（需替换 SSH 密码）：
|
||||
```bash
|
||||
# 运行自动化脚本
|
||||
python ssh_model_operate.py
|
||||
```
|
||||
|
||||
## 项目结构
|
||||
```
|
||||
.
|
||||
├── download_model.py # 基座模型下载脚本
|
||||
├── train.sh # LoRA 微调脚本
|
||||
├── ssh_model_operate.py # 远程服务器自动化操作脚本
|
||||
├── data/ # 数据集目录（存放 cuda_train 数据集）
|
||||
├── local-model/ # 本地下载的基座模型
|
||||
├── saves/ # 训练检查点保存目录
|
||||
├── exported_qwen3_model/ # 导出的微调后完整模型
|
||||
└── README.md # 项目说明文档
|
||||
```
|
||||
|
||||
## 注意事项
|
||||
1. **PyTorch 版本**：恢复训练时需保证 PyTorch ≥2.6，否则会触发 `torch.load` 安全漏洞报错；
|
||||
2. **服务器环境**：若服务器无 GPU，需在加载模型时指定 `device_map='cpu'`；
|
||||
3. **大文件传输**：模型文件较大时，推荐使用 `rsync` 替代 SCP 以提升稳定性：
|
||||
```bash
|
||||
rsync -avz -e 'ssh -p 14788' root@connect.sha1.seetacloud.com:/root/exported_qwen3_model ./
|
||||
```
|
||||
4. **SSH 安全**：建议使用 SSH 密钥登录服务器，避免密码明文暴露。
|
||||
|
||||
## 致谢
|
||||
1. 感谢学校提供的 Seetacloud 训练服务器资源；
|
||||
2. 感谢 LLaMA Factory 团队提供的一站式大模型微调工具；
|
||||
3. 感谢阿里云通义千问团队开源的 Qwen3 系列模型。
|
||||
28
added_tokens.json
Normal file
28
added_tokens.json
Normal file
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"</think>": 151668,
|
||||
"</tool_call>": 151658,
|
||||
"</tool_response>": 151666,
|
||||
"<think>": 151667,
|
||||
"<tool_call>": 151657,
|
||||
"<tool_response>": 151665,
|
||||
"<|box_end|>": 151649,
|
||||
"<|box_start|>": 151648,
|
||||
"<|endoftext|>": 151643,
|
||||
"<|file_sep|>": 151664,
|
||||
"<|fim_middle|>": 151660,
|
||||
"<|fim_pad|>": 151662,
|
||||
"<|fim_prefix|>": 151659,
|
||||
"<|fim_suffix|>": 151661,
|
||||
"<|im_end|>": 151645,
|
||||
"<|im_start|>": 151644,
|
||||
"<|image_pad|>": 151655,
|
||||
"<|object_ref_end|>": 151647,
|
||||
"<|object_ref_start|>": 151646,
|
||||
"<|quad_end|>": 151651,
|
||||
"<|quad_start|>": 151650,
|
||||
"<|repo_name|>": 151663,
|
||||
"<|video_pad|>": 151656,
|
||||
"<|vision_end|>": 151653,
|
||||
"<|vision_pad|>": 151654,
|
||||
"<|vision_start|>": 151652
|
||||
}
|
||||
4
chat_template.jinja
Normal file
4
chat_template.jinja
Normal file
@@ -0,0 +1,4 @@
|
||||
{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% endif %}{% if system_message is defined %}{{ 'System: ' + system_message + '<|im_end|>' + '
|
||||
' }}{% endif %}{% for message in loop_messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ 'Human: ' + content + '<|im_end|>' + '
|
||||
Assistant:' }}{% elif message['role'] == 'assistant' %}{{ content + '<|im_end|>' + '
|
||||
' }}{% endif %}{% endfor %}
|
||||
62
config.json
Normal file
62
config.json
Normal file
@@ -0,0 +1,62 @@
|
||||
{
|
||||
"architectures": [
|
||||
"Qwen3ForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"eos_token_id": 151645,
|
||||
"head_dim": 128,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 1024,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 3072,
|
||||
"layer_types": [
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention"
|
||||
],
|
||||
"max_position_embeddings": 40960,
|
||||
"max_window_layers": 28,
|
||||
"model_type": "qwen3",
|
||||
"num_attention_heads": 16,
|
||||
"num_hidden_layers": 28,
|
||||
"num_key_value_heads": 8,
|
||||
"pad_token_id": 151654,
|
||||
"rms_norm_eps": 1e-06,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 1000000,
|
||||
"sliding_window": null,
|
||||
"tie_word_embeddings": true,
|
||||
"torch_dtype": "float16",
|
||||
"transformers_version": "4.55.0",
|
||||
"unsloth_fixed": true,
|
||||
"unsloth_version": "2025.4.7",
|
||||
"use_cache": true,
|
||||
"use_sliding_window": false,
|
||||
"vocab_size": 151936
|
||||
}
|
||||
1
configuration.json
Normal file
1
configuration.json
Normal file
@@ -0,0 +1 @@
|
||||
{"framework":"Pytorch","task":"text-generation"}
|
||||
14
generation_config.json
Normal file
14
generation_config.json
Normal file
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"bos_token_id": 151643,
|
||||
"do_sample": true,
|
||||
"eos_token_id": [
|
||||
151645,
|
||||
151643
|
||||
],
|
||||
"max_length": 40960,
|
||||
"pad_token_id": 151654,
|
||||
"temperature": 0.6,
|
||||
"top_k": 20,
|
||||
"top_p": 0.95,
|
||||
"transformers_version": "4.55.0"
|
||||
}
|
||||
BIN
merges.txt
(Stored with Git LFS)
Normal file
BIN
merges.txt
(Stored with Git LFS)
Normal file
Binary file not shown.
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:28dbe8330733912621f876cd0b6b40f3b1d782fc8e261ff863fe3b05e225ebb9
|
||||
size 1192134784
|
||||
31
special_tokens_map.json
Normal file
31
special_tokens_map.json
Normal file
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"eos_token": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<|vision_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
Binary file not shown.
240
tokenizer_config.json
Normal file
240
tokenizer_config.json
Normal file
@@ -0,0 +1,240 @@
|
||||
{
|
||||
"add_bos_token": false,
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"151643": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151644": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151645": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151646": {
|
||||
"content": "<|object_ref_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151647": {
|
||||
"content": "<|object_ref_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151648": {
|
||||
"content": "<|box_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151649": {
|
||||
"content": "<|box_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151650": {
|
||||
"content": "<|quad_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151651": {
|
||||
"content": "<|quad_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151652": {
|
||||
"content": "<|vision_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151653": {
|
||||
"content": "<|vision_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151654": {
|
||||
"content": "<|vision_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151655": {
|
||||
"content": "<|image_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151656": {
|
||||
"content": "<|video_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151657": {
|
||||
"content": "<tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151658": {
|
||||
"content": "</tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151659": {
|
||||
"content": "<|fim_prefix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151660": {
|
||||
"content": "<|fim_middle|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151661": {
|
||||
"content": "<|fim_suffix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151662": {
|
||||
"content": "<|fim_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151663": {
|
||||
"content": "<|repo_name|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151664": {
|
||||
"content": "<|file_sep|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151665": {
|
||||
"content": "<tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151666": {
|
||||
"content": "</tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151667": {
|
||||
"content": "<think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151668": {
|
||||
"content": "</think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"bos_token": null,
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|im_end|>",
|
||||
"errors": "replace",
|
||||
"extra_special_tokens": {},
|
||||
"model_max_length": 40960,
|
||||
"pad_token": "<|vision_pad|>",
|
||||
"padding_side": "left",
|
||||
"split_special_tokens": false,
|
||||
"tokenizer_class": "Qwen2Tokenizer",
|
||||
"unk_token": null
|
||||
}
|
||||
BIN
vocab.json
(Stored with Git LFS)
Normal file
BIN
vocab.json
(Stored with Git LFS)
Normal file
Binary file not shown.
Reference in New Issue
Block a user