初始化项目，由ModelHub XC社区提供模型
Model: zhihu/Zhi-Create-DSR1-14B-GPTQ-INT4 Source: Original Platform
This commit is contained in:
ModelHub XC
49
.gitattributes
vendored
Normal file
49
.gitattributes
vendored
Normal file
@@ -0,0 +1,49 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zstandard filter=lfs diff=lfs merge=lfs -text
|
||||
*.tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
*.db* filter=lfs diff=lfs merge=lfs -text
|
||||
*.ark* filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.gguf* filter=lfs diff=lfs merge=lfs -text
|
||||
*.ggml filter=lfs diff=lfs merge=lfs -text
|
||||
*.llamafile* filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
model-00001-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
model-00002-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
230
README.md
Normal file
230
README.md
Normal file
@@ -0,0 +1,230 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
datasets:
|
||||
- Congliu/Chinese-DeepSeek-R1-Distill-data-110k
|
||||
- cognitivecomputations/dolphin-r1
|
||||
- open-thoughts/OpenThoughts-114k
|
||||
- qihoo360/Light-R1-SFTData
|
||||
- qihoo360/Light-R1-DPOData
|
||||
language:
|
||||
- zh
|
||||
- en
|
||||
base_model:
|
||||
- deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
|
||||
tags:
|
||||
- qwen2
|
||||
library_name: transformers
|
||||
---
|
||||
# Zhi-Create-DSR1-14B
|
||||
|
||||
## 1. Introduction
|
||||
|
||||
Zhi-Create-DSR1-14B is a fine-tuned model based on DeepSeek-R1-Distill-Qwen-14B, specifically optimized for enhanced creative writing capabilities. Several benchmark evaluations indicate the model's improved creative writing performance.
|
||||
|
||||
In the [LLM Creative Story-Writing Benchmark](https://github.com/lechmazur/writing), the model achieved a score of **8.33** compared to its base model's **7.8**. In the [WritingBench](https://github.com/X-PLUG/WritingBench) evaluation framework, it scored **8.46**, showing improvement over DeepSeek-R1-Distill-Qwen-14B's **7.93**. The model was also evaluated using GPT-4o on the AlpacaEval dataset, achieving an **82.6%** win rate when compared with the base model.
|
||||
|
||||
The figure below shows the performance comparison across different domains in WritingBench:
|
||||
|
||||
|
||||
|
||||
<figcaption style="text-align:center; font-size:0.9em; color:#666">
|
||||
Figure 1: WritingBench performance of Zhi-Create-DSR1-14B and DeepSeek-R1-Distill-Qwen-14B across 6 domains and 3 writing requirements evaluated with WritingBench critic model (scale: 1-10). The six domains include: (D1) Academic & Engineering, (D2) Finance & Business, (D3) Politics & Law, (D4) Literature & Art, (D5) Education, and (D6) Advertising & Marketing. The three writing requirements assessed are: (R1) Style, (R2) Format, and (R3) Length. Here, "C" indicates category-specific scores.
|
||||
</figcaption>
|
||||
|
||||
## 2. Training Process
|
||||
|
||||
### Data
|
||||
|
||||
The model's training corpus comprises three primary data sources: rigorously filtered open-source datasets, chain-of-thought reasoning corpora, and curated question-answer pairs from Zhihu.
|
||||
|
||||
To achieve optimal domain coverage, we meticulously balanced the distribution of various datasets, including [Dolphin-r1](https://huggingface.co/datasets/cognitivecomputations/dolphin-r1), [Congliu/Chinese-DeepSeek-R1-Distill-data-110k](https://huggingface.co/datasets/Congliu/Chinese-DeepSeek-R1-Distill-data-110k), [OpenThoughts-114k](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k), [Light-R1-SFTData](https://huggingface.co/datasets/qihoo360/Light-R1-SFTData), and [Light-R1-DPOData](https://huggingface.co/datasets/qihoo360/Light-R1-DPOData), alongside high-quality content from Zhihu. All datasets underwent comprehensive quality assurance through our Reward Model (RM) filtering pipeline.
|
||||
|
||||
### Training
|
||||
**Supervised Fine-tuning (SFT)**: We employed a curriculum learning strategy for supervised fine-tuning. This methodical approach systematically enhances creative writing capabilities while incorporating diverse domain data to maintain core competencies and mitigate catastrophic forgetting.
|
||||
|
||||
**Direct Preference Optimization (DPO)**: For scenarios involving minimal edit distances, we utilized Step-DPO ([arxiv:2406.18629](https://arxiv.org/abs/2406.18629)) to selectively penalize incorrect tokens, while incorporating positive constraints in the loss function as proposed in DPOP ([arXiv:2402.13228](https://arxiv.org/abs/2402.13228)).
|
||||
|
||||
## 3. Evaluation Results
|
||||
|
||||
Our evaluation results suggest promising improvements in the model's creative writing capabilities. In the LLM Creative Story-Writing Benchmark evaluation, the model achieved a score of **8.33**, showing an improvement from the base model's **7.87**. When assessed on WritingBench, a comprehensive framework for evaluating large language model writing abilities, the model attained a score of **8.46**. This places it in proximity to DeepSeek-R1's performance and represents an advancement over DeepSeek-R1-Distill-Qwen-14B's score of **7.93**.
|
||||
|
||||
With respect to general capabilities, evaluations indicate modest improvements of **2%–5% in knowledge and reasoning tasks (CMMLU, MMLU-Pro)**, alongside encouraging progress in mathematical reasoning as measured by benchmarks such as **AIME-2024, AIME-2025, and GSM8K**. The results suggest that the model maintains a balanced performance profile, with improvements observed across creative writing, knowledge/reasoning, and mathematical tasks compared to DeepSeek-R1-Distill-Qwen-14B. These characteristics potentially make it suitable for a range of general-purpose applications. We conducted additional evaluations on the instruction-following ifeval benchmark, with experimental results demonstrating a performance improvement in model capabilities from an initial score of **71.43** to an enhanced score of **74.71**.
|
||||
|
||||
|
||||
|
||||
<figcaption style="text-align:center; font-size:0.9em; color:#666">
|
||||
Figure 2: When evaluating model performance, it is recommended to conduct multiple tests and average the results. (We use n=16 and max_tokens=32768 for mathematical tasks and n=2 for others)
|
||||
</figcaption>
|
||||
|
||||
## 4. How to Run Locally
|
||||
|
||||
Zhi-Create-DSR1-14B can be deployed on various hardware configurations, including GPUs with 80GB memory, a single H20/A800/H800, or dual RTX 4090. Additionally, the INT4 quantized version Zhi-Create-DSR1-14B-GPTQ-INT4 can be deployed on a single RTX 4090.
|
||||
|
||||
### Transformers
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
from transformers.generation import GenerationConfig
|
||||
|
||||
MODEL_NAME = "Zhihu-ai/Zhi-Create-DSR1-14B"
|
||||
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, trust_remote_code=True)
|
||||
|
||||
# use bf16
|
||||
# model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto", trust_remote_code=True, bf16=True).eval()
|
||||
# use fp16
|
||||
# model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto", trust_remote_code=True, fp16=True).eval()
|
||||
# use cpu only
|
||||
# model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="cpu", trust_remote_code=True).eval()
|
||||
# use auto mode, automatically select precision based on the device.
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
MODEL_NAME,
|
||||
device_map="auto",
|
||||
trust_remote_code=True
|
||||
).eval()
|
||||
|
||||
# Specify hyperparameters for generation. But if you use transformers>=4.32.0, there is no need to do this.
|
||||
# model.generation_config = GenerationConfig.from_pretrained(MODEL_NAME, trust_remote_code=True)
|
||||
|
||||
generate_configs = {
|
||||
"temperature": 0.6,
|
||||
"do_sample": True,
|
||||
"top_p": 0.95,
|
||||
"max_new_tokens": 4096
|
||||
}
|
||||
|
||||
prompt = "请你以鲁迅的口吻，写一篇介绍西湖醋鱼的文章"
|
||||
messages = [
|
||||
{"role": "user", "content": prompt}
|
||||
]
|
||||
text = tokenizer.apply_chat_template(
|
||||
messages,
|
||||
tokenize=False,
|
||||
add_generation_prompt=True
|
||||
)
|
||||
|
||||
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
||||
|
||||
generated_ids = model.generate(
|
||||
**model_inputs,
|
||||
**generate_configs
|
||||
)
|
||||
generated_ids = [
|
||||
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
|
||||
]
|
||||
|
||||
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
||||
print(response)
|
||||
```
|
||||
|
||||
### ZhiLight
|
||||
|
||||
You can easily start a service using [ZhiLight](https://github.com/zhihu/ZhiLight)
|
||||
|
||||
```bash
|
||||
docker run -it --net=host --gpus='"device=0"' -v /path/to/model:/mnt/models --entrypoints="" ghcr.io/zhihu/zhilight/zhilight:0.4.17-cu124 python -m zhilight.server.openai.entrypoints.api_server --model-path /mnt/models --port 8000 --enable-reasoning --reasoning-parser deepseek-r1 --served-model-name Zhi-Create-DSR1-14B
|
||||
|
||||
curl http://localhost:8000/v1/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "Zhi-Create-DSR1-14B",
|
||||
"prompt": "请你以鲁迅的口吻，写一篇介绍西湖醋鱼的文章",
|
||||
"max_tokens": 4096,
|
||||
"temperature": 0.6,
|
||||
"top_p": 0.95
|
||||
}'
|
||||
```
|
||||
|
||||
### vllm
|
||||
|
||||
For instance, you can easily start a service using [vLLM](https://github.com/vllm-project/vllm)
|
||||
|
||||
```bash
|
||||
# install vllm
|
||||
pip install vllm>=0.6.4.post1
|
||||
|
||||
# huggingface model id
|
||||
vllm serve Zhihu-ai/Zhi-Create-DSR1-14B --served-model-name Zhi-Create-DSR1-14B --port 8000
|
||||
|
||||
# local path
|
||||
vllm serve /path/to/model --served-model-name Zhi-Create-DSR1-14B --port 8000
|
||||
|
||||
curl http://localhost:8000/v1/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "Zhi-Create-DSR1-14B",
|
||||
"prompt": "请你以鲁迅的口吻，写一篇介绍西湖醋鱼的文章",
|
||||
"max_tokens": 4096,
|
||||
"temperature": 0.6,
|
||||
"top_p": 0.95
|
||||
}'
|
||||
```
|
||||
|
||||
|
||||
### SGLang
|
||||
|
||||
You can also easily start a service using [SGLang](https://github.com/sgl-project/sglang)
|
||||
|
||||
```bash
|
||||
# install SGLang
|
||||
pip install "sglang[all]>=0.4.5" --find-links https://flashinfer.ai/whl/cu124/torch2.5/flashinfer-python
|
||||
|
||||
# huggingface model id
|
||||
python -m sglang.launch_server --model-path Zhihu-ai/Zhi-Create-DSR1-14B --served-model-name Zhi-Create-DSR1-14B --port 8000
|
||||
|
||||
# local path
|
||||
python -m sglang.launch_server --model-path /path/to/model --served-model-name Zhi-Create-DSR1-14B --port 8000
|
||||
|
||||
# send request
|
||||
curl http://localhost:8000/v1/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "Zhi-Create-DSR1-14B",
|
||||
"prompt": "请你以鲁迅的口吻，写一篇介绍西湖醋鱼的文章",
|
||||
"max_tokens": 4096,
|
||||
"temperature": 0.6,
|
||||
"top_p": 0.95
|
||||
}'
|
||||
```
|
||||
|
||||
### ollama
|
||||
|
||||
You can download ollama using [this](https://ollama.com/download/)
|
||||
|
||||
* quantization: Q4_K_M
|
||||
|
||||
```bash
|
||||
ollama run zhihu/zhi-create-dsr1-14b
|
||||
```
|
||||
|
||||
* bf16
|
||||
|
||||
```bash
|
||||
ollama run zhihu/zhi-create-dsr1-14b:bf16
|
||||
```
|
||||
|
||||
## 5. Usage Recommendations
|
||||
|
||||
We recommend adhering to the following configurations when utilizing the Zhi-Create-DSR1-14B, including benchmarking, to achieve the expected performance:
|
||||
|
||||
* Set the temperature within the range of 0.5-0.7 (0.6 is recommended) to prevent endless repetitions or incoherent outputs.
|
||||
|
||||
* When evaluating model performance, it is recommended to conduct multiple tests and average the results. (We use `n=16` and `max_tokens=32768` for mathematical tasks and `n=2` for others)
|
||||
|
||||
* To ensure that the model engages in thorough reasoning like DeepSeek-R1 series models, we recommend enforcing the model to initiate its response with "\<think\>\n" at the beginning of every output.
|
||||
|
||||
## 6. Citation
|
||||
|
||||
```text
|
||||
@misc{Zhi-Create-DSR1-14B,
|
||||
title={Zhi-Create-DSR1-14B: Curriculum Reinforcement and Direct Preference Optimization for Robust Creative Writing in LLMs},
|
||||
author={Jiewu Wang, Xu Chen, Wenyuan Su, Chao Huang, Hongkui Gao, Lin Feng, Shan Wang, Lu Xu, Penghe Liu, Zebin Ou},
|
||||
year={2025},
|
||||
eprint={},
|
||||
archivePrefix={},
|
||||
url={https://huggingface.co/Zhihu-ai/Zhi-Create-DSR1-14B},
|
||||
}
|
||||
```
|
||||
|
||||
## 7. Contact
|
||||
|
||||
If you have any questions, please raise an issue or contact us at [ai@zhihu.com](mailto:ai@zhihu.com).
|
||||
47
config.json
Normal file
47
config.json
Normal file
@@ -0,0 +1,47 @@
|
||||
{
|
||||
"architectures": [
|
||||
"Qwen2ForCausalLM"
|
||||
],
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 151643,
|
||||
"eos_token_id": 151643,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 5120,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 13824,
|
||||
"max_position_embeddings": 131072,
|
||||
"max_window_layers": 48,
|
||||
"model_type": "qwen2",
|
||||
"num_attention_heads": 40,
|
||||
"num_hidden_layers": 48,
|
||||
"num_key_value_heads": 8,
|
||||
"pad_token_id": 151643,
|
||||
"quantization_config": {
|
||||
"bits": 4,
|
||||
"block_name_to_quantize": "model.layers",
|
||||
"checkpoint_format": "gptq",
|
||||
"damp_percent": 0.1,
|
||||
"desc_act": false,
|
||||
"group_size": 128,
|
||||
"meta": {
|
||||
"quantizer": [
|
||||
"optimum:1.24.0",
|
||||
"auto_gptq:0.7.1"
|
||||
]
|
||||
},
|
||||
"modules_in_block_to_quantize": null,
|
||||
"quant_method": "gptq",
|
||||
"sym": true,
|
||||
"true_sequential": true
|
||||
},
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 1000000.0,
|
||||
"sliding_window": null,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "float16",
|
||||
"transformers_version": "4.50.0",
|
||||
"use_cache": false,
|
||||
"use_sliding_window": false,
|
||||
"vocab_size": 152064
|
||||
}
|
||||
1
configuration.json
Normal file
1
configuration.json
Normal file
@@ -0,0 +1 @@
|
||||
{"framework":"Pytorch","task":"text-generation"}
|
||||
9
generation_config.json
Normal file
9
generation_config.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"bos_token_id": 151646,
|
||||
"eos_token_id": 151643,
|
||||
"pad_token_id": 151643,
|
||||
"max_new_tokens": 4096,
|
||||
"temperature": 0.6,
|
||||
"top_p": 0.95,
|
||||
"transformers_version": "4.49.0"
|
||||
}
|
||||
BIN
images/general_score.png
Normal file
BIN
images/general_score.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 211 KiB |
BIN
images/writingbench_score.png
Normal file
BIN
images/writingbench_score.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 134 KiB |
3
model-00001-of-00002.safetensors
Normal file
3
model-00001-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7fb3e7047fef96067d38638f351c9ac739fcc7304b544cd30298b2a43e94284f
|
||||
size 4997121424
|
||||
3
model-00002-of-00002.safetensors
Normal file
3
model-00002-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:64d3049455d0737f8337e2921a0e2038c4df51688986fbc2de14f2eb4218552f
|
||||
size 4991638208
|
||||
1594
model.safetensors.index.json
Normal file
1594
model.safetensors.index.json
Normal file
File diff suppressed because it is too large
Load Diff
19
quantize_config.json
Normal file
19
quantize_config.json
Normal file
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"bits": 4,
|
||||
"dataset": "",
|
||||
"group_size": 128,
|
||||
"damp_percent": 0.1,
|
||||
"desc_act": false,
|
||||
"sym": true,
|
||||
"true_sequential": true,
|
||||
"quant_method": "gptq",
|
||||
"modules_in_block_to_quantize": null,
|
||||
"checkpoint_format": "gptq",
|
||||
"meta": {
|
||||
"quantizer": [
|
||||
"optimum:1.24.0",
|
||||
"auto_gptq:0.7.1"
|
||||
]
|
||||
},
|
||||
"block_name_to_quantize": "model.layers"
|
||||
}
|
||||
23
special_tokens_map.json
Normal file
23
special_tokens_map.json
Normal file
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<｜begin▁of▁sentence｜>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<｜end▁of▁sentence｜>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<｜end▁of▁sentence｜>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
757492
tokenizer.json
Normal file
757492
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
199
tokenizer_config.json
Normal file
199
tokenizer_config.json
Normal file
@@ -0,0 +1,199 @@
|
||||
{
|
||||
"add_bos_token": true,
|
||||
"add_eos_token": false,
|
||||
"add_prefix_space": null,
|
||||
"added_tokens_decoder": {
|
||||
"151643": {
|
||||
"content": "<｜end▁of▁sentence｜>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151644": {
|
||||
"content": "<｜User｜>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151645": {
|
||||
"content": "<｜Assistant｜>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151646": {
|
||||
"content": "<｜begin▁of▁sentence｜>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151647": {
|
||||
"content": "<|EOT|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151648": {
|
||||
"content": "<think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151649": {
|
||||
"content": "</think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151650": {
|
||||
"content": "<|quad_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151651": {
|
||||
"content": "<|quad_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151652": {
|
||||
"content": "<|vision_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151653": {
|
||||
"content": "<|vision_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151654": {
|
||||
"content": "<|vision_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151655": {
|
||||
"content": "<|image_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151656": {
|
||||
"content": "<|video_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151657": {
|
||||
"content": "<tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151658": {
|
||||
"content": "</tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151659": {
|
||||
"content": "<|fim_prefix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151660": {
|
||||
"content": "<|fim_middle|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151661": {
|
||||
"content": "<|fim_suffix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151662": {
|
||||
"content": "<|fim_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151663": {
|
||||
"content": "<|repo_name|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151664": {
|
||||
"content": "<|file_sep|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
}
|
||||
},
|
||||
"bos_token": "<｜begin▁of▁sentence｜>",
|
||||
"chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='') %}{%- for message in messages %}{%- if message['role'] == 'system' %}{% set ns.system_prompt = message['content'] %}{%- endif %}{%- endfor %}{{bos_token}}{{ns.system_prompt}}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{{'<｜User｜>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is none %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls']%}{%- if not ns.is_first %}{{'<｜Assistant｜><｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<｜tool▁call▁end｜>'}}{%- set ns.is_first = true -%}{%- else %}{{'\\n' + '<｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<｜tool▁call▁end｜>'}}{{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}{%- endif %}{%- endfor %}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is not none %}{%- if ns.is_tool %}{{'<｜tool▁outputs▁end｜>' + message['content'] + '<｜end▁of▁sentence｜>'}}{%- set ns.is_tool = false -%}{%- else %}{% set content = message['content'] %}{% if '</think>' in content %}{% set content = content.split('</think>')[-1] %}{% endif %}{{'<｜Assistant｜>' + content + '<｜end▁of▁sentence｜>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_tool = true -%}{%- if ns.is_output_first %}{{'<｜tool▁outputs▁begin｜><｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- set ns.is_output_first = false %}{%- else %}{{'\\n<｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- endif %}{%- endif %}{%- endfor -%}{% if ns.is_tool %}{{'<｜tool▁outputs▁end｜>'}}{% endif %}{% if add_generation_prompt and not ns.is_tool %}{{'<｜Assistant｜><think>\\n'}}{% endif %}",
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<｜end▁of▁sentence｜>",
|
||||
"extra_special_tokens": {},
|
||||
"legacy": true,
|
||||
"max_length": 8096,
|
||||
"model_max_length": 16384,
|
||||
"pad_token": "<｜end▁of▁sentence｜>",
|
||||
"sp_model_kwargs": {},
|
||||
"stride": 0,
|
||||
"tokenizer_class": "LlamaTokenizerFast",
|
||||
"truncation_side": "right",
|
||||
"truncation_strategy": "longest_first",
|
||||
"unk_token": null,
|
||||
"use_default_system_prompt": false
|
||||
}
|
||||
Reference in New Issue
Block a user