105 lines
2.7 KiB
Markdown
105 lines
2.7 KiB
Markdown
|
---
|
||
|
|
base_model:
|
||
|
|
- meta-llama/Meta-Llama-3.1-8B-Instruct
|
||
|
|
- elyza/Llama-3-ELYZA-JP-8B
|
||
|
|
- nvidia/Llama3-ChatQA-1.5-8B
|
||
|
|
library_name: transformers
|
||
|
|
tags:
|
||
|
|
- mergekit
|
||
|
|
- merge
|
||
|
|
language:
|
||
|
|
- ja
|
||
|
|
license: llama3
|
||
|
|
---
|
||
|
|
|
||
|
|
## 概要
|
||
|
|
|
||
|
|
このモデルはllama3.1-8B-instructをもとに日本語性能を高めることを目的にMergekit&ファインチューニングを用いて作成されました。
|
||
|
|
|
||
|
|
meta,ELYZA,nvidiaの皆様に感謝します。
|
||
|
|
|
||
|
|
## how to use
|
||
|
|
|
||
|
|
|
||
|
|
```python
|
||
|
|
import torch
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
|
||
|
|
DEFAULT_SYSTEM_PROMPT = "あなたは誠実で優秀な日本人のアシスタントです。特に指示が無い場合は、常に日本語で回答してください。"
|
||
|
|
text = "Vtuberとして成功するために大切な5つのことを小学生にでもわかるように教えてください。"
|
||
|
|
|
||
|
|
model_name = "DataPilot/Llama3.1-ArrowSE-v0.4"
|
||
|
|
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
||
|
|
model_name,
|
||
|
|
torch_dtype="auto",
|
||
|
|
device_map="auto",
|
||
|
|
)
|
||
|
|
model.eval()
|
||
|
|
|
||
|
|
messages = [
|
||
|
|
{"role": "system", "content": DEFAULT_SYSTEM_PROMPT},
|
||
|
|
{"role": "user", "content": text},
|
||
|
|
]
|
||
|
|
prompt = tokenizer.apply_chat_template(
|
||
|
|
messages,
|
||
|
|
tokenize=False,
|
||
|
|
add_generation_prompt=True
|
||
|
|
)
|
||
|
|
token_ids = tokenizer.encode(
|
||
|
|
prompt, add_special_tokens=False, return_tensors="pt"
|
||
|
|
)
|
||
|
|
|
||
|
|
with torch.no_grad():
|
||
|
|
output_ids = model.generate(
|
||
|
|
token_ids.to(model.device),
|
||
|
|
max_new_tokens=1200,
|
||
|
|
do_sample=True,
|
||
|
|
temperature=0.6,
|
||
|
|
top_p=0.9,
|
||
|
|
)
|
||
|
|
output = tokenizer.decode(
|
||
|
|
output_ids.tolist()[0][token_ids.size(1):], skip_special_tokens=True
|
||
|
|
)
|
||
|
|
print(output)
|
||
|
|
```
|
||
|
|
|
||
|
|
|
||
|
|
## merge
|
||
|
|
|
||
|
|
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
||
|
|
|
||
|
|
## Merge Details
|
||
|
|
### Merge Method
|
||
|
|
|
||
|
|
This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using meta-llama/Meta-Llama-3.1-8B-Instruct as a base.
|
||
|
|
|
||
|
|
### Models Merged
|
||
|
|
|
||
|
|
The following models were included in the merge:
|
||
|
|
* [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)
|
||
|
|
* [elyza/Llama-3-ELYZA-JP-8B](https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B)
|
||
|
|
* [nvidia/Llama3-ChatQA-1.5-8B](https://huggingface.co/nvidia/Llama3-ChatQA-1.5-8B)
|
||
|
|
|
||
|
|
### Configuration
|
||
|
|
|
||
|
|
The following YAML configuration was used to produce this model:
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
models:
|
||
|
|
- model: meta-llama/Meta-Llama-3.1-8B-Instruct
|
||
|
|
parameters:
|
||
|
|
weight: 1
|
||
|
|
- model: elyza/Llama-3-ELYZA-JP-8B
|
||
|
|
parameters:
|
||
|
|
weight: 0.7
|
||
|
|
- model: nvidia/Llama3-ChatQA-1.5-8B
|
||
|
|
parameters:
|
||
|
|
weight: 0.15
|
||
|
|
merge_method: ties
|
||
|
|
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
|
||
|
|
parameters:
|
||
|
|
normalize: false
|
||
|
|
dtype: bfloat16
|
||
|
|
```
|