初始化项目，由ModelHub XC社区提供模型

Model: DataPilot/Llama3.1-ArrowSE-v0.4
Source: Original Platform

.gitattributes

@@ -0,0 +1,66 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
+*.tfevents* filter=lfs diff=lfs merge=lfs -text
+*.db* filter=lfs diff=lfs merge=lfs -text
+*.ark* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
+ 
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.gguf* filter=lfs diff=lfs merge=lfs -text
+*.ggml filter=lfs diff=lfs merge=lfs -text
+*.llamafile* filter=lfs diff=lfs merge=lfs -text
+*.pt2 filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+
+model-00003-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00007-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00009-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00012-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00015-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00005-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00002-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00011-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00004-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00006-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00010-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00013-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00008-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00017-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00014-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+model-00016-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00001-of-00017.safetensors filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,105 @@
+---
+base_model:
+- meta-llama/Meta-Llama-3.1-8B-Instruct
+- elyza/Llama-3-ELYZA-JP-8B
+- nvidia/Llama3-ChatQA-1.5-8B
+library_name: transformers
+tags:
+- mergekit
+- merge
+language:
+- ja
+license: llama3
+---
+
+## 概要　
+
+このモデルはllama3.1-8B-instructをもとに日本語性能を高めることを目的にMergekit&ファインチューニングを用いて作成されました。
+
+meta,ELYZA,nvidiaの皆様に感謝します。
+
+## how to use
+
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+DEFAULT_SYSTEM_PROMPT = "あなたは誠実で優秀な日本人のアシスタントです。特に指示が無い場合は、常に日本語で回答してください。"
+text = "Vtuberとして成功するために大切な5つのことを小学生にでもわかるように教えてください。"
+
+model_name = "DataPilot/Llama3.1-ArrowSE-v0.4"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto",
+)
+model.eval()
+
+messages = [
+    {"role": "system", "content": DEFAULT_SYSTEM_PROMPT},
+    {"role": "user", "content": text},
+]
+prompt = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+token_ids = tokenizer.encode(
+    prompt, add_special_tokens=False, return_tensors="pt"
+)
+
+with torch.no_grad():
+    output_ids = model.generate(
+        token_ids.to(model.device),
+        max_new_tokens=1200,
+        do_sample=True,
+        temperature=0.6,
+        top_p=0.9,
+    )
+output = tokenizer.decode(
+    output_ids.tolist()[0][token_ids.size(1):], skip_special_tokens=True
+)
+print(output)
+```
+
+
+## merge
+
+This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
+
+## Merge Details
+### Merge Method
+
+This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using meta-llama/Meta-Llama-3.1-8B-Instruct as a base.
+
+### Models Merged
+
+The following models were included in the merge:
+* [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)
+* [elyza/Llama-3-ELYZA-JP-8B](https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B)
+* [nvidia/Llama3-ChatQA-1.5-8B](https://huggingface.co/nvidia/Llama3-ChatQA-1.5-8B)
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+models:
+  - model: meta-llama/Meta-Llama-3.1-8B-Instruct
+    parameters:
+      weight: 1
+  - model: elyza/Llama-3-ELYZA-JP-8B
+    parameters:
+      weight: 0.7
+  - model: nvidia/Llama3-ChatQA-1.5-8B
+    parameters:
+      weight: 0.15
+merge_method: ties
+base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
+parameters:
+  normalize: false
+dtype: bfloat16
+```

config.json

@@ -0,0 +1,39 @@
+{
+  "_name_or_path": "./llama3.1-8B",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 8.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.43.1",
+  "use_cache": true,
+  "vocab_size": 128256
+}

configuration.json

@@ -0,0 +1 @@
+{"framework": "pytorch", "task": "text-generation", "allow_remote": true}

mergekit_config.yml

@@ -0,0 +1,15 @@
+models:
+  - model: ./llama3.1-8B
+    parameters:
+      weight: 1
+  - model: elyza/Llama-3-ELYZA-JP-8B
+    parameters:
+      weight: 0.7
+  - model: nvidia/Llama3-ChatQA-1.5-8B
+    parameters:
+      weight: 0.15
+merge_method: ties
+base_model: ./llama3.1-8B
+parameters:
+  normalize: false
+dtype: bfloat16

model-00001-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c10418a2f0a50b1d5d9d84bb8181456685eaa7ab4dbab7a4224458bb8d3682e
+size 1050673280

model-00002-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ef0d6429c3ad82a9a66fea43ab17558ba6c7e88b30199b8484cff20c00edd185
+size 1050673296

model-00003-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9ebfe14d01b4f1a6629381e8c467d2b1ddb4de77af865db678451a77abe9f7a
+size 989899000

model-00004-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:096e27347959fc9a4cbe6b253783506525bdce54c936a8a3d78dcdba737191d2
+size 989890728

model-00005-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7bffdb744507d08dcb31f196b0a4528d16f74a6801d52de359242cb4e9055b50
+size 998287760

model-00006-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2ca5e2d14e8d9f27ae9d8aa77253ce06ef9192e39bb8b8cafd1966ca183b9967
+size 947956216

model-00007-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b241dc9dd2c27457fe41db2df3fc7ba1519c74f742abd0fd5cd1df2483d39488
+size 989890728

model-00008-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f94d7ac3e70fd4facea8274f03eb8f15121bc9823489d5c0d2a40d99277823b
+size 989890720

model-00009-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2732f53532b41d1b7db3dd94823889cacfdf6f484c0292e6529d56524fdd243
+size 998287760

model-00010-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:167cf7ddb8f480bdd3e6fce59c6efe6acd6d78abf9715e5ff7608b69575f8946
+size 947956216

model-00011-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab0faaf2ed11f6daf4527056e8f4d6736dffd4a1c41f8220525d8509ed557c1f
+size 989890728

model-00012-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b25bb48e9edd32a34bb1adbdbbb2f6647813514b5e2966f0f50a3744ddf2c685
+size 989890728

model-00013-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a665dd3d5deeac2796a7e14f4c273d47c2a4786f000059ad529dff41060b3dac
+size 998287752

model-00014-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aaeec1d4bcb8ddcee251db2008dfc1bcc207b37b98cf7cffa3f86da3689859a2
+size 947956208

model-00015-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76deded112fa3c1878d31df5ec4ab41e5641717e4d5eeb3d0a9afb734440c708
+size 989890712

model-00016-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75fd387ce27607f3b9e3e7eda14b1d94882b20eb8857c2742068f37a2ced0ee7
+size 989890712

model-00017-of-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60a0484f3275c72de671ebc52aaac7defd6dc76aae1db4b5423289c72633d6a7
+size 201343792

special_tokens_map.json

@@ -0,0 +1,4 @@
+{
+  "bos_token": "<|begin_of_text|>",
+  "eos_token": "<|eot_id|>"
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bbc1904d35169c542dffbe1f7589a5994ec7426d9e5b609d07bab876f32e97ab
+size 9084449