初始化项目，由ModelHub XC社区提供模型

Model: Dampfinchen/Llama-3-8B-Ultra-Instruct
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,120 @@
+---
+base_model:
+- Undi95/Meta-Llama-3-8B-Instruct-hf
+- Undi95/Llama-3-LewdPlay-8B-evo
+- jondurbin/bagel-8b-v1.0
+- Weyaxi/Einstein-v6.1-Llama3-8B
+- VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
+- aaditya/OpenBioLLM-Llama3-8B
+library_name: transformers
+tags:
+- mergekit
+- merge
+license: llama3
+---
+# Llama-3-8B-Ultra-Instruct
+
+This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
+
+Hello everyone this is Dampf, creator of the Destroyer series! 
+
+This time, I'm introducing you to 8B-Ultra-Instruct. It is a small general purpose model that combines the most powerful instruct models with enticing roleplaying models. It will introduce better RAG capabilities in the form of Bagel to Llama 3 8B Instruct as well as German multilanguage, higher general intelligence and vision support. A model focused on Biology adds knowledge in the medical field.
+
+As for roleplay, it features two of the hottest models right now. Those are known to be high quality and for being uncensored. So this model might put out harmful responses. We are not responsible for what you do with this model and please take everything the model says with a huge grain of salt.
+Lastly, you might notice I'm conversative with the weight values in the final merge. This is because I believe L8B Instruct is a very dense model that's already great and doesn't need a lot more data. So instead of reaching the weight value of 1 in a ties merge, I'm only using a total of 0,65. This is to preserve Llama Instruct's intelligence and knowledge, while adding a little bit of the aforementioned models as salt in the soup. 
+
+A huge thank you for all the creators of the datasets. Those include Undi95, Jon Durbin, Aaditya, VAGOsolutions, Teknium, Camel and many more. They deserve all the credit. And of course, thank you Elinas for providing the compute. 
+
+## Quants
+[GGUF](https://huggingface.co/bartowski/Llama-3-8B-Ultra-Instruct-GGUF)
+
+[exllama2](https://huggingface.co/bartowski/Llama-3-8B-Ultra-Instruct-exl2)
+
+## Merge Details
+### Merge Method
+
+This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [Undi95/Meta-Llama-3-8B-Instruct-hf](https://huggingface.co/Undi95/Meta-Llama-3-8B-Instruct-hf) as a base.
+
+### Models Merged
+
+The following models were included in the merge:
+* Undi95/Meta-Llama-3-8B-Instruct-hf
+* Undi95/Llama-3-LewdPlay-8B-evo
+* jondurbin/bagel-8b-v1.0
+* Weyaxi/Einstein-v6.1-Llama3-8B
+* VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
+* aaditya/OpenBioLLM-Llama3-8B
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+models:
+  - model: ChaoticNeutrals/Poppy_Porpoise-v0.7-L3-8B
+    parameters:
+      weight: 0.4
+  - model: Undi95/Llama-3-LewdPlay-8B-evo
+    parameters:
+      weight: 0.5
+  - model: jondurbin/bagel-8b-v1.0
+    parameters:
+      weight: 0.1
+merge_method: dare_ties
+dtype: bfloat16
+base_model: Undi95/Meta-Llama-3-8B-hf
+name: RPPart
+---
+
+models:
+  - model: Weyaxi/Einstein-v6.1-Llama3-8B
+    parameters:
+      weight: 0.6
+  - model: VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
+    parameters:
+      weight: 0.3
+  - model: aaditya/OpenBioLLM-Llama3-8B
+    parameters:
+      weight: 0.1
+merge_method: dare_ties
+base_model: Undi95/Meta-Llama-3-8B-hf
+dtype: bfloat16
+name: InstructPart
+---
+
+models:
+  - model: RPPart
+    parameters:
+      weight: 0.39
+  - model: InstructPart
+    parameters:
+      weight: 0.26
+merge_method: dare_ties
+base_model: Undi95/Meta-Llama-3-8B-Instruct-hf
+dtype: bfloat16
+name: Llama-3-8B-Ultra-Instruct 
+```
+
+### Chat Template (Llama 3 Official)
+```
+<|begin_of_text|><|start_header_id|>system<|end_header_id|>
+
+{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
+
+{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
+
+{output}<|eot_id|>
+```
+
+### [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
+Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_elinas__Llama-3-8B-Ultra-Instruct)
+
+|             Metric              |Value|
+|---------------------------------|----:|
+|Avg.                             |69.11|
+|AI2 Reasoning Challenge (25-Shot)|64.59|
+|HellaSwag (10-Shot)              |81.63|
+|MMLU (5-Shot)                    |68.32|
+|TruthfulQA (0-shot)              |52.80|
+|Winogrande (5-shot)              |76.95|
+|GSM8k (5-shot)                   |70.36|

config.json

@@ -0,0 +1,27 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "eos_token_id": 128001,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 8192,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.40.0.dev0",
+  "use_cache": true,
+  "vocab_size": 128256
+}

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "bos_token_id": 128000,
+  "eos_token_id": [128001, 128009],
+  "do_sample": true,
+  "temperature": 0.6,
+  "max_length": 4096,
+  "top_p": 0.9,
+  "transformers_version": "4.40.0.dev0"
+}

mergekit_config.yml

@@ -0,0 +1,43 @@
+models:
+  - model: ChaoticNeutrals/Poppy_Porpoise-v0.7-L3-8B
+    parameters:
+      weight: 0.4
+  - model: Undi95/Llama-3-LewdPlay-8B-evo
+    parameters:
+      weight: 0.5
+  - model: jondurbin/bagel-8b-v1.0
+    parameters:
+      weight: 0.1
+merge_method: dare_ties
+dtype: bfloat16
+base_model: Undi95/Meta-Llama-3-8B-hf
+name: RPPart
+---
+
+models:
+  - model: Weyaxi/Einstein-v6.1-Llama3-8B
+    parameters:
+      weight: 0.6
+  - model: VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
+    parameters:
+      weight: 0.3
+  - model: aaditya/OpenBioLLM-Llama3-8B
+    parameters:
+      weight: 0.1
+merge_method: dare_ties
+base_model: Undi95/Meta-Llama-3-8B-hf
+dtype: bfloat16
+name: InstructPart
+---
+
+models:
+  - model: RPPart
+    parameters:
+      weight: 0.39
+  - model: InstructPart
+    parameters:
+      weight: 0.26
+merge_method: dare_ties
+base_model: Undi95/Meta-Llama-3-8B-Instruct-hf
+dtype: bfloat16
+name: Llama-3-8B-Ultra-Instruct

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff896501d33cae1931c532c7649f82b1e9b56bf5257d8a94fd8abb3286004209
+size 9953405736

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99caf1e7df9a1f0faa826fc9abe50c48257c6c8ab0f78ffb04b5c85ebb71115b
+size 6107150624

special_tokens_map.json

@@ -0,0 +1,4 @@
+{
+  "bos_token": "<|begin_of_text|>",
+  "eos_token": "<|end_of_text|>"
+}