初始化项目，由ModelHub XC社区提供模型

Model: abideen/NexoNimbus-7B
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,99 @@
+---
+license: apache-2.0
+tags:
+- merge
+- abideen/DareVox-7B
+- udkai/Garrulus
+language:
+- en
+---
+
+# NexoNimbus-7B
+
+
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/64e380b2e12618b261fa6ba0/9lIzCPqDYR6nnLgoH6kMp.png)
+
+
+NexoNimbus-7B is a merge of the following models:
+* [abideen/DareVox-7B](https://huggingface.co/abideen/DareVox-7B)
+* [udkai/Garrulus](https://huggingface.co/udkai/Garrulus)
+
+🏆 Evaluation
+NexoNimbus-7B is the 5th best-performing 7B LLM on the Open LLM Leaderboard:
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/64e380b2e12618b261fa6ba0/MIkOaXVGJ0T5UVYIEhtYA.png)
+
+
+|    Task     |Version| Metric |Value|   |Stderr|
+|-------------|------:|--------|----:|---|-----:|
+|arc_challenge|      0|acc     |68.25|±  |  1.36|
+|             |       |acc_norm|70.81|±  |  1.38|
+|hellaswag    |      0|acc     |70.86|±  |  0.45|
+|             |       |acc_norm|87.86|±  |  0.32|
+|gsm8k        |      0|acc     |70.35|±  |  1.25|
+|winogrande   |      0|acc     |84.84|±  |  1.00|
+|mmlu         |      0|acc     |64.69|±  |  1.00|
+
+Average: 73.5%
+
+### TruthfulQA
+|    Task     |Version|Metric|Value|   |Stderr|
+|-------------|------:|------|----:|---|-----:|
+|truthfulqa_mc|      1|mc1   |46.26|±  |  1.74|
+|             |       |mc2   |62.42|±  |  1.54|
+
+
+## 🧩 Configuration
+
+```yaml
+slices:
+  - sources:
+      - model: abideen/DareVox-7B 
+        layer_range: [0, 32]
+      - model: udkai/Garrulus
+        layer_range: [0, 32]
+merge_method: slerp
+base_model: abideen/DareVox-7B 
+parameters:
+  t:
+    - filter: self_attn
+      value: [0, 0.5, 0.3, 0.7, 1]
+    - filter: mlp
+      value: [1, 0.5, 0.7, 0.3, 0]
+    - value: 0.5
+dtype: bfloat16
+
+```
+
+## 💻 Usage
+
+Here's a [Colab notebook](https://colab.research.google.com/drive/1F9lzL1IeZRMgiSbY9UbgCR__RreIflJh?usp=sharing) to run NexoNimbus-7B in 4-bit precision on a free T4 GPU.
+
+```python
+!pip install -qU transformers accelerate
+
+from transformers import AutoTokenizer
+import transformers
+import torch
+
+model = "abideen/NexoNimbus-7B"
+messages = [{"role": "user", "content": "Explain what is Machine learning."}]
+
+tokenizer = AutoTokenizer.from_pretrained(model)
+prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+pipeline = transformers.pipeline(
+    "text-generation",
+    model=model,
+    torch_dtype=torch.float16,
+    device_map="auto",
+)
+
+outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
+print(outputs[0]["generated_text"])
+```
+
+"Machine learning is a subfield of artificial intelligence that focuses on developing algorithms and models that allow computers to learn and improve their performance over time, without being explicitly programmed. It involves the use of statistical techniques and data analysis to identify patterns and make predictions based on input data.
+In machine learning, data is fed into a model, which then adjusts its internal parameters to minimize the difference between the predicted output and the actual output. This process is called training, and as the model is exposed to more data, it becomes better at making predictions or classifications.
+Machine learning can be divided into several categories, including supervised learning, unsupervised learning, and reinforcement learning. Supervised learning involves using labeled data, where the desired output is known, and the model learns to map inputs to outputs. Unsupervised learning, on the other hand, does not have a predefined output, and the model learns to identify patterns or relationships within the data. Reinforcement learning involves learning through trial and error, with the model receiving feedback in the form of rewards or penalties based on its actions.
+Some common applications of machine learning include image recognition, natural language processing, recommendation systems, fraud detection, and self-driving."

config.json

@@ -0,0 +1,25 @@
+{
+  "_name_or_path": "abideen/DareVox-7B",
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "rms_norm_eps": 1e-05,
+  "rope_theta": 10000.0,
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.35.2",
+  "use_cache": true,
+  "vocab_size": 32000
+}

mergekit_config.yml

@@ -0,0 +1,18 @@
+
+slices:
+  - sources:
+      - model: abideen/DareVox-7B 
+        layer_range: [0, 32]
+      - model: udkai/Garrulus
+        layer_range: [0, 32]
+merge_method: slerp
+base_model: abideen/DareVox-7B 
+parameters:
+  t:
+    - filter: self_attn
+      value: [0, 0.5, 0.3, 0.7, 1]
+    - filter: mlp
+      value: [1, 0.5, 0.7, 0.3, 0]
+    - value: 0.5
+dtype: bfloat16
+

model-00001-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:509725751c348c8d1a13277327f09c97a101f8246eaea0860fb7b143b1b425d2
+size 1889595352

model-00002-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5153a5ad91d6a74f0c569e532af864823c07735b67d5c2b8f7365f775986948
+size 1979789712

model-00003-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6038ee8f1aee8a8ac9332115f1226367dbcb67019f5308eafe8fa42989cd6baa
+size 1988186784

model-00004-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9269b6e34f57e4931345658d45d98a7c14796e28175430692b231f460f7132e
+size 1937846944

model-00005-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:395ee503efbbf46c16536baf1b415cab2496cb27d1b124b511f43d75b154bda9
+size 1988178496

model-00006-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9552076160f2fe1c0d3d0dc9ce41a3aba85016f5e2c0117632e0eaeb29919ec6
+size 1998655576

model-00007-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6cd5fe52d11f5194b9d6f7cce78435ad4ab890f7b70cc5e7465ac124004bb92d
+size 1946235664

model-00008-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:307a493e603e7b5bea26fe93f58f6519612085291119342c52d848a21229c202
+size 755009456

special_tokens_map.json

@@ -0,0 +1,23 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,40 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": null,
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}