初始化项目，由ModelHub XC社区提供模型

Model: SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP
Source: Original Platform

.gitattributes

@@ -0,0 +1,53 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+ 
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
+*.tfevents* filter=lfs diff=lfs merge=lfs -text
+*.db* filter=lfs diff=lfs merge=lfs -text
+*.ark* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
+ 
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.gguf* filter=lfs diff=lfs merge=lfs -text
+*.ggml filter=lfs diff=lfs merge=lfs -text
+*.llamafile* filter=lfs diff=lfs merge=lfs -text
+*.pt2 filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+
+tokenizer.model filter=lfs diff=lfs merge=lfs -text
+model-00001-of-00004.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00002-of-00004.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00004-of-00004.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00003-of-00004.safetensors filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,152 @@
+---
+language:
+- en
+- he
+license: apache-2.0
+widget:
+  - text: "Zion_Alpha"
+    output:
+      url: https://huggingface.co/SicariusSicariiStuff/Zion_Alpha/resolve/main/Zion_Alpha.png
+---
+
+<div align="center">
+  <b style="font-size: 40px;">Zion_Alpha_Instruction_Tuned_SLERP</b>
+
+
+</div>
+
+
+<img src="https://i.imgur.com/e1LEQ18.png" alt="Zion_Alpha_Instruction_Tuned_SLERP" style="width: 50%; min-width: 400px; display: block; margin: auto;">
+
+
+
+# Model Details
+
+Zion_Alpha is the first **REAL** Hebrew model in the world. This version WAS fine tuned for tasks. I did the finetune using SOTA techniques and using my insights from years of underwater basket weaving. If you wanna offer me a job, just add me on Facebook.
+
+
+# Another world record broken by Zion_Alpha!
+On **June 10th, 2024**, this model achieved the **highest sentiment analysis score in the world** for Hebrew LLMs, with an impressive **70.3**, surpassing even a **35B** model that's five times its size!
+<div align="center">
+<img src="https://i.imgur.com/yg6CJoz.png" alt="Zion_Alpha SNLI Score" style="width: 80%; min-width: 700px; display: block; margin: auto;">
+</div>
+
+
+# Future Plans
+My previous LLM, Zion_Alpha, set a world record on Hugging Face by achieving the highest SNLI score for Hebrew open LLMs at 84.05. The current model, a SLERP merge, achieved a lower SNLI score but still surprised everyone by securing the highest sentiment analysis score of 70.3. This demonstrates significant untapped potential in optimizing the training process, showing that 7B models can deliver far more performance in Hebrew than previously thought possible. This will be my last Hebrew model for a while, as I have other adventures to pursue.
+# Looking for Sponsors
+Since all my work is done on-premises, I am constrained by my current hardware. I would greatly appreciate any support in acquiring an A6000, which would enable me to train significantly larger models much faster.
+
+# Papers?
+Maybe. We'll see. No promises here 🤓
+
+# Contact Details
+I'm not great at self-marketing (to say the least) and don't have any social media accounts. If you'd like to reach out to me, you can email me at sicariussicariistuff@gmail.com. Please note that this email might receive more messages than I can handle, so I apologize in advance if I can't respond to everyone.
+
+# Versions and QUANTS
+- Base model: [FP16](https://huggingface.co/SicariusSicariiStuff/Zion_Alpha)
+- Instruction tuned: [FP16](https://huggingface.co/SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned) | [GGUF](https://huggingface.co/SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_GGUF)
+
+
+# Model architecture
+Based on Mistral 7B. I didn't even bother to alter the tokenizer.
+
+# The recommended prompt setting is Debug-deterministic:
+```
+temperature: 1
+top_p: 1
+top_k: 1
+typical_p: 1
+min_p: 1
+repetition_penalty: 1
+```
+
+# The recommended instruction template is Mistral:
+```
+{%- for message in messages %}
+    {%- if message['role'] == 'system' -%}
+        {{- message['content'] -}}
+    {%- else -%}
+        {%- if message['role'] == 'user' -%}
+            {{-'[INST] ' + message['content'].rstrip() + ' [/INST]'-}}
+        {%- else -%}
+            {{-'' + message['content'] + '</s>' -}}
+        {%- endif -%}
+    {%- endif -%}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{-''-}}
+{%- endif -%}
+```
+# English to hebrew example:
+
+
+<div align="center">
+  <b style="font-size: 40px;">Zion_Alpha English to Hebrew example</b>
+
+
+</div>
+
+
+<img src="https://i.imgur.com/JnTuawF.png" alt="Zion_Alpha" style="width: 40%; min-width: 600px; display: block; margin: auto;">
+
+
+# English to hebrew example:
+
+
+<div align="center">
+  <b style="font-size: 40px;">Zion_Alpha Hebrew to English example</b>
+
+
+</div>
+
+
+<img src="https://i.imgur.com/Wm2igLJ.png" alt="Zion_Alpha" style="width: 40%; min-width: 600px; display: block; margin: auto;">
+
+
+<div align="center">
+  <b style="font-size: 30px;">Unscripted video: live zero shot demonstration at story writing capabilities in Hebrew</b>
+
+[![Zion_Alpha Story writing](https://img.youtube.com/vi/YYKeovnS0do/0.jpg)](https://www.youtube.com/watch?v=YYKeovnS0do)
+</div>
+
+<div align="center">
+  <b style="font-size: 30px;">Zion_Alpha VS Mistral 'Hebrew' Live & unscripted in real time</b>
+
+[![Zion_Alpha Story writing](https://img.youtube.com/vi/YYKeovnS0do/0.jpg)](https://www.youtube.com/watch?v=DQFtx8M2txc)
+</div>
+
+<div align="center">
+  <b style="font-size: 30px;">Zion_Alpha VS Mistral 'Hebrew' Live & unscripted in real time Long text translation</b>
+
+[![Zion_Alpha Story writing](https://img.youtube.com/vi/YYKeovnS0do/0.jpg)](https://www.youtube.com/watch?v=w5fz3Ot6tH8)
+</div>
+
+### History
+The model was originally trained about 2 month after Mistral (v0.1) was released.
+As of 04 June 2024, Zion_Alpha got the **Highest SNLI score in the world** among open source models in Hebrew, surpassing most of the models by a huge margin. (**84.05** score)
+
+<img src="https://huggingface.co/SicariusSicariiStuff/Zion_Alpha/resolve/main/Images/Zion_Alpha.png" alt="Zion_Alpha SNLI Score" style="width: 100%; min-width: 700px; display: block; margin: auto;">
+
+---
+
+## Citation Information
+
+```
+@llm{Zion_Alpha_Instruction_Tuned_SLERP,
+  author = {SicariusSicariiStuff},
+  title = {Zion_Alpha_Instruction_Tuned_SLERP},
+  year = {2024},
+  publisher = {Hugging Face},
+  url = {https://huggingface.co/SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP}
+}
+```
+
+---
+
+### Support
+<img src="https://i.imgur.com/0lHHN95.png" alt="GPUs too expensive" style="width: 10%; min-width: 100px; display: block; margin: left;">
+
+- [My Ko-fi page](https://ko-fi.com/sicarius) ALL donations will go for research resources and compute, every bit counts 🙏🏻
+- [My Patreon](https://patreon.com/TenebraAI) ALL donations will go for research resources and compute, every bit counts 🙏🏻
+

config.json

@@ -0,0 +1,26 @@
+{
+  "_name_or_path": "SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP",
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 32000,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "rms_norm_eps": 1e-05,
+  "rope_theta": 1000000.0,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.40.0.dev0",
+  "use_cache": false,
+  "vocab_size": 32000
+}

configuration.json

@@ -0,0 +1 @@
+{"framework": "pytorch", "task": "text-generation", "allow_remote": true}

model-00001-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9696c8132190cd98b0f13b769380dbd7c170ae9c5b4e387d0023183d7401c19
+size 3972145320

model-00002-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac8c1c2913f03ff80ec845a126d0f68705cfcaa37ec7dd2d56a0850a3b309e98
+size 3967977016

model-00003-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:344eae72800a04f1e407c46bed155e0ec2f1ee012315fdc820efc296efb00586
+size 3926025448

model-00004-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63fcaa83bfdcd991cef795736eb4be395cbbde0f799b1e05fa4ae4c4deacc064
+size 2617350232

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,60 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": true,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32000": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32001": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "bos_token": "<s>",
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false,
+  "use_fast": true
+}