初始化项目，由ModelHub XC社区提供模型

Model: mshojaei77/gemma-2-2b-fa-v2
Source: Original Platform

.gitattributes

@@ -0,0 +1,37 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+gemma-2-2b-fa-v2_q8_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,96 @@
+---
+base_model:
+- mshojaei77/Gemma-2-2b-fa
+library_name: transformers
+tags:
+- mergekit
+- merge
+license: gemma
+datasets:
+- mshojaei77/Persian_sft
+language:
+- fa
+---
+
+# Persian Gemma 2b (v2)
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/6556b1bb85d43542fa1a8f91/NOSivA-mvcG_DhVkJQY3p.png)
+
+This repository hosts **Persian Gemma 2b v2**, an optimized version of the initial Persian Gemma 2b model. This version has undergone further experimental improvements through techniques like **self-merging** to enhance its performance and efficiency for Persian language conversational tasks.
+
+This model builds upon the foundation of Google's Gemma-2-2b-it and the initial fine-tuning efforts of `mshojaei77/Gemma-2b-fa`.
+
+## Key Improvements in v2
+
+* **Optimized Performance:**  This version incorporates techniques to improve the model's performance in generating Persian text and engaging in conversations.
+* **Self-Merged:** The model has been merged with itself, potentially leading to a more robust and coherent representation.
+
+**Important Note:** This is still an **experimental model** and is under active development. While optimizations have been applied, it's crucial to understand that it retains the limitations inherent in a 2 billion parameter model and the early-stage nature of this project.  **Output quality may vary, and critical evaluation is still necessary.**
+
+## How to Use
+
+You can use this model just like the original `mshojaei77/Gemma-2b-fa`, using the `transformers` library and the `pipeline`.
+
+```python
+import torch
+from transformers import pipeline
+
+# Initialize the text generation pipeline
+pipe = pipeline(
+    "text-generation",
+    model="mshojaei77/gemma-2-2b-fa-v2",
+    model_kwargs={"torch_dtype": torch.bfloat16},
+    device="cuda",  # Or "mps" for Macs with Apple Silicon
+)
+
+# Prepare input messages (using the gemma chat template implicitly)
+messages = [
+    {"role": "user", "content": "سلام چطوری؟"},
+]
+
+# Generate a response with a maximum of 512 new tokens
+outputs = pipe(messages, max_new_tokens=512, chat_template="gemma") # Explicitly using chat_template for clarity
+assistant_response = outputs[0]["generated_text"][-1]["content"].strip()
+
+print(assistant_response)
+# Example Output (Illustrative - Output quality may vary significantly):
+# سلام! من خوبم، ممنون. شما چطوری؟ 😊
+```
+
+**Key Usage Notes:**
+
+* **`model="mshojaei77/gemma-2-2b-fa-v2"`:**  Make sure to specify the correct model name (`v2`) in your code.
+* **`chat_template="gemma"`:**  Continue to use the `gemma` chat template for proper formatting.
+* **Experimental Nature:** Expect variable output quality. This is still an experimental model.
+
+## Model Details (v2)
+
+* **Base Model:** google/gemma-2-2b-it
+* **Previous Version:** mshojaei77/Gemma-2b-fa
+* **Optimization Techniques:** Self-Merging, Further Optimization.
+* **Architecture:** Gemma2ForCausalLM (same as base model)
+* **Model Size:** 2 billion parameters
+
+## Intended Use
+
+This model is intended for:
+
+* **Research and Experimentation:**  Exploring the impact of Self-Merging techniques on Persian Gemma models.
+* **Educational Purposes:** Demonstrating advanced fine-tuning and optimization methods.
+* **Community Development:**  Contributing to the growing ecosystem of Persian language models.
+* **Prototyping (with caution):** For early-stage prototyping, acknowledging its experimental nature.
+
+## Limitations
+
+While optimized, this model still has limitations:
+
+* **Experimental Stage:**  Under ongoing development and refinement.
+* **2b Parameter Model:**  Performance is limited by the model size compared to larger models.
+* **Variable Output Quality:** Output quality can still be inconsistent and require careful evaluation.
+* **Potential for Imperfections:** May still exhibit issues like fluency problems, factual inaccuracies, or biases.
+
+**Use this model responsibly and be aware of its experimental nature.**
+
+**Citation:**
+
+If you use this model in your research or applications, please cite it using the following DOI: [10.57967/hf/4772](https://doi.org/10.57967/hf/4772)

config.json

@@ -0,0 +1,33 @@
+{
+  "_name_or_path": "mshojaei77/gemma-2-2b-fa-v3",
+  "architectures": [
+    "Gemma2ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attn_logit_softcapping": 50.0,
+  "bos_token_id": 2,
+  "cache_implementation": "hybrid",
+  "eos_token_id": 1,
+  "final_logit_softcapping": 30.0,
+  "head_dim": 256,
+  "hidden_act": "gelu_pytorch_tanh",
+  "hidden_activation": "gelu_pytorch_tanh",
+  "hidden_size": 2304,
+  "initializer_range": 0.02,
+  "intermediate_size": 9216,
+  "max_position_embeddings": 8192,
+  "model_type": "gemma2",
+  "num_attention_heads": 8,
+  "num_hidden_layers": 26,
+  "num_key_value_heads": 4,
+  "pad_token_id": 0,
+  "query_pre_attn_scalar": 256,
+  "rms_norm_eps": 1e-06,
+  "rope_theta": 10000.0,
+  "sliding_window": 4096,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.47.0",
+  "use_cache": false,
+  "vocab_size": 256000
+}

gemma-2-2b-fa-v2_q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e514bfb551c318e85ad8d8e78a5460e9f73be2bb7cd174627680141dfa291277
+size 2784495776

mergekit_config.yml

@@ -0,0 +1,10 @@
+
+models:
+  - model: mshojaei77/Gemma-2-2b-fa
+  - model: mshojaei77/gemma-2-2b-fa-v3
+merge_method: slerp
+base_model: mshojaei77/gemma-2-2b-fa-v3
+dtype: bfloat16
+parameters:
+  t: [0, 0.5, 1, 0.5, 0]
+

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ea2312bba56074d49d73e5a6d77c1afd1b7e9c24212ca11b5a9ec87d83f51758
+size 4959718480

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b27fbba0e33f32f0b5c48c25a39d6fafdd79e5e3a7ed7f5b6eead10c15e426b3
+size 268999016

special_tokens_map.json

@@ -0,0 +1,34 @@
+{
+  "additional_special_tokens": [
+    "<start_of_turn>",
+    "<end_of_turn>"
+  ],
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f7eee611703c5ce5d1eee32d9cdcfe465647b8aff0c1dfb3bed7ad7dbb05060
+size 34362873

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61a7b147390c64585d6c3543dd6fc636906c9af3865a5548f27f31aee1d4c8e2
+size 4241003