Initial release

README.md

@@ -1,3 +1,65 @@
 ---
+base_model:
+- princeton-nlp/Llama-3-Instruct-8B-SimPO
+- UCLA-AGI/Llama-3-Instruct-8B-SPPO-Iter3
+library_name: transformers
+tags:
+- mergekit
+- merge
 license: llama3
+pipeline_tag: text-generation
 ---
+# Llama-3-Instruct-8B-SPPO-Iter3-SimPO-merge
+
+This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
+
+Built with Meta Llama 3.
+
+## Merge Details
+### Merge Method
+
+This model was merged using the SLERP merge method.
+
+### Models Merged
+
+The following models were included in the merge:
+* [princeton-nlp/Llama-3-Instruct-8B-SimPO](https://huggingface.co/princeton-nlp/Llama-3-Instruct-8B-SimPO)
+* [UCLA-AGI/Llama-3-Instruct-8B-SPPO-Iter3](https://huggingface.co/UCLA-AGI/Llama-3-Instruct-8B-SPPO-Iter3)
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+slices:
+- sources:
+  - model: princeton-nlp/Llama-3-Instruct-8B-SimPO
+    layer_range:
+    - 0
+    - 32
+  - model: UCLA-AGI/Llama-3-Instruct-8B-SPPO-Iter3
+    layer_range:
+    - 0
+    - 32
+merge_method: slerp
+base_model: UCLA-AGI/Llama-3-Instruct-8B-SPPO-Iter3
+parameters:
+  t:
+  - filter: self_attn
+    value:
+    - 0
+    - 0.5
+    - 0.3
+    - 0.7
+    - 1
+  - filter: mlp
+    value:
+    - 1
+    - 0.5
+    - 0.7
+    - 0.3
+    - 0
+  - value: 0.5
+dtype: bfloat16
+
+```