diff --git a/README.md b/README.md new file mode 100644 index 0000000..e2fbbf5 --- /dev/null +++ b/README.md @@ -0,0 +1,30 @@ +--- +tags: +- mistral +--- +# Model Card for Silicon-Medley + + + +Slerp merge of Silicon-Maid-7B and piano-medley-7b. + +## Model Details + +``` +slices: + - sources: + - model: SanjiWatsuki/Silicon-Maid-7B + layer_range: [0, 32] + - model: chargoddard/piano-medley-7b + layer_range: [0, 32] +merge_method: slerp +base_model: mistralai/Mistral-7B-v0.1 +parameters: + t: + - filter: self_attn + value: [0.22, 0.61, 0.46, 0.77, 1] + - filter: mlp + value: [0.78, 0.39, 0.54, 0.23, 0] + - value: 0.5 # fallback for rest of tensors +dtype: float16 +``` \ No newline at end of file