初始化项目,由ModelHub XC社区提供模型
Model: athirdpath/CleverMommy-mix-20b Source: Original Platform
This commit is contained in:
28
README.md
Normal file
28
README.md
Normal file
@@ -0,0 +1,28 @@
|
||||
---
|
||||
license: cc-by-nc-4.0
|
||||
---
|
||||
An extended part of my effort to create Eileithyia-20B. This model is made by following the recipe below, inverting it, then SLERPing the models back together at 0.5, hopefully fusing the models into one block for use with Harmonia.
|
||||
|
||||
slices:
|
||||
- sources:
|
||||
- model: microsoft/Orca-2-13b
|
||||
-
|
||||
layer_range: [0, 16]
|
||||
- sources:
|
||||
- model: athirdpath/Eileithyia-13B
|
||||
-
|
||||
layer_range: [8, 24]
|
||||
- sources:
|
||||
- model: microsoft/Orca-2-13b
|
||||
-
|
||||
layer_range: [17, 32]
|
||||
- sources:
|
||||
- model: athirdpath/Eileithyia-13B
|
||||
-
|
||||
layer_range: [25, 40]
|
||||
|
||||
merge_method: passthrough
|
||||
|
||||
dtype: float16
|
||||
|
||||
Thanks to Undi95 for pioneering the recipe.
|
||||
Reference in New Issue
Block a user