初始化项目,由ModelHub XC社区提供模型

Model: dddsaty/SOLAR_Merge_Adapter_DPO_Orca
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-01 17:52:10 +08:00
commit d8b7f17ce0
14 changed files with 91783 additions and 0 deletions

57
README.md Normal file
View File

@@ -0,0 +1,57 @@
---
license: cc-by-nc-4.0
datasets:
- Intel/orca_dpo_pairs
language:
- en
pipeline_tag: text-generation
---
**Explanation**
- Merge two base models using [mergekit](https://github.com/arcee-ai/mergekit) (slerp)
- Apply DPO to the merged model, just an adapter part is saved
- merge the adpater and the merged model
**Base Model**
- [upstage/SOLAR-10.7B-Instruct-v1.0](https://huggingface.co/upstage/SOLAR-10.7B-Instruct-v1.0)
- [beomi/OPEN-SOLAR-KO-10.7B](https://huggingface.co/beomi/OPEN-SOLAR-KO-10.7B)
**Training Corpus**
- [Intel/orca_dpo_pairs](https://huggingface.co/datasets/Intel/orca_dpo_pairs)
**Score**
|Average|ARC|HellaSwag|MMLU|TruthfulQA|Winogrande|GSM8K|
|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|65.96|63.91|84.58|63.18|51.49|82|50.57|
**Log**
- 2024.02.05: Initial version Upload
- 2024.02.10: Readme update
**LICENSE**
Following the upstage/SOLAR-10.7B-Instruct-v1.0 License
- cc-by-nc-4.0
**Citation**
- beomi/OPEN-SOLAR-KO-10.7B
```
@misc {solar_ko_junbum_2023,
author = { {L. Junbum} },
title = { Solar-Ko-10.7b },
year = 2024,
url = { https://huggingface.co/beomi/SOLAR-KO-10.7B },
publisher = { Hugging Face }
}
```
- upstage/SOLAR-10.7B-Instruct-v1.0
```
@misc{kim2023solar,
title={SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling},
author={Dahyun Kim and Chanjun Park and Sanghoon Kim and Wonsung Lee and Wonho Song and Yunsu Kim and Hyeonwoo Kim and Yungi Kim and Hyeonju Lee and Jihoo Kim and Changbae Ahn and Seonghoon Yang and Sukyung Lee and Hyunbyung Park and Gyoungjin Gim and Mikyoung Cha and Hwalsuk Lee and Sunghun Kim},
year={2023},
eprint={2312.15166},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```