初始化项目,由ModelHub XC社区提供模型
Model: KaeriJenti/kaori-34b-v4 Source: Original Platform
This commit is contained in:
47
README.md
Normal file
47
README.md
Normal file
@@ -0,0 +1,47 @@
|
||||
---
|
||||
license: llama2
|
||||
---
|
||||
|
||||
<h1>kaori-34b-v4 Model Card</h1>
|
||||
|
||||
This Model was Finetuned By Kaeri and Jenti.
|
||||
|
||||
<h3>Datasets</h3>
|
||||
|
||||
- Open-Platypus
|
||||
- Dolphin
|
||||
- OpenOrca
|
||||
|
||||
We trained the model with <b>100%</b> Open-Platypus data, <b>5%</b> Dolphin data and <b>10%</b> OpenOrca data and applied SFT strategy.
|
||||
|
||||
We did not use GSM8k samples when generating data.
|
||||
Also we were careful of data contamination by similarity filtering
|
||||
the training data if the data correspond to any of the following list.
|
||||
|
||||
<pre>
|
||||
filtering_tasks = [
|
||||
'cot_gsm8k',
|
||||
'cot_gsm8k_ii',
|
||||
'drop:2.0.0',
|
||||
'winogrande:1.1.0'
|
||||
'task228_arc_answer_generation_easy',
|
||||
'ai2_arc/ARC-Challenge:1.0.0',
|
||||
'ai2_arc/ARC-Easy:1.0.0',
|
||||
'task229_arc_answer_generation_hard',
|
||||
'hellaswag:1.1.0',
|
||||
'task1389_hellaswag_completion'
|
||||
]
|
||||
</pre>
|
||||
|
||||
|
||||
<h3>Framework:</h3>
|
||||
|
||||
- https://github.com/hiyouga/LLaMA-Factory
|
||||
|
||||
|
||||
<h3>Parameters:</h3>
|
||||
|
||||
- Finetune_Type : LoRA
|
||||
- GPUs : A100x4(80GB)
|
||||
- Epochs : 3
|
||||
- Batchsize : 8
|
||||
Reference in New Issue
Block a user