初始化项目，由ModelHub XC社区提供模型
Model: HenryJJ/Instruct_Yi-6B_Dolly15K Source: Original Platform
This commit is contained in:
ModelHub XC
448
README.md
Normal file
448
README.md
Normal file
@@ -0,0 +1,448 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
datasets:
|
||||
- databricks/databricks-dolly-15k
|
||||
---
|
||||
|
||||
|
||||
# Instruct_Yi-6B_Dolly15K
|
||||
Fine-tuned from Yi-6B， used Dolly15k for the dataset. 90% for training, 10% validation. Trained for 2.0 epochs using Lora. Trained with 1024 context window.
|
||||
|
||||
# Model Details
|
||||
* **Trained by**: trained by HenryJJ.
|
||||
* **Model type:** **Instruct_Yi-6B_Dolly15K** is an auto-regressive language model based on the Llama 2 transformer architecture.
|
||||
* **Language(s)**: English
|
||||
* **License for Instruct_Yi-6B_Dolly15K**: apache-2.0 license
|
||||
|
||||
|
||||
# Prompting
|
||||
|
||||
## Prompt Template With Context
|
||||
<|startoftext|>[INST]{instruction} {context}[/INST]{response}<|endoftext|>
|
||||
|
||||
```
|
||||
<|startoftext|>[INST]
|
||||
Write a 10-line poem about a given topic
|
||||
The topic is about racecars
|
||||
[/INST]
|
||||
```
|
||||
## Prompt Template Without Context
|
||||
```
|
||||
<|startoftext|>[INST]
|
||||
Who was the was the second president of the United States?
|
||||
[/INST]
|
||||
```
|
||||
|
||||
# Training script:
|
||||
Fully opensourced at: https://github.com/hengjiUSTC/learn-llm/blob/main/trl_finetune.py. Run on aws g4dn.12xlarge instance for 4 hours.
|
||||
|
||||
```
|
||||
python3 trl_finetune.py --config configs/yi_6b.yml
|
||||
```
|
||||
|
||||
# Dataset Card for Evaluation run of HenryJJ/Instruct_Yi-6B_Dolly15K
|
||||
|
||||
<!-- Provide a quick summary of the dataset. -->
|
||||
|
||||
Dataset automatically created during the evaluation run of model [HenryJJ/Instruct_Yi-6B_Dolly15K](https://huggingface.co/HenryJJ/Instruct_Yi-6B_Dolly15K) on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
|
||||
|
||||
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
|
||||
|
||||
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results.
|
||||
|
||||
An additional configuration "results" store all the aggregated results of the run (and is used to compute and display the aggregated metrics on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)).
|
||||
|
||||
To load the details from a run, you can for instance do the following:
|
||||
```python
|
||||
from datasets import load_dataset
|
||||
data = load_dataset("open-llm-leaderboard/details_HenryJJ__Instruct_Yi-6B_Dolly15K",
|
||||
"harness_winogrande_5",
|
||||
split="train")
|
||||
```
|
||||
|
||||
## Latest results
|
||||
|
||||
These are the [latest results from run 2024-01-06T09:45:44.755529](https://huggingface.co/datasets/open-llm-leaderboard/details_HenryJJ__Instruct_Yi-6B_Dolly15K/blob/main/results_2024-01-06T09-45-44.755529.json)(note that their might be results for other tasks in the repos if successive evals didn't cover the same tasks. You find each in the results and the "latest" split for each eval):
|
||||
|
||||
```python
|
||||
{
|
||||
"all": {
|
||||
"acc": 0.6267070831158695,
|
||||
"acc_stderr": 0.03222713761046951,
|
||||
"acc_norm": 0.6343965374667763,
|
||||
"acc_norm_stderr": 0.032887983229700546,
|
||||
"mc1": 0.28886168910648713,
|
||||
"mc1_stderr": 0.01586634640138431,
|
||||
"mc2": 0.42839602626744816,
|
||||
"mc2_stderr": 0.014270024501714959
|
||||
},
|
||||
"harness|arc:challenge|25": {
|
||||
"acc": 0.5,
|
||||
"acc_stderr": 0.014611390804670088,
|
||||
"acc_norm": 0.5486348122866894,
|
||||
"acc_norm_stderr": 0.014542104569955265
|
||||
},
|
||||
"harness|hellaswag|10": {
|
||||
"acc": 0.5654252141007767,
|
||||
"acc_stderr": 0.004946879874422681,
|
||||
"acc_norm": 0.7587134037044413,
|
||||
"acc_norm_stderr": 0.00426989301158892
|
||||
},
|
||||
"harness|hendrycksTest-abstract_algebra|5": {
|
||||
"acc": 0.35,
|
||||
"acc_stderr": 0.0479372485441102,
|
||||
"acc_norm": 0.35,
|
||||
"acc_norm_stderr": 0.0479372485441102
|
||||
},
|
||||
"harness|hendrycksTest-anatomy|5": {
|
||||
"acc": 0.562962962962963,
|
||||
"acc_stderr": 0.04284958639753401,
|
||||
"acc_norm": 0.562962962962963,
|
||||
"acc_norm_stderr": 0.04284958639753401
|
||||
},
|
||||
"harness|hendrycksTest-astronomy|5": {
|
||||
"acc": 0.6776315789473685,
|
||||
"acc_stderr": 0.03803510248351585,
|
||||
"acc_norm": 0.6776315789473685,
|
||||
"acc_norm_stderr": 0.03803510248351585
|
||||
},
|
||||
"harness|hendrycksTest-business_ethics|5": {
|
||||
"acc": 0.7,
|
||||
"acc_stderr": 0.046056618647183814,
|
||||
"acc_norm": 0.7,
|
||||
"acc_norm_stderr": 0.046056618647183814
|
||||
},
|
||||
"harness|hendrycksTest-clinical_knowledge|5": {
|
||||
"acc": 0.690566037735849,
|
||||
"acc_stderr": 0.028450154794118637,
|
||||
"acc_norm": 0.690566037735849,
|
||||
"acc_norm_stderr": 0.028450154794118637
|
||||
},
|
||||
"harness|hendrycksTest-college_biology|5": {
|
||||
"acc": 0.6666666666666666,
|
||||
"acc_stderr": 0.039420826399272135,
|
||||
"acc_norm": 0.6666666666666666,
|
||||
"acc_norm_stderr": 0.039420826399272135
|
||||
},
|
||||
"harness|hendrycksTest-college_chemistry|5": {
|
||||
"acc": 0.41,
|
||||
"acc_stderr": 0.049431107042371025,
|
||||
"acc_norm": 0.41,
|
||||
"acc_norm_stderr": 0.049431107042371025
|
||||
},
|
||||
"harness|hendrycksTest-college_computer_science|5": {
|
||||
"acc": 0.44,
|
||||
"acc_stderr": 0.04988876515698589,
|
||||
"acc_norm": 0.44,
|
||||
"acc_norm_stderr": 0.04988876515698589
|
||||
},
|
||||
"harness|hendrycksTest-college_mathematics|5": {
|
||||
"acc": 0.36,
|
||||
"acc_stderr": 0.04824181513244218,
|
||||
"acc_norm": 0.36,
|
||||
"acc_norm_stderr": 0.04824181513244218
|
||||
},
|
||||
"harness|hendrycksTest-college_medicine|5": {
|
||||
"acc": 0.6069364161849711,
|
||||
"acc_stderr": 0.03724249595817731,
|
||||
"acc_norm": 0.6069364161849711,
|
||||
"acc_norm_stderr": 0.03724249595817731
|
||||
},
|
||||
"harness|hendrycksTest-college_physics|5": {
|
||||
"acc": 0.3235294117647059,
|
||||
"acc_stderr": 0.04655010411319617,
|
||||
"acc_norm": 0.3235294117647059,
|
||||
"acc_norm_stderr": 0.04655010411319617
|
||||
},
|
||||
"harness|hendrycksTest-computer_security|5": {
|
||||
"acc": 0.77,
|
||||
"acc_stderr": 0.04229525846816507,
|
||||
"acc_norm": 0.77,
|
||||
"acc_norm_stderr": 0.04229525846816507
|
||||
},
|
||||
"harness|hendrycksTest-conceptual_physics|5": {
|
||||
"acc": 0.6212765957446809,
|
||||
"acc_stderr": 0.03170995606040655,
|
||||
"acc_norm": 0.6212765957446809,
|
||||
"acc_norm_stderr": 0.03170995606040655
|
||||
},
|
||||
"harness|hendrycksTest-econometrics|5": {
|
||||
"acc": 0.35964912280701755,
|
||||
"acc_stderr": 0.045144961328736334,
|
||||
"acc_norm": 0.35964912280701755,
|
||||
"acc_norm_stderr": 0.045144961328736334
|
||||
},
|
||||
"harness|hendrycksTest-electrical_engineering|5": {
|
||||
"acc": 0.6482758620689655,
|
||||
"acc_stderr": 0.0397923663749741,
|
||||
"acc_norm": 0.6482758620689655,
|
||||
"acc_norm_stderr": 0.0397923663749741
|
||||
},
|
||||
"harness|hendrycksTest-elementary_mathematics|5": {
|
||||
"acc": 0.4470899470899471,
|
||||
"acc_stderr": 0.02560672399577703,
|
||||
"acc_norm": 0.4470899470899471,
|
||||
"acc_norm_stderr": 0.02560672399577703
|
||||
},
|
||||
"harness|hendrycksTest-formal_logic|5": {
|
||||
"acc": 0.38095238095238093,
|
||||
"acc_stderr": 0.04343525428949098,
|
||||
"acc_norm": 0.38095238095238093,
|
||||
"acc_norm_stderr": 0.04343525428949098
|
||||
},
|
||||
"harness|hendrycksTest-global_facts|5": {
|
||||
"acc": 0.4,
|
||||
"acc_stderr": 0.04923659639173309,
|
||||
"acc_norm": 0.4,
|
||||
"acc_norm_stderr": 0.04923659639173309
|
||||
},
|
||||
"harness|hendrycksTest-high_school_biology|5": {
|
||||
"acc": 0.7774193548387097,
|
||||
"acc_stderr": 0.023664216671642525,
|
||||
"acc_norm": 0.7774193548387097,
|
||||
"acc_norm_stderr": 0.023664216671642525
|
||||
},
|
||||
"harness|hendrycksTest-high_school_chemistry|5": {
|
||||
"acc": 0.4975369458128079,
|
||||
"acc_stderr": 0.03517945038691063,
|
||||
"acc_norm": 0.4975369458128079,
|
||||
"acc_norm_stderr": 0.03517945038691063
|
||||
},
|
||||
"harness|hendrycksTest-high_school_computer_science|5": {
|
||||
"acc": 0.64,
|
||||
"acc_stderr": 0.04824181513244218,
|
||||
"acc_norm": 0.64,
|
||||
"acc_norm_stderr": 0.04824181513244218
|
||||
},
|
||||
"harness|hendrycksTest-high_school_european_history|5": {
|
||||
"acc": 0.7393939393939394,
|
||||
"acc_stderr": 0.034277431758165236,
|
||||
"acc_norm": 0.7393939393939394,
|
||||
"acc_norm_stderr": 0.034277431758165236
|
||||
},
|
||||
"harness|hendrycksTest-high_school_geography|5": {
|
||||
"acc": 0.8181818181818182,
|
||||
"acc_stderr": 0.0274796030105388,
|
||||
"acc_norm": 0.8181818181818182,
|
||||
"acc_norm_stderr": 0.0274796030105388
|
||||
},
|
||||
"harness|hendrycksTest-high_school_government_and_politics|5": {
|
||||
"acc": 0.9015544041450777,
|
||||
"acc_stderr": 0.021500249576033456,
|
||||
"acc_norm": 0.9015544041450777,
|
||||
"acc_norm_stderr": 0.021500249576033456
|
||||
},
|
||||
"harness|hendrycksTest-high_school_macroeconomics|5": {
|
||||
"acc": 0.617948717948718,
|
||||
"acc_stderr": 0.02463554916390823,
|
||||
"acc_norm": 0.617948717948718,
|
||||
"acc_norm_stderr": 0.02463554916390823
|
||||
},
|
||||
"harness|hendrycksTest-high_school_mathematics|5": {
|
||||
"acc": 0.31851851851851853,
|
||||
"acc_stderr": 0.028406533090608463,
|
||||
"acc_norm": 0.31851851851851853,
|
||||
"acc_norm_stderr": 0.028406533090608463
|
||||
},
|
||||
"harness|hendrycksTest-high_school_microeconomics|5": {
|
||||
"acc": 0.7647058823529411,
|
||||
"acc_stderr": 0.027553614467863797,
|
||||
"acc_norm": 0.7647058823529411,
|
||||
"acc_norm_stderr": 0.027553614467863797
|
||||
},
|
||||
"harness|hendrycksTest-high_school_physics|5": {
|
||||
"acc": 0.36423841059602646,
|
||||
"acc_stderr": 0.03929111781242742,
|
||||
"acc_norm": 0.36423841059602646,
|
||||
"acc_norm_stderr": 0.03929111781242742
|
||||
},
|
||||
"harness|hendrycksTest-high_school_psychology|5": {
|
||||
"acc": 0.8348623853211009,
|
||||
"acc_stderr": 0.01591955782997604,
|
||||
"acc_norm": 0.8348623853211009,
|
||||
"acc_norm_stderr": 0.01591955782997604
|
||||
},
|
||||
"harness|hendrycksTest-high_school_statistics|5": {
|
||||
"acc": 0.5694444444444444,
|
||||
"acc_stderr": 0.03376922151252335,
|
||||
"acc_norm": 0.5694444444444444,
|
||||
"acc_norm_stderr": 0.03376922151252335
|
||||
},
|
||||
"harness|hendrycksTest-high_school_us_history|5": {
|
||||
"acc": 0.8088235294117647,
|
||||
"acc_stderr": 0.027599174300640766,
|
||||
"acc_norm": 0.8088235294117647,
|
||||
"acc_norm_stderr": 0.027599174300640766
|
||||
},
|
||||
"harness|hendrycksTest-high_school_world_history|5": {
|
||||
"acc": 0.7932489451476793,
|
||||
"acc_stderr": 0.026361651668389094,
|
||||
"acc_norm": 0.7932489451476793,
|
||||
"acc_norm_stderr": 0.026361651668389094
|
||||
},
|
||||
"harness|hendrycksTest-human_aging|5": {
|
||||
"acc": 0.695067264573991,
|
||||
"acc_stderr": 0.030898610882477515,
|
||||
"acc_norm": 0.695067264573991,
|
||||
"acc_norm_stderr": 0.030898610882477515
|
||||
},
|
||||
"harness|hendrycksTest-human_sexuality|5": {
|
||||
"acc": 0.7480916030534351,
|
||||
"acc_stderr": 0.03807387116306085,
|
||||
"acc_norm": 0.7480916030534351,
|
||||
"acc_norm_stderr": 0.03807387116306085
|
||||
},
|
||||
"harness|hendrycksTest-international_law|5": {
|
||||
"acc": 0.7768595041322314,
|
||||
"acc_stderr": 0.03800754475228733,
|
||||
"acc_norm": 0.7768595041322314,
|
||||
"acc_norm_stderr": 0.03800754475228733
|
||||
},
|
||||
"harness|hendrycksTest-jurisprudence|5": {
|
||||
"acc": 0.7777777777777778,
|
||||
"acc_stderr": 0.040191074725573483,
|
||||
"acc_norm": 0.7777777777777778,
|
||||
"acc_norm_stderr": 0.040191074725573483
|
||||
},
|
||||
"harness|hendrycksTest-logical_fallacies|5": {
|
||||
"acc": 0.7852760736196319,
|
||||
"acc_stderr": 0.03226219377286775,
|
||||
"acc_norm": 0.7852760736196319,
|
||||
"acc_norm_stderr": 0.03226219377286775
|
||||
},
|
||||
"harness|hendrycksTest-machine_learning|5": {
|
||||
"acc": 0.4375,
|
||||
"acc_stderr": 0.04708567521880525,
|
||||
"acc_norm": 0.4375,
|
||||
"acc_norm_stderr": 0.04708567521880525
|
||||
},
|
||||
"harness|hendrycksTest-management|5": {
|
||||
"acc": 0.8155339805825242,
|
||||
"acc_stderr": 0.03840423627288276,
|
||||
"acc_norm": 0.8155339805825242,
|
||||
"acc_norm_stderr": 0.03840423627288276
|
||||
},
|
||||
"harness|hendrycksTest-marketing|5": {
|
||||
"acc": 0.8974358974358975,
|
||||
"acc_stderr": 0.01987565502786744,
|
||||
"acc_norm": 0.8974358974358975,
|
||||
"acc_norm_stderr": 0.01987565502786744
|
||||
},
|
||||
"harness|hendrycksTest-medical_genetics|5": {
|
||||
"acc": 0.76,
|
||||
"acc_stderr": 0.042923469599092816,
|
||||
"acc_norm": 0.76,
|
||||
"acc_norm_stderr": 0.042923469599092816
|
||||
},
|
||||
"harness|hendrycksTest-miscellaneous|5": {
|
||||
"acc": 0.8007662835249042,
|
||||
"acc_stderr": 0.014283378044296417,
|
||||
"acc_norm": 0.8007662835249042,
|
||||
"acc_norm_stderr": 0.014283378044296417
|
||||
},
|
||||
"harness|hendrycksTest-moral_disputes|5": {
|
||||
"acc": 0.708092485549133,
|
||||
"acc_stderr": 0.024476994076247333,
|
||||
"acc_norm": 0.708092485549133,
|
||||
"acc_norm_stderr": 0.024476994076247333
|
||||
},
|
||||
"harness|hendrycksTest-moral_scenarios|5": {
|
||||
"acc": 0.33519553072625696,
|
||||
"acc_stderr": 0.015788007190185884,
|
||||
"acc_norm": 0.33519553072625696,
|
||||
"acc_norm_stderr": 0.015788007190185884
|
||||
},
|
||||
"harness|hendrycksTest-nutrition|5": {
|
||||
"acc": 0.7222222222222222,
|
||||
"acc_stderr": 0.025646863097137897,
|
||||
"acc_norm": 0.7222222222222222,
|
||||
"acc_norm_stderr": 0.025646863097137897
|
||||
},
|
||||
"harness|hendrycksTest-philosophy|5": {
|
||||
"acc": 0.6913183279742765,
|
||||
"acc_stderr": 0.026236965881153262,
|
||||
"acc_norm": 0.6913183279742765,
|
||||
"acc_norm_stderr": 0.026236965881153262
|
||||
},
|
||||
"harness|hendrycksTest-prehistory|5": {
|
||||
"acc": 0.7191358024691358,
|
||||
"acc_stderr": 0.025006469755799208,
|
||||
"acc_norm": 0.7191358024691358,
|
||||
"acc_norm_stderr": 0.025006469755799208
|
||||
},
|
||||
"harness|hendrycksTest-professional_accounting|5": {
|
||||
"acc": 0.48226950354609927,
|
||||
"acc_stderr": 0.02980873964223777,
|
||||
"acc_norm": 0.48226950354609927,
|
||||
"acc_norm_stderr": 0.02980873964223777
|
||||
},
|
||||
"harness|hendrycksTest-professional_law|5": {
|
||||
"acc": 0.4876140808344198,
|
||||
"acc_stderr": 0.012766317315473565,
|
||||
"acc_norm": 0.4876140808344198,
|
||||
"acc_norm_stderr": 0.012766317315473565
|
||||
},
|
||||
"harness|hendrycksTest-professional_medicine|5": {
|
||||
"acc": 0.6213235294117647,
|
||||
"acc_stderr": 0.02946513363977613,
|
||||
"acc_norm": 0.6213235294117647,
|
||||
"acc_norm_stderr": 0.02946513363977613
|
||||
},
|
||||
"harness|hendrycksTest-professional_psychology|5": {
|
||||
"acc": 0.6568627450980392,
|
||||
"acc_stderr": 0.019206606848825365,
|
||||
"acc_norm": 0.6568627450980392,
|
||||
"acc_norm_stderr": 0.019206606848825365
|
||||
},
|
||||
"harness|hendrycksTest-public_relations|5": {
|
||||
"acc": 0.6909090909090909,
|
||||
"acc_stderr": 0.044262946482000985,
|
||||
"acc_norm": 0.6909090909090909,
|
||||
"acc_norm_stderr": 0.044262946482000985
|
||||
},
|
||||
"harness|hendrycksTest-security_studies|5": {
|
||||
"acc": 0.7306122448979592,
|
||||
"acc_stderr": 0.02840125202902294,
|
||||
"acc_norm": 0.7306122448979592,
|
||||
"acc_norm_stderr": 0.02840125202902294
|
||||
},
|
||||
"harness|hendrycksTest-sociology|5": {
|
||||
"acc": 0.8159203980099502,
|
||||
"acc_stderr": 0.027403859410786862,
|
||||
"acc_norm": 0.8159203980099502,
|
||||
"acc_norm_stderr": 0.027403859410786862
|
||||
},
|
||||
"harness|hendrycksTest-us_foreign_policy|5": {
|
||||
"acc": 0.84,
|
||||
"acc_stderr": 0.03684529491774708,
|
||||
"acc_norm": 0.84,
|
||||
"acc_norm_stderr": 0.03684529491774708
|
||||
},
|
||||
"harness|hendrycksTest-virology|5": {
|
||||
"acc": 0.4578313253012048,
|
||||
"acc_stderr": 0.0387862677100236,
|
||||
"acc_norm": 0.4578313253012048,
|
||||
"acc_norm_stderr": 0.0387862677100236
|
||||
},
|
||||
"harness|hendrycksTest-world_religions|5": {
|
||||
"acc": 0.8070175438596491,
|
||||
"acc_stderr": 0.030267457554898458,
|
||||
"acc_norm": 0.8070175438596491,
|
||||
"acc_norm_stderr": 0.030267457554898458
|
||||
},
|
||||
"harness|truthfulqa:mc|0": {
|
||||
"mc1": 0.28886168910648713,
|
||||
"mc1_stderr": 0.01586634640138431,
|
||||
"mc2": 0.42839602626744816,
|
||||
"mc2_stderr": 0.014270024501714959
|
||||
},
|
||||
"harness|winogrande|5": {
|
||||
"acc": 0.7490134175217048,
|
||||
"acc_stderr": 0.012185776220516148
|
||||
},
|
||||
"harness|gsm8k|5": {
|
||||
"acc": 0.2926459438968916,
|
||||
"acc_stderr": 0.012532334368242888
|
||||
}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user