We introduce ADELIE (Aligning large language moDELs on Information Extraction), an aligned LLM that effectively solves various IE tasks, including closed IE, open IE, and on-demand IE. We first collect and construct a high-quality alignment corpus IEInstruct for IE. Then we train ADELIESFT using instruction tuning on IEInstruct. We further train ADELIESFT with direct preference optimization (DPO) objective, resulting in ADELIEDPO. Extensive experiments on various held-out IE datasets demonstrate that our models (ADELIESFT and ADELIEDPO) achieve state-of-the-art (SoTA) performance among open-source models. We further explore the general capabilities of ADELIE, and experimental results reveal that their general capabilities do not exhibit a noticeable decline.
The table below presents the average F1 scores (%) of the ADELIE model across closed IE, open IE, and on-demand IE tasks, as well as its overall performance (%) on general benchmarks. For dataset details, please refer to the paper.
Model
Closed IE
Open IE
On-demand IE
General Average Score
Llama2 7B
5.7
5.6
22.4
52.2
ADELIE-SFT
42.6
46.9
60.4
53.5
ADELIE-DPO
42.7
47.6
60.5
53.8
-----------------
-----------
---------
--------------
-----------------------
Llama3.2 3B
19.1
18.5
20.8
55.5
ADELIE-SFT-3B
41.8
47.6
60.8
55.6
ADELIE-DPO-3B
39.2
47.8
60.7
55.6
-----------------
-----------
---------
--------------
-----------------------
Qwen2.5 1.5B
16.5
14.2
20.5
54.6
ADELIE-SFT-1.5B
37.7
44.6
58.9
55.0
ADELIE-DPO-1.5B
38.5
45.6
59.2
55.1
Model Description
Developed by: Yunjia Qi, Hao Peng, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li
