初始化项目，由ModelHub XC社区提供模型
Model: tu-ericngo/llama-3.1-8B-StructuredIE Source: Original Platform
This commit is contained in:
ModelHub XC
138
README.md
Normal file
138
README.md
Normal file
@@ -0,0 +1,138 @@
|
||||
---
|
||||
library_name: transformers
|
||||
tags:
|
||||
- unsloth
|
||||
- trl
|
||||
- sft
|
||||
license: llama3.1
|
||||
language:
|
||||
- en
|
||||
base_model:
|
||||
- unsloth/Llama-3.1-8B-Instruct
|
||||
---
|
||||
|
||||
# Model Card for Model ID
|
||||
|
||||
<!-- Section that provides information about the model -->
|
||||
|
||||
## Model Details
|
||||
|
||||
### Model Description
|
||||
|
||||
<!-- Provide a longer summary of what this model is. -->
|
||||
|
||||
This is an early fine-tuned version of Llama-3.1-8B-Instruct for structured information extraction (IE).
|
||||
Particularly, the target task involve joint named entity recognition (NER) and relation extraction (RE) to identify & extract information about politicla elites, their educational and professional associations, events and timeframes, and family members. The extracted information is generated in a structured JSON output.
|
||||
The fine-tuning process is adopted from Unsloth's procedure (unsloth/Llama-3.1-8B-Instruct).
|
||||
Data for the fine-tuning comes from 2 sources: (1) mannual collection and (2) synthetic data generated by GPT-4.
|
||||
|
||||
- **Developed by:** Tu 'Eric' Ngo.
|
||||
- **Language(s) (NLP):** English.
|
||||
- **Finetuned from:** Unsloth's Llama-3.1-8B-Instruct.
|
||||
|
||||
### Model Sources
|
||||
|
||||
<!-- Provide the basic links for the model. -->
|
||||
|
||||
- **Repository:** [To be added]
|
||||
- **Paper:** [To be added]
|
||||
|
||||
## Uses
|
||||
|
||||
<!--Description of intended use -->
|
||||
The model is fine-tuned to structured information extraction from political elite biographies in a very specific way.
|
||||
It follows a particular template that is very specific to the author's research project.
|
||||
The actual JSON schema and prompt for this fine-tuned task will be published in the future.
|
||||
|
||||
### Out-of-Scope Use
|
||||
|
||||
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
||||
While the fine-tuned model may be able to perform similar structured IE tasks (especially for the simpler tasks with simpler JSON schema), the model is only trained with a specific task in mine.
|
||||
However, in the future, the author intends to expand the range of structured IE tasks that the model can be used for.
|
||||
|
||||
## Bias, Risks, and Limitations
|
||||
|
||||
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
||||
|
||||
[To be added]
|
||||
|
||||
### Recommendations
|
||||
|
||||
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
||||
|
||||
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
||||
|
||||
## How to Get Started with the Model
|
||||
|
||||
[To be added]
|
||||
|
||||
## Training Details
|
||||
|
||||
### Training Data
|
||||
|
||||
<!-- Description of training data -->
|
||||
Data for the fine-tuning comes from 2 sources: (1) mannual collection and (2) synthetic data generated by GPT-4.
|
||||
The data is structured in an Alpaca format, with each training example consisting of Prompt (description of task, JSON schema, and one-shot example), Input (an elite's biographical text), and Output (JSON record).
|
||||
|
||||
### Training Procedure
|
||||
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
||||
|
||||
#### Training Hyperparameters
|
||||
|
||||
- **Training regime:** <!-- bf16 non-mixed precision --> bf16 non-mixed precision
|
||||
|
||||
#### Speeds, Sizes, Times [optional]
|
||||
<!-- This section provides information about throughput, start/end time, etc. -->
|
||||
- Num Epochs = 3 | Total steps = 99
|
||||
- Batch size per device = 2 | Gradient accumulation steps = 4
|
||||
- Data Parallel GPUs = 1 | Total batch size (2 x 4 x 1) = 8
|
||||
- Trainable parameters = 83,886,080/8,000,000,000 (1.05% trained)
|
||||
- 38.48 minutes used for training.
|
||||
- Peak reserved memory = 10.107 GB.
|
||||
- Peak reserved memory for training = 4.189 GB.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Evaluation
|
||||
|
||||
<!-- This section describes the evaluation protocols and provides the results. -->
|
||||
|
||||
### Testing
|
||||
|
||||
#### Metrics
|
||||
|
||||
<!-- Description of metrics and why -->
|
||||
F1, Precision, and Recall are used to evaluate the fine-tuned model.
|
||||
Since the intended JSON schema is highly complex and include multiple levels of nested components, the evaluation metrics are calculated for each of the root-level (broad) fields.
|
||||
|
||||
### Results
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
| Metric | Value |
|
||||
|-------------------------|---------|
|
||||
| JSON Valid (%) | 70.270 |
|
||||
| Exact Match (%) | 2.700 |
|
||||
| Avg Jaccard Similarity | 0.535 |
|
||||
| Avg Cosine Similarity | 0.672 |
|
||||
|
||||
|
||||
## Environmental Impact
|
||||
|
||||
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
||||
|
||||
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
||||
|
||||
- **Hardware Type:** [More Information Needed]
|
||||
- **Hours used:** [More Information Needed]
|
||||
- **Cloud Provider:** [More Information Needed]
|
||||
- **Compute Region:** [More Information Needed]
|
||||
- **Carbon Emitted:** [More Information Needed]
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user