初始化项目，由ModelHub XC社区提供模型
Model: xaviergillard/digita Source: Original Platform
This commit is contained in:
ModelHub XC
136
README.md
Normal file
136
README.md
Normal file
@@ -0,0 +1,136 @@
|
||||
---
|
||||
base_model: unsloth/meta-llama-3.1-8b-instruct-bnb-4bit
|
||||
library_name: peft
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- base_model:adapter:unsloth/meta-llama-3.1-8b-instruct-bnb-4bit
|
||||
- lora
|
||||
- transformers
|
||||
- unsloth
|
||||
---
|
||||
|
||||
# Model Card for Digita
|
||||
|
||||
Digita has been fine-tuned (qlora) on a proprietary dataset so as to respond
|
||||
similarly to the Belgian State Archive customer support. It was essentially
|
||||
trained using data from the "digit" mailbox from the DiVa section.
|
||||
|
||||
|
||||
## Model Details
|
||||
|
||||
### Model Description
|
||||
|
||||
Digita has been trained in the context of the
|
||||
[Arkey project](https://www.uclouvain.be/fr/instituts-recherche/ilc/miil/arkey)
|
||||
conjointly held by the Belgian State Archive and UCLouvain. The purpose of this
|
||||
project is to ease access to the archives held at both institutions for the
|
||||
general public.
|
||||
|
||||
In that context, Digita has been trained as an experiment to tune a chatbot
|
||||
that would be able to talk and reply to user request in a way that feels
|
||||
similar to the actual people dealing with the matter at the BSA.
|
||||
|
||||
|
||||
- **Developed by:** Xavier GILLARD
|
||||
- **Funded by BELSPO:** [Arkey project](https://www.uclouvain.be/fr/instituts-recherche/ilc/miil/arkey)
|
||||
- **Model type:** Conversational (Fine Tuned from Llama-3.1-8b-Instruct-bnb-4bit)
|
||||
- **Language(s) (NLP):** Dutch, French, German, English
|
||||
- **License:** MIT License
|
||||
- **Finetuned from model Llama-3.1-8b-Instruct:** [unsloth/meta-llama-3.1-8b-instruct-bnb-4bit](https://huggingface.co/unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit)
|
||||
|
||||
### Model Sources
|
||||
|
||||
All scripts, notebooks, etc.. (except the dataset data) that have been used
|
||||
to prepare the dataset and fine tune the model are [available on github](https://github.com/xgillard/corpus_digit)
|
||||
|
||||
|
||||
**Repository:** [xgillard/corpus_digit](https://github.com/xgillard/corpus_digit)
|
||||
|
||||
## Uses
|
||||
|
||||
Prototype a chatbot to help the users of the Belgian State Archive in the
|
||||
context of the Arkey project.
|
||||
|
||||
## Limitations
|
||||
|
||||
This model has been fine tuned on data from the 2022-2024 period. Which means
|
||||
the majority of the data predates the introduction of the AGATHA platform.
|
||||
This means that, while some of the info provided by this model can be useful,
|
||||
it is essentially going to be stale at this point.
|
||||
|
||||
The second major limitation I envision for this model stems from the sheer size
|
||||
of the model: 8b is probably not going to cut it for a majority of cases.
|
||||
Especially if one envisions to use the quantized versions.
|
||||
However, I think that this model is a good proof of concept and that it shows
|
||||
what could be achieved if resources were allocated to perform the same task
|
||||
on a more up-to-date dataset (or limited to 2024) using a larger model.
|
||||
This was however not feasible on my personal machine.
|
||||
|
||||
### Recommendations
|
||||
|
||||
If anyone is serious about using this model as a basis to help the BSA personnel
|
||||
of to deploy an user facing chatbot at the BSA, I would recommend to completely
|
||||
redo all the finetuning using a larger model and an updated (more recent)
|
||||
version of the dataset.
|
||||
|
||||
## How to Get Started with the Model
|
||||
|
||||
I do recommend that you use the QLoRA adapter directly using unsloth as this is
|
||||
the asset that seemed to yield the best result while maintaining a similar
|
||||
VRAM footprint (about 7G). Here is how you get started:
|
||||
|
||||
```
|
||||
from unsloth import (
|
||||
FastLanguageModel,
|
||||
train_on_responses_only,
|
||||
)
|
||||
from unsloth.chat_templates import (
|
||||
get_chat_template,
|
||||
)
|
||||
from transformers import TextStreamer
|
||||
|
||||
model, tok = FastLanguageModel.from_pretrained("xaviergillard/digita", load_in_4bit=True)
|
||||
model = FastLanguageModel.for_inference(model)
|
||||
```
|
||||
|
||||
## Training Details
|
||||
|
||||
### Training Data
|
||||
|
||||
Original training data consists of a curated dataset of response emails sent
|
||||
by the DiVa cusomer service. The details of this dataset will not be disclosed.
|
||||
|
||||
### Training Procedure
|
||||
|
||||
#### Preprocessing
|
||||
|
||||
1. Cleaning up the encoding
|
||||
2. Conversation restructuration
|
||||
3. Conversation classification
|
||||
4. Data Augmentation
|
||||
5. Filtering
|
||||
|
||||
#### Training Hyperparameters
|
||||
|
||||
- **Training regime:** QLoRA adapter based on a 4bit quantization of Llama-3.1-8b-Instruct by unsloth.
|
||||
- **LoRA rank:** `rank = 16`
|
||||
- **LoRA alpha:** `alpha = 16`
|
||||
- **Base model quantization:** `4 bits`
|
||||
|
||||
All training/eval metrics can be consulted [here](https://www.comet.com/xaviergillard/digita/view/new/panels).
|
||||
|
||||
|
||||
## Evaluation
|
||||
|
||||
So far, evaluation has not been very thorough: it mostly consisted of
|
||||
* comparing the eval loss to the training loss during training
|
||||
* ensuring the curves seemed ok
|
||||
* peforming a few manual tests on the trained & quantized models.
|
||||
|
||||
## Model Card Authors
|
||||
|
||||
Xavier Gillard
|
||||
|
||||
### Framework versions
|
||||
|
||||
- PEFT 0.18.0
|
||||
Reference in New Issue
Block a user