初始化项目,由ModelHub XC社区提供模型
Model: flax-community/gpt2-base-thai Source: Original Platform
This commit is contained in:
69
README.md
Normal file
69
README.md
Normal file
@@ -0,0 +1,69 @@
|
||||
---
|
||||
language: th
|
||||
tags:
|
||||
- gpt2-base-thai
|
||||
license: mit
|
||||
datasets:
|
||||
- oscar
|
||||
widget:
|
||||
- text: "สวัสดีตอนเช้า"
|
||||
---
|
||||
|
||||
## GPT-2 Base Thai
|
||||
|
||||
GPT-2 Base Thai is a causal language model based on the [OpenAI GPT-2](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) model. It was trained on the [OSCAR](https://huggingface.co/datasets/oscar) dataset, specifically the `unshuffled_deduplicated_th` subset. The model was trained from scratch and achieved an evaluation loss of 1.708 and an evaluation perplexity of 5.516.
|
||||
|
||||
This model was trained using HuggingFace's Flax framework and is part of the [JAX/Flax Community Week](https://discuss.huggingface.co/t/open-to-the-community-community-week-using-jax-flax-for-nlp-cv/7104) organized by HuggingFace. All training was done on a TPUv3-8 VM, sponsored by the Google Cloud team.
|
||||
|
||||
All necessary scripts used for training could be found in the [Files and versions](https://hf.co/flax-community/gpt2-base-thai/tree/main) tab, as well as the [Training metrics](https://hf.co/flax-community/gpt2-base-thai/tensorboard) logged via Tensorboard.
|
||||
|
||||
## Model
|
||||
|
||||
| Model | #params | Arch. | Training/Validation data (text) |
|
||||
| ---------------- | ------- | ----- | ------------------------------------ |
|
||||
| `gpt2-base-thai` | 124M | GPT-2 | `unshuffled_deduplicated_th` Dataset |
|
||||
|
||||
## Evaluation Results
|
||||
|
||||
The model was trained for 3 epochs and the following is the final result once the training ended.
|
||||
|
||||
| train loss | valid loss | valid PPL | total time |
|
||||
| ---------- | ---------- | --------- | ---------- |
|
||||
| 1.638 | 1.708 | 5.516 | 6:12:34 |
|
||||
|
||||
## How to Use
|
||||
|
||||
### As Causal Language Model
|
||||
|
||||
```python
|
||||
from transformers import pipeline
|
||||
|
||||
pretrained_name = "flax-community/gpt2-base-thai"
|
||||
|
||||
nlp = pipeline(
|
||||
"text-generation",
|
||||
model=pretrained_name,
|
||||
tokenizer=pretrained_name
|
||||
)
|
||||
|
||||
nlp("สวัสดีตอนเช้า")
|
||||
```
|
||||
|
||||
### Feature Extraction in PyTorch
|
||||
|
||||
```python
|
||||
from transformers import GPT2Model, GPT2TokenizerFast
|
||||
|
||||
pretrained_name = "flax-community/gpt2-base-thai"
|
||||
model = GPT2Model.from_pretrained(pretrained_name)
|
||||
tokenizer = GPT2TokenizerFast.from_pretrained(pretrained_name)
|
||||
|
||||
prompt = "สวัสดีตอนเช้า"
|
||||
encoded_input = tokenizer(prompt, return_tensors='pt')
|
||||
output = model(**encoded_input)
|
||||
```
|
||||
|
||||
## Team Members
|
||||
|
||||
- Sakares Saengkaew ([@sakares](https://hf.co/sakares))
|
||||
- Wilson Wongso ([@w11wo](https://hf.co/w11wo))
|
||||
Reference in New Issue
Block a user