初始化项目,由ModelHub XC社区提供模型
Model: cambridgeltl/simctg_rocstories Source: Original Platform
This commit is contained in:
60
README.md
Normal file
60
README.md
Normal file
@@ -0,0 +1,60 @@
|
||||
This model provides a GPT-2 language model trained with SimCTG on the ROCStories benchmark [(Mostafazadeh et al., 2016)](https://aclanthology.org/N16-1098.pdf) based on our paper [_A Contrastive Framework for Neural Text Generation_](https://arxiv.org/abs/2202.06417).
|
||||
|
||||
We provide a detailed tutorial on how to apply SimCTG and Contrastive Search in our [project repo](https://github.com/yxuansu/SimCTG#4-huggingface-style-tutorials-back-to-top). In the following, we illustrate a brief tutorial on how to use our approach to perform text generation.
|
||||
|
||||
## 1. Installation of SimCTG:
|
||||
```yaml
|
||||
pip install simctg --upgrade
|
||||
```
|
||||
|
||||
## 2. Initialize SimCTG Model:
|
||||
```python
|
||||
import torch
|
||||
# load SimCTG language model
|
||||
from simctg.simctggpt import SimCTGGPT
|
||||
model_name = r'cambridgeltl/simctg_rocstories'
|
||||
model = SimCTGGPT(model_name)
|
||||
model.eval()
|
||||
tokenizer = model.tokenizer
|
||||
```
|
||||
|
||||
## 3. Prepare the Text Prefix:
|
||||
```python
|
||||
prompt = r"Accident in the Lab <|endoftext|>"
|
||||
print ('Prefix is: {}'.format(prompt))
|
||||
tokens = model.tokenizer.tokenize(prompt)
|
||||
input_ids = model.tokenizer.convert_tokens_to_ids(tokens)
|
||||
input_ids = torch.LongTensor(input_ids).view(1,-1)
|
||||
```
|
||||
|
||||
## 4. Generate Text with Contrastive Search:
|
||||
```python
|
||||
beam_width, alpha, decoding_len = 5, 0.65, 45
|
||||
output = model.fast_contrastive_search(input_ids=input_ids, beam_width=beam_width,
|
||||
alpha=alpha, decoding_len=decoding_len)
|
||||
print("Output:\n" + 100 * '-')
|
||||
print(tokenizer.decode(output).split(model.tokenizer.eos_token)[1].strip())
|
||||
'''
|
||||
Prefix is: Accident in the Lab <|endoftext|>
|
||||
Output:
|
||||
----------------------------------------------------------------------------------------------------
|
||||
Tom went to work one day. He noticed a lab accident in the lab. Tom was worried about his safety at work.
|
||||
Unfortunately the accident didn't go well. Tom wound up leaving early to get back on the job.
|
||||
'''
|
||||
```
|
||||
|
||||
For more details of our work, please refer to our main [project repo](https://github.com/yxuansu/SimCTG).
|
||||
|
||||
## 5. Citation:
|
||||
If you find our paper and resources useful, please kindly leave a star and cite our paper. Thanks!
|
||||
|
||||
```bibtex
|
||||
@article{su2022contrastive,
|
||||
title={A Contrastive Framework for Neural Text Generation},
|
||||
author={Su, Yixuan and Lan, Tian and Wang, Yan and Yogatama, Dani and Kong, Lingpeng and Collier, Nigel},
|
||||
journal={arXiv preprint arXiv:2202.06417},
|
||||
year={2022}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user