初始化项目,由ModelHub XC社区提供模型
Model: cambridgeltl/simctg_wikitext103 Source: Original Platform
This commit is contained in:
67
README.md
Normal file
67
README.md
Normal file
@@ -0,0 +1,67 @@
|
||||
This model provides a GPT-2 language model trained with SimCTG on the Wikitext-103 benchmark [(Merity et al., 2016)](https://arxiv.org/abs/1609.07843) based on our paper [_A Contrastive Framework for Neural Text Generation_](https://arxiv.org/abs/2202.06417).
|
||||
|
||||
We provide a detailed tutorial on how to apply SimCTG and Contrastive Search in our [project repo](https://github.com/yxuansu/SimCTG#4-huggingface-style-tutorials-back-to-top). In the following, we illustrate a brief tutorial on how to use our approach to perform text generation.
|
||||
|
||||
## 1. Installation of SimCTG:
|
||||
```yaml
|
||||
pip install simctg --upgrade
|
||||
```
|
||||
|
||||
## 2. Initialize SimCTG Model:
|
||||
```python
|
||||
import torch
|
||||
# load SimCTG language model
|
||||
from simctg.simctggpt import SimCTGGPT
|
||||
model_name = r'cambridgeltl/simctg_wikitext103'
|
||||
model = SimCTGGPT(model_name)
|
||||
model.eval()
|
||||
tokenizer = model.tokenizer
|
||||
```
|
||||
|
||||
## 3. Prepare the Text Prefix:
|
||||
```python
|
||||
prefix_text = r"Butt criticized Donald 's controls in certain situations in the game , as well as the difficulty of some levels and puzzles .
|
||||
Buchanan also criticized the controls , calling"
|
||||
print ('Prefix is: {}'.format(prefix_text))
|
||||
tokens = tokenizer.tokenize(prefix_text)
|
||||
input_ids = tokenizer.convert_tokens_to_ids(tokens)
|
||||
input_ids = torch.LongTensor(input_ids).view(1,-1)
|
||||
```
|
||||
|
||||
## 4. Generate Text with Contrastive Search:
|
||||
```python
|
||||
beam_width, alpha, decoding_len = 8, 0.6, 128
|
||||
output = model.fast_contrastive_search(input_ids=input_ids, beam_width=beam_width,
|
||||
alpha=alpha, decoding_len=decoding_len)
|
||||
|
||||
print("Output:\n" + 100 * '-')
|
||||
print(tokenizer.decode(output))
|
||||
'''
|
||||
Prefix is: Butt criticized Donald 's controls in certain situations in the game , as well as the difficulty of some levels and puzzles .
|
||||
Buchanan also criticized the controls , calling
|
||||
Output:
|
||||
----------------------------------------------------------------------------------------------------
|
||||
Butt criticized Donald's controls in certain situations in the game, as well as the difficulty of some levels and puzzles. Buchanan also
|
||||
criticized the controls, calling them " unimpressive " and a " nightmare " of an experience to play with players unfamiliar with Tetris.
|
||||
On the other hand, his opinion was shared by other reviewers, and some were critical of the game's technical design for the Wii version
|
||||
of Tetris. In addition, Tintin's review included a quote from Roger Ebert, who said that Tetris was better than the original game due to
|
||||
its simplicity and ease of play. Ebert's comments were included in the game's DVD commentary, released on March 22, 2010. It is unclear
|
||||
if any of the video commentary was taken from the DVD
|
||||
'''
|
||||
```
|
||||
|
||||
For more details of our work, please refer to our main [project repo](https://github.com/yxuansu/SimCTG).
|
||||
|
||||
## 5. Citation:
|
||||
If you find our paper and resources useful, please kindly leave a star and cite our paper. Thanks!
|
||||
|
||||
```bibtex
|
||||
@article{su2022contrastive,
|
||||
title={A Contrastive Framework for Neural Text Generation},
|
||||
author={Su, Yixuan and Lan, Tian and Wang, Yan and Yogatama, Dani and Kong, Lingpeng and Collier, Nigel},
|
||||
journal={arXiv preprint arXiv:2202.06417},
|
||||
year={2022}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user