初始化项目，由ModelHub XC社区提供模型

Model: cahya/gpt2-small-indonesian-522M
Source: Original Platform

.gitattributes

@@ -0,0 +1,9 @@
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tar.gz filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,65 @@
+---
+license: mit
+datasets:
+- indonesian-nlp/wikipedia-id
+language:
+- id
+metrics:
+- perplexity
+---
+
+# Indonesian GPT2 small model 
+
+## Model description
+It is GPT2-small model pre-trained with indonesian Wikipedia using a causal language modeling (CLM) objective. This 
+model is uncased: it does not make a difference between indonesia and Indonesia.
+
+This is one of several other language models that have been pre-trained with indonesian datasets. More detail about 
+its usage on downstream tasks (text classification, text generation, etc) is available at [Transformer based Indonesian Language Models](https://github.com/cahya-wirawan/indonesian-language-models/tree/master/Transformers)
+
+## Intended uses & limitations
+
+### How to use
+You can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, 
+we set a seed for reproducibility:
+```python
+>>> from transformers import pipeline, set_seed
+>>> generator = pipeline('text-generation', model='cahya/gpt2-small-indonesian-522M')
+>>> set_seed(42)
+>>> generator("Kerajaan Majapahit adalah", max_length=30, num_return_sequences=5, num_beams=10)
+
+[{'generated_text': 'Kerajaan Majapahit adalah sebuah kerajaan yang pernah berdiri di Jawa Timur pada abad ke-14 hingga abad ke-15. Kerajaan ini berdiri pada abad ke-14'}, 
+{'generated_text': 'Kerajaan Majapahit adalah sebuah kerajaan yang pernah berdiri di Jawa Timur pada abad ke-14 hingga abad ke-16. Kerajaan ini berdiri pada abad ke-14'}, 
+{'generated_text': 'Kerajaan Majapahit adalah sebuah kerajaan yang pernah berdiri di Jawa Timur pada abad ke-14 hingga abad ke-15. Kerajaan ini berdiri pada abad ke-15'}, 
+{'generated_text': 'Kerajaan Majapahit adalah sebuah kerajaan yang pernah berdiri di Jawa Timur pada abad ke-14 hingga abad ke-16. Kerajaan ini berdiri pada abad ke-15'}, 
+{'generated_text': 'Kerajaan Majapahit adalah sebuah kerajaan yang pernah berdiri di Jawa Timur pada abad ke-14 hingga abad ke-15. Kerajaan ini merupakan kelanjutan dari Kerajaan Majapahit yang'}]
+
+```
+Here is how to use this model to get the features of a given text in PyTorch:
+```python
+from transformers import GPT2Tokenizer, GPT2Model
+
+model_name='cahya/gpt2-small-indonesian-522M'
+tokenizer = GPT2Tokenizer.from_pretrained(model_name)
+model = GPT2Model.from_pretrained(model_name)
+text = "Silakan diganti dengan text apa saja."
+encoded_input = tokenizer(text, return_tensors='pt')
+output = model(**encoded_input)
+```
+and in Tensorflow:
+```python
+from transformers import GPT2Tokenizer, TFGPT2Model
+
+model_name='cahya/gpt2-small-indonesian-522M'
+tokenizer = GPT2Tokenizer.from_pretrained(model_name)
+model = TFGPT2Model.from_pretrained(model_name)
+text = "Silakan diganti dengan text apa saja."
+encoded_input = tokenizer(text, return_tensors='tf')
+output = model(encoded_input)
+```
+
+## Training data
+
+This model was pre-trained with 522MB of indonesian Wikipedia.
+The texts are tokenized using a byte-level version of Byte Pair Encoding (BPE) (for unicode characters) and 
+a vocabulary size of 52,000. The inputs are sequences of 128 consecutive tokens.

config.json

@@ -0,0 +1,26 @@
+{
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": null,
+  "n_layer": 12,
+  "n_positions": 1024,
+  "resid_pdrop": 0.1,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "vocab_size": 50257
+}

flax_model.msgpack

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79a9a8688f751b3f9ae98f1dda8716ce0432c4570ee7064ecc7cb1207555e0fa
+size 497764120

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9775e440708adbba087e2efe1bce5660b4cfdd3f22958e26b1c70528f33b75ea
+size 510378682

special_tokens_map.json

@@ -0,0 +1 @@
+{"bos_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "eos_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "unk_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}}

tf_model.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70d86c182b94efcbf32e16abcee220290040331e74d1e1bdadeb611cb2680544
+size 497933648

tokenizer_config.json

@@ -0,0 +1 @@
+{"max_len": 512}