初始化项目，由ModelHub XC社区提供模型

Model: cglez/gpt2-ag_news
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,97 @@
+---
+library_name: transformers
+language: en
+license: mit
+datasets:
+- fancyzhx/ag_news
+base_model:
+- openai-community/gpt2
+---
+
+# Model Card: GPT-2-AG-News
+
+An in-domain GPT-2, pre-trained from scratch on the AG-News dataset texts.
+
+## Model Details
+
+### Description
+
+This model is based on the [GPT-2](https://huggingface.co/openai-community/gpt2)
+architecture and was pre-trained from scratch (in-domain) using the text in AG-News dataset, excluding its test split.
+
+- **Developed by:** [Cesar Gonzalez-Gutierrez](https://ceguel.es)
+- **Funded by:** [ERC](https://erc.europa.eu)
+- **Architecture:** GPT-2
+- **Language:** English
+- **License:** MIT
+- **Base model:** [GPT-2](https://huggingface.co/openai-community/gpt2)
+
+### Checkpoints
+
+Intermediate checkpoints from the pre-training process are available and can be accessed using specific tags,
+which correspond to training epochs and steps:
+
+| Epoch | Step | Tags | |
+|---|---|---|---|
+| 1 | 1125 | epoch-1 | step-1125 |
+| 5 | 5625 | epoch-5 | step-5625 |
+| 10 | 11250 | epoch-10 | step-11250 |
+| 20 | 22500 | epoch-20 | step-22500 |
+| 30 | 33750 | epoch-30 | step-33750 |
+| 40 | 45000 | epoch-40 | step-45000 |
+| 50 | 56250 | epoch-50 | step-56250 |
+| 60 | 67500 | epoch-60 | step-67500 |
+| 70 | 78750 | epoch-70 | step-78750 |
+| 80 | 90000 | epoch-80 | step-90000 |
+| 90 | 101250 | epoch-90 | step-101250 |
+| 100 | 112500 | epoch-100 | step-112500 |
+
+To load a model from a specific intermediate checkpoint, use the `revision` parameter with the corresponding tag:
+```python
+from transformers import AutoModelForCausalLM
+
+model = AutoModelForMaskedLM.from_pretrained("<model-name>", revision="<checkpoint-tag>")
+```
+
+### Sources
+
+- **Paper:** [Information pending]
+
+## Training Details
+
+For more details on the training procedure, please refer to the base model's documentation:
+[Training procedure](https://huggingface.co/openai-community/gpt2#training-procedure).
+
+### Training Data
+
+All texts from AG-News dataset, excluding the test partition.
+
+#### Training Hyperparameters
+
+- **Precision:** fp16
+- **Batch size:** 8
+- **Gradient accumulation steps:** 12
+
+## Uses
+
+For typical use cases and limitations, please refer to the base model's guidance: 
+[Inteded uses & limitations](https://huggingface.co/openai-community/gpt2#intended-uses--limitations).
+
+## Bias, Risks, and Limitations
+
+This model inherits potential risks and limitations from the base model. Refer to:
+[Limitations and bias](https://huggingface.co/openai-community/gpt2#limitations-and-bias).
+
+## Environmental Impact
+
+- **Hardware Type:** NVIDIA A100 PCIE 40GB
+- **Hours used:** 15
+- **Cluster Provider:** [Artemisa](https://artemisa.ific.uv.es/web/)
+- **Compute Region:** EU
+- **Carbon Emitted:** 1.62 kg CO2 eq.
+
+## Citation
+
+**BibTeX:**
+
+[More Information Needed]

config.json

@@ -0,0 +1,32 @@
+{
+  "_name_or_path": "output/pretrained_model/gpt2-ag_news/checkpoint-112500",
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": null,
+  "n_layer": 12,
+  "n_positions": 1024,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.37.2",
+  "use_cache": true,
+  "vocab_size": 50257
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "transformers_version": "4.37.2"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48cd81178fd5471d4d30f3a5b8aa8d794d21ba0ac8412467ac505d471078027d
+size 497774208

runs/tensorboard/events.out.tfevents.1755456105.mlwn16.ific.uv.es.138483.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:167174021be4f37e6f591472bb67ae7cc31736d37e754283ec70ae3c1c6cec9b
+size 22225

runs/tensorboard/events.out.tfevents.1755681376.mlwn05.ific.uv.es.2283059.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7163d2f8e081ad4463f2db694120d4e882c2c98ad02510197b99a89eb3e5c639
+size 4898

runs/tensorboard/events.out.tfevents.1755690372.mlwn11.ific.uv.es.2530255.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4013a10cceae0bf1400d18f8542493214ee2f392b176ecdae08a8b42b787947d
+size 4897

runs/tensorboard/events.out.tfevents.1755690722.mlwn11.ific.uv.es.2530518.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16e90c189357575c984ebe18112c051b11b814d64a32a148886a479279f586f4
+size 14438

runs/tensorboard/events.out.tfevents.1755864350.mlwn28.ific.uv.es.2654249.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5a5c6af813f4ef9b3e784b2b77144f4050b20f3d26a64ae3e67a789dcfe8ee9
+size 35386

runs/tensorboard/events.out.tfevents.1756045772.mlwn27.ific.uv.es.598775.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5082f55a3c6fd147c150f8d77609217cf110687a7012fb6687feb82258874c4
+size 10743

special_tokens_map.json

@@ -0,0 +1,23 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 1024,
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}