初始化项目，由ModelHub XC社区提供模型

Model: ethzanalytics/distilgpt2-tiny-conversational
Source: Original Platform

.gitattributes

@@ -0,0 +1,28 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text 
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+model.safetensors filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1 @@
+checkpoint-*/

README.md

@@ -0,0 +1,130 @@
+---
+license: apache-2.0
+tags:
+- text-generation
+- chatbot
+- dialogue
+- distilgpt2
+- gpt2
+- ai-msgbot
+
+widget:
+- text: "I know you're tired, but can we go for another walk this evening?\nperson beta:\n\n" 
+  example_title: "walk"
+- text: "Have you done anything exciting lately?\nperson beta:\n\n"
+  example_title: "activities"
+- text: "hey - do you have a favorite grocery store around here?\nperson beta:\n\n"
+  example_title: "grocery"
+- text: "Can you take me for dinner somewhere nice this time?\nperson beta:\n\n"
+  example_title: "dinner"
+- text: "What's your favorite form of social media?\nperson beta:\n\n"
+  example_title: "social media"
+- text: "Hi, how are you?\nperson beta:\n\n"
+  example_title: "greeting"
+- text: "I am the best; my sister is the worst. What am I?\nperson beta:\n\n" 
+  example_title: "sister"
+- text: "What do you call an alligator who's just had surgery to remove his left arm?\nperson beta:\n\n" 
+  example_title: "alligator"
+- text: "A man walks into a bar and asks for a drink. The bartender asks for $10, and he pays him $1. What did he pay him with?\nperson beta:\n\n" 
+  example_title: "dollar"
+- text: "What did I say was in the mailbox when it was actually in the cabinet?\nperson beta:\n\n" 
+  example_title: "mailbox"
+- text: "My friend says that she knows every language, but she doesn't speak any of them.. what's wrong with her?\nperson beta:\n\n" 
+  example_title: "language"
+
+inference:
+  parameters:
+    min_length: 2
+    max_length: 64
+    length_penalty: 0.7
+    no_repeat_ngram_size: 2
+    do_sample: True
+    top_p: 0.95
+    top_k: 20
+    temperature: 0.3
+    repetition_penalty: 3.5
+    
+---
+
+
+# distilgpt2-tiny-conversational
+
+This model is a fine-tuned version of [distilgpt2](https://huggingface.co/distilgpt2) on a parsed version of Wizard of Wikipedia. Persona alpha/beta framework designed for use with [ai-msgbot](https://github.com/pszemraj/ai-msgbot).
+It achieves the following results on the evaluation set:
+- Loss: 2.2461
+
+## Model description
+
+- a basic dialogue model for conversation. It can be used as a chatbot.
+- check out a [simple demo here](https://huggingface.co/spaces/ethzanalytics/dialogue-demo)
+
+
+## Intended uses & limitations
+
+- usage is designed for integrating with this repo: [ai-msgbot](https://github.com/pszemraj/ai-msgbot)
+- the main specific information to know is that the model generates whole conversations between two entities, `person alpha` and `person beta`. These entity names are used functionally as custom `<bos>` tokens to extract when one response ends and another begins. 
+
+## Training and evaluation data
+
+- [wizard of Wikipedia](https://parl.ai/projects/wizard_of_wikipedia/) parsed, from parlAI
+
+## Training procedure
+
+- deepspeed + huggingface trainer, an example notebook is in [ai-msgbot](https://github.com/pszemraj/ai-msgbot)
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 32
+- eval_batch_size: 32
+- seed: 42
+- distributed_type: multi-GPU
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 128
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.05
+- num_epochs: 30
+
+### Training results
+
+| Training Loss | Epoch | Step  | Validation Loss |
+|:-------------:|:-----:|:-----:|:---------------:|
+| No log        | 1.0   | 418   | 2.7793          |
+| 2.9952        | 2.0   | 836   | 2.6914          |
+| 2.7684        | 3.0   | 1254  | 2.6348          |
+| 2.685         | 4.0   | 1672  | 2.5938          |
+| 2.6243        | 5.0   | 2090  | 2.5625          |
+| 2.5816        | 6.0   | 2508  | 2.5332          |
+| 2.5816        | 7.0   | 2926  | 2.5098          |
+| 2.545         | 8.0   | 3344  | 2.4902          |
+| 2.5083        | 9.0   | 3762  | 2.4707          |
+| 2.4793        | 10.0  | 4180  | 2.4551          |
+| 2.4531        | 11.0  | 4598  | 2.4395          |
+| 2.4269        | 12.0  | 5016  | 2.4238          |
+| 2.4269        | 13.0  | 5434  | 2.4102          |
+| 2.4051        | 14.0  | 5852  | 2.3945          |
+| 2.3777        | 15.0  | 6270  | 2.3848          |
+| 2.3603        | 16.0  | 6688  | 2.3711          |
+| 2.3394        | 17.0  | 7106  | 2.3613          |
+| 2.3206        | 18.0  | 7524  | 2.3516          |
+| 2.3206        | 19.0  | 7942  | 2.3398          |
+| 2.3026        | 20.0  | 8360  | 2.3301          |
+| 2.2823        | 21.0  | 8778  | 2.3203          |
+| 2.2669        | 22.0  | 9196  | 2.3105          |
+| 2.2493        | 23.0  | 9614  | 2.3027          |
+| 2.2334        | 24.0  | 10032 | 2.2930          |
+| 2.2334        | 25.0  | 10450 | 2.2852          |
+| 2.2194        | 26.0  | 10868 | 2.2754          |
+| 2.2014        | 27.0  | 11286 | 2.2695          |
+| 2.1868        | 28.0  | 11704 | 2.2598          |
+| 2.171         | 29.0  | 12122 | 2.2539          |
+| 2.1597        | 30.0  | 12540 | 2.2461          |
+
+
+### Framework versions
+
+- Transformers 4.16.1
+- Pytorch 1.10.0+cu111
+- Tokenizers 0.11.0

config.json

@@ -0,0 +1,46 @@
+{
+  "_name_or_path": "/content/final_model",
+  "_num_labels": 1,
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": null,
+  "n_layer": 6,
+  "n_positions": 1024,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.16.1",
+  "use_cache": false,
+  "vocab_size": 50257
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8016f417451190da784b7eaff1bc38f546a9198ca3532e454bb8ac1198597b21
+size 333950622

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c2e3c90b1093fb37976deaed18fcf36c913a59de30f0cb8dbb33c2bc7b36672
+size 333971677

special_tokens_map.json

@@ -0,0 +1 @@
+{"bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "unk_token": "<|endoftext|>"}

tokenizer_config.json

@@ -0,0 +1 @@
+{"unk_token": "<|endoftext|>", "bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "add_prefix_space": false, "model_max_length": 1024, "special_tokens_map_file": null, "name_or_path": "distilgpt2", "tokenizer_class": "GPT2Tokenizer"}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83d8354bcc1c4902bf8b8dbff4bd5a46adcf8432ce6e93a791b460f3959caf98
+size 4143