初始化项目,由ModelHub XC社区提供模型
Model: Pomni/OWoTGPT-1.3 Source: Original Platform
This commit is contained in:
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
55
README.md
Normal file
55
README.md
Normal file
@@ -0,0 +1,55 @@
|
||||
---
|
||||
language:
|
||||
- en
|
||||
base_model:
|
||||
- openai-community/gpt2
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- gpt2
|
||||
- slm
|
||||
- owot
|
||||
- gpt
|
||||
---
|
||||
|
||||
# 📜 OWoTGPT-1.3
|
||||
**OWoTGPT-1.3** is a small GPT-2 model fine-tuned on chat logs from the website [Our World of Text](https://ourworldoftext.com/). It can generate new messages based on prior chat history.
|
||||
|
||||
## Model Details
|
||||
|
||||
### Model Description
|
||||
**OWoTGPT-1.3** is a 135M parameter GPT-2 model fine-tuned on chat logs from the website [Our World of Text](https://ourworldoftext.com/). When given prior chat history, it can predict and generate what messages might come after. OWoTGPT-1.3 can also mimic a specific user if a username and ID are specified. It may attempt to mimic their tone, speaking style, and interests. If no information is supplied for the next message, the model will randomly generate it on its own and mimic the user it has picked.
|
||||
|
||||
- **Developed by:** gimmickcellar ([@PizzaTowerFanGD](https://github.com/PizzaTowerFanGD) on GitHub)
|
||||
- **Model type:** GPT-2 Small Language Model
|
||||
- **Language(s) (NLP):** English
|
||||
- **Finetuned from model:** [openai-community/gpt2](https://huggingface.co/openai-community/gpt2)
|
||||
|
||||
### Model Sources
|
||||
- **Repository:** [PizzaTowerFanGD/owotgpt](https://github.com/PizzaTowerFanGD/owotgpt) (GitHub)
|
||||
|
||||
## Uses
|
||||
This model was built for entertainment purposes and is not meant to be used in any serious applications.
|
||||
|
||||
You can either tell the model to imitate a specific user or to choose a user on its own. To let it randomly pick a user, put a random ID (a number from 1 usually up to 9999) surrounded by square brackets as the input, plus an asterisk placed in front of the ID. The model will then generate a username (for non-anon messages) after.
|
||||
|
||||
For example, putting `[*1201]` may result in `[*1201] KCUCiznob: `, while `[*719]` may result in `[*719] CarlBatman:`.
|
||||
|
||||
If you want to generate an anonymous message, put a random ID surrounded by square brackets **without** the asterisk, and add a colon right after, like this: `[2481]:`.
|
||||
|
||||
To demonstrate, putting `[3465]:` may result in the message `[3465]: your training data already exists`.
|
||||
|
||||
The model will also generate more messages after.
|
||||
|
||||
### Out-of-Scope Use
|
||||
This model will not work for any purpose other than generating OWOT chat logs.
|
||||
|
||||
## Bias, Risks, and Limitations
|
||||
* As the chat logs it was trained on include spam, the model may end up generating spam
|
||||
* It may generate offensive content (including racial slurs) as OWOT doesn't have a lot of rules
|
||||
* Its messages will likely not be very relevant to the context
|
||||
* It may not generate coherent messages at times
|
||||
* The model cannot give factual knowledge of any sort
|
||||
* The messages generated may not properly address the prior chat history
|
||||
|
||||
### Recommendations
|
||||
Follow the instructions and take note of the limitations above, and you'll be fine.
|
||||
34
config.json
Normal file
34
config.json
Normal file
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"activation_function": "gelu_new",
|
||||
"add_cross_attention": false,
|
||||
"architectures": [
|
||||
"GPT2LMHeadModel"
|
||||
],
|
||||
"attn_pdrop": 0.1,
|
||||
"bos_token_id": 50256,
|
||||
"embd_pdrop": 0.1,
|
||||
"eos_token_id": 50256,
|
||||
"initializer_range": 0.02,
|
||||
"layer_norm_epsilon": 1e-05,
|
||||
"model_type": "gpt2",
|
||||
"n_ctx": 1024,
|
||||
"n_embd": 768,
|
||||
"n_head": 12,
|
||||
"n_inner": null,
|
||||
"n_layer": 12,
|
||||
"n_positions": 1024,
|
||||
"pad_token_id": null,
|
||||
"reorder_and_upcast_attn": false,
|
||||
"resid_pdrop": 0.1,
|
||||
"scale_attn_by_inverse_layer_idx": false,
|
||||
"scale_attn_weights": true,
|
||||
"summary_activation": null,
|
||||
"summary_first_dropout": 0.1,
|
||||
"summary_proj_to_labels": true,
|
||||
"summary_type": "cls_index",
|
||||
"summary_use_proj": true,
|
||||
"tie_word_embeddings": true,
|
||||
"transformers_version": "5.3.0",
|
||||
"use_cache": true,
|
||||
"vocab_size": 50257
|
||||
}
|
||||
50001
merges.txt
Normal file
50001
merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:229a2f16b503886284db36de863fab108bc4277e40d950a62f055b4a39293f84
|
||||
size 497772432
|
||||
3
pytorch_model.bin
Normal file
3
pytorch_model.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e09b87c47c7747a8e5f6addc6809a7fcc53c2528aed7a0b2bf756398246b0c69
|
||||
size 497810035
|
||||
1
vocab.json
Normal file
1
vocab.json
Normal file
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user