初始化项目，由ModelHub XC社区提供模型

Model: Playingyoyo/GPepT
Source: Original Platform

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+TOC.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,102 @@
+---
+license: apache-2.0
+pipeline_tag: text-generation
+widget:
+- text: <|endoftext|>
+inference:
+  parameters:
+    top_k: 950
+    repetition_penalty: 1.2
+---
+
+# **GPepT: A Language Model for Peptides and Peptidomimetics**
+![alt text](TOC.png)
+
+GPepT is a cutting-edge language model designed to understand and generate sequences in the specialized domain of peptides and peptidomimetics. It serves as a powerful tool for _de novo_ protein design and engineering. As demonstrated in our research, the incorporation of peptidomimetics significantly broadens the chemical space accessible through generated sequences, enabling innovative approaches to peptide-based therapeutics.
+
+## **Model Overview**
+GPepT builds upon the GPT-2 Transformer architecture, comprising 36 layers and a model dimensionality of 1280, with a total of 738 million parameters. This decoder-only model has been pre-trained on a curated dataset of peptides and peptidomimetics mined from bioactivity-labeled chemical formulas in ChEMBL.
+
+To leverage GPepT’s pre-trained weights, input molecules must be converted into a standardized sequence-like representation of peptidomimetics using [**Monomerizer**](https://github.com/tsudalab/Monomerizer/tree/main). Detailed insights into the training process and datasets are provided in our accompanying publication.
+
+Unlike traditional protein design models, GPepT is trained in a self-supervised manner, using raw sequence data without explicit annotation. This design enables the model to generalize across diverse sequence spaces, producing functional antimicrobial peptidomimetics upon fine-tuning.
+
+SMILES representation, and selected chemical properties of each token, which corresponds to a non-canonical amino acid or terminal modification.
+
+---
+
+## **Using GPepT for Sequence Generation**
+GPepT is fully compatible with the HuggingFace Transformers Python library. Installation instructions can be found [here](https://huggingface.co/docs/transformers/installation).
+
+The model excels at generating peptidomimetic sequences in a zero-shot fashion, but it can also be fine-tuned on custom datasets to generate sequences tailored to specific requirements.
+
+
+### **Example 1: Zero-Shot Sequence Generation**
+GPepT generates sequences that extend from a specified input token (e.g., `<|endoftext|>`). If no input is provided, it selects the start token automatically and generates likely sequences. Here’s a Python example:
+
+```python
+from transformers import pipeline
+
+# Initialize GPepT for text generation
+GPepT = pipeline('text-generation', model="Playingyoyo/GPepT")
+
+# Generate sequences
+sequences = GPepT("<|endoftext|>", 
+                   max_length=25, 
+                   do_sample=True, 
+                   top_k=950, 
+                   repetition_penalty=1.5, 
+                   num_return_sequences=5, 
+                   eos_token_id=0)
+
+# Print generated sequences
+for seq in sequences:
+    print(seq['generated_text'])
+```
+
+Sample output:
+```
+<|endoftext|>R K A L E Z1649
+<|endoftext|>G K A L Z341
+<|endoftext|>G V A G K X4097 V A P
+```
+
+---
+
+### **Example 2: Fine-Tuning for Directed Sequence Generation**
+Fine-tuning enables GPepT to generate sequences with user-defined properties. To prepare training data:
+1. ```git clone https://github.com/tsudalab/Monomerizer/tree/main```
+2. ```cd Monomerizer```
+3. ```python3 Monomerizer/run_pipeline.py --input_file path_to_your_smiles_file.txt```. Check the repo for the required format.
+4. 3. will monomerize the SMILES and split the resulting sequences into training (`output/datetime/for_GPepT/train90.txt`) and validation (`output/datetime/for_GPepT/val10.txt`) files.
+
+To fine-tune the model:
+
+```bash
+python run_clm.py --model_name_or_path Playingyoyo/GPepT \
+                  --train_file path_to_train90.txt \
+                  --validation_file path_to_val10.txt \
+                  --tokenizer_name Playingyoyo/GPepT \
+                  --do_train \
+                  --do_eval \
+                  --output_dir ./output \
+                  --learning_rate 1e-5
+```
+
+Refer to the HuggingFace [script run_clm.py](https://github.com/huggingface/transformers/blob/master/examples/pytorch/language-modeling/run_clm.py) and [requirements.txt](https://huggingface.co/Playingyoyo/GPepT/blob/main/requirements.txt).
+Note that train90.txt and val10.txt need to be at least 50 samples long.
+
+The fine-tuned model will be saved in the `./output` directory, ready to generate tailored sequences.
+
+---
+
+## **Selecting Valid Sequences**
+While GPepT generates diverse peptidomimetic sequences, not all are chemically valid. For example:
+- **Invalid Sequences:** Those with terminal modifications (e.g., `Z`) embedded within the sequence.
+- **Valid Sequences:** Should adhere to standard peptidomimetic rules.
+
+By filtering out invalid sequences, GPepT users can ensure the generation of high-quality candidates for further study.
+
+--- 
+
+GPepT stands as a powerful tool for researchers at the forefront of peptide and peptidomimetic innovation, enabling both exploration and application in vast chemical and biological spaces.

TOC.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b665351aceafb2aabce68afb8178806a28486137ec02d5533fb3bc7072873053
+size 581854

config.json

@@ -0,0 +1,39 @@
+{
+  "_name_or_path": "Playingyoyo/GPepT",
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 0,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 0,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 1280,
+  "n_head": 20,
+  "n_inner": null,
+  "n_layer": 36,
+  "n_positions": 1024,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.0.dev0",
+  "use_cache": true,
+  "vocab_size": 50257
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "eos_token_id": 0,
+  "transformers_version": "4.48.0.dev0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:758c70c7fe9fb3e451481e9bcfb3143f4e9ea088096803293f6ef1762708ec7f
+size 3096165928

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:26eca24280cb0530e481f217c471ad960f2a38daf793967598d0ba97d62c7280
+size 3134027901

requirements.txt

@@ -0,0 +1,171 @@
+absl-py==2.0.0
+accelerate==1.1.1
+aiohappyeyeballs==2.4.3
+aiohttp==3.11.8
+aiosignal==1.3.1
+alembic==1.14.0
+appnope==0.1.4
+asttokens==2.4.1
+astunparse==1.6.3
+async-timeout==5.0.1
+attrs==23.2.0
+backcall==0.2.0
+beautifulsoup4==4.13.4
+bleach==6.2.0
+blobfile==3.0.0
+bokeh==3.4.2
+cachetools==5.3.1
+cattrs==23.2.3
+certifi==2023.7.22
+charset-normalizer==3.3.0
+chembl-webresource-client==0.10.9
+click==8.1.7
+cloudpickle==3.0.0
+colorlog==6.9.0
+contourpy==1.2.1
+cycler==0.12.0
+datasets==3.1.0
+decorator==5.1.1
+defusedxml==0.7.1
+dill==0.3.6
+docopt==0.6.2
+docstring_parser==0.16
+easydict==1.13
+einops==0.8.1
+evaluate==0.4.3
+exceptiongroup==1.2.1
+executing==2.0.1
+fastjsonschema==2.21.1
+filelock==3.12.4
+flatbuffers==23.5.26
+fonttools==4.43.0
+frozenlist==1.5.0
+fsspec==2024.9.0
+future==1.0.0
+gast==0.4.0
+google-auth==2.23.2
+google-auth-oauthlib==0.4.6
+google-pasta==0.2.0
+greenlet==3.1.1
+grpcio==1.59.0
+h5py==3.9.0
+huggingface-hub==0.26.3
+hyperopt==0.2.7
+idna==3.4
+ipython==8.12.3
+jedi==0.19.1
+Jinja2==3.1.2
+joblib==1.4.0
+JPype1==1.4.1
+jsonpickle==3.0.4
+jsonschema==4.23.0
+jsonschema-specifications==2025.4.1
+jupyter_client==8.6.3
+jupyter_core==5.7.2
+jupyterlab_pygments==0.3.0
+keras==2.10.0
+Keras-Preprocessing==1.1.2
+kiwisolver==1.4.5
+libclang==16.0.6
+lxml==5.3.0
+Mako==1.3.8
+Markdown==3.4.4
+MarkupSafe==2.1.3
+matplotlib==3.8.0
+matplotlib-inline==0.1.7
+mistune==3.1.3
+mpmath==1.3.0
+multidict==6.1.0
+multiprocess==0.70.14
+mypy-extensions==1.0.0
+nbclient==0.10.2
+nbconvert==7.16.6
+nbformat==5.10.4
+networkx==3.1
+numpy==1.26.0
+oauthlib==3.2.2
+openTSNE==1.0.1
+opt-einsum==3.3.0
+optimum @ git+https://github.com/huggingface/optimum.git@a6c696c7de105e7691d432dd80102beec78d8fd4
+optuna==4.1.0
+packaging==23.2
+pandas==2.2.2
+pandocfilters==1.5.1
+parso==0.8.4
+pexpect==4.9.0
+pickleshare==0.7.5
+Pillow==10.0.1
+pipreqs==0.5.0
+platformdirs==4.2.2
+prompt-toolkit==3.0.43
+propcache==0.2.0
+protobuf==3.20.3
+psutil==6.1.0
+ptyprocess==0.7.0
+pure-eval==0.2.2
+py4j==0.10.9.7
+pyarrow==18.1.0
+pyasn1==0.5.0
+pyasn1-modules==0.3.0
+pycryptodomex==3.21.0
+Pygments==2.17.2
+pyparsing==3.1.1
+pystow==0.5.0
+python-dateutil==2.8.2
+python-louvain==0.16
+pytz==2024.1
+pyvis==0.3.2
+PyYAML==6.0.1
+pyzmq==26.4.0
+rdkit-pypi==2022.9.5
+referencing==0.36.2
+regex==2024.11.6
+requests==2.32.3
+requests-cache==1.2.0
+requests-oauthlib==1.3.1
+responses==0.18.0
+rpds-py==0.24.0
+rsa==4.9
+safetensors==0.4.5
+scikit-learn==1.4.2
+scipy==1.13.0
+seaborn==0.13.2
+sentencepiece==0.2.0
+six==1.16.0
+soupsieve==2.7
+SQLAlchemy==2.0.36
+stack-data==0.6.3
+STOUT-pypi==2.0.5
+sympy==1.12
+tensorboard==2.10.1
+tensorboard-data-server==0.6.1
+tensorboard-plugin-wit==1.8.1
+tensorboardX==2.6.2.2
+tensorflow==2.10.1
+tensorflow-estimator==2.10.0
+tensorflow-io-gcs-filesystem==0.34.0
+termcolor==2.3.0
+threadpoolctl==3.4.0
+tiktoken==0.8.0
+tinycss2==1.4.0
+tokenizers==0.20.3
+torch==2.0.1
+tornado==6.4.1
+tqdm==4.67.1
+traitlets==5.14.3
+transformers==4.46.3
+typed-argument-parser==1.10.0
+typing-inspect==0.9.0
+typing_extensions==4.8.0
+tzdata==2024.1
+unicodedata2==15.1.0
+url-normalize==1.4.3
+urllib3==2.0.5
+wcwidth==0.2.13
+webencodings==0.5.1
+Werkzeug==3.0.0
+wrapt==1.15.0
+xxhash==3.5.0
+xyzservices==2024.6.0
+yarg==0.1.9
+yarl==1.18.0

special_tokens_map.json

@@ -0,0 +1,9 @@
+{
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": false,
+  "extra_special_tokens": {},
+  "max_length": 42,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|endoftext|>",
+  "padding": "max_length",
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}