初始化项目,由ModelHub XC社区提供模型
Model: hoskinson-center/proofGPT-v0.1 Source: Original Platform
This commit is contained in:
22
README.md
Normal file
22
README.md
Normal file
@@ -0,0 +1,22 @@
|
||||
---
|
||||
language:
|
||||
- en
|
||||
tags:
|
||||
- text generation
|
||||
- pytorch
|
||||
- causal-lm
|
||||
- gpt_neox
|
||||
license: mit
|
||||
datasets:
|
||||
- hoskinson-center/proof-pile
|
||||
---
|
||||
|
||||
# ProofGPT-v0.1
|
||||
|
||||
# Model Description
|
||||
ProofGPT-v0.1 is a 1.3B parameter language model based on the GPT-NeoX architecture and trained on the [proof-pile](https://huggingface.co/datasets/hoskinson-center/proof-pile) (v1.1).
|
||||
We initiailized training with pythia-1.3b weights, a precursor to the [pythia-1.4b](https://huggingface.co/EleutherAI/pythia-1.3b) model that has roughly equivalent performance.
|
||||
|
||||
Detailed evaluations coming soon :)
|
||||
|
||||
**Note**: Commit `3bcdc4e` replaced the weights with a model trained on proof-pile v1.1, as opposed to previous commits which were trained on v1.0. Commit `9695b51` updated the tokenizer to have bos, eos, and unk tokens.
|
||||
Reference in New Issue
Block a user