初始化项目,由ModelHub XC社区提供模型

Model: gvadhul/gpt2-2layer-1m
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-28 12:58:18 +08:00
commit 2ad796c129
7 changed files with 853 additions and 0 deletions

48
README.md Normal file
View File

@@ -0,0 +1,48 @@
---
language:
- en
license: apache-2.0
tags:
- gpt2
- causal-lm
- text-generation
- tiny
- testing
pipeline_tag: text-generation
---
# gpt2-2layer-1m
A **randomly-initialized**, 2-layer GPT-2 model for functional / integration testing.
It produces incoherent text — that is by design.
## Architecture
| Hyperparameter | Value |
|---|---|
| Architecture | GPT-2 (decoder-only) |
| Layers | 2 |
| Hidden size | 256 |
| Attention heads | 4 |
| FFN inner size | 1 024 |
| Context length | 512 |
| Vocabulary size | 183 (byte-level BPE) |
| Total params | ~1.76 M |
| Non-embedding params | ~1.58 M |
## Usage
```python
from transformers import GPT2LMHeadModel, PreTrainedTokenizerFast
tokenizer = PreTrainedTokenizerFast.from_pretrained("gvadhul/gpt2-2layer-1m")
model = GPT2LMHeadModel.from_pretrained("gvadhul/gpt2-2layer-1m")
inputs = tokenizer("Hello world", return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(out[0]))
```
## Intended use
Functional / unit testing of pipelines that need a *tiny* causal LM.
**Not suitable for any real NLP task** — weights are random.