OnurDemircioglu/OmniGPT-355M
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: OnurDemircioglu/OmniGPT-355M
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-06 16:33:31 +08:00
commit e0d4e3e57e
19 changed files with 502561 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
.gitattributes vendored Normal file
View File

@@ -0,0 +1,35 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

57
README.md Normal file
View File

@@ -0,0 +1,57 @@
---
library_name: transformers
license: mit
base_model: OnurDemircioglu/OmniGPT-355M
tags:
- generated_from_trainer
model-index:
- name: OmniGPT-355M
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# OmniGPT-355M
This model is a fine-tuned version of [OnurDemircioglu/OmniGPT-355M](https://huggingface.co/OnurDemircioglu/OmniGPT-355M) on an unknown dataset.
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 32
- optimizer: Use OptimizerNames.ADAFACTOR and the args are:
No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 1
- mixed_precision_training: Native AMP
### Training results
### Framework versions
- Transformers 5.0.0
- Pytorch 2.10.0+cu128
- Datasets 4.8.3
- Tokenizers 0.22.2

34
config.json Normal file
View File

@@ -0,0 +1,34 @@
{
"activation_function": "gelu_new",
"add_cross_attention": false,
"architectures": [
"GPT2LMHeadModel"
],
"attn_pdrop": 0.1,
"bos_token_id": 50256,
"dtype": "float32",
"embd_pdrop": 0.1,
"eos_token_id": 50256,
"initializer_range": 0.02,
"layer_norm_epsilon": 1e-05,
"model_type": "gpt2",
"n_embd": 1024,
"n_head": 16,
"n_inner": 4096,
"n_layer": 24,
"n_positions": 256,
"pad_token_id": 50256,
"reorder_and_upcast_attn": false,
"resid_pdrop": 0.1,
"scale_attn_by_inverse_layer_idx": false,
"scale_attn_weights": true,
"summary_activation": null,
"summary_first_dropout": 0.1,
"summary_proj_to_labels": true,
"summary_type": "cls_index",
"summary_use_proj": true,
"tie_word_embeddings": true,
"transformers_version": "5.0.0",
"use_cache": false,
"vocab_size": 50257
}

10
generation_config.json Normal file
View File

@@ -0,0 +1,10 @@
{
"_from_model_config": true,
"bos_token_id": 50256,
"eos_token_id": 50256,
"output_attentions": false,
"output_hidden_states": false,
"pad_token_id": 50256,
"transformers_version": "5.0.0",
"use_cache": true
}

34
last-checkpoint/config.json Normal file
View File

@@ -0,0 +1,34 @@
{
"activation_function": "gelu_new",
"add_cross_attention": false,
"architectures": [
"GPT2LMHeadModel"
],
"attn_pdrop": 0.1,
"bos_token_id": 50256,
"dtype": "float32",
"embd_pdrop": 0.1,
"eos_token_id": 50256,
"initializer_range": 0.02,
"layer_norm_epsilon": 1e-05,
"model_type": "gpt2",
"n_embd": 1024,
"n_head": 16,
"n_inner": 4096,
"n_layer": 24,
"n_positions": 256,
"pad_token_id": 50256,
"reorder_and_upcast_attn": false,
"resid_pdrop": 0.1,
"scale_attn_by_inverse_layer_idx": false,
"scale_attn_weights": true,
"summary_activation": null,
"summary_first_dropout": 0.1,
"summary_proj_to_labels": true,
"summary_type": "cls_index",
"summary_use_proj": true,
"tie_word_embeddings": true,
"transformers_version": "5.0.0",
"use_cache": false,
"vocab_size": 50257
}

10
last-checkpoint/generation_config.json Normal file
View File

@@ -0,0 +1,10 @@
{
"_from_model_config": true,
"bos_token_id": 50256,
"eos_token_id": 50256,
"output_attentions": false,
"output_hidden_states": false,
"pad_token_id": 50256,
"transformers_version": "5.0.0",
"use_cache": true
}

3
last-checkpoint/model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6810aa86eb3583146f230bca1aa2688ecacf9e5c73867ee8ae476ea19e44f9a3
size 1416177152

3
last-checkpoint/optimizer.pt Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d2c716fc654048b0b6501ad5afb5838c8fb45d6fbe8499d961df7eaba9dce115
size 3263499

3
last-checkpoint/rng_state.pth Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7e6949b0369630a7b27cbe408bc2eda535fe6230c94463f16c249620dba2c375
size 14645

3
last-checkpoint/scaler.pt Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4b7316cbdaabfb127fe62edf882e64cc9cc03e628fa8969a9ffb1f4a9528c303
size 1383

3
last-checkpoint/scheduler.pt Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:832d0dc34bcd2a4080a6288ca40823bec904bbcf6b5d8ea7c75eb7dfac61a0b7
size 1465

250306
last-checkpoint/tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

12
last-checkpoint/tokenizer_config.json Normal file
View File

@@ -0,0 +1,12 @@
{
"add_prefix_space": false,
"backend": "tokenizers",
"bos_token": "<|endoftext|>",
"eos_token": "<|endoftext|>",
"errors": "replace",
"is_local": false,
"model_max_length": 1024,
"pad_token": "<|endoftext|>",
"tokenizer_class": "GPT2Tokenizer",
"unk_token": "<|endoftext|>"
}

1721
last-checkpoint/trainer_state.json Normal file
View File

File diff suppressed because it is too large Load Diff

3
last-checkpoint/training_args.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:aefa1a2d28a7234129e6f96653865e46363957dbaed20d364831c0c0bdf8148d
size 5201

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6810aa86eb3583146f230bca1aa2688ecacf9e5c73867ee8ae476ea19e44f9a3
size 1416177152

250306
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

12
tokenizer_config.json Normal file
View File

@@ -0,0 +1,12 @@
{
"add_prefix_space": false,
"backend": "tokenizers",
"bos_token": "<|endoftext|>",
"eos_token": "<|endoftext|>",
"errors": "replace",
"is_local": false,
"model_max_length": 1024,
"pad_token": "<|endoftext|>",
"tokenizer_class": "GPT2Tokenizer",
"unk_token": "<|endoftext|>"
}

3
training_args.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:aefa1a2d28a7234129e6f96653865e46363957dbaed20d364831c0c0bdf8148d
size 5201