初始化项目，由ModelHub XC社区提供模型

Model: ctxnn1/gpt2-124m-fineweb-edu-10b
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,80 @@
+---
+license: mit
+language:
+- en
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- gpt2
+- fineweb-edu
+- pretrained
+---
+
+# GPT-2 124M FineWeb-Edu 10B
+
+This is a GPT-2 124M **base text-completion model trained from scratch** on the FineWeb-Edu `sample-10BT` dataset. It is not instruction-tuned and is not a chatbot.
+
+## Try it live
+
+Run this model in a free public Gradio Space:
+
+**[Open the live demo →](https://huggingface.co/spaces/ctxnn1/gpt2-from-scratch-demo)**
+
+> The Space runs on CPU-basic (free) hardware and accepts a prompt with
+> temperature, top-k, top-p, repetition penalty, and seed controls.
+
+## Model description
+
+- Architecture: GPT-2 decoder-only Transformer
+- Parameters: 124,439,808 after export-vocabulary trimming
+- Layers / heads / hidden size: 12 / 12 / 768
+- Context length: 1,024 tokens
+- Export vocabulary: 50,257 GPT-2 tokens
+- Training tokens: 9,999,745,024
+- Hardware: one NVIDIA H100
+
+The native trainer padded its embedding/output matrix from 50,257 to 50,304 rows for efficient kernels. The final 47 rows were never tokenizer-addressable. This export keeps rows 0–50,256, sets `config.vocab_size=50257`, and preserves tied input/output embeddings, so generation cannot emit a padded ID.
+
+## Final evaluation
+
+| Metric | Value |
+|---|---:|
+| Training step | 19,073 |
+| Train loss | 3.103327 |
+| Validation loss | 3.030832 |
+| Validation perplexity | 20.714451 |
+| HellaSwag accuracy | 30.0339% (3,016/10,042) |
+
+## Intended use
+
+The model is intended for research, education, reproducibility studies, and experiments with small pretrained language models. It performs ordinary next-token completion.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "ctxnn1/gpt2-124m-fineweb-edu-10b"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id)
+inputs = tokenizer("The future of artificial intelligence is", return_tensors="pt")
+output = model.generate(**inputs, max_new_tokens=64, do_sample=True, top_k=50)
+print(tokenizer.decode(output[0], skip_special_tokens=True))
+```
+
+## Limitations, risks, and biases
+
+- This is a 124M-parameter base model and is not competitive with modern large language models.
+- It is not instruction-tuned, preference-aligned, safety-tuned, or suitable as a chatbot.
+- Outputs can be inaccurate, incoherent, biased, offensive, unsafe, or memorized from pretraining data.
+- HellaSwag accuracy is only modestly above the 25% random-choice baseline.
+- The run did not include comprehensive safety, fairness, memorization, or downstream-task evaluation.
+- Users must evaluate outputs and suitability for their own domain before deployment.
+
+## Training and conversion provenance
+
+- Source code: https://github.com/ctxnn/gpt-2
+- W&B: https://wandb.ai/ctxnn-thapar-university/gpt2-from-scratch/runs/65e78f54c14046ef99e04e12e7b3e810
+- Native checkpoint SHA-256: `e519d993d20c98c841ef061f76a1dec3e6ee24d5e55162bdea2a3e2da280fd40`
+- Training Git SHA: `36bfc9edd044eb828e118d49c79532eef8440a2a`
+- Cloud execution Git SHA: `9862792ab4024f9ebec758be73ebe7e75419d09b`
+
+The native Linear weights for attention and MLP projections were transposed into Hugging Face GPT-2 `Conv1D` orientation. Positional embeddings, LayerNorm parameters, attention/MLP projections, and tied token embeddings were preserved and validated with native-versus-Hugging-Face logit and loss comparisons.

config.json

@@ -0,0 +1,32 @@
+{
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.0,
+  "bos_token_id": 50256,
+  "dtype": "float32",
+  "embd_pdrop": 0.0,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": null,
+  "n_layer": 12,
+  "n_positions": 1024,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.0,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "transformers_version": "4.57.6",
+  "use_cache": true,
+  "vocab_size": 50257
+}

export_metadata.json

@@ -0,0 +1,132 @@
+{
+  "schema_version": 1,
+  "created_at": "2026-07-31T12:37:34.664037+00:00",
+  "source": {
+    "training_run_id": "gpt2-124m-fineweb10b-20260729t103500z-36bfc9e",
+    "checkpoint_filename": "final_step_019073.pt",
+    "checkpoint_step": 19073,
+    "checkpoint_size_bytes": 1493919114,
+    "checkpoint_sha256": "e519d993d20c98c841ef061f76a1dec3e6ee24d5e55162bdea2a3e2da280fd40",
+    "checkpoint_git_sha": "36bfc9edd044eb828e118d49c79532eef8440a2a",
+    "complete_verified": true
+  },
+  "architecture": {
+    "model_type": "GPT2LMHeadModel",
+    "n_layer": 12,
+    "n_head": 12,
+    "n_embd": 768,
+    "n_positions": 1024,
+    "native_vocab_size": 50304,
+    "export_vocab_size": 50257,
+    "parameter_count": 124439808,
+    "tied_embeddings": true
+  },
+  "mapping": {
+    "native_parameter_tensors": 149,
+    "huggingface_parameter_tensors": 149,
+    "transposed_weight_tensors": 48,
+    "transposed_weight_names": [
+      "transformer.h.0.attn.c_attn.weight",
+      "transformer.h.0.attn.c_proj.weight",
+      "transformer.h.0.mlp.c_fc.weight",
+      "transformer.h.0.mlp.c_proj.weight",
+      "transformer.h.1.attn.c_attn.weight",
+      "transformer.h.1.attn.c_proj.weight",
+      "transformer.h.1.mlp.c_fc.weight",
+      "transformer.h.1.mlp.c_proj.weight",
+      "transformer.h.2.attn.c_attn.weight",
+      "transformer.h.2.attn.c_proj.weight",
+      "transformer.h.2.mlp.c_fc.weight",
+      "transformer.h.2.mlp.c_proj.weight",
+      "transformer.h.3.attn.c_attn.weight",
+      "transformer.h.3.attn.c_proj.weight",
+      "transformer.h.3.mlp.c_fc.weight",
+      "transformer.h.3.mlp.c_proj.weight",
+      "transformer.h.4.attn.c_attn.weight",
+      "transformer.h.4.attn.c_proj.weight",
+      "transformer.h.4.mlp.c_fc.weight",
+      "transformer.h.4.mlp.c_proj.weight",
+      "transformer.h.5.attn.c_attn.weight",
+      "transformer.h.5.attn.c_proj.weight",
+      "transformer.h.5.mlp.c_fc.weight",
+      "transformer.h.5.mlp.c_proj.weight",
+      "transformer.h.6.attn.c_attn.weight",
+      "transformer.h.6.attn.c_proj.weight",
+      "transformer.h.6.mlp.c_fc.weight",
+      "transformer.h.6.mlp.c_proj.weight",
+      "transformer.h.7.attn.c_attn.weight",
+      "transformer.h.7.attn.c_proj.weight",
+      "transformer.h.7.mlp.c_fc.weight",
+      "transformer.h.7.mlp.c_proj.weight",
+      "transformer.h.8.attn.c_attn.weight",
+      "transformer.h.8.attn.c_proj.weight",
+      "transformer.h.8.mlp.c_fc.weight",
+      "transformer.h.8.mlp.c_proj.weight",
+      "transformer.h.9.attn.c_attn.weight",
+      "transformer.h.9.attn.c_proj.weight",
+      "transformer.h.9.mlp.c_fc.weight",
+      "transformer.h.9.mlp.c_proj.weight",
+      "transformer.h.10.attn.c_attn.weight",
+      "transformer.h.10.attn.c_proj.weight",
+      "transformer.h.10.mlp.c_fc.weight",
+      "transformer.h.10.mlp.c_proj.weight",
+      "transformer.h.11.attn.c_attn.weight",
+      "transformer.h.11.attn.c_proj.weight",
+      "transformer.h.11.mlp.c_fc.weight",
+      "transformer.h.11.mlp.c_proj.weight"
+    ],
+    "trimmed_embedding_rows": 47,
+    "missing_parameters": [],
+    "unexpected_parameters": []
+  },
+  "validation": {
+    "finite_parameters": {
+      "status": "passed",
+      "values_checked": 163037184,
+      "nonfinite_tensors": []
+    },
+    "native_huggingface_equivalence": {
+      "status": "passed",
+      "seed": 19073,
+      "input_shape": [
+        2,
+        24
+      ],
+      "comparison_vocabulary_size": 50257,
+      "max_absolute_logit_difference": 2.09808349609375e-05,
+      "mean_absolute_logit_difference": 8.717573223293584e-07,
+      "absolute_tolerance": 5e-05,
+      "relative_tolerance": 5e-05,
+      "native_trimmed_next_token_loss": 13.709978103637695,
+      "huggingface_next_token_loss": 13.709978103637695,
+      "absolute_loss_difference": 0.0,
+      "loss_note": "Native logits are trimmed to the exported 50,257-token vocabulary before cross-entropy; the 47 training-only padding rows are intentionally excluded."
+    },
+    "tokenizer": {
+      "status": "passed",
+      "prompts_checked": 3,
+      "vocab_size": 50257
+    },
+    "clean_process": {
+      "status": "passed",
+      "model_class": "GPT2LMHeadModel",
+      "tokenizer_class": "GPT2TokenizerFast",
+      "max_generated_id": 11666,
+      "generated_tokens": 18
+    },
+    "generated_ids_below_vocab_size": true
+  },
+  "model_safetensors_size_bytes": 497774208,
+  "model_safetensors_sha256": "6260e630dd15c0f942423d8319428922155b03fbd37e38dc84470a88b95afe6d",
+  "license": "mit",
+  "publication_ready": true,
+  "target_repo_id": "ctxnn1/gpt2-124m-fineweb-edu-10b",
+  "huggingface_validation": {
+    "status": "passed",
+    "repo_id": "ctxnn1/gpt2-124m-fineweb-edu-10b",
+    "url": "https://huggingface.co/ctxnn1/gpt2-124m-fineweb-edu-10b",
+    "model_revision": "c0b09743d3a5604a510135a7a6c0572c510a62db",
+    "downloaded_model_safetensors_sha256": "6260e630dd15c0f942423d8319428922155b03fbd37e38dc84470a88b95afe6d",
+    "generated_max_token_id": 11666
+  }
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "pad_token_id": 50256,
+  "transformers_version": "4.57.6"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6260e630dd15c0f942423d8319428922155b03fbd37e38dc84470a88b95afe6d
+size 497774208

special_tokens_map.json

@@ -0,0 +1,5 @@
+{
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "unk_token": "<|endoftext|>"
+}

tokenizer_config.json

@@ -0,0 +1,20 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": {},
+  "model_max_length": 1024,
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}