初始化项目，由ModelHub XC社区提供模型

Model: lorcannrauzduel/gpt2-citations Source: Original Platform
2026-08-08 02:35:17 +08:00
commit 843cfb54f8
7 changed files with 250573 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,170 @@
+---
+library_name: transformers
+tags:
+- text-generation
+- gpt2
+- fine-tuned
+- citations
+- causal-lm
+---
+
+# GPT‑2 Fine‑tuned on English Quotes
+
+## Model Description
+
+This model is a fine‑tuned version of [GPT‑2 small](https://huggingface.co/gpt2) (124M parameters) on the [Abirate/english_quotes](https://huggingface.co/datasets/Abirate/english_quotes) dataset.  
+The goal is to generate text in the style of philosophical or literary quotes, including the author’s name.
+
+**⚠️ This model was created for educational and research purposes only. It is not intended for production use.**  
+It demonstrates full fine‑tuning of a causal language model on a small dataset and the improvements in generation quality compared to the base model.
+
+**Base model**: `gpt2`  
+**Task**: Causal language modelling (text generation)  
+**Fine‑tuning type**: Full fine‑tuning (all parameters updated)
+
+## Intended Uses & Limitations
+
+### Direct Use (Research / Experimentation)
+
+You can use this model to generate short quotes given a prompt. The model expects prompts to start with the special token `<|startoftext|>` and will learn to produce a quote followed by an author and the `<|endoftext|>` token.
+
+**Example**:
+```python
+from transformers import pipeline
+
+generator = pipeline("text-generation", model="lorcannrauzduel/gpt2-citations")
+output = generator("<|startoftext|> The secret to", max_new_tokens=50, do_sample=True)
+print(output[0]['generated_text'])
+```
+
+### Limitations
+
+- The model is small (124M) and was trained on only ~2,500 quotes. It may sometimes produce repetitive or nonsensical outputs.
+- It only generates English text.
+- It does not have factual knowledge about the authors; it merely mimics the style of the training quotes.
+- **Not suitable for any commercial or critical application.**
+
+## Training Details
+
+### Training Data
+
+- **Dataset**: [Abirate/english_quotes](https://huggingface.co/datasets/Abirate/english_quotes) – 2,508 quotes, each with a `quote` and an `author` field.
+- **Preprocessing**: Each example was formatted as:
+  ```
+  <|startoftext|> "quote" — author <|endoftext|>
+  ```
+  The special tokens help the model learn where a quote starts and ends.
+
+### Training Procedure
+
+The model was trained for 5 epochs using the Hugging Face `Trainer` with the following hyperparameters:
+
+| Hyperparameter          | Value |
+|------------------------|-------|
+| Learning rate          | 5e-5  |
+| Batch size (per device)| 8     |
+| Gradient accumulation  | 2     |
+| Effective batch size   | 16    |
+| Warmup steps           | 100   |
+| Weight decay           | 0.01  |
+| Optimizer              | AdamW |
+| Precision              | fp16  |
+| Max sequence length    | 128   |
+| Training steps         | 1410  |
+
+**Hardware**: NVIDIA Tesla T4 (15 GB VRAM) on Google Colab / Kaggle.  
+**Training time**: ~5 minutes.
+
+### Evaluation Results
+
+The final training loss was **2.506**, corresponding to a perplexity of **12.26**.  
+Validation loss stagnated around 2.30, indicating a slight overfitting after 3‑4 epochs – acceptable for a small generative model.
+
+## How to Use the Model
+
+### With 🤗 Transformers
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("lorcannrauzduel/gpt2-citations")
+model = AutoModelForCausalLM.from_pretrained("lorcannrauzduel/gpt2-citations")
+
+prompt = "<|startoftext|> Life is"
+inputs = tokenizer(prompt, return_tensors="pt")
+output = model.generate(**inputs, max_new_tokens=50, do_sample=True, temperature=0.9)
+print(tokenizer.decode(output[0], skip_special_tokens=False))
+```
+
+### With Pipeline
+
+```python
+from transformers import pipeline
+
+pipe = pipeline("text-generation", model="lorcannrauzduel/gpt2-citations")
+print(pipe("<|startoftext|> You can never", max_new_tokens=50)[0]['generated_text'])
+```
+
+### With vLLM (for high‑throughput inference)
+
+```bash
+pip install vllm
+vllm serve "lorcannrauzduel/gpt2-citations"
+```
+
+Then query with curl:
+
+```bash
+curl -X POST "http://localhost:8000/v1/completions" \
+     -H "Content-Type: application/json" \
+     --data '{
+         "model": "lorcannrauzduel/gpt2-citations",
+         "prompt": "<|startoftext|> The secret to",
+         "max_tokens": 50,
+         "temperature": 0.8
+     }'
+```
+
+### With Ollama (local deployment after GGUF conversion)
+
+1. Download the GGUF version from the repository (if available) or convert it yourself using `llama.cpp`.
+2. Create a `Modelfile`:
+   ```
+   FROM ./gpt2-citations-q4km.gguf
+   SYSTEM "You are a quote generator."
+   PARAMETER temperature 0.8
+   PARAMETER stop "<|endoftext|>"
+   ```
+3. Import and run:
+   ```bash
+   ollama create gpt2-citations -f Modelfile
+   ollama run gpt2-citations "<|startoftext|> Life is"
+   ```
+
+## Model Comparison (Base vs Fine‑tuned)
+
+| Prompt | GPT‑2 Base (no fine‑tuning) | GPT‑2 Fine‑tuned |
+|--------|-----------------------------|------------------|
+| `<|startoftext|> The secret to` | "The secret to making the most of your life... [[...]]" | "The secret to happiness is to trust your instincts rather than your brain.” — Albert Einstein" |
+| `<|startoftext|> Life is` | "Life is precious and life is precious..." | "Life is full of opportunities, but few opportunities are worth the time..." — Jodi Picoult |
+| `<|startoftext|> You can never` | "You can never get around to finding out what you want..." | "You can never lose your way because you are still thinking about the things you've done..." |
+
+The fine‑tuned model consistently produces coherent quotes with an author attribution, while the base model generates irrelevant or repetitive text.
+
+## Environmental Impact
+
+Training was performed on a cloud GPU (Tesla T4) for about 5 minutes. Estimated CO₂ emissions are negligible (< 0.01 kg CO₂eq).
+
+## Acknowledgements
+
+- The [Hugging Face](https://huggingface.co) team for `transformers` and `datasets`.
+- The original GPT‑2 paper by Radford et al. (2019).
+- Dataset provided by [Abirate](https://huggingface.co/Abirate).
+
+## License
+
+This model is released under the MIT license (same as the original GPT‑2 small).
+
+---
+
+**Model card created by [lorcannrauzduel](https://huggingface.co/lorcannrauzduel) for research and experimentation purposes.**
--- a/config.json
+++ b/config.json
@@ -0,0 +1,41 @@
+{
+  "activation_function": "gelu_new",
+  "add_cross_attention": false,
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "dtype": "float32",
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": null,
+  "n_layer": 12,
+  "n_positions": 1024,
+  "pad_token_id": null,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.0.0",
+  "use_cache": false,
+  "vocab_size": 50257
+}
--- a/generation_config.json
+++ b/generation_config.json
@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "transformers_version": "5.0.0"
+}
--- a/model.safetensors
+++ b/model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:553c77589397bc6e11471dd939f5cac0683e226464591f4a7a7889a930816a12
+size 497774208
--- a/tokenizer.json
+++ b/tokenizer.json
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
@@ -0,0 +1,12 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "is_local": false,
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}