初始化项目，由ModelHub XC社区提供模型

Model: Pranilllllll/finetuned_gpt2_45krows_10epochs Source: Original Platform
2026-07-10 16:55:23 +08:00
commit 750599c3e1
10 changed files with 300683 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,241 @@
+---
+library_name: transformers
+tags:
+  - gpt2
+  - text-generation
+  - causal-lm
+  - fine-tuned
+  - mental-health
+  - psychology
+  - counseling
+  - conversational-ai
+license: mit
+language:
+  - en
+pipeline_tag: text-generation
+---
+
+# Model Card — Fine-tuned GPT-2 on Mental Health & Psychology Datasets (45K rows, 10 Epochs)
+
+## Model Description
+
+This model is a fine-tuned version of **GPT-2** on a combined dataset of ~45,000 mental health and psychology conversation samples across 6 datasets. It is a **causal language model** trained to generate empathetic, contextually appropriate responses to mental health-related prompts — making it suitable for counseling conversation research, mental health chatbot prototyping, and psychology NLP tasks.
+
+- **Developed by:** [praniil](https://github.com/praniil)
+- **Model type:** Causal Language Model (GPT-2)
+- **Language(s):** English
+- **License:** MIT
+- **Finetuned from model:** `gpt2` (OpenAI GPT-2 124M, via Hugging Face)
+
+### Model Sources
+
+- **Repository:** [https://github.com/praniil/finetuned_gpt2_45krows_n5](https://github.com/praniil/finetuned_gpt2_45krows_n5)
+- **HuggingFace Hub:** `Pranilllllll/finetuned_gpt2_45krows_10epochs`
+
+---
+
+## Uses
+
+### Direct Use
+
+This model can be used out-of-the-box for **mental health and psychology text generation** — given a user message or question as a prompt, it generates a response in the style of a counseling conversation.
+```python
+from transformers import GPT2LMHeadModel, GPT2Tokenizer
+import torch
+
+model_name = "Pranilllllll/finetuned_gpt2_45krows_10epochs"
+tokenizer = GPT2Tokenizer.from_pretrained(model_name)
+model = GPT2LMHeadModel.from_pretrained(model_name)
+model.eval()
+
+prompt = "I have been feeling very anxious and overwhelmed lately."
+inputs = tokenizer(prompt, return_tensors="pt")
+
+with torch.no_grad():
+    outputs = model.generate(
+        **inputs,
+        max_new_tokens=100,
+        do_sample=True,
+        temperature=0.9,
+        top_p=0.95,
+        pad_token_id=tokenizer.eos_token_id
+    )
+
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+### Downstream Use
+
+This model can be plugged into larger pipelines for:
+- Mental health chatbot or virtual counselor prototyping
+- Generating synthetic counseling conversation data
+- Psychology NLP research and benchmarking
+- Empathetic response generation systems
+
+### Out-of-Scope Use
+
+- **Not a substitute for professional mental health care.** This model should never be used as a replacement for licensed therapists or clinical diagnosis.
+- Not suitable for crisis intervention or emergency mental health situations.
+- Not designed for factual question answering or knowledge retrieval tasks.
+- Should not be deployed in production-facing mental health applications without thorough safety evaluation.
+
+---
+
+## Bias, Risks, and Limitations
+
+- **Clinical risk:** The model may generate responses that sound plausible but are clinically incorrect, harmful, or inappropriate for vulnerable users. Always include human oversight.
+- **Data bias:** The model reflects patterns and biases present across the 6 source datasets. Some datasets may over-represent specific demographics or therapeutic styles.
+- **Hallucination:** GPT-2 based models may generate fluent but factually incorrect or contextually inappropriate text.
+- **Short context window:** Sequences were truncated to 128 tokens during training, so very long conversations may lose context.
+- **Small model size:** At 124M parameters, GPT-2 has limited capacity for nuanced reasoning compared to larger modern LLMs.
+
+### Recommendations
+
+This model is intended for **research and prototyping only**. It should not be deployed in any real-world mental health support context without rigorous safety evaluation, content filtering, and human-in-the-loop oversight.
+
+---
+
+## How to Get Started with the Model
+
+Install dependencies:
+```bash
+pip install transformers torch
+```
+
+Then use the inference script in the Direct Use section above.
+
+---
+
+## Training Details
+
+### Training Data
+
+The model was trained on a combined dataset of **~45,000 rows** sourced from 6 public mental health and psychology datasets on Hugging Face:
+
+| # | Dataset | Description |
+|---|---------|-------------|
+| 1 | [marmikpandya/mental-health](https://huggingface.co/datasets/marmikpandya/mental-health) | Mental health Q&A pairs |
+| 2 | [fadodr/mental_health_therapy](https://huggingface.co/datasets/fadodr/mental_health_therapy) | Therapy conversation pairs |
+| 3 | [Amod/mental_health_counseling_conversations](https://huggingface.co/datasets/Amod/mental_health_counseling_conversations) | Counseling context-response pairs |
+| 4 | [jkhedri/psychology-dataset](https://huggingface.co/datasets/jkhedri/psychology-dataset) | Psychology Q&A pairs |
+| 5 | [samhog/psychology-6k](https://huggingface.co/datasets/samhog/psychology-6k) | Psychology input-output pairs |
+| 6 | [RAJJ18/mental_health_dataset](https://huggingface.co/datasets/RAJJ18/mental_health_dataset) | Mental health conversations (3,000 rows sampled) |
+
+All datasets were standardized to a unified `input` / `output` column format before concatenation. Dataset 6 was randomly sampled to 3,000 rows (seed=42) for balance.
+
+### Training Procedure
+
+#### Preprocessing
+
+- All datasets normalized to `input` and `output` columns
+- Input and output concatenated as a single string: `"{input} {output}"`
+- Tokenized using the GPT-2 BPE tokenizer (`AutoTokenizer` from `gpt2`)
+- `pad_token` set to `eos_token`
+- Sequences truncated and padded to **max length of 128 tokens**
+- Labels set equal to `input_ids` for causal language modelling (next-token prediction)
+
+#### Training Hyperparameters
+
+| Hyperparameter              | Value                        |
+|-----------------------------|------------------------------|
+| Base model                  | `gpt2` (124M parameters)     |
+| Epochs                      | 10                           |
+| Training rows               | ~45,000                      |
+| Per-device train batch size | 4                            |
+| Per-device eval batch size  | 4                            |
+| Learning rate               | 3e-5                         |
+| Warmup steps                | 100                          |
+| Weight decay                | 0.01                         |
+| Max sequence length         | 128 tokens                   |
+| Training regime             | fp16 mixed precision         |
+| Evaluation strategy         | Every 5,000 steps            |
+| Save strategy               | Every 5,000 steps            |
+| Logging steps               | Every 50 steps               |
+| Best model metric           | Validation loss (lower is better) |
+| Checkpoints kept            | 2 (save_total_limit=2)       |
+| Optimizer                   | AdamW (Hugging Face default) |
+
+#### Evaluation Dataset
+
+The test split of `fadodr/mental_health_therapy` (dataset 2) was used as the held-out validation set during training.
+
+---
+
+## Evaluation
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+The test split of `fadodr/mental_health_therapy` — held out from training and used for validation loss tracking.
+
+#### Metrics
+
+- **Training Loss:** Tracked every 50 steps via TensorBoard logging
+- **Validation Loss:** Evaluated every 5,000 steps; best model checkpoint selected based on lowest validation loss
+- **Perplexity:** Derived from validation loss — lower perplexity indicates better language modelling
+
+### Results
+
+Training and validation loss curves are available in the [`new_graph/`](https://github.com/praniil/finetuned_gpt2_45krows_n5/tree/main/new_graph) directory. Full training logs are stored in [`new_logs/`](https://github.com/praniil/finetuned_gpt2_45krows_n5/tree/main/new_logs).
+
+---
+
+## Technical Specifications
+
+### Model Architecture and Objective
+
+- **Architecture:** GPT-2 (decoder-only transformer)
+- **Objective:** Causal Language Modelling (next-token prediction)
+- **Parameters:** 124M
+- **Layers:** 12 transformer blocks
+- **Attention heads:** 12
+- **Hidden size:** 768
+- **Max context length:** 1024 tokens (128 tokens used during training)
+- **Tokenizer:** GPT-2 BPE tokenizer (vocab size: 50,257)
+
+### Compute Infrastructure
+
+#### Hardware
+
+- CUDA-enabled GPU (local machine)
+
+#### Software
+
+- Python 3.8+
+- PyTorch
+- Hugging Face `transformers`
+- Hugging Face `datasets`
+- TensorBoard (for logging)
+
+---
+
+## Environmental Impact
+
+- **Hardware Type:** CUDA-enabled GPU
+- **Cloud Provider:** Not applicable (local training)
+- **Compute Region:** Nepal
+- **Carbon Emitted:** Not measured
+
+---
+
+## Citation
+```bibtex
+@misc{praniil2024finetuned-gpt2-mentalhealth-10epochs,
+  author       = {praniil},
+  title        = {Fine-tuned GPT-2 on Mental Health and Psychology Datasets (45K rows, 10 Epochs)},
+  year         = {2024},
+  publisher    = {HuggingFace},
+  howpublished = {\url{https://huggingface.co/Pranilllllll/finetuned_gpt2_45krows_10epochs}},
+}
+```
+
+---
+
+## Model Card Authors
+
+[praniil](https://github.com/praniil)
+
+## Model Card Contact
+
+Open an issue at [https://github.com/praniil/finetuned_gpt2_45krows_n5/issues](https://github.com/praniil/finetuned_gpt2_45krows_n5/issues)
--- a/config.json
+++ b/config.json
@@ -0,0 +1,39 @@
+{
+  "_name_or_path": "/home/nil/python_projects/gpt2_finetuned_45k_10epochs/results/checkpoint-105000",
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": null,
+  "n_layer": 12,
+  "n_positions": 1024,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.1",
+  "use_cache": true,
+  "vocab_size": 50257
+}
--- a/generation_config.json
+++ b/generation_config.json
@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "transformers_version": "4.48.1"
+}
--- a/merges.txt
+++ b/merges.txt
--- a/model.safetensors
+++ b/model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:05249f08045ee5d31ca113a15d296084bcbd803e6c78d6c31d50e1a2116f3545
+size 497774208
--- a/special_tokens_map.json
+++ b/special_tokens_map.json
@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}
--- a/tokenizer.json
+++ b/tokenizer.json
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
@@ -0,0 +1,21 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": {},
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}
--- a/vocab.json
+++ b/vocab.json