初始化项目,由ModelHub XC社区提供模型

Model: singtan/my-llm-finetuned-pdf
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-23 09:25:23 +08:00
commit fc51872a5c
7 changed files with 250469 additions and 0 deletions

66
README.md Normal file
View File

@@ -0,0 +1,66 @@
---
license: apache-2.0
library_name: transformers
base_model: gpt2
tags:
- text-generation
- pdf-extraction
- fine-tuned
datasets:
- custom
language:
- en
pipeline_tag: text-generation
---
# Fine-tuned Language Model (PDF Optimized)
This model is a specialized version of **gpt2**, fine-tuned for high-context coherence based on technical documentation extracted from PDF sources.
## Model Summary
This model was developed to improve text generation accuracy and contextual understanding for specific domains covered in the provided PDF assets. It leverages the underlying architecture of `gpt2` and is optimized for the structure and vocabulary found in document-based datasets.
### Key Specifications
| Attribute | Value |
| :--- | :--- |
| **Base Architecture** | gpt2 |
| **Format** | PyTorch (Transformers) |
| **Task** | Causal Language Modeling |
| **Language** | English (en) |
## Training Configuration
The model was fine-tuned using the following high-level hyperparameters to ensure stability and convergence:
- **Epochs:** 3
- **Batch Size:** 1
- **Learning Rate:** 5e-05
- **Optimized for:** Cross-Entropy Loss
- **Hardware:** cuda
### Performance Metrics
- **Total Training Loss:** 3.680997848510742
- **Training Runtime:** 3.6704 seconds
## Data Preprocessing
The training data was sourced from `dummy.pdf`. The pipeline included:
1. **Extraction:** Text recovery using `pypdf`.
2. **Normalization:** Regex-based whitespace cleaning and token normalization.
3. **Tokenization:** Model-specific subword tokenization with a max sequence length of 512 tokens.
## Usage Instructions
To utilize this model for inference, use the following snippet:
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
checkpoint = "singtan/my-llm-finetuned-pdf"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint)
prompt = "Insert your context here"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=150)
print(tokenizer.decode(outputs[0]))
```
## Contact
Developed by **Bibek** - Senior AI Engineering Portfolio.