初始化项目,由ModelHub XC社区提供模型
Model: singtan/my-llm-finetuned-pdf Source: Original Platform
This commit is contained in:
66
README.md
Normal file
66
README.md
Normal file
@@ -0,0 +1,66 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
library_name: transformers
|
||||
base_model: gpt2
|
||||
tags:
|
||||
- text-generation
|
||||
- pdf-extraction
|
||||
- fine-tuned
|
||||
datasets:
|
||||
- custom
|
||||
language:
|
||||
- en
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# Fine-tuned Language Model (PDF Optimized)
|
||||
|
||||
This model is a specialized version of **gpt2**, fine-tuned for high-context coherence based on technical documentation extracted from PDF sources.
|
||||
|
||||
## Model Summary
|
||||
This model was developed to improve text generation accuracy and contextual understanding for specific domains covered in the provided PDF assets. It leverages the underlying architecture of `gpt2` and is optimized for the structure and vocabulary found in document-based datasets.
|
||||
|
||||
### Key Specifications
|
||||
| Attribute | Value |
|
||||
| :--- | :--- |
|
||||
| **Base Architecture** | gpt2 |
|
||||
| **Format** | PyTorch (Transformers) |
|
||||
| **Task** | Causal Language Modeling |
|
||||
| **Language** | English (en) |
|
||||
|
||||
## Training Configuration
|
||||
The model was fine-tuned using the following high-level hyperparameters to ensure stability and convergence:
|
||||
|
||||
- **Epochs:** 3
|
||||
- **Batch Size:** 1
|
||||
- **Learning Rate:** 5e-05
|
||||
- **Optimized for:** Cross-Entropy Loss
|
||||
- **Hardware:** cuda
|
||||
|
||||
### Performance Metrics
|
||||
- **Total Training Loss:** 3.680997848510742
|
||||
- **Training Runtime:** 3.6704 seconds
|
||||
|
||||
## Data Preprocessing
|
||||
The training data was sourced from `dummy.pdf`. The pipeline included:
|
||||
1. **Extraction:** Text recovery using `pypdf`.
|
||||
2. **Normalization:** Regex-based whitespace cleaning and token normalization.
|
||||
3. **Tokenization:** Model-specific subword tokenization with a max sequence length of 512 tokens.
|
||||
|
||||
## Usage Instructions
|
||||
To utilize this model for inference, use the following snippet:
|
||||
```python
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
|
||||
checkpoint = "singtan/my-llm-finetuned-pdf"
|
||||
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
|
||||
model = AutoModelForCausalLM.from_pretrained(checkpoint)
|
||||
|
||||
prompt = "Insert your context here"
|
||||
inputs = tokenizer(prompt, return_tensors="pt")
|
||||
outputs = model.generate(**inputs, max_length=150)
|
||||
print(tokenizer.decode(outputs[0]))
|
||||
```
|
||||
|
||||
## Contact
|
||||
Developed by **Bibek** - Senior AI Engineering Portfolio.
|
||||
Reference in New Issue
Block a user