58 lines
1.9 KiB
Markdown
58 lines
1.9 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
datasets:
|
||
|
|
- huggingface-course/codeparrot-ds-train
|
||
|
|
- huggingface-course/codeparrot-ds-valid
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
metrics:
|
||
|
|
- code_eval
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
tags:
|
||
|
|
- code
|
||
|
|
- gpt2
|
||
|
|
- pytorch
|
||
|
|
- causal-lm
|
||
|
|
---
|
||
|
|
|
||
|
|
# python-ds-accelerate (GPT-2 124M)
|
||
|
|
|
||
|
|
This model is a GPT-2 (124M parameter) causal language model trained from scratch specifically for **Python code completion** in Data Science contexts.
|
||
|
|
|
||
|
|
## Model Details
|
||
|
|
|
||
|
|
### Model Description
|
||
|
|
|
||
|
|
This model is an implementation of the GPT-2 architecture optimized for generating functional Python code snippets. It was trained using a custom training pipeline that incorporates a **keytoken weighted loss** function to prioritize important programming keywords (like `plt`, `pd`, `fit`, `predict`), making it more effective at suggesting Data Science-related code.
|
||
|
|
|
||
|
|
- **Developed by:** [Pranav Guhan R](https://github.com/PranavGuhanR)
|
||
|
|
- **Model type:** Transformer-based Causal Language Model
|
||
|
|
- **Language(s):** Python (English comments)
|
||
|
|
- **License:** Apache 2.0
|
||
|
|
- **Finetuned from model:** Trained from scratch
|
||
|
|
|
||
|
|
### Model Sources
|
||
|
|
|
||
|
|
- **Repository:** [GPT-2-124M-pretraining-for-code-completion](https://github.com/PranavGuhanR/GPT-2-124M-pretraining-for-code-completion)
|
||
|
|
|
||
|
|
## Uses
|
||
|
|
|
||
|
|
### Direct Use
|
||
|
|
The model is intended to be used for code completion tasks, specifically for completing Python scripts involving libraries like `pandas`, `matplotlib`, and `scikit-learn`.
|
||
|
|
|
||
|
|
### Out-of-Scope Use
|
||
|
|
The model is not suitable for general-purpose natural language conversation or generating code in languages other than Python.
|
||
|
|
|
||
|
|
## How to Get Started with the Model
|
||
|
|
|
||
|
|
You can use the model directly with a Hugging Face pipeline:
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import pipeline
|
||
|
|
|
||
|
|
pipe = pipeline("text-generation", model="PranavGuhan/python-ds-accelerate")
|
||
|
|
|
||
|
|
txt = """# create dataframe from x and y
|
||
|
|
df = pd.DataFrame({'x':x, 'y':y})
|
||
|
|
"""
|
||
|
|
print(pipe(txt, num_return_sequences=1)[0]["generated_text"])
|