123 lines
3.6 KiB
Markdown
123 lines
3.6 KiB
Markdown
---
|
|
license: gemma
|
|
datasets:
|
|
- DireDreadlord/magicoder-glaive-code-instruct
|
|
language:
|
|
- en
|
|
base_model:
|
|
- google/gemma-3-270m-it
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- text-generation-inference
|
|
- code
|
|
- gemma3
|
|
- SLM
|
|
- chat
|
|
---
|
|
|
|
|
|
# GemCod270M - Jade (gemma-270m-it-code v4.1.0)
|
|
|
|
|
|
|
|
GemCod is a lightweight code generation model finetuned using SFT on the base gemma-270m-it model(https://huggingface.co/google/gemma-3-270m-it). It offers accurate and quick(ish) code snippet and long-form code generation in all major programming languages.
|
|
It's small size (270M parameters) allows it to run comfortably on laptop grade GPUs.
|
|
|
|
The Jade model serves as an upgrade from the previous Aquamarine model(https://huggingface.co/DireDreadlord/GemCod-codegen-270M), as it provides facilities for superior long-form code generation and explainability of generated snippets whilst keeping the inference time and space requirements roughly the same.
|
|
|
|
This model also offers rudimentary Q/A and subject matter expert capabilities on code related subjects.
|
|
|
|
---
|
|
|
|
|
|
**Estimated parameters:** ~270M
|
|
|
|
**Architecture:** Gemma3
|
|
|
|
**Intended use:** Code snippet and long-form code generation from natural language
|
|
|
|
---
|
|
|
|
|
|
## Training data
|
|
- Source: magicoder-glaive-code-instruct dataset (https://huggingface.co/datasets/DireDreadlord/magicoder-glaive-code-instruct)
|
|
- Rows: ~350,000 rows templated with a custom .jinja chat format
|
|
- Training: trained for 3,000 steps on an RTX 3050 (4GB VRAM)
|
|
|
|
|
|
## Usage
|
|
|
|
Install requirements:
|
|
|
|
```bash
|
|
pip install -r requirements.txt
|
|
pip install transformers datasets accelerate safetensors
|
|
```
|
|
|
|
|
|
## Usage (Hugging Face Hub)
|
|
You can load it directly from HuggingFace:
|
|
|
|
```python
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained("DireDreadlord/GemCod-Jade-270M")
|
|
model = AutoModelForCausalLM.from_pretrained("DireDreadlord/GemCod-Jade-270M")
|
|
model.to(device)
|
|
model.eval()
|
|
model.resize_token_embeddings(len(tokenizer))
|
|
|
|
|
|
if tokenizer.pad_token is None:
|
|
tokenizer.pad_token = tokenizer.eos_token
|
|
|
|
chat_template = """{% for message in messages %}{% if message['role'] == 'user' %}User: {{ message['content'] }}
|
|
{% elif message['role'] == 'assistant' %}Assistant: {{ message['content'] }}
|
|
{% endif %}{% endfor %}"""
|
|
tokenizer.chat_template = chat_template
|
|
|
|
def generate_code(prompt, max_tokens) -> str:
|
|
messages = [
|
|
{
|
|
"role": "user",
|
|
"content": prompt
|
|
}
|
|
]
|
|
|
|
formatted_prompt = tokenizer.apply_chat_template(
|
|
messages,
|
|
tokenize=False,
|
|
add_generation_prompt=True
|
|
)
|
|
|
|
|
|
inputs = tokenizer(formatted_prompt, return_tensors="pt").to(device)
|
|
input_length = inputs["input_ids"].shape[1]
|
|
|
|
with torch.no_grad():
|
|
outputs = model.generate(
|
|
**inputs,
|
|
max_new_tokens=512,
|
|
do_sample=False,
|
|
num_beams=1,
|
|
pad_token_id=tokenizer.eos_token_id,
|
|
eos_token_id=tokenizer.eos_token_id,
|
|
use_cache=False,
|
|
)
|
|
|
|
generated_tokens = outputs[0][input_length:]
|
|
generated_text = tokenizer.decode(generated_tokens, skip_special_tokens=True)
|
|
return generated_text
|
|
|
|
|
|
prompt = "WAP to train a sklearn model to predict the price of a house based on its size and location"
|
|
print("Prompt: ", prompt)
|
|
|
|
result = generate_code(prompt)
|
|
print(result)
|
|
```
|
|
|
|
|
|
## Limitations
|
|
- Model for experimental use only; users should employ it as such under license. |