120 lines
3.2 KiB
Markdown
120 lines
3.2 KiB
Markdown
---
|
|
base_model: unsloth/gemma-3-270m-it
|
|
library_name: transformers
|
|
tags:
|
|
- text-generation-inference
|
|
- transformers
|
|
- unsloth
|
|
- gemma
|
|
- trl
|
|
- sft
|
|
- energyplus
|
|
- eppy
|
|
- building-simulation
|
|
license: apache-2.0
|
|
language:
|
|
- en
|
|
datasets:
|
|
- self-created
|
|
pipeline_tag: text-generation
|
|
---
|
|
|
|
# Gemma-3-270M Fine-tuned for EnergyPlus & eppy
|
|
|
|
**Developed by:** meftah416
|
|
**Model License:** apache-2.0
|
|
**Base Model:** [unsloth/gemma-3-270m-it](https://huggingface.co/unsloth/gemma-3-270m-it)
|
|
|
|
This model was trained **2x faster with [Unsloth](https://github.com/unslothai/unsloth)** ⚡
|
|
|
|
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
|
|
|
---
|
|
|
|
## Model Overview
|
|
A specialized Gemma-3-270M model fine-tuned on **EnergyPlus building simulation and eppy Python API** tasks. This model understands and generates text related to building energy simulation, HVAC systems, thermal zones, schedules, and eppy scripting.
|
|
|
|
### Key Features
|
|
- ✨ Fine-tuned on 2,700 high-quality training examples
|
|
- ✅ Validated on 300 test examples
|
|
- 🚀 2x faster training with Unsloth
|
|
- 💾 Lightweight (270M parameters)
|
|
- 🎯 Domain-specific for EnergyPlus + eppy
|
|
---
|
|
|
|
## Training Details
|
|
|
|
| Metric | Value |
|
|
|--------|-------|
|
|
| **Base Model** | unsloth/gemma-3-270m-it |
|
|
| **Training Samples** | 2,700 |
|
|
| **Validation Samples** | 300 |
|
|
| **Total Data** | 3,000 examples |
|
|
| **Data Source** | Self-created |
|
|
| **Domain** | EnergyPlus IDF + eppy Python |
|
|
| **Batch Size** | 2 |
|
|
| **Learning Rate** | 2e-5 |
|
|
| **Optimizer** | adamw_8bit |
|
|
| **Epochs** | 1 |
|
|
| **Max Sequence Length** | 2600 |
|
|
| **Training Framework** | Unsloth + TRL (SFTTrainer) |
|
|
| **Precision** | float16 |
|
|
|
|
---
|
|
## Model Capabilities
|
|
This model is trained to handle:
|
|
|
|
### EnergyPlus Tasks
|
|
- Generate EnergyPlus IDF snippets from descriptions
|
|
- Explain EnergyPlus object syntax
|
|
- Create thermal zone definitions
|
|
- Define HVAC system configurations
|
|
- Generate occupancy and schedule objects
|
|
### eppy Tasks
|
|
- Generate eppy Python code for building simulations
|
|
- Explain eppy API usage
|
|
- Create building objects programmatically
|
|
- Manipulate IDF files with eppy
|
|
---
|
|
## Usage
|
|
### Basic Usage
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("meftah416/gemma-eppy-270m")
|
|
tokenizer = AutoTokenizer.from_pretrained("meftah416/gemma-eppy-270m")
|
|
|
|
# Create messages in correct format
|
|
messages = [
|
|
{"role": "system", "content": "Set infiltration to 0.4 ACH"},
|
|
{"role": "user", "content": ""},
|
|
]
|
|
|
|
# Apply chat template (IMPORTANT!)
|
|
prompt = tokenizer.apply_chat_template(
|
|
messages,
|
|
tokenize=False,
|
|
add_generation_prompt=True
|
|
).removeprefix('<bos>')
|
|
|
|
# Generate
|
|
inputs = tokenizer(prompt, return_tensors="pt")
|
|
outputs = model.generate(**inputs, max_length=2600)
|
|
result = tokenizer.decode(outputs[0])
|
|
```
|
|
# Save to model README on Hub
|
|
---
|
|
## Performance
|
|
- **Inference Speed:** ~50-100 tokens/sec (A10 GPU)
|
|
- **Memory Usage:** 3-4 GB VRAM (float16)
|
|
- **Context Window:** 8192 tokens
|
|
---
|
|
## Limitations
|
|
⚠️ Always validate generated EnergyPlus IDF files before running simulations. Model may occasionally generate incorrect syntax.
|
|
|
|
---
|
|
## License
|
|
Apache 2.0 License
|
|
|
|
---
|
|
**Created by:** meftah416 |