初始化项目,由ModelHub XC社区提供模型
Model: meftah416/gemma-eppy-270m Source: Original Platform
This commit is contained in:
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||||
120
README.md
Normal file
120
README.md
Normal file
@@ -0,0 +1,120 @@
|
|||||||
|
---
|
||||||
|
base_model: unsloth/gemma-3-270m-it
|
||||||
|
library_name: transformers
|
||||||
|
tags:
|
||||||
|
- text-generation-inference
|
||||||
|
- transformers
|
||||||
|
- unsloth
|
||||||
|
- gemma
|
||||||
|
- trl
|
||||||
|
- sft
|
||||||
|
- energyplus
|
||||||
|
- eppy
|
||||||
|
- building-simulation
|
||||||
|
license: apache-2.0
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
datasets:
|
||||||
|
- self-created
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
---
|
||||||
|
|
||||||
|
# Gemma-3-270M Fine-tuned for EnergyPlus & eppy
|
||||||
|
|
||||||
|
**Developed by:** meftah416
|
||||||
|
**Model License:** apache-2.0
|
||||||
|
**Base Model:** [unsloth/gemma-3-270m-it](https://huggingface.co/unsloth/gemma-3-270m-it)
|
||||||
|
|
||||||
|
This model was trained **2x faster with [Unsloth](https://github.com/unslothai/unsloth)** ⚡
|
||||||
|
|
||||||
|
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Model Overview
|
||||||
|
A specialized Gemma-3-270M model fine-tuned on **EnergyPlus building simulation and eppy Python API** tasks. This model understands and generates text related to building energy simulation, HVAC systems, thermal zones, schedules, and eppy scripting.
|
||||||
|
|
||||||
|
### Key Features
|
||||||
|
- ✨ Fine-tuned on 2,700 high-quality training examples
|
||||||
|
- ✅ Validated on 300 test examples
|
||||||
|
- 🚀 2x faster training with Unsloth
|
||||||
|
- 💾 Lightweight (270M parameters)
|
||||||
|
- 🎯 Domain-specific for EnergyPlus + eppy
|
||||||
|
---
|
||||||
|
|
||||||
|
## Training Details
|
||||||
|
|
||||||
|
| Metric | Value |
|
||||||
|
|--------|-------|
|
||||||
|
| **Base Model** | unsloth/gemma-3-270m-it |
|
||||||
|
| **Training Samples** | 2,700 |
|
||||||
|
| **Validation Samples** | 300 |
|
||||||
|
| **Total Data** | 3,000 examples |
|
||||||
|
| **Data Source** | Self-created |
|
||||||
|
| **Domain** | EnergyPlus IDF + eppy Python |
|
||||||
|
| **Batch Size** | 2 |
|
||||||
|
| **Learning Rate** | 2e-5 |
|
||||||
|
| **Optimizer** | adamw_8bit |
|
||||||
|
| **Epochs** | 1 |
|
||||||
|
| **Max Sequence Length** | 2600 |
|
||||||
|
| **Training Framework** | Unsloth + TRL (SFTTrainer) |
|
||||||
|
| **Precision** | float16 |
|
||||||
|
|
||||||
|
---
|
||||||
|
## Model Capabilities
|
||||||
|
This model is trained to handle:
|
||||||
|
|
||||||
|
### EnergyPlus Tasks
|
||||||
|
- Generate EnergyPlus IDF snippets from descriptions
|
||||||
|
- Explain EnergyPlus object syntax
|
||||||
|
- Create thermal zone definitions
|
||||||
|
- Define HVAC system configurations
|
||||||
|
- Generate occupancy and schedule objects
|
||||||
|
### eppy Tasks
|
||||||
|
- Generate eppy Python code for building simulations
|
||||||
|
- Explain eppy API usage
|
||||||
|
- Create building objects programmatically
|
||||||
|
- Manipulate IDF files with eppy
|
||||||
|
---
|
||||||
|
## Usage
|
||||||
|
### Basic Usage
|
||||||
|
```python
|
||||||
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||||
|
|
||||||
|
model = AutoModelForCausalLM.from_pretrained("meftah416/gemma-eppy-270m")
|
||||||
|
tokenizer = AutoTokenizer.from_pretrained("meftah416/gemma-eppy-270m")
|
||||||
|
|
||||||
|
# Create messages in correct format
|
||||||
|
messages = [
|
||||||
|
{"role": "system", "content": "Set infiltration to 0.4 ACH"},
|
||||||
|
{"role": "user", "content": ""},
|
||||||
|
]
|
||||||
|
|
||||||
|
# Apply chat template (IMPORTANT!)
|
||||||
|
prompt = tokenizer.apply_chat_template(
|
||||||
|
messages,
|
||||||
|
tokenize=False,
|
||||||
|
add_generation_prompt=True
|
||||||
|
).removeprefix('<bos>')
|
||||||
|
|
||||||
|
# Generate
|
||||||
|
inputs = tokenizer(prompt, return_tensors="pt")
|
||||||
|
outputs = model.generate(**inputs, max_length=2600)
|
||||||
|
result = tokenizer.decode(outputs[0])
|
||||||
|
```
|
||||||
|
# Save to model README on Hub
|
||||||
|
---
|
||||||
|
## Performance
|
||||||
|
- **Inference Speed:** ~50-100 tokens/sec (A10 GPU)
|
||||||
|
- **Memory Usage:** 3-4 GB VRAM (float16)
|
||||||
|
- **Context Window:** 8192 tokens
|
||||||
|
---
|
||||||
|
## Limitations
|
||||||
|
⚠️ Always validate generated EnergyPlus IDF files before running simulations. Model may occasionally generate incorrect syntax.
|
||||||
|
|
||||||
|
---
|
||||||
|
## License
|
||||||
|
Apache 2.0 License
|
||||||
|
|
||||||
|
---
|
||||||
|
**Created by:** meftah416
|
||||||
47
chat_template.jinja
Normal file
47
chat_template.jinja
Normal file
@@ -0,0 +1,47 @@
|
|||||||
|
{{ bos_token }}
|
||||||
|
{%- if messages[0]['role'] == 'system' -%}
|
||||||
|
{%- if messages[0]['content'] is string -%}
|
||||||
|
{%- set first_user_prefix = messages[0]['content'] + '
|
||||||
|
|
||||||
|
' -%}
|
||||||
|
{%- else -%}
|
||||||
|
{%- set first_user_prefix = messages[0]['content'][0]['text'] + '
|
||||||
|
|
||||||
|
' -%}
|
||||||
|
{%- endif -%}
|
||||||
|
{%- set loop_messages = messages[1:] -%}
|
||||||
|
{%- else -%}
|
||||||
|
{%- set first_user_prefix = "" -%}
|
||||||
|
{%- set loop_messages = messages -%}
|
||||||
|
{%- endif -%}
|
||||||
|
{%- for message in loop_messages -%}
|
||||||
|
{%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
|
||||||
|
{{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if (message['role'] == 'assistant') -%}
|
||||||
|
{%- set role = "model" -%}
|
||||||
|
{%- else -%}
|
||||||
|
{%- set role = message['role'] -%}
|
||||||
|
{%- endif -%}
|
||||||
|
{{ '<start_of_turn>' + role + '
|
||||||
|
' + (first_user_prefix if loop.first else "") }}
|
||||||
|
{%- if message['content'] is string -%}
|
||||||
|
{{ message['content'] | trim }}
|
||||||
|
{%- elif message['content'] is iterable -%}
|
||||||
|
{%- for item in message['content'] -%}
|
||||||
|
{%- if item['type'] == 'image' -%}
|
||||||
|
{{ '<start_of_image>' }}
|
||||||
|
{%- elif item['type'] == 'text' -%}
|
||||||
|
{{ item['text'] | trim }}
|
||||||
|
{%- endif -%}
|
||||||
|
{%- endfor -%}
|
||||||
|
{%- else -%}
|
||||||
|
{{ raise_exception("Invalid content type") }}
|
||||||
|
{%- endif -%}
|
||||||
|
{{ '<end_of_turn>
|
||||||
|
' }}
|
||||||
|
{%- endfor -%}
|
||||||
|
{%- if add_generation_prompt -%}
|
||||||
|
{{ '<start_of_turn>model
|
||||||
|
' }}
|
||||||
|
{%- endif -%}
|
||||||
65
config.json
Normal file
65
config.json
Normal file
@@ -0,0 +1,65 @@
|
|||||||
|
{
|
||||||
|
"_sliding_window_pattern": 6,
|
||||||
|
"architectures": [
|
||||||
|
"Gemma3ForCausalLM"
|
||||||
|
],
|
||||||
|
"attention_bias": false,
|
||||||
|
"attention_dropout": 0.0,
|
||||||
|
"attn_logit_softcapping": null,
|
||||||
|
"bos_token_id": 2,
|
||||||
|
"dtype": "bfloat16",
|
||||||
|
"eos_token_id": 106,
|
||||||
|
"final_logit_softcapping": null,
|
||||||
|
"head_dim": 256,
|
||||||
|
"hidden_activation": "gelu_pytorch_tanh",
|
||||||
|
"hidden_size": 640,
|
||||||
|
"initializer_range": 0.02,
|
||||||
|
"intermediate_size": 2048,
|
||||||
|
"layer_types": [
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"full_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"full_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"full_attention"
|
||||||
|
],
|
||||||
|
"max_position_embeddings": 32768,
|
||||||
|
"model_name": "unsloth/gemma-3-270m-it",
|
||||||
|
"model_type": "gemma3_text",
|
||||||
|
"num_attention_heads": 4,
|
||||||
|
"num_hidden_layers": 18,
|
||||||
|
"num_key_value_heads": 1,
|
||||||
|
"pad_token_id": 0,
|
||||||
|
"query_pre_attn_scalar": 256,
|
||||||
|
"rms_norm_eps": 1e-06,
|
||||||
|
"rope_parameters": {
|
||||||
|
"full_attention": {
|
||||||
|
"rope_theta": 1000000.0,
|
||||||
|
"rope_type": "default"
|
||||||
|
},
|
||||||
|
"sliding_attention": {
|
||||||
|
"rope_theta": 10000.0,
|
||||||
|
"rope_type": "default"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"sliding_window": 512,
|
||||||
|
"tie_word_embeddings": true,
|
||||||
|
"transformers_version": "5.5.0",
|
||||||
|
"unsloth_fixed": true,
|
||||||
|
"unsloth_version": "2026.5.10",
|
||||||
|
"use_bidirectional_attention": false,
|
||||||
|
"use_cache": false,
|
||||||
|
"vocab_size": 262144
|
||||||
|
}
|
||||||
14
generation_config.json
Normal file
14
generation_config.json
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
{
|
||||||
|
"bos_token_id": 2,
|
||||||
|
"cache_implementation": "hybrid",
|
||||||
|
"do_sample": true,
|
||||||
|
"eos_token_id": [
|
||||||
|
1,
|
||||||
|
106
|
||||||
|
],
|
||||||
|
"max_length": 32768,
|
||||||
|
"pad_token_id": 0,
|
||||||
|
"top_k": 64,
|
||||||
|
"top_p": 0.95,
|
||||||
|
"transformers_version": "5.5.0"
|
||||||
|
}
|
||||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:f8bfc4d2316eeb1faeae2e69834fffd7c046725793b727803fab1b32bd4ec18d
|
||||||
|
size 536334056
|
||||||
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:daab2354f8a74e70d70b4d1f804939b68a8c9624dd06cb7858e52dd8970e9726
|
||||||
|
size 33384567
|
||||||
3
tokenizer.model
Normal file
3
tokenizer.model
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
|
||||||
|
size 4689074
|
||||||
51346
tokenizer_config.json
Normal file
51346
tokenizer_config.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user