115 lines
2.9 KiB
Markdown
115 lines
2.9 KiB
Markdown
---
|
|
license: gemma
|
|
language:
|
|
- en
|
|
base_model: kth8/gemma-3-270m-it-homeowner-classifier
|
|
datasets:
|
|
- kth8/homeowner-classification
|
|
pipeline_tag: text-generation
|
|
library_name: transformers
|
|
tags:
|
|
- sft
|
|
- trl
|
|
- unsloth
|
|
- gemma
|
|
- gemma3
|
|
- gemma3_text
|
|
---
|
|
|
|
A supervised fine-tune of [unsloth/gemma-3-270m-it](https://huggingface.co/unsloth/gemma-3-270m-it) on the [kth8/homeowner-classification](https://huggingface.co/datasets/kth8/homeowner-classification) dataset.
|
|
|
|
Inspired by https://www.teachmecoolstuff.com/viewarticle/fine-tuning-a-local-llm-to-categorize-questions
|
|
|
|
Use `temperature=0.0` for optimal results.
|
|
|
|
## Usage example
|
|
|
|
**System prompt**
|
|
```
|
|
Classify the homeowner question into a category from the list below.
|
|
The answer must be exactly one category name from the list in JSON format.
|
|
Choose the best category based on the meaning of the question.
|
|
|
|
Valid categories:
|
|
- appliances
|
|
- brick work
|
|
- car
|
|
- cooking
|
|
- doorbell
|
|
- electric
|
|
- fence
|
|
- fountain
|
|
- garden lights
|
|
- gutters
|
|
- hvac
|
|
- irrigation
|
|
- mosquito
|
|
- painting
|
|
- pool
|
|
- tree service
|
|
- water heater
|
|
- window service
|
|
```
|
|
**User prompt**
|
|
```
|
|
What is the CYA level supposed to be in the pool water?
|
|
```
|
|
**Assistant response**
|
|
```
|
|
{"category": "pool"}
|
|
```
|
|
## Model Details
|
|
- Base Model: `unsloth/gemma-3-270m-it`
|
|
- Parameter Count: 268,098,176
|
|
- Precision: torch.bfloat16
|
|
|
|
## Training Settings
|
|
|
|
### PEFT
|
|
- Rank: 32
|
|
- LoRA alpha: 64
|
|
- Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
|
|
- Gradient checkpointing: unsloth
|
|
|
|
### SFT
|
|
- Epoch: 1
|
|
- Batch size: 8
|
|
- Gradient Accumulation steps: 2
|
|
- Learning rate: 0.0002
|
|
- Optimizer: adamw_torch_fused
|
|
- Learning rate scheduler: cosine
|
|
- Warmup steps: 10
|
|
- Weight decay: 0.01
|
|
|
|
## Training stats
|
|
- Date: 2026-06-17T01:06:10.026211
|
|
- GPU: NVIDIA L4
|
|
- Peak VRAM usage: 2.205 GB
|
|
- Global step: 126
|
|
- Training runtime (seconds): 334.7612
|
|
- Best validation loss: 0.013801434077322483
|
|
|
|
| Step | Training Loss | Validation Loss |
|
|
|------|---------------|-----------------|
|
|
| 0 | No log | 2.182445 |
|
|
| 12 | 1.131100 | 0.195152 |
|
|
| 24 | 0.122900 | 0.082139 |
|
|
| 36 | 0.076800 | 0.034620 |
|
|
| 48 | 0.075600 | 0.027673 |
|
|
| 60 | 0.039300 | 0.031881 |
|
|
| 72 | 0.038300 | 0.020789 |
|
|
| 84 | 0.031200 | 0.015834 |
|
|
| 96 | 0.018400 | 0.013921 |
|
|
| 108 | 0.030300 | 0.014498 |
|
|
| 120 | 0.023700 | 0.013801 |
|
|
|
|
## Framework versions
|
|
- Unsloth: 2026.6.7
|
|
- TRL: 0.22.2
|
|
- Transformers: 4.56.2
|
|
- Pytorch: 2.11.0+cu128
|
|
- Datasets: 5.0.0
|
|
- Tokenizers: 0.22.2
|
|
|
|
## License
|
|
This model is released under the Gemma license. See the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) and [Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy) regarding the use of Gemma-generated content. |