---
license: gemma
language:
- en
base_model: kth8/gemma-3-270m-it-homeowner-classifier
datasets:
- kth8/homeowner-classification
pipeline_tag: text-generation
library_name: transformers
tags:
- sft
- trl
- unsloth
- gemma
- gemma3
- gemma3_text
---
![logo](https://storage.googleapis.com/gweb-developer-goog-blog-assets/images/gemma-3_2.original.png)
A supervised fine-tune of [unsloth/gemma-3-270m-it](https://huggingface.co/unsloth/gemma-3-270m-it) on the [kth8/homeowner-classification](https://huggingface.co/datasets/kth8/homeowner-classification) dataset.

Inspired by https://www.teachmecoolstuff.com/viewarticle/fine-tuning-a-local-llm-to-categorize-questions

Use `temperature=0.0` for optimal results.

## Usage example

**System prompt**
```
Classify the homeowner question into a category from the list below.
The answer must be exactly one category name from the list in JSON format.
Choose the best category based on the meaning of the question.

Valid categories:
- appliances
- brick work
- car
- cooking
- doorbell
- electric
- fence
- fountain
- garden lights
- gutters
- hvac
- irrigation
- mosquito
- painting
- pool
- tree service
- water heater
- window service
```
**User prompt**
```
What is the CYA level supposed to be in the pool water?
```
**Assistant response**
```
{"category": "pool"}
```
## Model Details
- Base Model: `unsloth/gemma-3-270m-it`
- Parameter Count: 268,098,176
- Precision: torch.bfloat16

## Training Settings

### PEFT
- Rank: 32
- LoRA alpha: 64
- Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Gradient checkpointing: unsloth

### SFT
- Epoch: 1
- Batch size: 8
- Gradient Accumulation steps: 2
- Learning rate: 0.0002
- Optimizer: adamw_torch_fused
- Learning rate scheduler: cosine
- Warmup steps: 10
- Weight decay: 0.01

## Training stats
- Date: 2026-06-17T01:06:10.026211
- GPU: NVIDIA L4
- Peak VRAM usage: 2.205 GB
- Global step: 126
- Training runtime (seconds): 334.7612
- Best validation loss: 0.013801434077322483

| Step | Training Loss | Validation Loss |
|------|---------------|-----------------|
| 0    | No log        | 2.182445        |
| 12   | 1.131100      | 0.195152        |
| 24   | 0.122900      | 0.082139        |
| 36   | 0.076800      | 0.034620        |
| 48   | 0.075600      | 0.027673        |
| 60   | 0.039300      | 0.031881        |
| 72   | 0.038300      | 0.020789        |
| 84   | 0.031200      | 0.015834        |
| 96   | 0.018400      | 0.013921        |
| 108  | 0.030300      | 0.014498        |
| 120  | 0.023700      | 0.013801        |

## Framework versions
- Unsloth: 2026.6.7
- TRL: 0.22.2
- Transformers: 4.56.2
- Pytorch: 2.11.0+cu128
- Datasets: 5.0.0
- Tokenizers: 0.22.2

## License
This model is released under the Gemma license. See the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) and [Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy) regarding the use of Gemma-generated content.