---
license: gemma
language:
- en
base_model: kth8/gemma-3-270m-it-Conversation
datasets:
- kth8/multi-turn-conversation-50000x
pipeline_tag: text-generation
library_name: transformers
tags:
- sft
- trl
- unsloth
- gemma
- gemma3
- gemma3_text
---
![logo](https://storage.googleapis.com/gweb-developer-goog-blog-assets/images/gemma-3_2.original.png)
A supervised fine-tune of [unsloth/gemma-3-270m-it](https://huggingface.co/unsloth/gemma-3-270m-it) on the [kth8/multi-turn-conversation-50000x](https://huggingface.co/datasets/kth8/multi-turn-conversation-50000x) dataset.

## Usage example

**System prompt**
```
You are a helpful assistant.
```
**User prompt**
```
Hey there! How's it going?
```
**Assistant response**
```
Hey! I'm doing great, thanks for asking! I'm here and ready to help with whatever you need. What's on your mind today?
```
## Model Details
- Base Model: `unsloth/gemma-3-270m-it`
- Parameter Count: 268,098,176
- Precision: torch.bfloat16

## Training Settings

### PEFT
- Rank: 32
- LoRA alpha: 64
- Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Gradient checkpointing: unsloth

### SFT
- Epoch: 1
- Batch size: 8
- Gradient Accumulation steps: 2
- Learning rate: 0.0002
- Optimizer: adamw_torch_fused
- Learning rate scheduler: cosine
- Warmup steps: 100
- Weight decay: 0.01

## Training stats
- Date: 2026-06-16T15:15:17.789826
- GPU: NVIDIA L4
- Peak VRAM usage: 16.455 GB
- Global step: 3120
- Training runtime (seconds): 18261.5241
- Best validation loss: 1.666245937347412

| Step | Training Loss | Validation Loss |
|------|---------------|-----------------|
| 0    | No log        | 2.784440        |
| 155  | 1.882700      | 1.881819        |
| 310  | 1.805000      | 1.832387        |
| 465  | 1.803100      | 1.804098        |
| 620  | 1.781600      | 1.782886        |
| 775  | 1.785700      | 1.765646        |
| 930  | 1.776400      | 1.749293        |
| 1085 | 1.753500      | 1.736082        |
| 1240 | 1.732600      | 1.725711        |
| 1395 | 1.703100      | 1.715472        |
| 1550 | 1.730700      | 1.705917        |
| 1705 | 1.713500      | 1.697924        |
| 1860 | 1.725500      | 1.690107        |
| 2015 | 1.707200      | 1.684427        |
| 2170 | 1.687700      | 1.678853        |
| 2325 | 1.675800      | 1.674952        |
| 2480 | 1.723100      | 1.671108        |
| 2635 | 1.684300      | 1.668909        |
| 2790 | 1.692800      | 1.667304        |
| 2945 | 1.663200      | 1.666461        |
| 3100 | 1.676500      | 1.666246        |

## Framework versions
- Unsloth: 2026.6.7
- TRL: 0.22.2
- Transformers: 4.56.2
- Pytorch: 2.11.0+cu128
- Datasets: 5.0.0
- Tokenizers: 0.22.2

## License
This model is released under the Gemma license. See the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) and [Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy) regarding the use of Gemma-generated content.