license, language, base_model, datasets, pipeline_tag, library_name, tags
| license | language | base_model | datasets | pipeline_tag | library_name | tags | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| gemma |
|
kth8/gemma-3-270m-it-Conversation |
|
text-generation | transformers |
|
A supervised fine-tune of unsloth/gemma-3-270m-it on the kth8/multi-turn-conversation-50000x dataset.
Usage example
System prompt
You are a helpful assistant.
User prompt
Hey there! How's it going?
Assistant response
Hey! I'm doing great, thanks for asking! I'm here and ready to help with whatever you need. What's on your mind today?
Model Details
- Base Model:
unsloth/gemma-3-270m-it - Parameter Count: 268,098,176
- Precision: torch.bfloat16
Training Settings
PEFT
- Rank: 32
- LoRA alpha: 64
- Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Gradient checkpointing: unsloth
SFT
- Epoch: 1
- Batch size: 8
- Gradient Accumulation steps: 2
- Learning rate: 0.0002
- Optimizer: adamw_torch_fused
- Learning rate scheduler: cosine
- Warmup steps: 100
- Weight decay: 0.01
Training stats
- Date: 2026-06-16T15:15:17.789826
- GPU: NVIDIA L4
- Peak VRAM usage: 16.455 GB
- Global step: 3120
- Training runtime (seconds): 18261.5241
- Best validation loss: 1.666245937347412
| Step | Training Loss | Validation Loss |
|---|---|---|
| 0 | No log | 2.784440 |
| 155 | 1.882700 | 1.881819 |
| 310 | 1.805000 | 1.832387 |
| 465 | 1.803100 | 1.804098 |
| 620 | 1.781600 | 1.782886 |
| 775 | 1.785700 | 1.765646 |
| 930 | 1.776400 | 1.749293 |
| 1085 | 1.753500 | 1.736082 |
| 1240 | 1.732600 | 1.725711 |
| 1395 | 1.703100 | 1.715472 |
| 1550 | 1.730700 | 1.705917 |
| 1705 | 1.713500 | 1.697924 |
| 1860 | 1.725500 | 1.690107 |
| 2015 | 1.707200 | 1.684427 |
| 2170 | 1.687700 | 1.678853 |
| 2325 | 1.675800 | 1.674952 |
| 2480 | 1.723100 | 1.671108 |
| 2635 | 1.684300 | 1.668909 |
| 2790 | 1.692800 | 1.667304 |
| 2945 | 1.663200 | 1.666461 |
| 3100 | 1.676500 | 1.666246 |
Framework versions
- Unsloth: 2026.6.7
- TRL: 0.22.2
- Transformers: 4.56.2
- Pytorch: 2.11.0+cu128
- Datasets: 5.0.0
- Tokenizers: 0.22.2
License
This model is released under the Gemma license. See the Gemma Terms of Use and Prohibited Use Policy regarding the use of Gemma-generated content.