137 lines
3.3 KiB
Markdown
137 lines
3.3 KiB
Markdown
---
|
|
base_model: unsloth/gemma-3-270m-it
|
|
library_name: transformers
|
|
pipeline_tag: text-generation
|
|
license: gemma
|
|
tags:
|
|
- gemma3
|
|
- lora
|
|
- sft
|
|
- unsloth
|
|
- gguf
|
|
---
|
|
|
|
# gemma-3-270m-lora-finetuned
|
|
|
|
A fine-tuned Gemma 3 270M model with LoRA adapters, trained on instruction-following data.
|
|
|
|
## Model Details
|
|
|
|
- **Base Model:** unsloth/gemma-3-270m-it
|
|
- **Model Type:** LoRA fine-tuned (merged)
|
|
- **Language:** English (instruction-tuned)
|
|
- **License:** Gemma License
|
|
|
|
## Available Formats
|
|
|
|
| Format | File | Use Case |
|
|
|--------|------|----------|
|
|
| Safetensors | `model.safetensors` | Transformers, vLLM |
|
|
| GGUF Q8_0 | `gguf/gemma-3-270m-Q8_0.gguf` | llama.cpp, Ollama |
|
|
| GGUF BF16 | `gguf/gemma-3-270m-auto.gguf` | llama.cpp (full precision) |
|
|
|
|
## Uses
|
|
|
|
### Direct Use
|
|
- Instruction following
|
|
- Conversational AI
|
|
- Question answering
|
|
- Text generation
|
|
|
|
### Out-of-Scope Use
|
|
- Medical or legal advice
|
|
- Generating harmful content
|
|
- Production use without evaluation
|
|
|
|
## Training Details
|
|
|
|
### Training Data
|
|
- **Dataset:** mlabonne/FineTome-100k (full dataset)
|
|
- **Samples:** ~100,000
|
|
- **Data Processing:** Standardized format with Gemma 3 chat template
|
|
|
|
### Training Hyperparameters
|
|
|
|
| Parameter | Value |
|
|
|-----------|-------|
|
|
| Precision | bfloat16 |
|
|
| Max Sequence Length | 1024 |
|
|
| Per Device Batch Size | 2 |
|
|
| Gradient Accumulation Steps | 4 |
|
|
| Effective Batch Size | 8 |
|
|
| Max Steps | 12,500 |
|
|
| Learning Rate | 2e-4 |
|
|
| Warmup Steps | 2 |
|
|
| Optimizer | adamw_8bit |
|
|
| Weight Decay | 0.001 |
|
|
| LR Scheduler | linear |
|
|
| Seed | 3407 |
|
|
|
|
### LoRA Configuration
|
|
|
|
| Parameter | Value |
|
|
|-----------|-------|
|
|
| LoRA Rank (r) | 128 |
|
|
| LoRA Alpha | 128 |
|
|
| LoRA Dropout | 0 |
|
|
| Bias | none |
|
|
| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
|
|
|
### Training Stats
|
|
- **Training Time:** ~98 minutes (1h 38m)
|
|
- **Peak VRAM Usage:** 12.4 GB (77.9% of 16 GB)
|
|
|
|
### Compute Infrastructure
|
|
- **Hardware:** NVIDIA RTX 5070 Ti (16 GB VRAM)
|
|
- **Framework:** Unsloth, Transformers, TRL
|
|
|
|
## How to Get Started
|
|
|
|
### Using Transformers (Safetensors)
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("chanifrusydi/gemma-3-270m-lora-finetuned")
|
|
tokenizer = AutoTokenizer.from_pretrained("chanifrusydi/gemma-3-270m-lora-finetuned")
|
|
|
|
messages = [{"role": "user", "content": "Hello, how are you?"}]
|
|
inputs = tokenizer.apply_chat_template(
|
|
messages,
|
|
tokenize=True,
|
|
add_generation_prompt=True,
|
|
return_tensors="pt"
|
|
)
|
|
|
|
output = model.generate(inputs, max_new_tokens=64, temperature=0.7)
|
|
print(tokenizer.decode(output[0], skip_special_tokens=True))
|
|
```
|
|
|
|
### Using GGUF (llama.cpp / Ollama)
|
|
```bash
|
|
# Download GGUF
|
|
wget https://huggingface.co/chanifrusydi/gemma-3-270m-lora-finetuned/resolve/main/gguf/gemma-3-270m-Q8_0.gguf
|
|
|
|
# Run with llama.cpp
|
|
./llama-cli -m gemma-3-270m-Q8_0.gguf -p "Hello, how are you?"
|
|
|
|
# Or import to Ollama
|
|
ollama create gemma3-finetuned -f Modelfile
|
|
```
|
|
|
|
## Citation
|
|
|
|
```bibtex
|
|
@misc{gemma-3-270m-lora-finetuned,
|
|
author = {chanifrusydi},
|
|
title = {gemma-3-270m-lora-finetuned},
|
|
publisher = {HuggingFace},
|
|
year = {2025}
|
|
}
|
|
```
|
|
|
|
## Acknowledgments
|
|
|
|
- [Unsloth](https://github.com/unslothai/unsloth) for the fast fine-tuning framework
|
|
- [Google](https://huggingface.co/google) for the Gemma 3 base model
|
|
- [mlabonne](https://huggingface.co/mlabonne) for the FineTome-100k dataset
|