---
language:
- en
license: apache-2.0
base_model: Qwen/Qwen3-4B
tags:
- qwen3
- lora
- fine-tuned
- alpaca
- chatml
- instruction-following
datasets:
- yahma/alpaca-cleaned
pipeline_tag: text-generation
---

# ChatWithMe — Qwen3-4B Fine-tuned on Alpaca

A Qwen3-4B model fine-tuned with LoRA on the Alpaca dataset for instruction-following conversations.

## Model Details

- **Base model:** Qwen/Qwen3-4B
- **Fine-tuning method:** LoRA (r=8, alpha=16)
- **Dataset:** yahma/alpaca-cleaned (~52k examples)
- **Chat format:** ChatML
- **Training:** 1 epoch, SFTTrainer, 4-bit quantization, bf16, cosine scheduler
- **Hardware:** NVIDIA A100 80GB
- **Final training loss:** 1.0875
- **Final validation loss:** 1.0976
- **Perplexity:** ~3.00

## LoRA Configuration

- **Target modules:** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- **Rank:** 8
- **Alpha:** 16
- **Dropout:** 0.05
- **Trainable parameters:** ~13M / 4B total (0.3%)

## Usage

```python