qwen3-4b-alpaca-chatwithme/README.md

language, license, base_model, tags, datasets, pipeline_tag

language	license	base_model	tags	datasets	pipeline_tag
en	apache-2.0	Qwen/Qwen3-4B	qwen3 lora fine-tuned alpaca chatml instruction-following	yahma/alpaca-cleaned	text-generation

ChatWithMe — Qwen3-4B Fine-tuned on Alpaca

A Qwen3-4B model fine-tuned with LoRA on the Alpaca dataset for instruction-following conversations.

Model Details

• Base model: Qwen/Qwen3-4B
• Fine-tuning method: LoRA (r=8, alpha=16)
• Dataset: yahma/alpaca-cleaned (~52k examples)
• Chat format: ChatML
• Training: 1 epoch, SFTTrainer, 4-bit quantization, bf16, cosine scheduler
• Hardware: NVIDIA A100 80GB
• Final training loss: 1.0875
• Final validation loss: 1.0976
• Perplexity: ~3.00

LoRA Configuration

• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Rank: 8
• Alpha: 16
• Dropout: 0.05
• Trainable parameters: ~13M / 4B total (0.3%)

Usage