初始化项目,由ModelHub XC社区提供模型
Model: welyty/qwen3-4b-alpaca-chatwithme Source: Original Platform
This commit is contained in:
44
README.md
Normal file
44
README.md
Normal file
@@ -0,0 +1,44 @@
|
||||
---
|
||||
language:
|
||||
- en
|
||||
license: apache-2.0
|
||||
base_model: Qwen/Qwen3-4B
|
||||
tags:
|
||||
- qwen3
|
||||
- lora
|
||||
- fine-tuned
|
||||
- alpaca
|
||||
- chatml
|
||||
- instruction-following
|
||||
datasets:
|
||||
- yahma/alpaca-cleaned
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# ChatWithMe — Qwen3-4B Fine-tuned on Alpaca
|
||||
|
||||
A Qwen3-4B model fine-tuned with LoRA on the Alpaca dataset for instruction-following conversations.
|
||||
|
||||
## Model Details
|
||||
|
||||
- **Base model:** Qwen/Qwen3-4B
|
||||
- **Fine-tuning method:** LoRA (r=8, alpha=16)
|
||||
- **Dataset:** yahma/alpaca-cleaned (~52k examples)
|
||||
- **Chat format:** ChatML
|
||||
- **Training:** 1 epoch, SFTTrainer, 4-bit quantization, bf16, cosine scheduler
|
||||
- **Hardware:** NVIDIA A100 80GB
|
||||
- **Final training loss:** 1.0875
|
||||
- **Final validation loss:** 1.0976
|
||||
- **Perplexity:** ~3.00
|
||||
|
||||
## LoRA Configuration
|
||||
|
||||
- **Target modules:** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
|
||||
- **Rank:** 8
|
||||
- **Alpha:** 16
|
||||
- **Dropout:** 0.05
|
||||
- **Trainable parameters:** ~13M / 4B total (0.3%)
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
Reference in New Issue
Block a user