29 lines
804 B
Markdown
29 lines
804 B
Markdown
|
---
|
|||
|
|
base_model: Qwen/Qwen3-4B-Instruct-2507
|
|||
|
|
datasets:
|
|||
|
|
- u-10bei/structured_data_with_cot_dataset_512_v2
|
|||
|
|
language:
|
|||
|
|
- en
|
|||
|
|
license: apache-2.0
|
|||
|
|
library_name: transformers
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
tags:
|
|||
|
|
- structured-output
|
|||
|
|
- merged
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# qwen3-4b-structured-sft-lora-v07-merged
|
|||
|
|
|
|||
|
|
Fully merged model (base + LoRA) fine-tuned from **Qwen/Qwen3-4B-Instruct-2507**.
|
|||
|
|
|
|||
|
|
## v07 変更点
|
|||
|
|
- **SFT LR**: 2e-6 → **2e-5**（v03比10倍。ガイド著者実績値）
|
|||
|
|
- 他はv03と同一（データセット・MASK_COT=1・LoRA r=64・Epoch=2）
|
|||
|
|
|
|||
|
|
## Training Configuration
|
|||
|
|
- Base model: Qwen/Qwen3-4B-Instruct-2507
|
|||
|
|
- Method: QLoRA (4-bit) → merged
|
|||
|
|
- LR: 2e-05 / Epochs: 2 / LoRA: r=64, alpha=128
|
|||
|
|
- Dataset: u-10bei/structured_data_with_cot_dataset_512_v2（3933件）
|
|||
|
|
- MASK_COT: 1（CoT保持・lossマスク）
|