初始化项目，由ModelHub XC社区提供模型

Model: wei123602/llama2-13b-fintune2
Source: Original Platform

README.md

@@ -0,0 +1,51 @@
+---
+license: llama2
+datasets:
+- huangyt/FINETUNE2
+---
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+在llama-2-13b上使用huangyt/FINETUNE2資料集進行訓練，採用與platypus相近參數
+
+# Fine-Tuning Information
+- **GPU:** RTX4090 (single core / 24564MiB)
+- **base model:** meta-llama/Llama-2-13b-hf
+- **dataset:** huangyt/FINETUNE2 (共約3w筆訓練集)
+- **peft_type:** LoRA
+- **lora_rank:** 16
+- **lora alpha:** 8
+- **lora dropout:** 0.05
+- **lora_target:** gate_proj, up_proj, down_proj
+- **per_device_train_batch_size:** 8
+- **gradient_accumulation_steps:** 8
+- **learning_rate :** 4e-4
+- **epoch:** 1
+- **batch size:** 8
+- **microbatch size:** 4
+- **warmup steps:** 100
+- **weight decay:** cosine
+- **cutoff length:** 2048
+- **precision:** bf16
+- **group_by_length:** True
+- **load_in_8bit:** True
+
+# Fine-Tuning Detail
+- **train_loss:** 0.0823
+- **train_runtime:** 02:40:01
+
+# Evaluation
+- 評估結果來自**HuggingFaceH4/open_llm_leaderboard**
+- 與Llama-2-13b比較4種Benchmark，包含**ARC**、**HellaSwag**、**MMLU**、**TruthfulQA**
+
+| Model                                                  |Average|  ARC  |HellaSwag| MMLU  |TruthfulQA|
+|--------------------------------------------------------|-------|-------|---------|-------|----------|
+|meta-llama/Llama-2-13b-hf                               | 56.9  | 58.11 |  80.97  | 54.34 |  34.17   |
+|meta-llama/Llama-2-13b-chat-hf                          | 59.93 | 59.04 |  81.94  | 54.64 |  44.12   |
+|CHIH-HUNG/llama-2-13b-FINETUNE2_3w                      | 58.24 | 58.62 |  82.32  | 54.25 |  38.17   |
+|CHIH-HUNG/llama-2-13b-huangyt_Fintune_1_17w-q_k_v_o_proj| 58.49 | 59.73 |  81.06  | 54.53 |  38.64   |
+|CHIH-HUNG/llama-2-13b-FINETUNE2_3w-q_k_v_o_proj         | 58.21 | 58.53 |  82.47  | 53.9  |  37.92   |
+|CHIH-HUNG/llama-2-13b-FINETUNE2_3w-gate_up_down_proj    | 58.81 | 57.42 |  82.42  | 55.57 |  39.19   |
+|wei123602/llama2-13b-fintune2                           | wait  | wait  |  wait   | wait  |  wait    |
+