sayhan/OpenHermes-2.5-Strix-Philosophy-Mistral-7B-LoRA
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Sayhan Yalvaçer
2024-02-18 15:53:23 +00:00
committed by system
parent 4b36bc80c7
commit d82f3d8e4b
1 changed files with 23 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
README.md
View File

@@ -19,3 +19,25 @@ library_name: transformers
- **License:** [apache-2.0](https://choosealicense.com/licenses/apache-2.0/) - **License:** [apache-2.0](https://choosealicense.com/licenses/apache-2.0/)
- **Finetuned from model :** [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) - **Finetuned from model :** [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B)
- **Dataset:** [sayhan/strix-philosophy-qa](https://huggingface.co/datasets/sayhan/strix-philosophy-qa) - **Dataset:** [sayhan/strix-philosophy-qa](https://huggingface.co/datasets/sayhan/strix-philosophy-qa)
---
**LoRA rank:** 8
**LoRA alpha:** 16
**LoRA dropout:** 0
**Rank-stabilized LoRA:** Yes
**Number of epochs:** 3
**Learning rate:** 1e-5
**Batch size:** 2
**Gradient accumulation steps:** 4
**Weight decay:** 0.01
**Target modules:**
```
- Query projection (`q_proj`)
- Key projection (`k_proj`)
- Value projection (`v_proj`)
- Output projection (`o_proj`)
- Gate projection (`gate_proj`)
- Up projection (`up_proj`)
- Down projection (`down_proj`)
```