初始化项目,由ModelHub XC社区提供模型
Model: sayhan/OpenHermes-2.5-Strix-Philosophy-Mistral-7B-LoRA Source: Original Platform
This commit is contained in:
43
README.md
Normal file
43
README.md
Normal file
@@ -0,0 +1,43 @@
|
||||
---
|
||||
language:
|
||||
- en
|
||||
license: apache-2.0
|
||||
tags:
|
||||
- trl
|
||||
- text-generation-inference
|
||||
- unsloth
|
||||
- mistral
|
||||
- gguf
|
||||
base_model: teknium/OpenHermes-2.5-Mistral-7B
|
||||
datasets:
|
||||
- sayhan/strix-philosophy-qa
|
||||
library_name: transformers
|
||||
---
|
||||

|
||||
# OpenHermes 2.5 Stix Philosophy Mistral 7B
|
||||
- **Finetuned by:** [sayhan](https://huggingface.co/sayhan)
|
||||
- **License:** [apache-2.0](https://choosealicense.com/licenses/apache-2.0/)
|
||||
- **Finetuned from model :** [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B)
|
||||
- **Dataset:** [sayhan/strix-philosophy-qa](https://huggingface.co/datasets/sayhan/strix-philosophy-qa)
|
||||
---
|
||||
**LoRA rank:** 8
|
||||
**LoRA alpha:** 16
|
||||
**LoRA dropout:** 0
|
||||
**Rank-stabilized LoRA:** Yes
|
||||
**Number of epochs:** 3
|
||||
**Learning rate:** 1e-5
|
||||
**Batch size:** 2
|
||||
**Gradient accumulation steps:** 4
|
||||
**Weight decay:** 0.01
|
||||
**Target modules:**
|
||||
```
|
||||
- Query projection (`q_proj`)
|
||||
- Key projection (`k_proj`)
|
||||
- Value projection (`v_proj`)
|
||||
- Output projection (`o_proj`)
|
||||
- Gate projection (`gate_proj`)
|
||||
- Up projection (`up_proj`)
|
||||
- Down projection (`down_proj`)
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user