32 lines
1.1 KiB
Markdown
32 lines
1.1 KiB
Markdown
---
|
|
license: other
|
|
license_name: yi-license
|
|
license_link: LICENSE
|
|
datasets:
|
|
- jondurbin/airoboros-3.1
|
|
---
|
|
|
|
|
|
|
|
# Instruction tune of Yi-34b-200k with Airoboros-3.1 (fp16)
|
|
|
|
|
|
## Overview
|
|
|
|
This is [larryvrh/Yi-34B-200K-Llamafied](https://huggingface.co/larryvrh/Yi-34B-200K-Llamafied), with instruction tuning performed with Jon Durbin's [jondurbin/airoboros-3.1](https://huggingface.co/datasets/jondurbin/airoboros-3.1) dataset. That base model is [01-ai/Yi-34B-200k](https://huggingface.co/01-ai/Yi-34B-200k), but using llama2 model definitions and tokenizer to remove any remote code requirements.
|
|
|
|
**This is a (merged) QLoRA fine-tune (rank 64)**.
|
|
|
|
The finetune was performed with 1x RTX 6000 Ada (~80 hours to this checkpoint). Prompts were truncated to 4096 tokens (for speed and VRAM headroom).
|
|
|
|
I have done very little testing with this model, so feedback on real world performance is appreciated!
|
|
|
|
## How to Use
|
|
|
|
Use as you would any other Hugging Face fp16 llama-2 model.
|
|
|
|
## Prompting:
|
|
|
|
Model was trained with llama-2 chat prompt format. See [jondurbin/airoboros-l2-13b-3.1.1](https://huggingface.co/jondurbin/airoboros-l2-13b-3.1.1) model card for details.
|
|
|