47824a2c8d
Model: afrideva/TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1-GGUF Source: Original Platform
93 lines
4.0 KiB
Markdown
93 lines
4.0 KiB
Markdown
---
|
|
base_model: habanoz/TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1
|
|
datasets:
|
|
- OpenAssistant/oasst_top1_2023-08-25
|
|
inference: false
|
|
language:
|
|
- en
|
|
license: apache-2.0
|
|
model_creator: habanoz
|
|
model_name: TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1
|
|
pipeline_tag: text-generation
|
|
quantized_by: afrideva
|
|
tags:
|
|
- gguf
|
|
- ggml
|
|
- quantized
|
|
- q2_k
|
|
- q3_k_m
|
|
- q4_k_m
|
|
- q5_k_m
|
|
- q6_k
|
|
- q8_0
|
|
---
|
|
# habanoz/TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1-GGUF
|
|
|
|
Quantized GGUF model files for [TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1](https://huggingface.co/habanoz/TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1) from [habanoz](https://huggingface.co/habanoz)
|
|
|
|
|
|
| Name | Quant method | Size |
|
|
| ---- | ---- | ---- |
|
|
| [tinyllama-1.1b-step-2t-lr-5-5ep-oasst1-top1-instruct-v1.fp16.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1-GGUF/resolve/main/tinyllama-1.1b-step-2t-lr-5-5ep-oasst1-top1-instruct-v1.fp16.gguf) | fp16 | 2.20 GB |
|
|
| [tinyllama-1.1b-step-2t-lr-5-5ep-oasst1-top1-instruct-v1.q2_k.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1-GGUF/resolve/main/tinyllama-1.1b-step-2t-lr-5-5ep-oasst1-top1-instruct-v1.q2_k.gguf) | q2_k | 483.12 MB |
|
|
| [tinyllama-1.1b-step-2t-lr-5-5ep-oasst1-top1-instruct-v1.q3_k_m.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1-GGUF/resolve/main/tinyllama-1.1b-step-2t-lr-5-5ep-oasst1-top1-instruct-v1.q3_k_m.gguf) | q3_k_m | 550.82 MB |
|
|
| [tinyllama-1.1b-step-2t-lr-5-5ep-oasst1-top1-instruct-v1.q4_k_m.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1-GGUF/resolve/main/tinyllama-1.1b-step-2t-lr-5-5ep-oasst1-top1-instruct-v1.q4_k_m.gguf) | q4_k_m | 668.79 MB |
|
|
| [tinyllama-1.1b-step-2t-lr-5-5ep-oasst1-top1-instruct-v1.q5_k_m.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1-GGUF/resolve/main/tinyllama-1.1b-step-2t-lr-5-5ep-oasst1-top1-instruct-v1.q5_k_m.gguf) | q5_k_m | 783.02 MB |
|
|
| [tinyllama-1.1b-step-2t-lr-5-5ep-oasst1-top1-instruct-v1.q6_k.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1-GGUF/resolve/main/tinyllama-1.1b-step-2t-lr-5-5ep-oasst1-top1-instruct-v1.q6_k.gguf) | q6_k | 904.39 MB |
|
|
| [tinyllama-1.1b-step-2t-lr-5-5ep-oasst1-top1-instruct-v1.q8_0.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1-GGUF/resolve/main/tinyllama-1.1b-step-2t-lr-5-5ep-oasst1-top1-instruct-v1.q8_0.gguf) | q8_0 | 1.17 GB |
|
|
|
|
|
|
|
|
## Original Model Card:
|
|
TinyLlama/TinyLlama-1.1B-intermediate-step-955k-token-2T finetuned using OpenAssistant/oasst_top1_2023-08-25 dataset.
|
|
|
|
Trained for 5 epochs using Qlora. Adapter is merged.
|
|
|
|
SFT code:
|
|
https://github.com/habanoz/qlora.git
|
|
|
|
Command used:
|
|
```bash
|
|
accelerate launch $BASE_DIR/qlora/train.py \
|
|
--model_name_or_path $BASE_MODEL \
|
|
--working_dir $BASE_DIR/$OUTPUT_NAME-checkpoints \
|
|
--output_dir $BASE_DIR/$OUTPUT_NAME-peft \
|
|
--merged_output_dir $BASE_DIR/$OUTPUT_NAME \
|
|
--final_output_dir $BASE_DIR/$OUTPUT_NAME-final \
|
|
--num_train_epochs 5 \
|
|
--logging_steps 1 \
|
|
--save_strategy steps \
|
|
--save_steps 75 \
|
|
--save_total_limit 2 \
|
|
--data_seed 11422 \
|
|
--evaluation_strategy steps \
|
|
--per_device_eval_batch_size 4 \
|
|
--eval_dataset_size 0.01 \
|
|
--eval_steps 75 \
|
|
--max_new_tokens 1024 \
|
|
--dataloader_num_workers 3 \
|
|
--logging_strategy steps \
|
|
--do_train \
|
|
--do_eval \
|
|
--lora_r 64 \
|
|
--lora_alpha 16 \
|
|
--lora_modules all \
|
|
--bits 4 \
|
|
--double_quant \
|
|
--quant_type nf4 \
|
|
--lr_scheduler_type constant \
|
|
--dataset oasst1-top1 \
|
|
--dataset_format oasst1 \
|
|
--model_max_len 1024 \
|
|
--per_device_train_batch_size 4 \
|
|
--gradient_accumulation_steps 4 \
|
|
--learning_rate 1e-5 \
|
|
--adam_beta2 0.999 \
|
|
--max_grad_norm 0.3 \
|
|
--lora_dropout 0.0 \
|
|
--weight_decay 0.0 \
|
|
--seed 11422 \
|
|
--gradient_checkpointing \
|
|
--use_flash_attention_2 \
|
|
--ddp_find_unused_parameters False
|
|
``` |