mesolitica/llama-1b-hf-32768-fpf
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
1 Commit 1 Branch 0 Tags
e9b6888a83dc92f5c463e5b3fe610cdb75b53519
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE

README.md

language
language
ms

Full Parameter Finetuning 1B 32768 context length Llama2 on Malaysian text

1B derived from first 4 layers 7B model.

README at https://github.com/mesolitica/malaya/tree/5.1/session/llama2#1b-32768-context-length-flash-attention-2

WandB, https://wandb.ai/mesolitica/fpf-Llama-2-1b-32k-hf

Description
Model synced from source: mesolitica/llama-1b-hf-32768-fpf
Readme 513 KiB