mesolitica/malaysian-llama-3-8b-262k
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
5f489e3dcb8f8ae6101d015a17353b9a690b7b6a
malaysian-llama-3-8b-262k/README.md
ModelHub XC 5f489e3dcb 初始化项目，由ModelHub XC社区提供模型 
Model: mesolitica/malaysian-llama-3-8b-262k
Source: Original Platform
2026-06-26 18:04:12 +08:00

465 B
Raw Blame History

library_name, tags
library_name tags
transformers

Malaysian Llama-3 8B 262144 context length

262144 context length and 207100000 RoPE Theta.

WanDB, https://wandb.ai/huseinzol05/EasyContext-262144?nw=nwuserhuseinzol05

Source code, https://github.com/mesolitica/malaya/tree/master/session/llama3#extend-1m-context-length

Special thanks to https://github.com/jzhang38/EasyContext for wrapping https://github.com/zhuzilin/ring-flash-attention for distributed training!