mesolitica/malaysian-llama-3-8b-262k
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mesolitica/malaysian-llama-3-8b-262k
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-26 18:04:12 +08:00
commit 5f489e3dcb
13 changed files with 2492 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
README.md Normal file
View File

@@ -0,0 +1,14 @@
---
library_name: transformers
tags: []
---
# Malaysian Llama-3 8B 262144 context length
262144 context length and 207100000 RoPE Theta.
WanDB, https://wandb.ai/huseinzol05/EasyContext-262144?nw=nwuserhuseinzol05
Source code, https://github.com/mesolitica/malaya/tree/master/session/llama3#extend-1m-context-length
Special thanks to https://github.com/jzhang38/EasyContext for wrapping https://github.com/zhuzilin/ring-flash-attention for distributed training!