adamo1139/LWM-7B-1M-1000000ctx-AEZAKMI-3_1-1702
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: adamo1139/LWM-7B-1M-1000000ctx-AEZAKMI-3_1-1702
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-23 08:29:17 +08:00
commit 796558edab
12 changed files with 93843 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
README.md Normal file
View File

@@ -0,0 +1,7 @@
---
license: llama2
---
LargeWorldModel 7B 1000000 ctx finetuned on AEZAKMI v3.1 dataset for epochs at max_seq_len of 4000 using QLoRA with lora_r 32 and cosine lr decaying from 0.00015.
I will be uploading exl2 quants and base model in safetensors format soon.
Fine-tuned with unsloth, FA2 on local RTX 3090 Ti. Training took around 6 hours. I think most of the long ctx capabilities remain.