9 lines
272 B
Markdown
9 lines
272 B
Markdown
|
||
## version： v20240318
|
||
- train 12B tokens with 256k context window
|
||
- recall of "Needle in A HayStack": 97.5% [](./images/v20240318.png)
|
||
|
||
## version: initial
|
||
- train 6B tokens with 256k context window
|
||
- recall of "Needle in A HayStack": 87.1% [](./images/initail.png)
|