9 lines
272 B
Markdown
9 lines
272 B
Markdown
|
|
|||
|
|
## version： v20240318
|
|||
|
|
- train 12B tokens with 256k context window
|
|||
|
|
- recall of "Needle in A HayStack": 97.5% [](./images/v20240318.png)
|
|||
|
|
|
|||
|
|
## version: initial
|
|||
|
|
- train 6B tokens with 256k context window
|
|||
|
|
- recall of "Needle in A HayStack": 87.1% [](./images/initail.png)
|