vaiv/GeM2-Llamion-14B-LongChat
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: vaiv/GeM2-Llamion-14B-LongChat
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-16 19:37:52 +08:00
commit 31f500fd7f
17 changed files with 170447 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
README.md Normal file
View File

@@ -0,0 +1,25 @@
---
license: apache-2.0
---
# **GeM2-Llamion-14B**
We have released **Llamion** as **GeM 2.0**, the second series of generative models developed by VAIV Company to address the our principal business needs.
**Llamion** (Llamafied Orion) is derived from transforming the [Orion model](https://huggingface.co/OrionStarAI/Orion-14B-LongChat)
into [the standard LLaMA architecture](https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py)
through parameter mapping and offline knowledge transfer.
Further technical specifications and study results are detailed in [our paper](https://arxiv.org/abs/2605.25676).
![vaiv_png](./vaiv.png)
Notably, the LongChat model supports an extensive text range of 200K tokens.
The following figure shows the perplexity of models
on [English Wikipedia corpus](https://huggingface.co/datasets/wikimedia/wikipedia/viewer/20231101.en)
and [Korean Wikipedia corpus](https://huggingface.co/datasets/wikimedia/wikipedia/viewer/20231101.ko), respectively.
![ppl_wiki_enko](./ppl_wiki_enko.png)
### Contributors
- VAIV Company AI Lab ([vaiv.kr](https://www.vaiv.kr/))