114 lines
3.1 KiB
Markdown
114 lines
3.1 KiB
Markdown
|
---
|
|||
|
|
language:
|
|||
|
|
- ru
|
|||
|
|
library_name: transformers
|
|||
|
|
tags:
|
|||
|
|
- text-generation
|
|||
|
|
- gpt3
|
|||
|
|
- russian
|
|||
|
|
- causal-lm
|
|||
|
|
license: mit
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
base_model: evilfreelancer/ruGPT3XL
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# ruGPT-3 XL (HuggingFace format) GGUF
|
|||
|
|
|
|||
|
|
A 1.3B-parameter GPT-3-style language model for Russian, converted from the original
|
|||
|
|
[ai-forever/rugpt3xl](https://huggingface.co/ai-forever/rugpt3xl) Megatron-LM checkpoint
|
|||
|
|
into a native HuggingFace `transformers` format.
|
|||
|
|
|
|||
|
|
This is a **base (pretrained) model**, not instruction-tuned. It performs text completion
|
|||
|
|
and can be fine-tuned for downstream tasks.
|
|||
|
|
|
|||
|
|
Details in "[A family of pretrained transformer language models for Russian](https://scholar.google.com/citations?view_op=view_citation&hl=en&user=yPayeJIAAAAJ&citation_for_view=yPayeJIAAAAJ:Se3iqnhoufwC)" paper.
|
|||
|
|
|
|||
|
|
## Model Details
|
|||
|
|
|
|||
|
|
| Parameter | Value |
|
|||
|
|
|---|---|
|
|||
|
|
| Parameters | 1.3B |
|
|||
|
|
| Architecture | GPT-3 (decoder-only transformer) |
|
|||
|
|
| Hidden size | 2048 |
|
|||
|
|
| Layers | 24 |
|
|||
|
|
| Attention heads | 16 |
|
|||
|
|
| FFN intermediate size | 8192 |
|
|||
|
|
| Max sequence length | 2048 |
|
|||
|
|
| Vocabulary | 50,264 tokens (BPE) |
|
|||
|
|
| Activation | GELU |
|
|||
|
|
| Normalization | Pre-LayerNorm |
|
|||
|
|
| Position encoding | Learned absolute |
|
|||
|
|
| Precision | float16 |
|
|||
|
|
| Training data | 80B tokens of Russian text (4 epochs) |
|
|||
|
|
| Test perplexity | 12.05 |
|
|||
|
|
|
|||
|
|
## Quick Start
|
|||
|
|
|
|||
|
|
Example with Q4_K_M:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
./llama.cpp/build/bin/llama-cli \
|
|||
|
|
-m ./ruGPT3XL-GGUF/ruGPT3XL-q4_k_m.gguf \
|
|||
|
|
-c 2048 \
|
|||
|
|
-p "Москва - столица" \
|
|||
|
|
-n 128 \
|
|||
|
|
--temp 0.7 \
|
|||
|
|
--top-p 0.9 \
|
|||
|
|
--repeat-penalty 1.2
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Notes:
|
|||
|
|
- Use `-c 2048` for the native context length.
|
|||
|
|
- Prefer `ruGPT3XL-q4_k_m.gguf` or `ruGPT3XL-q8_0.gguf` for CPU inference.
|
|||
|
|
- Use `ruGPT3XL-f16.gguf` mainly for GPU.
|
|||
|
|
|
|||
|
|
Start server:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
./llama.cpp/build/bin/llama-server \
|
|||
|
|
-m ./ruGPT3XL-GGUF/ruGPT3XL-q4_k_m.gguf \
|
|||
|
|
-c 2048 \
|
|||
|
|
--host 127.0.0.1 \
|
|||
|
|
--port 8080
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Example request:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
curl http://127.0.0.1:8080/completion \
|
|||
|
|
-H "Content-Type: application/json" \
|
|||
|
|
-d '{
|
|||
|
|
"prompt": "Вопрос: Какая столица России?\n\nОтвет: ",
|
|||
|
|
"n_predict": 128,
|
|||
|
|
"temperature": 0.7,
|
|||
|
|
"top_p": 0.9,
|
|||
|
|
"repeat_penalty": 1.2
|
|||
|
|
}'
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Limitations
|
|||
|
|
|
|||
|
|
- This is a **base model** trained on Russian internet text. It may generate biased, factually
|
|||
|
|
incorrect, or offensive content.
|
|||
|
|
- The model was trained primarily on Russian text. It has limited capability in other languages.
|
|||
|
|
- Maximum context length is 2048 tokens. Inputs longer than this will be truncated.
|
|||
|
|
- The model is not instruction-tuned and works best for text completion rather than
|
|||
|
|
following specific instructions.
|
|||
|
|
|
|||
|
|
## Citation
|
|||
|
|
|
|||
|
|
```bibtex
|
|||
|
|
@misc{rugpt3xl-gguf,
|
|||
|
|
title={ruGPT3XL-GGUF},
|
|||
|
|
author={Pavel Rykov},
|
|||
|
|
year={2026},
|
|||
|
|
publisher={Hugging Face},
|
|||
|
|
url={https://huggingface.co/evilfreelancer/ruGPT3XL-GGUF}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Links
|
|||
|
|
|
|||
|
|
- [A family of pretrained transformer language models for Russian](https://scholar.google.com/citations?view_op=view_citation&hl=en&user=yPayeJIAAAAJ&citation_for_view=yPayeJIAAAAJ:Se3iqnhoufwC) - paper on Google Scholar
|
|||
|
|
- [ai-forever/rugpt3xl](https://huggingface.co/ai-forever/rugpt3xl) - original model
|
|||
|
|
- [ai-forever/ru-gpts](https://github.com/ai-forever/ru-gpts) - original training codebase
|