初始化项目,由ModelHub XC社区提供模型
Model: dikiyplayerpig/dpp-gpt-v2.0-flash-135m Source: Original Platform
This commit is contained in:
41
.gitattributes
vendored
Normal file
41
.gitattributes
vendored
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
dpp-gpt-v2.0-flash-f16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
dpp-gpt-v2.0-flash-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
dpp-gpt-v2.0-flash-q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
dpp-gpt-v2.0-flash-q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
dpp-gpt-v2.0-flash-q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
dpp-gpt-v2.0-flash-q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
131
README.md
Normal file
131
README.md
Normal file
@@ -0,0 +1,131 @@
|
|||||||
|
---
|
||||||
|
tags:
|
||||||
|
- text-generation
|
||||||
|
- custom-framework
|
||||||
|
- gguf
|
||||||
|
- llama-3
|
||||||
|
- ru
|
||||||
|
- en
|
||||||
|
license: apache-2.0
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
- ru
|
||||||
|
---
|
||||||
|
|
||||||
|
# 🧠 dpp-gpt v2.0 Flash (135M)
|
||||||
|
|
||||||
|
*(🇺🇸 English / 🇷🇺 Русский)*
|
||||||
|
|
||||||
|
This is a compact language model trained **from scratch** on the Llama 3 architecture. The model is part of the dpp-gpt small language models (SLM) family.
|
||||||
|
|
||||||
|
## ⚙️ Model Details
|
||||||
|
* **Architecture:** Llama 3
|
||||||
|
* **Parameters:** 136.9M
|
||||||
|
* **Layers:** 16
|
||||||
|
* **Hidden size (Embeddings):** 768
|
||||||
|
* **Attention heads:** 12
|
||||||
|
* **Vocab size:** ~24,000
|
||||||
|
* **Context:** 1024 tokens
|
||||||
|
* **Format:** GGUF
|
||||||
|
* **License:** Apache 2.0
|
||||||
|
|
||||||
|
## 🚀 Features & Training
|
||||||
|
* **Pre-training:** Weights were randomly initialized, and the architecture was built entirely from scratch.
|
||||||
|
* **Fine-Tuning (SFT):** The model underwent Supervised Fine-Tuning (SFT) on a custom dataset to understand dialogue structure and answer questions.
|
||||||
|
|
||||||
|
## 📊 Training Data
|
||||||
|
The model was pre-trained on a corpus of **~6.93 Billion tokens**. The dataset is a carefully balanced mix of languages, code, and reasoning data:
|
||||||
|
* **49%** — CulturaX (Russian text corpus)
|
||||||
|
* **28%** — Russian Wikipedia
|
||||||
|
* **11%** — Source code in various programming languages
|
||||||
|
* **9%** — Cosmopedia
|
||||||
|
* **3%** — Mathematics
|
||||||
|
|
||||||
|
## 💻 How to Use
|
||||||
|
The model is optimized for low-end devices and smartphones. It can be easily run using:
|
||||||
|
* **LM Studio**
|
||||||
|
* **llama.cpp**
|
||||||
|
* **KoboldCPP**
|
||||||
|
|
||||||
|
* recommended temperature: **0.3**
|
||||||
|
* recommended top-k: **10**
|
||||||
|
|
||||||
|
**Standard Prompt Format (ChatML):**
|
||||||
|
```text
|
||||||
|
<|im_start|>user
|
||||||
|
{prompt}<|im_end|>
|
||||||
|
<|im_start|>assistant
|
||||||
|
```
|
||||||
|
|
||||||
|
**Reasoning Mode Prompt:**
|
||||||
|
```text
|
||||||
|
<|im_start|>user
|
||||||
|
[THINK] {prompt}<|im_end|>
|
||||||
|
<|im_start|>assistant
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# 🇷🇺 Описание на русском
|
||||||
|
|
||||||
|
Это компактная языковая модель, обученная **с нуля (from scratch)** на архитектуре Llama 3. Модель является частью семейства малых языковых моделей (SLM) проекта dpp-gpt.
|
||||||
|
|
||||||
|
## ⚙️ Детали модели
|
||||||
|
* **Архитектура:** Llama 3
|
||||||
|
* **Параметры:** 136.9M
|
||||||
|
* **Слои (Layers):** 16
|
||||||
|
* **Размерность (Hidden size):** 768
|
||||||
|
* **Головы внимания (Attention heads):** 12
|
||||||
|
* **Размер словаря (Vocab size):** ~24,000
|
||||||
|
* **Контекст:** 1024 токена
|
||||||
|
* **Формат весов:** GGUF
|
||||||
|
* **Лицензия:** Apache 2.0
|
||||||
|
|
||||||
|
## 🚀 Особенности и обучение
|
||||||
|
* **Pre-training:** Веса были инициализированы случайным образом, архитектура полностью собрана с нуля.
|
||||||
|
* **Fine-Tuning (SFT):** Модель прошла стадию SFT на авторском датасете, чтобы понимать структуру диалога и отвечать на вопросы.
|
||||||
|
|
||||||
|
## 📊 Данные для обучения
|
||||||
|
Модель обучалась на корпусе объемом **~6.93 млрд токенов**. Состав датасета был сбалансирован для получения базовых навыков языка, логики и программирования:
|
||||||
|
* **49%** — CulturaX (очищенный русскоязычный корпус)
|
||||||
|
* **28%** — Русская Википедия
|
||||||
|
* **11%** — Исходный код на различных языках программирования
|
||||||
|
* **9%** — Cosmopedia
|
||||||
|
* **3%** — Математика
|
||||||
|
|
||||||
|
## 💻 Как использовать
|
||||||
|
Модель оптимизирована для работы на слабых устройствах и телефонах. Она легко запускается через:
|
||||||
|
* **LM Studio**
|
||||||
|
* **llama.cpp**
|
||||||
|
* **KoboldCPP**
|
||||||
|
|
||||||
|
* рекомендованная temperature: **0.3**
|
||||||
|
* рекомендованный top-k: **10**
|
||||||
|
|
||||||
|
**Стандартный формат промпта:**
|
||||||
|
```text
|
||||||
|
<|im_start|>user
|
||||||
|
{промпт}<|im_end|>
|
||||||
|
<|im_start|>assistant
|
||||||
|
```
|
||||||
|
|
||||||
|
**Режим размышления:**
|
||||||
|
```text
|
||||||
|
<|im_start|>user
|
||||||
|
[THINK] {промпт}<|im_end|>
|
||||||
|
<|im_start|>assistant
|
||||||
|
```
|
||||||
|
|
||||||
|
<!-- lmstudio-hotfix -->
|
||||||
|
|
||||||
|
## Hotfix
|
||||||
|
|
||||||
|
The GGUF files in this repository were re-uploaded as a hotfix.
|
||||||
|
|
||||||
|
An LM Studio update changed the way special tokens are handled, which made the
|
||||||
|
previously published GGUF files generate broken output. The conversion has been
|
||||||
|
fixed and the quantizations here were rebuilt from the corrected model. The
|
||||||
|
weights are unchanged — only the token metadata inside the GGUF files.
|
||||||
|
|
||||||
|
If you downloaded a GGUF from this repository before this commit, please
|
||||||
|
download it again.
|
||||||
3
dpp-gpt-v2.0-flash-f16.gguf
Normal file
3
dpp-gpt-v2.0-flash-f16.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:95b6ceb860f80dbc2682d9e1228cc32473c989f5513f1e3086a97210742be3e7
|
||||||
|
size 275138624
|
||||||
3
dpp-gpt-v2.0-flash-q2_K.gguf
Normal file
3
dpp-gpt-v2.0-flash-q2_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:5f84f9389cae2c6ad6c0584c71485af64d1e4cb3e91721f69c5bc149131257e3
|
||||||
|
size 59717728
|
||||||
3
dpp-gpt-v2.0-flash-q4_K_M.gguf
Normal file
3
dpp-gpt-v2.0-flash-q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:f10712abb0b63b5007c769a1e00f6db0e4a24918a99ba6d2dd01f14719fc9f7f
|
||||||
|
size 86846560
|
||||||
3
dpp-gpt-v2.0-flash-q6_K.gguf
Normal file
3
dpp-gpt-v2.0-flash-q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:1155f25b143123dc7e5119462aaceeb96794b285617303a1bedd779a6e1d123e
|
||||||
|
size 113711200
|
||||||
3
dpp-gpt-v2.0-flash-q8_0.gguf
Normal file
3
dpp-gpt-v2.0-flash-q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:4a8d81d313a70aed50ba3d3029640729b18fc792156d1f9a19c416fe642db265
|
||||||
|
size 146851936
|
||||||
122357
tokenizer.json
Normal file
122357
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user