初始化项目，由ModelHub XC社区提供模型

Model: ai-sage/GigaChat3-10B-A1.8B-base Source: Original Platform
2026-10-10 14:55:15 +08:00
commit 556f224823
24 changed files with 5810 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,63 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
+*.tfevents* filter=lfs diff=lfs merge=lfs -text
+*.db* filter=lfs diff=lfs merge=lfs -text
+*.ark* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
+ 
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.gguf* filter=lfs diff=lfs merge=lfs -text
+*.ggml filter=lfs diff=lfs merge=lfs -text
+*.llamafile* filter=lfs diff=lfs merge=lfs -text
+*.pt2 filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+
+model-00005-of-00013.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00002-of-00013.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00000-of-00013.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00003-of-00013.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00007-of-00013.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00009-of-00013.safetensors filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+model-00006-of-00013.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00004-of-00013.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00013-of-00013.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00010-of-00013.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00011-of-00013.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00012-of-00013.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00008-of-00013.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00001-of-00013.safetensors filter=lfs diff=lfs merge=lfs -text
--- a/LICENSE.txt
+++ b/LICENSE.txt
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Salute Developers
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
--- a/README.md
+++ b/README.md
@@ -0,0 +1,152 @@
+---
+license: mit
+language:
+- ru
+- en
+pipeline_tag: text-generation
+tags:
+- moe
+---
+# GigaChat3-10B-A1.8B-base
+
+Представляем `GigaChat3-10B-A1.8B-base` — базовую (pretrain) модель семейства GigaChat. Модель основана на архитектуре Mixture-of-Experts (MoE) с 10B общих и 1.8B активных параметров.
+Архитектура включает **Multi-head Latent Attention (MLA)** и **Multi-Token Prediction (MTP)**, за счет чего модель оптимизирована для высокой пропускной способности (throughput) при инференсе.
+Больше подробностей [в хабр статье](https://habr.com/en/companies/sberdevices/articles/968904/).
+
+## Важное замечание: Base vs. Instruct
+
+Данная карточка описывает **базовую** (pretrain) модель.
+Для диалоговых задач и выполнения инструкций, пожалуйста, используйте нашу [**instruct**-версию](https://huggingface.co/ai-sage/GigaChat3-10B-A1.8B).
+
+## Архитектура модели
+
+`GigaChat3-10B-A1.8B-base` использует кастомную MoE-архитектуру:
+
+### Multi-head Latent Attention (MLA)
+
+Вместо стандартного Multi-head Attention модель использует MLA. MLA обеспечивает эффективный инференс за счет сжатия Key-Value (KV) кэша в латентный вектор, что значительно снижает требования к памяти и ускоряет обработку.
+
+### Multi-Token Prediction (MTP)
+
+Модель обучена с использованием задачи Multi-Token Prediction (MTP). Это позволяет модели предсказывать несколько токенов за один проход, что ускоряет генерацию до 40% с помощью техник спекулятивной/параллельной генерации.
+
+## Данные для обучения
+
+Модель обучена на 20Т токенов.
+Мы добавили 10 языков — от китайского и арабского до узбекского и казахского, а также расширили набор источников: книги, академические данные, датасеты по коду и математике. Все данные проходят дедупликацию, языковую фильтрацию и автоматические проверки качества при помощи эвристик и классификаторов.
+Ключевой вклад в качество внесла синтетика: мы сгенерировали около 5,5 триллионов токенов синтетических данных. В корпус входят вопросы-ответы к текстам, цепочки reverse-prompt для структурирования данных, LLM-заметки с комментариями от модели внутри текстов, миллионы синтетических задач с решениями по математике и олимпиадному программированию (с синтетическими тестами) на основе PromptCot.
+
+## Инференс
+
+Одно из ключевых преимуществ `GigaChat3-10B-A1.8B-base` — скорость инференса. Модель (особенно в режиме MTP) демонстрирует пропускную способность, сопоставимую с пропускной способностью значительно меньших dense‑моделей.
+Измерения проводились с помощью vLLM v0.11.0, на типе bfloat16 c  `batch_size=1`.
+[Ссылка на код.](https://gist.github.com/ajpqs/ce941aa6f0f48ef36a65cb87a2a1d726)
+
+|**Модель**|**request_throughput**|**output_throughput**|**total_token_throughput**|**mean_ttft_ms**|
+|---|---|---|---|---|
+|`Qwen3-1.7B`|1.689|357.308|726.093|11.824|
+|**`mtp-GigaChat3-10B-A1.8B-base`**|**1.533**|**333.620**|**678.894**|**26.345**|
+|**`GigaChat3-10B-A1.8B-base`**|**1.077**|**234.363**|**476.912**|**31.053**|
+|`Qwen3-4B`|0.978|206.849|420.341|14.947|
+|`Qwen3-8B`|0.664|140.432|285.375|16.663|
+|`YandexGPT-5-Lite-8B-pretrain`|0.641|147.305|300.269|16.711|
+
+
+## Бенчмарки
+
+Хотя модель имеет 10 миллиардов параметров, её прямые аналоги — модели размером 3–4 миллиарда параметров. Однако благодаря высокой скорости генерации мы также сравниваем её с ещё более компактными моделями.
+
+![image](https://cdn-uploads.huggingface.co/production/uploads/66356a710508bbbb61184fd2/3lxW7o5sJtzcIRpVMg15c.png)
+
+
+## Как проверить метрики модели
+
+```shell
+# lm-eval[api]==0.4.9.1
+# sglang[all]==0.5.5
+# или 
+# vllm==0.11.2
+
+export HF_ALLOW_CODE_EVAL=1
+
+# sglang server up
+
+# 10B
+python -m sglang.launch_server --model-path <path_to_model> --host 127.0.0.1 --port 30000 --dtype auto --mem-fraction-static 0.88 --trust-remote-code --allow-auto-truncate --speculative-algorithm EAGLE --speculative-num-steps 1 --speculative-eagle-topk 1 --speculative-num-draft-tokens 2
+
+# mmlu pro check
+python -m lm_eval --model sglang-generate --output_path <path_to_model> --batch_size 16 --model_args base_url=http://127.0.0.1:30000/generate,num_concurrent=16,tokenized_requests=True,max_length=131072,tokenizer=<path_to_model> --trust_remote_code --confirm_run_unsafe_code --num_fewshot 5 --tasks mmlu_pro
+
+```
+
+## Пример использования (Quickstart)
+
+### 1.  `transformers`
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig
+
+model_name = "ai-sage/GigaChat3-10B-A1.8B-base"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto")
+model.generation_config = GenerationConfig.from_pretrained(model_name)
+
+messages = (
+    "Ниже я написал подробное доказательство теоремы о неподвижной точке:"
+)
+inputs = tokenizer(messages, return_tensors="pt").to(model.device)
+outputs = model.generate(inputs.input_ids, attention_mask=inputs.attention_mask)
+
+result = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=False)
+print(result)
+```
+
+### 2. `vLLM`
+
+Запуск сервера
+```shell
+vllm serve ai-sage/GigaChat3-10B-A1.8B-base \
+  --dtype "auto" \
+  --speculative-config '{"method": "mtp", "num_speculative_tokens": 1, "disable_padded_drafter_batch": false}'
+```
+
+Пример запроса
+```shell
+curl http://localhost:8000/v1/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "ai-sage/GigaChat3-10B-A1.8B-base",
+    "prompt": "Ниже я написал подробное доказательство теоремы о неподвижной точке:",
+    "max_tokens": 400,
+    "temperature": 0
+  }'
+```
+
+### 3. `SGLang`
+
+Запуск сервера
+```shell
+python -m sglang.launch_server \
+  --model-path ai-sage/GigaChat3-10B-A1.8B-base \
+  --host 0.0.0.0 \
+  --port 30000 \
+  --dtype auto \
+  --mem-fraction-static 0.88 \
+  --speculative-algorithm EAGLE \
+  --speculative-num-steps 1 \
+  --speculative-eagle-topk 1 \
+  --speculative-num-draft-tokens 2
+```
+
+Пример запроса
+```shell
+curl http://localhost:30000/v1/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "ai-sage/GigaChat3-10B-A1.8B-base",
+    "prompt": "Ниже я написал подробное доказательство теоремы о неподвижной точке:",
+    "max_tokens": 400,
+    "temperature": 0
+  }'
+```
--- a/config.json
+++ b/config.json
@@ -0,0 +1,53 @@
+{
+  "vocab_size": 128256,
+  "max_position_embeddings": 262144,
+  "hidden_size": 1536,
+  "intermediate_size": 8960,
+  "moe_intermediate_size": 1280,
+  "num_hidden_layers": 26,
+  "num_nextn_predict_layers": 1,
+  "num_attention_heads": 32,
+  "n_shared_experts": 1,
+  "n_routed_experts": 64,
+  "ep_size": 1,
+  "routed_scaling_factor": 1,
+  "kv_lora_rank": 512,
+  "q_lora_rank": null,
+  "qk_rope_head_dim": 64,
+  "v_head_dim": 192,
+  "qk_nope_head_dim": 128,
+  "topk_method": "noaux_tc",
+  "n_group": 1,
+  "topk_group": 1,
+  "num_experts_per_tok": 4,
+  "moe_layer_freq": 1,
+  "first_k_dense_replace": 1,
+  "norm_topk_prob": true,
+  "scoring_func": "sigmoid",
+  "num_key_value_heads": 32,
+  "hidden_act": "silu",
+  "initializer_range": 0.006,
+  "rms_norm_eps": 1e-06,
+  "use_cache": true,
+  "rope_theta": 100000,
+  "rope_scaling": {
+    "beta_fast": 32,
+    "beta_slow": 1,
+    "factor": 64.0,
+    "mscale": 1.0,
+    "mscale_all_dim": 1.0,
+    "original_max_position_embeddings": 4096,
+    "type": "yarn"
+  },
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "tie_word_embeddings": false,
+  "architectures": [
+    "DeepseekV3ForCausalLM"
+  ],
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "transformers_version": "4.53.2",
+  "model_type": "deepseek_v3",
+  "torch_dtype": "bfloat16"
+}
--- a/configuration.json
+++ b/configuration.json
@@ -0,0 +1 @@
+{"framework": "pytorch", "task": "text-generation", "allow_remote": true}
--- a/generation_config.json
+++ b/generation_config.json
@@ -0,0 +1,6 @@
+{
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "pad_token_id": 2,
+  "transformers_version": "4.53.2"
+}
--- a/model-00000-of-00013.safetensors
+++ b/model-00000-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dbde17d32011e58148063b22e08d509de074bca874a88d75ad8ca02c0496b67f
+size 2535052360
--- a/model-00001-of-00013.safetensors
+++ b/model-00001-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:28b366ea5cbcf69638ea7a3ca50edbc84fed62a20a082a7b61a8a205433d36d0
+size 1634008152
--- a/model-00002-of-00013.safetensors
+++ b/model-00002-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87a0e9430c6a59480962d41a07eb400db77e9ceb94198f2acc245e9f06d1ded6
+size 1634008152
--- a/model-00003-of-00013.safetensors
+++ b/model-00003-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73f93cc6cd2f10be1dd5a68952cc584d57e52ee6e756d6dc6e508bd42890a175
+size 1634008152
--- a/model-00004-of-00013.safetensors
+++ b/model-00004-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c85ce81f3aca0e858e714a7a9bc71963d07b99aeb33372992b9e623a4a6664e
+size 1634008152
--- a/model-00005-of-00013.safetensors
+++ b/model-00005-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39850871731e238984ca3e81b046ab16b4b438910dd09afa7656bec985feeb1f
+size 1634008352
--- a/model-00006-of-00013.safetensors
+++ b/model-00006-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:996a4e6acd3cf59c4f2b31e952e93a4091a767ee0c001dd56afd5db37bccbedb
+size 1634008560
--- a/model-00007-of-00013.safetensors
+++ b/model-00007-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:992bfce2672518d189554a4784b4393c69d7e47090265f9409e28619c6b72f7a
+size 1634008560
--- a/model-00008-of-00013.safetensors
+++ b/model-00008-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f74593b7f5ba04481cae3a7663cd038da37d224c0996b3a6ca487f23ebdc15d5
+size 1634008560
--- a/model-00009-of-00013.safetensors
+++ b/model-00009-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eda8bbcb68bf5b3855e80be49479e8bd39c52a485ae3947c2d5f80901b043d32
+size 1634008560
--- a/model-00010-of-00013.safetensors
+++ b/model-00010-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:14d01008aed42014e5ac5ca35db667a14b7b31d45dbac12128b284a231618bac
+size 1634008560
--- a/model-00011-of-00013.safetensors
+++ b/model-00011-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7763658738bdb8f340800e9bc64b152bc80e0396f701f0e369c962306ddd12d6
+size 1634008560
--- a/model-00012-of-00013.safetensors
+++ b/model-00012-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:423742c221924dc034276768b34c8d7d76cb2dcc11946bb167d48118bdf9e0bc
+size 1634008560
--- a/model-00013-of-00013.safetensors
+++ b/model-00013-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:68091a164dacaf2ea42c73538bdefe4a7073e0657f152104a87dab82c4879718
+size 817004096
--- a/model.safetensors.index.json
+++ b/model.safetensors.index.json
--- a/special_tokens_map.json
+++ b/special_tokens_map.json
@@ -0,0 +1,16 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}
--- a/tokenizer.json
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fdb3c045850a565109244ef3b89475cc2582769417647a41c46da14a3782aa5
+size 10680605
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
@@ -0,0 +1,123 @@
+{
+  "added_tokens_decoder": {
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128000": {
+      "content": "<|role_sep|>\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|message_sep|>\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|file|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|/file|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "[image_token]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "[video_image_token]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "[audio_token]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "[video_audio_token]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<point>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "</point>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<bbox>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "</bbox>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "</s>",
+  "extra_special_tokens": {},
+  "model_max_length": 1000000000000000019884624838656,
+  "tokenizer_class": "PreTrainedTokenizer",
+  "unk_token": null
+}
				`@@ -0,0 +1 @@`
				`{"framework": "pytorch", "task": "text-generation", "allow_remote": true}`