初始化项目，由ModelHub XC社区提供模型

Model: NickupAI/LFM2.5-1.2B-Saiga-It-v3-GGUF Source: Original Platform
2026-11-03 09:04:16 +08:00
commit f20905b579
9 changed files with 192 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,42 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+LFM2.5-1.2B-Saiga-It-v3-BF16.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2.5-1.2B-Saiga-It-v3-F16.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2.5-1.2B-Saiga-It-v3-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2.5-1.2B-Saiga-It-v3-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2.5-1.2B-Saiga-It-v3-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2.5-1.2B-Saiga-It-v3-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2.5-1.2B-Saiga-It-v3-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
--- a/LFM2.5-1.2B-Saiga-It-v3-BF16.gguf
+++ b/LFM2.5-1.2B-Saiga-It-v3-BF16.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2078702a092068971fc3247186f6db3374c27ed08043964d95758fd590b1ef27
+size 2343326016
--- a/LFM2.5-1.2B-Saiga-It-v3-F16.gguf
+++ b/LFM2.5-1.2B-Saiga-It-v3-F16.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1483bec190f2ea38fcf842e33a80a0a2bc5f84702b2b12728531745e34a4c86c
+size 2343326016
--- a/LFM2.5-1.2B-Saiga-It-v3-IQ3_M.gguf
+++ b/LFM2.5-1.2B-Saiga-It-v3-IQ3_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e79dc0df935ab86cb54fa22b5421ef1af8bcc72a07f8235d0d04ae1393aa31a9
+size 566792512
--- a/LFM2.5-1.2B-Saiga-It-v3-Q4_K_M.gguf
+++ b/LFM2.5-1.2B-Saiga-It-v3-Q4_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b32ae11554ca0a75ee5c81d8f98615e10d2657a0b9fdce16a5f4b4835e83bc8
+size 731976000
--- a/LFM2.5-1.2B-Saiga-It-v3-Q5_K_M.gguf
+++ b/LFM2.5-1.2B-Saiga-It-v3-Q5_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8775cf2c2316bf4f1431dff47f42581c4018e3a3f5739a4d16eea0b581b4973b
+size 843911488
--- a/LFM2.5-1.2B-Saiga-It-v3-Q6_K.gguf
+++ b/LFM2.5-1.2B-Saiga-It-v3-Q6_K.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1472e2f699c9490a20d26c56d273fa10b719c2a6474acc1bfb5129bde1a1e4aa
+size 962842944
--- a/LFM2.5-1.2B-Saiga-It-v3-Q8_0.gguf
+++ b/LFM2.5-1.2B-Saiga-It-v3-Q8_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d865e6109b2efefd36e912c34dbf323c62edf553693ae465175fb468c027eef4
+size 1246253376
--- a/README.md
+++ b/README.md
@@ -0,0 +1,129 @@
+---
+language:
+- ru
+- en
+license: other
+base_model: NickupAI/LFM2.5-1.2B-Saiga-It-v3
+tags:
+- russian
+- saiga
+- continued-pretraining
+- sft
+- lfm
+- liquid-ai
+datasets:
+- IlyaGusev/saiga_scored
+- d0rj/alpaca-cleaned-ru
+- IlyaGusev/ru_sharegpt_cleaned
+- IlyaGusev/ru_turbo_saiga
+- lksy/ru_instruct_gpt4
+- wikimedia/wikipedia
+- uonlp/CulturaX
+- allenai/c4
+- IlyaGusev/habr
+---
+
+# LFM2.5-1.2B-Saiga-It-v3
+
+Continued attempt to make a Russian-language Saiga out of [LFM2.5 1.2B](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct) by Liquid AI. This is v3 — a significant upgrade over [v2](https://huggingface.co/NickupAI/LFM2.5-1.2B-Saiga-It-v2) with much more CPT data.
+
+Inspired by the amazing work of [Ilya Gusev](https://huggingface.co/IlyaGusev) and the [Saiga project](https://huggingface.co/IlyaGusev/saiga_llama3_8b).
+
+## What changed vs v2?
+
+**CPT dataset grew from ~1.25M to ~2.69M examples:**
+- Full Russian Wikipedia (~1.9M articles, up from 350k)
+- Added Habr (~295k technical articles in Russian)
+- Replaced c4 ru with CulturaX ru (200k)
+- Kept English c4 (300k) for retention
+
+**SFT dataset expanded:**
+- Added `IlyaGusev/ru_turbo_saiga`
+- Added `lksy/ru_instruct_gpt4`
+
+## Training
+
+**Stage 1 — Continued Pre-Training (CPT):**
+- `wikimedia/wikipedia` (Russian, ~1.9M articles)
+- `IlyaGusev/habr` (Russian, ~295k technical articles)
+- `uonlp/CulturaX` (Russian, 200k)
+- `allenai/c4` (English, 300k for retention)
+- 10,000 steps, lr=3e-5, loss 2.26 → 2.13
+
+**Stage 2 — Supervised Fine-Tuning (SFT):**
+- `IlyaGusev/saiga_scored` (opus_score ≥ 8, ~27k)
+- `d0rj/alpaca-cleaned-ru` (15k)
+- `IlyaGusev/ru_sharegpt_cleaned`
+- `IlyaGusev/ru_turbo_saiga`
+- `lksy/ru_instruct_gpt4`
+- 2 epochs, lr=5e-6, final loss ~1.21
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+model = AutoModelForCausalLM.from_pretrained(
+    "NickupAI/LFM2.5-1.2B-Saiga-It-v3",
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+    trust_remote_code=True,
+)
+tokenizer = AutoTokenizer.from_pretrained(
+    "NickupAI/LFM2.5-1.2B-Saiga-It-v3",
+    trust_remote_code=True,
+)
+
+messages = [{"role": "user", "content": "Привет! Как дела?"}]
+text = tokenizer.apply_chat_template(
+    messages, tokenize=False, add_generation_prompt=True
+)
+inputs = tokenizer(text, return_tensors="pt").to(model.device)
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=200,
+    temperature=0.1,
+    top_k=50,
+    top_p=0.1,
+    repetition_penalty=1.05,
+    do_sample=True,
+)
+print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
+```
+
+## Honest Warning ⚠️
+
+Still hallucinates. Training a model of this size properly requires significantly more compute
+and time than was available for this project — the CPT data barely scratches the surface of
+what would be needed for reliable factual knowledge. The model knows the *format* of answers
+but sometimes invents the content.
+
+**Recommended for:**
+- Creative nonsense generation, lol
+- Experiments and research
+- Russian language generation tasks
+
+**Not recommended for:**
+- Factual questions
+- Medicine, law or any serious topics
+- Astronomy (the dwarf planet situation has not improved)
+
+> — Назови все карликовые планеты Солнечной системы
+>
+> 1. Марс, 2. Венус, 3. Земля, 4. Юпитер... 8. Эритрея
+
+v2 had Kvass and Gamma-Tit. v3 replaced them with Eritrea. Progress.
+
+## GGUF versions
+
+Quantized versions available at [NickupAI/LFM2.5-1.2B-Saiga-It-v3-GGUF](https://huggingface.co/NickupAI/LFM2.5-1.2B-Saiga-It-v3-GGUF).
+
+## What's next
+
+LoRA experiments on larger base models (7B+). The 1.2B size has fundamental limitations for factual knowledge — time to scale up.
+
+
+## GGUF Versions
+
+Quantized versions of the model.