初始化项目，由ModelHub XC社区提供模型

Model: dogukanvzr/Kumru-2B-Thinking-v2.0 Source: Original Platform
2026-06-16 06:05:18 +08:00
commit 953860da6a
20 changed files with 261106 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,42 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/benchmark_average.png filter=lfs diff=lfs merge=lfs -text
+assets/benchmark_tasks_greedy.png filter=lfs diff=lfs merge=lfs -text
+assets/benchmark_tasks_sampling.png filter=lfs diff=lfs merge=lfs -text
+assets/positioning_cards.png filter=lfs diff=lfs merge=lfs -text
+assets/v2_vs_v1_greedy_delta.png filter=lfs diff=lfs merge=lfs -text
+assets/kumru_header.png filter=lfs diff=lfs merge=lfs -text
+assets/model_specs_strip.png filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,476 @@
+---
+license: apache-2.0
+language:
+- tr
+pipeline_tag: text-generation
+library_name: transformers
+base_model: vngrs-ai/Kumru-2B
+datasets:
+- dogukanvzr/kumru-v5-dataset
+tags:
+- turkish
+- türkçe
+- reasoning
+- thinking
+- chain-of-thought
+- full-finetune
+- fine-tuned
+- text-generation
+- conversational
+---
+
+<p align="center">
+  <img src="./assets/kumru_header.png" alt="Kumru-2B-Thinking-v2.0 header" width="100%">
+</p>
+
+<p align="center">
+  <img src="https://img.shields.io/badge/Language-Turkish-red?style=for-the-badge" />
+  <img src="https://img.shields.io/badge/Model-2.375B-blue?style=for-the-badge" />
+  <img src="https://img.shields.io/badge/Fine--tune-Full--FT-purple?style=for-the-badge" />
+  <img src="https://img.shields.io/badge/Context-8192-orange?style=for-the-badge" />
+  <img src="https://img.shields.io/badge/License-Apache--2.0-green?style=for-the-badge" />
+</p>
+
+<p align="center">
+  <b>A full-parameter Turkish reasoning model fine-tuned to generate structured <code>&lt;think&gt;...&lt;/think&gt;</code> traces.</b>
+</p>
+
+<p align="center">
+  <img src="./assets/model_specs_strip.png" alt="Model specs strip" width="100%">
+</p>
+
+# Kumru-2B-Thinking-v2.0
+
+**Kumru-2B-Thinking-v2.0** is a Turkish reasoning-oriented language model built on top of **[`vngrs-ai/Kumru-2B`](https://huggingface.co/vngrs-ai/Kumru-2B)**.
+
+It is the **full-parameter fine-tune** successor to **[`Kumru-2B-Thinking-v1.0`](https://huggingface.co/dogukanvzr/Kumru-2B-Thinking-v1.0)**. While v1.0 used LoRA r=128, v2.0 updates the full 2.375B-parameter backbone and keeps the same `<think>...</think>` reasoning interface.
+
+This release is aimed at Turkish reasoning experiments, educational demos, benchmark studies, and further alignment / fine-tuning research. It is not meant to be a high-stakes expert system.
+
+---
+
+## Table of contents
+
+- [Model summary](#model-summary)
+- [Overview](#overview)
+- [Türkçe açıklama](#türkçe-açıklama)
+- [Quick start](#quick-start)
+- [Recommended generation settings](#recommended-generation-settings)
+- [Reasoning format](#reasoning-format)
+- [Benchmarks](#benchmarks)
+- [v2.0 vs v1.0](#v20-vs-v10)
+- [Training recipe](#training-recipe)
+- [Training data](#training-data)
+- [Intended use](#intended-use)
+- [Limitations](#limitations)
+- [Citation](#citation)
+- [Acknowledgments](#acknowledgments)
+- [License](#license)
+
+---
+
+## Model summary
+
+| Field | Value |
+|---|---|
+| Model name | `dogukanvzr/Kumru-2B-Thinking-v2.0` |
+| Base model | `vngrs-ai/Kumru-2B` |
+| Approx. size | 2.375B parameters |
+| Fine-tuning method | Full-parameter fine-tuning |
+| Main language | Turkish |
+| Context length | 8192 tokens |
+| Main output style | `<think>...</think>` + final answer |
+| License | Apache-2.0 |
+| Predecessor | `Kumru-2B-Thinking-v1.0` (LoRA) |
+
+---
+
+## Overview
+
+Kumru-2B-Thinking-v2.0 is the **full fine-tune evolution** of Kumru-2B-Thinking-v1.0.
+
+The goal of the model is to produce **structured and inspectable Turkish reasoning-style outputs** while preserving a practical chat interface. Compared with v1.0, this release expands the adaptation surface from LoRA layers to the entire backbone, which leads to richer reasoning traces and better performance on several commonsense and reading-comprehension style tasks.
+
+When to choose v2.0:
+
+- You want richer Turkish reasoning traces.
+- You care more about HellaSwag-TR, Belebele-TR, ARC-TR, and TruthfulQA-TR style performance.
+- You want a compact full-FT research release rather than a LoRA adaptation.
+
+When v1.0 may still be preferable:
+
+- You need stricter multiple-choice format adherence.
+- You care more about MMLU-TR and Winogrande-TR style tasks.
+- You want the narrower behavior of the LoRA-tuned version.
+
+---
+
+## Türkçe açıklama
+
+**Kumru-2B-Thinking-v2.0**, Kumru-2B-Thinking-v1.0'ın **full-parameter fine-tune** devam sürümüdür.
+
+v1.0 LoRA r=128 ile daha dar bir güncelleme yüzeyi kullanırken, v2.0 modelin tüm 2.375B parametresini günceller. Böylece aynı `<think>...</think>` arayüzünü korurken daha zengin reasoning izleri ve bazı görevlerde daha güçlü performans sunar.
+
+Özellikle HellaSwag-TR, Belebele-TR, ARC-TR ve TruthfulQA-TR gibi görevlerde v2.0 daha güçlüdür. Buna karşılık, MMLU-TR ve Winogrande-TR gibi daha format-hassas çoktan seçmeli görevlerde v1.0 bazı durumlarda daha iyi kalabilir.
+
+---
+
+## Quick start
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_id = "dogukanvzr/Kumru-2B-Thinking-v2.0"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+
+# Recommended for faster generation during inference
+model.config.use_cache = True
+
+messages = [
+    {
+        "role": "user",
+        "content": (
+            "Bir trende 96 yolcu var. İlk durakta yolcuların dörtte biri iniyor "
+            "ve 18 yolcu biniyor. İkinci durakta mevcut yolcuların üçte biri iniyor. "
+            "Son durumda trende kaç yolcu kalır?
+
+"
+            "Adım adım düşün. Sonucu son satırda 'FINAL_ANSWER: <sayı>' formatında yaz."
+        ),
+    }
+]
+
+inputs = tokenizer.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_tensors="pt",
+).to(model.device)
+
+outputs = model.generate(
+    inputs,
+    max_new_tokens=1024,
+    temperature=0.6,
+    do_sample=True,
+    top_p=0.9,
+    repetition_penalty=1.05,
+    eos_token_id=tokenizer.eos_token_id,
+    pad_token_id=tokenizer.pad_token_id,
+)
+
+generated = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=False)
+print(generated)
+```
+
+---
+
+## Recommended generation settings
+
+For reasoning-style interactive use:
+
+```python
+generation_config = {
+    "max_new_tokens": 1024,
+    "temperature": 0.6,
+    "top_p": 0.9,
+    "do_sample": True,
+    "repetition_penalty": 1.05,
+}
+```
+
+For deterministic benchmark-style evaluation:
+
+```python
+generation_config = {
+    "max_new_tokens": 1024,
+    "do_sample": False,
+}
+```
+
+Practical notes:
+
+- Use Turkish prompts for best results.
+- Put the full instruction in the `user` message; avoid relying on a system prompt.
+- Use `max_new_tokens=1024` when you want the reasoning trace to complete.
+- If you are compute-constrained, `768` can work, but expect some reasoning traces to be cut.
+- If generation is slow, make sure `model.config.use_cache = True` is enabled.
+
+---
+
+## Reasoning format
+
+The model was trained to produce a visible reasoning block followed by the final answer:
+
+```text
+<think>
+Burada model problemi Türkçe olarak adım adım değerlendirir.
+</think>
+
+Burada final cevap yer alır.
+```
+
+For tasks where you need reliable extraction, explicitly request a final answer marker.
+
+Example for multiple-choice tasks:
+
+```text
+Aşağıdaki soruyu çöz. Seçenekleri dikkatlice değerlendir.
+Cevabı düşünüp son satırda yalnızca şu formatta yaz:
+
+FINAL_ANSWER: <A/B/C/D>
+```
+
+Example for numerical tasks:
+
+```text
+Problemi adım adım çöz.
+Son satırda yalnızca şu formatta yaz:
+
+FINAL_ANSWER: <sayı>
+```
+
+### Splitting reasoning and final answer
+
+```python
+import re
+
+text = generated
+match = re.search(r"<think>(.*?)</think>\s*(.*?)(?:<\|eot_id\|>|$)", text, re.DOTALL)
+reasoning, answer = (match.group(1).strip(), match.group(2).strip()) if match else ("", text)
+```
+
+---
+
+## Benchmarks
+
+Setup:
+
+```text
+7 Turkish tasks × 250 samples × 3 seeds × 2 decoding strategies × max_new_tokens=1024
+```
+
+Metric:
+
+```text
+Accuracy (exact-match on final answer letter or final number)
+```
+
+Why 1024 tokens?
+
+> The original v1.0 release reported numbers at 768 tokens, but a substantial portion of reasoning traces was being cut mid-thought. Both v1.0 and v2.0 were evaluated at 1024 tokens here for a fairer comparison.
+
+<p align="center">
+  <img src="./assets/benchmark_average.png" alt="Average benchmark accuracy comparison" width="92%">
+</p>
+
+<p align="center">
+  <img src="./assets/benchmark_tasks_sampling.png" alt="Per-task benchmark results with sampling" width="92%">
+</p>
+
+### Greedy decoding
+
+<p align="center">
+  <img src="./assets/benchmark_tasks_greedy.png" alt="Per-task benchmark results with greedy decoding" width="92%">
+</p>
+
+| Model | MMLU-TR | ARC-TR | TruthQA-TR | Wino-TR | HSwag-TR | Belebele-TR | GSM8K-TR | Avg |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| Kumru-2B-Instruct | 6.7 | 5.1 | 22.0 | 7.6 | 4.3 | 6.7 | 2.3 | 7.8 |
+| Kara-Kumru-v1.0 | 9.9 | 7.2 | 24.7 | 25.7 | 7.1 | 3.5 | 2.1 | 11.5 |
+| Kumru-2B-Thinking-v1.0 | 19.3 | 17.1 | 18.1 | 13.3 | 17.9 | 14.8 | 1.3 | 14.5 |
+| **Kumru-2B-Thinking-v2.0** | **14.3** | **19.3** | **20.9** | **10.8** | **22.3** | **18.4** | **1.7** | **15.4** |
+
+### Sampling, temperature 0.7
+
+| Model | MMLU-TR | ARC-TR | TruthQA-TR | Wino-TR | HSwag-TR | Belebele-TR | GSM8K-TR | Avg |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| Kumru-2B-Instruct | 8.6 | 4.7 | 23.9 | 18.8 | 6.8 | 8.0 | 1.1 | 10.3 |
+| Kara-Kumru-v1.0 | 10.7 | 8.9 | 27.9 | 28.1 | 7.9 | 6.5 | 2.9 | 13.3 |
+| Kumru-2B-Thinking-v1.0 | 18.4 | 19.7 | 20.9 | 19.3 | 20.7 | 16.5 | 1.9 | 16.8 |
+| **Kumru-2B-Thinking-v2.0** | **17.3** | **21.1** | **22.0** | **16.1** | **18.9** | **19.1** | **1.3** | **16.5** |
+
+### Interpretation
+
+v2.0 is **not** a strict replacement for v1.0. It is a broader full-parameter adaptation that wins on several tasks tied to commonsense and reading-comprehension style reasoning, while v1.0 remains stronger on some format-strict multiple-choice tasks.
+
+---
+
+## v2.0 vs v1.0
+
+<p align="center">
+  <img src="./assets/v2_vs_v1_greedy_delta.png" alt="v2 vs v1 greedy delta" width="86%">
+</p>
+
+Key takeaways:
+
+- **v2.0 wins** on HellaSwag-TR, Belebele-TR, TruthfulQA-TR, ARC-TR, and slightly on GSM8K-TR.
+- **v1.0 wins** on MMLU-TR and Winogrande-TR.
+- **Average greedy score:** 15.4% for v2.0 vs 14.5% for v1.0.
+- **Average sampling score:** 16.5% for v2.0 vs 16.8% for v1.0.
+
+---
+
+## Training recipe
+
+<p align="center">
+  <img src="./assets/training_pipeline.png" alt="Training pipeline" width="96%">
+</p>
+
+| Component | Value |
+|---|---|
+| Base model | `vngrs-ai/Kumru-2B` |
+| Fine-tuning method | Full-parameter fine-tuning |
+| Trainable parameters | All 2.375B parameters |
+| Optimizer | `torchao.optim.AdamW8bit(bf16_stochastic_round=True)` |
+| Learning rate | `2e-5`, cosine, warmup 5% |
+| Effective batch size | 16 |
+| Epochs | 1 |
+| Max sequence length | 8192 |
+| Precision | bfloat16 + stochastic rounding |
+| Loss masking | Assistant-only |
+| Hardware | 1 × RTX 5090 (32 GB VRAM) |
+| Runtime | 2 h 19 m 44 s |
+| Final train loss | 1.609 |
+| Best eval loss | 1.6505 |
+| Weight delta verification | 165 / 165 tracked parameter groups changed |
+
+Key engineering decisions:
+
+1. **No Unsloth full-FT path** was used for the final release.
+2. **HF Trainer** was used directly for the full fine-tune.
+3. **Stochastic rounding** was used to better preserve small RMSNorm updates.
+4. A **post-training weight-change verification** step confirmed that the intended parameter groups were actually updated.
+
+---
+
+## Training data
+
+The model was trained on the same **25.8K curated Turkish reasoning traces** used for v1.0.
+
+Dataset:
+
+```text
+dogukanvzr/kumru-v5-dataset
+```
+
+The training format encourages the model to produce:
+
+1. A Turkish reasoning section inside `<think>...</think>`
+2. A concise final answer after the reasoning block
+
+---
+
+## Intended use
+
+<p align="center">
+  <img src="./assets/positioning_cards.png" alt="Model positioning cards" width="96%">
+</p>
+
+This model is suitable for:
+
+- Turkish reasoning experiments
+- Educational assistant prototypes
+- Benchmark and evaluation studies
+- Chain-of-thought style output experiments
+- Further full-FT / alignment research
+- Small-model reasoning behavior analysis
+
+### Out-of-scope use
+
+This model should not be used as the sole source of truth for:
+
+- Medical, legal, financial, or safety-critical decisions
+- Fully automated expert systems
+- High-stakes grading or evaluation
+- Sensitive personal data processing
+- Reliable math solving without external verification
+- Production systems that require strong factual reliability
+
+---
+
+## Limitations
+
+### Mathematical reasoning
+
+The model can produce step-by-step reasoning, but numerical accuracy is not guaranteed. Mathematical answers should be checked with a calculator, symbolic tool, or a stronger verifier model.
+
+### Factual accuracy
+
+The model may hallucinate facts, dates, names, or explanations. It does not have live internet access and should not be treated as a current knowledge system.
+
+### Long reasoning traces
+
+The model may overthink simple questions or produce longer reasoning than necessary. If you need shorter answers, reduce `max_new_tokens`.
+
+### English and code-mixed prompts
+
+The model is Turkish-first. It may respond to English or mixed prompts, but quality is expected to be better on Turkish prompts.
+
+### Formatting
+
+The model is trained to use `<think>...</think>`, but downstream applications should still validate the output format if strict parsing is required.
+
+### Safety
+
+The model has not been extensively safety-aligned beyond the behavior inherited from the base model and the fine-tuning data. Additional safety filtering is recommended for public-facing applications.
+
+---
+
+## Practical recommendations
+
+For best results:
+
+1. Use Turkish prompts.
+2. Put all task instructions in the `user` turn.
+3. Ask for a strict final answer format when evaluating.
+4. Use `max_new_tokens=1024` when the reasoning trace matters.
+5. Verify math and factual claims externally.
+6. Parse the final answer separately from the `<think>` block.
+7. Prefer v2.0 for richer reasoning; prefer v1.0 for stricter MC formatting.
+
+---
+
+## Citation
+
+```bibtex
+@misc{veziroglu2026kumru2bthinkingv2,
+  title        = {Kumru-2B-Thinking-v2.0: A Turkish Full-Fine-Tuned Reasoning Language Model},
+  author       = {Dogukan Veziroglu},
+  year         = {2026},
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/dogukanvzr/Kumru-2B-Thinking-v2.0}}
+}
+```
+
+---
+
+## Acknowledgments
+
+This model is based on [`vngrs-ai/Kumru-2B`](https://huggingface.co/vngrs-ai/Kumru-2B).
+
+Thanks to the VNGRS-AI team for releasing Turkish language models that make further Turkish NLP and reasoning experiments possible.
+
+---
+
+## License
+
+This model is released under the **Apache-2.0** license.
+
+Please also check the license and usage terms of the base model and any datasets used in downstream applications.
+
+---
+
+## Contact
+
+For questions, issues, or feedback, please use the Hugging Face community tab or contact the model author through the Hugging Face profile:
+
+```text
+dogukanvzr
+```
--- a/assets/benchmark_average.png
+++ b/assets/benchmark_average.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d4eba1f19ccd550bb1b45073a437267c8fbcb4fae368ecdee8377507ca2d4387
+size 113433
--- a/assets/benchmark_average.svg
+++ b/assets/benchmark_average.svg
--- a/assets/benchmark_tasks_greedy.png
+++ b/assets/benchmark_tasks_greedy.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98dc33889fd3ed97e5a516e0d31b73ff8390532119a4252a60a44952757db5e6
+size 158676
--- a/assets/benchmark_tasks_greedy.svg
+++ b/assets/benchmark_tasks_greedy.svg
--- a/assets/benchmark_tasks_sampling.png
+++ b/assets/benchmark_tasks_sampling.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a74733c53248306981b384b3b01b82d10a852bde9b072a1cc15a9a74c7e243a3
+size 166043
--- a/assets/benchmark_tasks_sampling.svg
+++ b/assets/benchmark_tasks_sampling.svg
--- a/assets/kumru_header.png
+++ b/assets/kumru_header.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:53afb0538b40fab1a0187ca968b7a55d91747b7eae10eb9c08ed9f7fa38f7818
+size 1196148
--- a/assets/model_specs_strip.png
+++ b/assets/model_specs_strip.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:658cc382e329be7e1fc8b6655d2332a6eaf1bbb99a4d8486c8a03349c7536784
+size 1040947
--- a/assets/positioning_cards.png
+++ b/assets/positioning_cards.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65ffc0f44b87c24b27037bbefd36636360b1c2a1e90526d77f1d95b659883726
+size 1283949
--- a/assets/training_pipeline.png
+++ b/assets/training_pipeline.png
--- a/assets/v2_vs_v1_greedy_delta.png
+++ b/assets/v2_vs_v1_greedy_delta.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a263269dff0ddce715c88347ad3cbf93af55d4a9248117c6e2e2d2fb5ff2619
+size 100204
--- a/assets/v2_vs_v1_greedy_delta.svg
+++ b/assets/v2_vs_v1_greedy_delta.svg
--- a/chat_template.jinja
+++ b/chat_template.jinja
@@ -0,0 +1,13 @@
+{% set loop_messages = messages %}{% for message in loop_messages %}{% set header = '<|start_header_id|>' + message['role'] + '<|end_header_id|>
+
+' %}{% if loop.index0 == 0 %}{{ bos_token }}{% endif %}{{ header }}{% if message['role'] == 'assistant' %}{% generation %}{% if message.get('reasoning_content') %}{{ '<think>
+' + message['reasoning_content'] | trim + '
+</think>
+
+' + message['content'] | trim + '<|eot_id|>' }}{% else %}{{ '<think>
+
+</think>
+
+' + message['content'] | trim + '<|eot_id|>' }}{% endif %}{% endgeneration %}{% else %}{{ message['content'] | trim + '<|eot_id|>' }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>
+
+' }}{% endif %}
--- a/config.json
+++ b/config.json
@@ -0,0 +1,30 @@
+{
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 2,
+  "dtype": "bfloat16",
+  "eos_token_id": 261,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 3072,
+  "initializer_range": 0.02,
+  "intermediate_size": 10752,
+  "max_position_embeddings": 8192,
+  "model_type": "mistral",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 18,
+  "num_key_value_heads": 4,
+  "pad_token_id": 0,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 500000,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": false,
+  "vocab_size": 50176
+}
--- a/generation_config.json
+++ b/generation_config.json
@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 2,
+  "eos_token_id": 261,
+  "pad_token_id": 0,
+  "transformers_version": "5.5.0"
+}
--- a/model.safetensors
+++ b/model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d32169bab08fa8a5e9b4c39551e3a8c0f50165f95b960789c5805a40a96582bd
+size 4750295696
--- a/tokenizer.json
+++ b/tokenizer.json
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
@@ -0,0 +1,11 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<BOS>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "is_local": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<PAD>",
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<UNK>"
+}