初始化项目，由ModelHub XC社区提供模型
Model: kizzlah/granite-4.2-3b Source: Original Platform
This commit is contained in:
ModelHub XC
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
892
README.md
Normal file
892
README.md
Normal file
@@ -0,0 +1,892 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
language: [en, de, es, fr, ja, pt, ar, cs, it, ko, nl, zh]
|
||||
tags: [granite, granite-4.2, reasoning, thinking, tool-calling, ibm]
|
||||
base_model: [ibm-granite/granite-4.1-3b-base]
|
||||
---
|
||||
|
||||
# Granite-4.2-3B
|
||||
|
||||
[](https://huggingface.co/collections/ibm-granite/granite-42-language-models)
|
||||
[](https://huggingface.co/blog/ibm-granite/granite-4-2)
|
||||
[](https://github.com/ibm-granite/granite-4.2-language-models)
|
||||
[](https://www.apache.org/licenses/LICENSE-2.0)
|
||||
|
||||
---
|
||||
|
||||
## Model Summary
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **Developers** | Granite Team, IBM |
|
||||
| **Model Type** | Decoder-only Dense Transformer (Reasoning) |
|
||||
| **Architecture** | GraniteForCausalLM |
|
||||
| **Base Model** | [Granite-4.1-3B-Base](https://huggingface.co/ibm-granite/granite-4.1-3b-base) |
|
||||
| **Parameters** | 3B |
|
||||
| **Context Length** | Natively Supports 128K (Long-context extension to 512K) |
|
||||
| **Precision** | bfloat16 |
|
||||
| **Tested Languages** | English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, Chinese (other languages may work but have not been fully tested) |
|
||||
| **Reasoning Mode** | Built-in `<think>...</think>` chain-of-thought |
|
||||
| **Best For** | Reasoning, Code Generation, Tool Calling, Agentic Workflows, Multilingual Dialog |
|
||||
| **License** | [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) |
|
||||
| **HF Collection** | [Granite 4.2 Language Models](https://huggingface.co/collections/ibm-granite/granite-42-language-models) |
|
||||
| **Release Date** | August 25, 2026 |
|
||||
|
||||
---
|
||||
|
||||
## Model Overview
|
||||
|
||||
### What is IBM Granite?
|
||||
|
||||
Granite is a family of open-source large language models developed by IBM, designed for enterprise and research use. Granite models are built to be versatile, safe, and efficient — covering a range of sizes and capabilities from compact edge-deployable models to large-scale reasoning systems. All Granite models are released under the Apache 2.0 license, enabling unrestricted commercial and academic use.
|
||||
|
||||
The Granite 4.2 generation introduces native reasoning (thinking) capabilities, allowing models to perform step-by-step chain-of-thought reasoning before producing final answers. This significantly improves performance on complex math, coding, multi-step logic, and agentic tool-calling tasks.
|
||||
|
||||
### Description
|
||||
|
||||
**Granite-4.2-3B** is the compact reasoning model in the Granite 4.2 family. Despite its small parameter count, it delivers strong performance on reasoning-intensive tasks by leveraging built-in `<think>...</think>` chain-of-thought. It supports flexible thinking modes — full thinking (default), non-thinking, and low-effort — allowing users to balance depth vs. latency on a per-query basis.
|
||||
|
||||
Key capabilities:
|
||||
- **Built-in Reasoning:** Native chain-of-thought that significantly improves performance on math, coding, and complex multi-step problems.
|
||||
- **Flexible Thinking Modes:** Seamlessly switch between full thinking, non-thinking, and low-effort modes within a single model.
|
||||
- **Reasoning-Augmented Tool Calling:** The model reasons about which tools to invoke and why, producing more accurate function calls.
|
||||
- **512K Context Window:** Supports long documents, multi-turn conversations, and complex agentic workflows.
|
||||
- **Apache 2.0 Licensed:** Fully open for commercial and research use.
|
||||
|
||||
---
|
||||
|
||||
## Model Design
|
||||
|
||||
Granite-4.2-3B is built on a decoder-only dense transformer architecture with the following core components:
|
||||
|
||||
* **Attention:** Grouped Query Attention (GQA) with 40 attention heads and 8 KV heads
|
||||
* **Position Embedding:** Rotary Position Embedding (RoPE) with θ = 10,000,000
|
||||
* **Feed-Forward:** MLP with SwiGLU activation (hidden size 8192)
|
||||
* **Normalization:** RMSNorm (ε = 1e-5)
|
||||
* **Embeddings:** Separate input/output embeddings (not tied)
|
||||
* **Precision:** bfloat16
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th style="text-align:left; background-color: #001d6c; color: white;">Component</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">3B Dense</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">8B Dense</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">30B Dense</th>
|
||||
</tr></thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td style="text-align:left;">Embedding size</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">2560</td>
|
||||
<td style="text-align:center;">4096</td>
|
||||
<td style="text-align:center;">4096</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">Number of layers</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">40</td>
|
||||
<td style="text-align:center;">40</td>
|
||||
<td style="text-align:center;">64</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">Attention head size</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">64</td>
|
||||
<td style="text-align:center;">128</td>
|
||||
<td style="text-align:center;">128</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">Number of attention heads</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">40</td>
|
||||
<td style="text-align:center;">32</td>
|
||||
<td style="text-align:center;">32</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">Number of KV heads</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">8</td>
|
||||
<td style="text-align:center;">8</td>
|
||||
<td style="text-align:center;">8</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">MLP hidden size</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">8192</td>
|
||||
<td style="text-align:center;">12800</td>
|
||||
<td style="text-align:center;">32768</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">MLP activation</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">SwiGLU</td>
|
||||
<td style="text-align:center;">SwiGLU</td>
|
||||
<td style="text-align:center;">SwiGLU</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">Sequence length</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">131072</td>
|
||||
<td style="text-align:center;">131072</td>
|
||||
<td style="text-align:center;">131072</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">Position embedding</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">RoPE</td>
|
||||
<td style="text-align:center;">RoPE</td>
|
||||
<td style="text-align:center;">RoPE</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;"># Parameters</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">3B</td>
|
||||
<td style="text-align:center;">8B</td>
|
||||
<td style="text-align:center;">30B</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
---
|
||||
|
||||
## Training Methodology
|
||||
|
||||
Granite-4.2-3B is post-trained from [Granite-4.1-3B-Base](https://huggingface.co/ibm-granite/granite-4.1-3b-base) through a rigorous multi-stage pipeline that progressively unlocks reasoning, tool use, and instruction-following capabilities. A full listing of training datasets is available in the [Granite 4.2 GitHub repository](https://github.com/ibm-granite/granite-4.2-language-models). The training pipeline consists of three stages:
|
||||
|
||||
### Stage 1: Pre-Training
|
||||
|
||||
<!-- TODO: Add pre-training details here -->
|
||||
|
||||
Granite-4.2-3B builds on [Granite-4.1-3B-Base](https://huggingface.co/ibm-granite/granite-4.1-3b-base), which was pre-trained on a large-scale English as well as multilingual corpus. For full pre-training details (data composition, training recipe, and infrastructure), refer to our [Granite 4.1 Technical Blog](https://huggingface.co/blog/ibm-granite/granite-4-1).
|
||||
|
||||
### Stage 2: Supervised Fine-Tuning
|
||||
|
||||
<!-- TODO: Add SFT details here -->
|
||||
|
||||
The SFT stage draws on instruction-following, chain-of-thought, and reasoning data to cultivate the model's reasoning and thinking abilities. For all the three, 3B, 8B and 30B models, the training corpus comprises four sources: (1) publicly available datasets under permissive licenses, (2) internally generated synthetic data targeting reasoning, tool calling, and chain-of-thought capabilities, (3) agentic traces collected across a diverse range of tasks, and (4) a curated selection of human-authored data. Hyperparameters were tuned before training was scaled to all three model sizes. For the 30B model, we conducted a second SFT phase, in which the agentic data was up-sampled while a smaller share of general replay data was retained. This phase trained for a single epoch, starting from a lower learning rate than Phase 1.
|
||||
|
||||
<!--...Please mention details on multi-phase SFT if necessary... This should be brief we will have more detailed version in technical blog. -->
|
||||
|
||||
### Stage 3: Reinforcement Learning
|
||||
|
||||
<!-- TODO: Add RL details here -->
|
||||
|
||||
The final stage of training applies multi-phase, multi-environment reinforcement learning using Group Relative Policy Optimization (GRPO). Training spans a broad mix of environments including math, code, science, instruction following, tool use, general chat and structured output. Most environments provide verifiable rewards, while open-ended prompts are scored by a generative reward model. Training runs asynchronously: generation and policy updates occupy separate GPU pools rather than proceeding in lockstep, and weights are refreshed in flight.
|
||||
|
||||
After the reward-driven phases, a preference-alignment (RLHF) phase tunes helpfulness, conversational quality, and safety. Reinforcement learning is carried out with NeMo RL, and the RL environments run on NeMo Gym.
|
||||
|
||||
<!-- After the reward-driven phases, a preference-alignment (RLHF) phase tunes helpfulness, conversational quality, and safety, and a closing phase shortens the model's chain-of-thought for improved token efficiency. Reinforcement learning is carried out with NeMo RL, and the RL environments run on NeMo Gym. -->
|
||||
|
||||
<!--
|
||||
The final stage of training applies multi-phase, multi-environment reinforcement learning using Group Relative Policy Optimization (GRPO). Training begins with several rounds of reinforcement learning across a broad mix of environments including math, code, science, instruction following, tool use, and structured output, followed by targeted phases that further sharpen key capabilities such as instruction following and coding. Most environments provide verifiable rewards, while open-ended prompts are scored by a generative reward model. Training runs asynchronously: generation and policy updates occupy separate GPU pools rather than proceeding in lockstep, and weights are refreshed in flight.
|
||||
|
||||
Several dedicated agentic RL phases then target multi-step tool use and long-horizon workflows across three settings: software engineering, where the model edits code and runs tests to resolve real repository issues in sandboxed containers; terminal use, where it operates a command-line shell to accomplish multi-step objectives; and web search, where it issues successive queries and reads results to gather evidence. Rollouts are complete agent trajectories that can span up to 200 tool-calling turns, so the model learns to plan, invoke tools, observe outcomes, and iterate toward a goal.
|
||||
|
||||
Finally, a preference-alignment (RLHF) phase tunes helpfulness, conversational quality, and safety. Reinforcement learning is carried out with NeMo RL, and the RL environments run on NeMo Gym.
|
||||
-->
|
||||
|
||||
<!-- (Initial placeholder description) The RL stage further aligns the model for instruction following and safety. An additional agentic RL stage specifically targets multi-step tool use and agentic workflows, improving the model's ability to reason about when and how to invoke tools. ...Mention some details on conventional RL and Agentic RL...This should be brief we will have more detailed version in technical blog. -->
|
||||
|
||||
---
|
||||
|
||||
**Infrastructure:**
|
||||
We trained the Granite 4.2 Language Models utilizing an NVIDIA GB200 NVL72 cluster hosted in CoreWeave. Intra-rack communication occurs via the 72-GPU NVLink domain, and a non-blocking, full Fat-Tree NDR 400 Gb/s InfiniBand network provides inter-rack communication.
|
||||
|
||||
For further details on the post-training methodology, please refer to the [Granite-4.2 Technical Blog](https://huggingface.co/blog/ibm-granite/granite-4-2).
|
||||
|
||||
---
|
||||
|
||||
## Evaluation Results
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th style="text-align:left; background-color: #001d6c; color: white;">Task</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">3B Dense</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">8B Dense</th>
|
||||
<th style="text-align:center; background-color: #001d6c; color: white;">30B Dense</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td colspan="4" style="text-align:center; font-style:italic;">Agentic (Coding)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SWE Bench Multilingual</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">NA</td>
|
||||
<td style="text-align:center;">30.78</td><td style="text-align:center;">41.89</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SWE Bench Pro</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">NA</td>
|
||||
<td style="text-align:center;">19.11</td><td style="text-align:center;">33.29</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SWE Bench Verified</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">NA</td>
|
||||
<td style="text-align:center;">47.67</td><td style="text-align:center;">57</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Terminal-Bench 2.1</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">NA</td>
|
||||
<td style="text-align:center;">20.56</td><td style="text-align:center;">29.24</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td colspan="4" style="text-align:center; font-style:italic;">Agentic (General)</td>
|
||||
</tr>
|
||||
<!-- <tr>
|
||||
<td>τ³-bench</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">50.99</td>
|
||||
<td style="text-align:center;">66.34</td><td style="text-align:center;">68.05</td>
|
||||
</tr> -->
|
||||
<tr>
|
||||
<td>τ³-bench (AVG)</td>
|
||||
<td style="text-align:center;background-color: #DAE8FF;">45.78</td>
|
||||
<td style="text-align:center;">58.06</td><td style="text-align:center;">62.00</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>BFCL (v4)</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">52.41</td>
|
||||
<td style="text-align:center;">52.39</td><td style="text-align:center;">61.39</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ProfBench</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">32.10</td>
|
||||
<td style="text-align:center;">41.20</td><td style="text-align:center;">42.90</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>BirdBench</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">NA</td>
|
||||
<td style="text-align:center;">41.07</td><td style="text-align:center;">41.85</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GDPval</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">NA</td>
|
||||
<td style="text-align:center;">1189</td><td style="text-align:center;">1225</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td colspan="4" style="text-align:center; font-style:italic;">Reasoning</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>AIME25</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">78.33</td>
|
||||
<td style="text-align:center;">86.67</td><td style="text-align:center;">89.17</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>HMMT Feb25</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">66.67</td>
|
||||
<td style="text-align:center;">78.33</td><td style="text-align:center;">89.17</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GPQA</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">54.80</td>
|
||||
<td style="text-align:center;">64.14</td><td style="text-align:center;">66.41</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>LiveCodeBench v6</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">69.71</td>
|
||||
<td style="text-align:center;">73.24</td><td style="text-align:center;">75.77</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SciCode</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">24.11</td>
|
||||
<td style="text-align:center;">36.09</td><td style="text-align:center;">38.76</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td colspan="4" style="text-align:center; font-style:italic;">Chat & Instruction Following</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMLU-Pro</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">67.84</td>
|
||||
<td style="text-align:center;">74.04</td><td style="text-align:center;">77.60</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMLU-ProX lite (IBM)</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">27.78</td>
|
||||
<td style="text-align:center;">61.06</td><td style="text-align:center;">66.64</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Arena-Hard-V2</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">34.96</td>
|
||||
<td style="text-align:center;">65.19</td><td style="text-align:center;">67.93</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>IFBench (prompt)</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">74.33</td>
|
||||
<td style="text-align:center;">79.33</td><td style="text-align:center;">77.17</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td colspan="4" style="text-align:center; font-style:italic;">Long Context</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RULER 64K</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">67.52</td>
|
||||
<td style="text-align:center;">80.99</td><td style="text-align:center;">89.96</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RULER 128K</td>
|
||||
<td style="text-align:center; background-color: #DAE8FF;">55.30</td>
|
||||
<td style="text-align:center;">71.41</td><td style="text-align:center;">81.38</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
Evaluations are run with an evaluation framework based on NeMo [Evaluator SDK](https://github.com/NVIDIA-NeMo/Evaluator).
|
||||
|
||||
---
|
||||
|
||||
## Inference
|
||||
|
||||
### Generation Parameters
|
||||
|
||||
> **Important:** Use `temperature=1.0` and `top_p=0.95` across **all tasks and serving backends**, including general chat, reasoning, and tool calling.
|
||||
|
||||
| Parameter | Value | Notes |
|
||||
|:----------|:------|:------|
|
||||
| `temperature` | `1.0` | Required for all modes |
|
||||
| `top_p` | `0.95` | Nucleus sampling threshold |
|
||||
| `max_new_tokens` | `8192` | Thinking mode (increase for complex reasoning) |
|
||||
| `max_new_tokens` | `2048` | Non-thinking mode |
|
||||
| `do_sample` | `True` | Required when temperature > 0 |
|
||||
|
||||
### Thinking Modes
|
||||
|
||||
| Mode | Template Parameters | Behavior |
|
||||
|:-----|:-------------------|:---------|
|
||||
| **Thinking** (default) | `enable_thinking=True` | Full chain-of-thought reasoning inside `<think>...</think>` |
|
||||
| **Non-thinking** | `enable_thinking=False` | Direct answer with no reasoning overhead |
|
||||
| **Low-effort** | `enable_thinking=True, low_effort=True` | Brief reasoning for simpler queries |
|
||||
|
||||
### How It Works
|
||||
|
||||
- **Thinking enabled** — The generation prompt ends with `<|im_start|>assistant\n<think>\n`, causing the model to reason until it emits `</think>`, then produce the final answer.
|
||||
- **Thinking disabled** — The prompt ends with `<|im_start|>assistant\n<think></think>`, bypassing reasoning entirely.
|
||||
- **Low-effort** — Appends `{reasoning effort: low}` to the user message, signaling shorter reasoning chains.
|
||||
|
||||
### History Truncation
|
||||
|
||||
In multi-turn conversations, thinking content from previous assistant turns is automatically stripped (`truncate_history_thinking=True` by default) to conserve context window space. Only the current generation produces full reasoning. Set `truncate_history_thinking=False` to preserve full reasoning history.
|
||||
|
||||
---
|
||||
|
||||
## Serving with vLLM
|
||||
|
||||
Granite-4.2-3B is optimized for deployment with [vLLM](https://github.com/vllm-project/vllm).
|
||||
|
||||
> **Reasoning parser:** Use the custom `granite_thinking_parser` included in this repository (requires vLLM v0.20+). The model also works with the built-in `nemotron_v3` parser, but `granite_thinking_parser` provides better formatting of reasoning output. Native support for granite_thinking_parser will be added to vLLM and SGLang very soon.
|
||||
> **Tool calling parser:** Use `qwen3_coder`.
|
||||
|
||||
### Starting the Server
|
||||
|
||||
```bash
|
||||
vllm serve ibm-granite/granite-4.2-3b \
|
||||
--served-model-name granite-4.2-3b \
|
||||
--dtype bfloat16 \
|
||||
--max-model-len 131072 \
|
||||
--reasoning-parser granite_thinking_parser \
|
||||
--reasoning-parser-plugin ./granite_thinking_parser.py \
|
||||
--tool-call-parser qwen3_coder \
|
||||
--enable-auto-tool-choice
|
||||
```
|
||||
|
||||
### OpenAI-Compatible API Usage
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model="granite-4.2-3b",
|
||||
messages=[{"role": "user", "content": "Explain the Riemann hypothesis in simple terms."}],
|
||||
temperature=1.0,
|
||||
top_p=0.95,
|
||||
max_tokens=8192,
|
||||
)
|
||||
|
||||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
### Tool Calling via vLLM
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
|
||||
|
||||
tools = [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_current_weather",
|
||||
"description": "Get the current weather for a specified city.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"city": {"type": "string", "description": "Name of the city"}
|
||||
},
|
||||
"required": ["city"]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model="granite-4.2-3b",
|
||||
messages=[{"role": "user", "content": "What's the weather like in Boston right now?"}],
|
||||
tools=tools,
|
||||
temperature=1.0,
|
||||
top_p=0.95,
|
||||
max_tokens=4096,
|
||||
)
|
||||
|
||||
print(response.choices[0].message.tool_calls)
|
||||
```
|
||||
|
||||
## Serving with SGLang
|
||||
|
||||
Granite-4.2-3B can also be served with [SGLang](https://github.com/sgl-project/sglang) (v0.5.18+) for high-throughput inference.
|
||||
|
||||
> **Reasoning parser:** Use `--reasoning-parser auto`, which resolves to the built-in `nemotron_3` parser for this checkpoint. It separates the thinking trace into `reasoning_content` and the final answer into `content`, and it handles all three thinking modes (`enable_thinking=True/False`, `low_effort=True`) described in [Thinking Modes](#thinking-modes).
|
||||
> **Tool calling parser:** Use `--tool-call-parser auto`, which resolves to `qwen3_coder` for this checkpoint.
|
||||
|
||||
### Starting the Server
|
||||
|
||||
```bash
|
||||
python3 -m sglang.launch_server \
|
||||
--model-path ibm-granite/granite-4.2-3b \
|
||||
--dtype bfloat16 \
|
||||
--context-length 131072 \
|
||||
--reasoning-parser auto \
|
||||
--tool-call-parser auto
|
||||
```
|
||||
|
||||
### OpenAI-Compatible API Usage
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(base_url="http://localhost:30000/v1", api_key="unused")
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model="ibm-granite/granite-4.2-3b",
|
||||
messages=[{"role": "user", "content": "Explain the Riemann hypothesis in simple terms."}],
|
||||
temperature=1.0,
|
||||
top_p=0.95,
|
||||
max_tokens=8192,
|
||||
)
|
||||
|
||||
print(response.choices[0].message.reasoning_content)
|
||||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
### Tool Calling via SGLang
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(base_url="http://localhost:30000/v1", api_key="unused")
|
||||
|
||||
tools = [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_current_weather",
|
||||
"description": "Get the current weather for a specified city.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"city": {"type": "string", "description": "Name of the city"}
|
||||
},
|
||||
"required": ["city"]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model="ibm-granite/granite-4.2-3b",
|
||||
messages=[{"role": "user", "content": "What's the weather like in Boston right now?"}],
|
||||
tools=tools,
|
||||
temperature=1.0,
|
||||
top_p=0.95,
|
||||
max_tokens=4096,
|
||||
)
|
||||
|
||||
print(response.choices[0].message.tool_calls)
|
||||
```
|
||||
|
||||
For a full deployment recipe (Docker, H200/B200 launch matrix, thinking-mode examples, and benchmark data), see the [SGLang Granite 4.2 cookbook](https://docs.sglang.io/cookbook/autoregressive/IBM/Granite-4.2).
|
||||
|
||||
---
|
||||
|
||||
## Using with Agentic Coding Harnesses
|
||||
|
||||
Granite-4.2-3B can be used as the backbone model for agentic coding tools. Since it supports reasoning and tool calling via the OpenAI-compatible API, it integrates with popular agentic harnesses out of the box. Start the vLLM server as shown in the [Serving with vLLM](#serving-with-vllm) section above, then follow the harness-specific instructions below.
|
||||
|
||||
### OpenCode
|
||||
|
||||
[OpenCode](https://opencode.ai) is an AI coding agent that runs in your terminal.
|
||||
|
||||
**Install:**
|
||||
|
||||
```bash
|
||||
curl -fsSL https://opencode.ai/install | bash
|
||||
```
|
||||
|
||||
**Configure** `~/.config/opencode/opencode.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"model": "local/granite-4.2-3b",
|
||||
"provider": {
|
||||
"local": {
|
||||
"npm": "@ai-sdk/openai-compatible",
|
||||
"name": "vLLM (local)",
|
||||
"options": {
|
||||
"baseURL": "http://localhost:8000/v1",
|
||||
"apiKey": "EMPTY"
|
||||
},
|
||||
"models": {
|
||||
"granite-4.2-3b": {
|
||||
"name": "Granite 4.2 3B",
|
||||
"limit": {
|
||||
"context": 131072,
|
||||
"output": 8192
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Run:**
|
||||
|
||||
```bash
|
||||
opencode
|
||||
opencode run "your task description"
|
||||
```
|
||||
|
||||
For full documentation, see [opencode.ai/docs](https://opencode.ai/docs).
|
||||
|
||||
### Pi
|
||||
|
||||
[Pi](https://pi.dev) is a minimal agent harness for AI-powered coding that runs in your terminal. It supports custom providers via a `models.json` configuration file.
|
||||
|
||||
**Install:**
|
||||
|
||||
```bash
|
||||
curl -fsSL https://pi.dev/install.sh | sh
|
||||
```
|
||||
|
||||
**Configure** `~/.pi/agent/models.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"providers": {
|
||||
"vllm": {
|
||||
"baseUrl": "http://localhost:8000/v1",
|
||||
"api": "openai-completions",
|
||||
"apiKey": "EMPTY",
|
||||
"compat": {
|
||||
"supportsDeveloperRole": false,
|
||||
"supportsReasoningEffort": false
|
||||
},
|
||||
"models": [
|
||||
{
|
||||
"id": "granite-4.2-3b",
|
||||
"name": "Granite 4.2 3B",
|
||||
"reasoning": true,
|
||||
"input": ["text"],
|
||||
"contextWindow": 131072,
|
||||
"maxTokens": 8192,
|
||||
"samplingParams": {
|
||||
"temperature": 1.0,
|
||||
"top_p": 0.95
|
||||
},
|
||||
"cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 }
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Run:**
|
||||
|
||||
```bash
|
||||
pi
|
||||
```
|
||||
|
||||
Then select the `granite-4.2-3b` model with `/model` or `Ctrl+L` in the interactive session.
|
||||
|
||||
For full documentation, see [pi.dev/docs](https://pi.dev/docs/latest).
|
||||
|
||||
### OpenHands
|
||||
|
||||
[OpenHands](https://www.openhands.dev) is an AI software engineer that can plan, write code, and execute commands.
|
||||
|
||||
1. **Install and launch OpenHands** following the [official installation guide](https://docs.openhands.dev/openhands/usage/agent-canvas/setup).
|
||||
|
||||
2. **Configure the LLM** in the OpenHands settings with:
|
||||
- **Model:** `granite-4.2-3b`
|
||||
- **Base URL:** `http://localhost:8000/v1`
|
||||
- **API Key:** your vLLM `--api-key` value
|
||||
|
||||
> **Note:** The `openai/` prefix is required when connecting to OpenAI-compatible endpoints like vLLM. Refer to the [OpenHands local LLM documentation](https://docs.openhands.dev/openhands/usage/llms/local-llms) for detailed setup instructions, troubleshooting, and alternative installation methods.
|
||||
|
||||
---
|
||||
|
||||
## Quick Start (Transformers)
|
||||
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
pip install torch torchvision torchaudio
|
||||
pip install accelerate transformers
|
||||
```
|
||||
|
||||
### Basic Inference (Thinking Mode)
|
||||
|
||||
```python
|
||||
import torch
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model_path = "ibm-granite/granite-4.2-3b"
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_path)
|
||||
model = AutoModelForCausalLM.from_pretrained(model_path, device_map="cuda", torch_dtype=torch.bfloat16)
|
||||
model.eval()
|
||||
|
||||
messages = [
|
||||
{"role": "user", "content": "How many r's are in the word 'strawberry'?"},
|
||||
]
|
||||
|
||||
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
|
||||
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
||||
|
||||
with torch.no_grad():
|
||||
output = model.generate(**inputs, max_new_tokens=8192, temperature=1.0, top_p=0.95, do_sample=True)
|
||||
|
||||
print(tokenizer.decode(output[0][inputs.input_ids.shape[-1]:], skip_special_tokens=False))
|
||||
```
|
||||
|
||||
<details>
|
||||
<summary><b>Example Output</b></summary>
|
||||
|
||||
```
|
||||
<think>
|
||||
Okay, let's see. The problem is to find how many 'r's are in the word 'strawberry'.
|
||||
|
||||
First, I need to write out the word: s t r a w b e r r y.
|
||||
|
||||
Now, I need to count the number of 'r' letters. Let's list each letter and check for 'r'.
|
||||
|
||||
1. s – not r
|
||||
2. t – not r
|
||||
3. r – yes, that's one
|
||||
4. a – no
|
||||
5. w – no
|
||||
6. b – no
|
||||
7. e – no
|
||||
8. r – yes, that's two
|
||||
9. r – yes, that's three
|
||||
10. y – no
|
||||
|
||||
Total r's = 3.
|
||||
</think>
|
||||
There are **3** r's in the word "strawberry".<|im_end|>
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
### Non-Thinking Mode
|
||||
|
||||
```python
|
||||
messages = [
|
||||
{"role": "user", "content": "What is the capital of France?"},
|
||||
]
|
||||
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
|
||||
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
||||
|
||||
output = model.generate(**inputs, max_new_tokens=2048, temperature=1.0, top_p=0.95, do_sample=True)
|
||||
print(tokenizer.decode(output[0][inputs.input_ids.shape[-1]:], skip_special_tokens=False))
|
||||
```
|
||||
|
||||
<details>
|
||||
<summary><b>Example Output</b></summary>
|
||||
|
||||
```
|
||||
<think></think>The capital of France is Paris.<|im_end|>
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
### Low-Effort Thinking
|
||||
|
||||
```python
|
||||
messages = [
|
||||
{"role": "user", "content": "What is 2 + 2?"},
|
||||
]
|
||||
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True,
|
||||
enable_thinking=True, low_effort=True)
|
||||
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
||||
|
||||
output = model.generate(**inputs, max_new_tokens=4096, temperature=1.0, top_p=0.95, do_sample=True)
|
||||
print(tokenizer.decode(output[0][inputs.input_ids.shape[-1]:], skip_special_tokens=False))
|
||||
```
|
||||
|
||||
<details>
|
||||
<summary><b>Example Output</b></summary>
|
||||
|
||||
```
|
||||
<think>
|
||||
Simple answer.
|
||||
</think>
|
||||
2 + 2 = 4.<|im_end|>
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## Tool Calling
|
||||
|
||||
Granite-4.2-3B supports tool calling with integrated reasoning — the model thinks about which tool to call and why before making the call. Tools are defined using the [OpenAI function definition schema](https://platform.openai.com/docs/guides/function-calling).
|
||||
|
||||
### Basic Tool Calling
|
||||
|
||||
```python
|
||||
tools = [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_current_weather",
|
||||
"description": "Get the current weather for a specified city.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"city": {"type": "string", "description": "Name of the city"}
|
||||
},
|
||||
"required": ["city"]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
|
||||
messages = [
|
||||
{"role": "user", "content": "What's the weather like in Boston right now?"},
|
||||
]
|
||||
text = tokenizer.apply_chat_template(messages, tokenize=False, tools=tools,
|
||||
add_generation_prompt=True, enable_thinking=True)
|
||||
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
||||
|
||||
output = model.generate(**inputs, max_new_tokens=4096, temperature=1.0, top_p=0.95, do_sample=True)
|
||||
print(tokenizer.decode(output[0][inputs.input_ids.shape[-1]:], skip_special_tokens=False))
|
||||
```
|
||||
|
||||
<details>
|
||||
<summary><b>Example Output</b></summary>
|
||||
|
||||
```
|
||||
<think>
|
||||
Okay, the user is asking for the weather in Boston right now. Let me check the tools
|
||||
available. There's a function called get_current_weather that takes a city parameter.
|
||||
I need to call that with the city set to Boston.
|
||||
</think>
|
||||
<tool_call>
|
||||
<function=get_current_weather>
|
||||
<parameter=city>
|
||||
Boston
|
||||
</parameter>
|
||||
</function>
|
||||
</tool_call>
|
||||
<|im_end|>
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
### Multi-Turn with Tool Response
|
||||
|
||||
```python
|
||||
messages = [
|
||||
{"role": "user", "content": "What's the weather like in Boston right now?"},
|
||||
{"role": "assistant", "content": "<think>\nThe user wants to know the current weather in Boston. I should call get_current_weather.\n</think>",
|
||||
"tool_calls": [{"function": {"name": "get_current_weather", "arguments": {"city": "Boston"}}}]},
|
||||
{"role": "tool", "content": '{"temperature": "72°F", "condition": "Partly cloudy", "humidity": "65%"}'},
|
||||
]
|
||||
text = tokenizer.apply_chat_template(messages, tokenize=False, tools=tools,
|
||||
add_generation_prompt=True, enable_thinking=True)
|
||||
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
||||
|
||||
output = model.generate(**inputs, max_new_tokens=4096, temperature=1.0, top_p=0.95, do_sample=True)
|
||||
print(tokenizer.decode(output[0][inputs.input_ids.shape[-1]:], skip_special_tokens=False))
|
||||
```
|
||||
|
||||
<details>
|
||||
<summary><b>Example Output</b></summary>
|
||||
|
||||
```
|
||||
<think>
|
||||
The tool returned the weather data for Boston: temperature 72°F, partly cloudy, humidity 65%.
|
||||
I need to present this information clearly to the user.
|
||||
</think>
|
||||
The current weather in Boston is 72°F, partly cloudy, with 65% humidity.<|im_end|>
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## Multi-Turn Conversations
|
||||
|
||||
### History Thinking Truncation
|
||||
|
||||
```python
|
||||
messages = [
|
||||
{"role": "user", "content": "What is 15 * 37?"},
|
||||
{"role": "assistant", "content": "<think>\nLet me calculate 15 * 37.\n15 * 37 = 15 * 30 + 15 * 7 = 450 + 105 = 555\n</think>\n15 * 37 = 555"},
|
||||
{"role": "user", "content": "Now divide that by 5"},
|
||||
]
|
||||
|
||||
# Default: previous thinking is stripped to save context
|
||||
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True,
|
||||
enable_thinking=True, truncate_history_thinking=True)
|
||||
|
||||
# To preserve full history:
|
||||
text_full = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True,
|
||||
enable_thinking=True, truncate_history_thinking=False)
|
||||
```
|
||||
|
||||
### Parsing Thinking vs. Final Answer
|
||||
|
||||
```python
|
||||
import re
|
||||
|
||||
def parse_model_output(text):
|
||||
"""Separate thinking content from final answer."""
|
||||
think_match = re.search(r'<think>(.*?)</think>', text, re.DOTALL)
|
||||
if think_match:
|
||||
thinking = think_match.group(1).strip()
|
||||
answer_start = text.find('</think>') + len('</think>')
|
||||
answer_end = text.find('<|im_end|>', answer_start)
|
||||
answer = text[answer_start:answer_end].strip() if answer_end != -1 else text[answer_start:].strip()
|
||||
else:
|
||||
thinking, answer = "", text.strip()
|
||||
return thinking, answer
|
||||
|
||||
thinking, answer = parse_model_output(output_text)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Ethical Considerations and Limitations
|
||||
|
||||
Granite 4.2 models are primarily finetuned using instruction-response pairs mostly in English, but also multilingual data covering the supported languages listed above. Although this model handles multilingual dialog, its performance may vary compared to English. Few-shot examples can help in such cases.
|
||||
|
||||
While aligned for safety, the model may occasionally produce inaccurate, biased, or unsafe responses. The content within `<think>...</think>` tags represents internal reasoning and may contain unpolished or intermediate thoughts that do not represent final conclusions.
|
||||
|
||||
To enhance safety in deployments, we recommend using Granite 4.2 alongside [Granite Guardian](https://huggingface.co/ibm-granite/granite-guardian-4.1-8b) to detect and flag risks across key dimensions outlined in the IBM AI Risk Atlas.
|
||||
|
||||
---
|
||||
|
||||
## Resources
|
||||
|
||||
- **Product Page:** https://www.ibm.com/granite
|
||||
- **Documentation:** https://www.ibm.com/granite/docs/
|
||||
- **Learning Resources:** https://ibm.biz/granite-learning-resources
|
||||
- **Technical Blog:** https://huggingface.co/blog/ibm-granite/granite-4-2
|
||||
- **GitHub:** https://github.com/ibm-granite/granite-4.2-language-models
|
||||
- **HF Collection:** https://huggingface.co/collections/ibm-granite/granite-42-language-models
|
||||
184
chat_template.jinja
Normal file
184
chat_template.jinja
Normal file
@@ -0,0 +1,184 @@
|
||||
{%- macro tool_to_json(tool) -%}
|
||||
{%- set ns_tool = namespace(first=true) -%}
|
||||
{{- '{' -}}
|
||||
{%- for k, v in tool.items() -%}
|
||||
{%- if k != 'defer_loading' and k != 'strict' -%}
|
||||
{%- if not ns_tool.first -%}{{- ', ' -}}{%- endif -%}
|
||||
{%- set ns_tool.first = false -%}
|
||||
{{- '"' ~ k ~ '": ' ~ (v | tojson(ensure_ascii=False)) -}}
|
||||
{%- endif -%}
|
||||
{%- endfor -%}
|
||||
{{- '}' -}}
|
||||
{%- endmacro -%}
|
||||
{%- set enable_thinking = enable_thinking if enable_thinking is defined else True %}
|
||||
{%- set low_effort = low_effort if low_effort is defined else False %}
|
||||
{%- if reasoning_effort is defined and reasoning_effort is not none %}
|
||||
{%- set low_effort = reasoning_effort == "low" %}
|
||||
{%- endif %}
|
||||
{%- set truncate_history_thinking = truncate_history_thinking if truncate_history_thinking is defined else True %}
|
||||
|
||||
{%- set ns = namespace(last_user_idx = -1) %}
|
||||
{%- set loop_messages = messages %}
|
||||
{%- for m in loop_messages %}
|
||||
{%- if m["role"] == "user" %}
|
||||
{%- set ns.last_user_idx = loop.index0 %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
|
||||
{%- if messages[0]["role"] == "system" %}
|
||||
{%- set system_message = messages[0]["content"] %}
|
||||
{%- set loop_messages = messages[1:] %}
|
||||
{%- else %}
|
||||
{%- set system_message = "" %}
|
||||
{%- set loop_messages = messages %}
|
||||
{%- endif %}
|
||||
{%- if not tools is defined %}
|
||||
{%- set tools = [] %}
|
||||
{%- endif %}
|
||||
{# Recompute last_user_idx relative to loop_messages after handling system #}
|
||||
{%- set ns = namespace(last_user_idx = -1) %}
|
||||
{%- for m in loop_messages %}
|
||||
{%- if m["role"] == "user" %}
|
||||
{%- set ns.last_user_idx = loop.index0 %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- if system_message is defined %}
|
||||
{{- "<|im_start|>system\n" + system_message }}
|
||||
{%- else %}
|
||||
{%- if tools is iterable and tools | length > 0 %}
|
||||
{{- "<|im_start|>system\n" }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- if tools is iterable and tools | length > 0 %}
|
||||
{%- if system_message is defined and system_message | length > 0 %}
|
||||
{{- "\n\n" }}
|
||||
{%- endif %}
|
||||
{{- "# Tools\n\nYou have access to the following functions:\n\n" }}
|
||||
{{- "<tools>" }}
|
||||
{%- for tool in tools %}
|
||||
{%- if tool.function is defined %}
|
||||
{%- set tool = tool.function %}
|
||||
{%- endif %}
|
||||
{%- if tool.defer_loading is not defined or not tool.defer_loading %}
|
||||
{{- '\n' ~ tool_to_json(tool) }}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{{- "\n</tools>" }}
|
||||
|
||||
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
|
||||
{%- endif %}
|
||||
|
||||
|
||||
{%- if system_message is defined %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- else %}
|
||||
{%- if tools is iterable and tools | length > 0 %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
|
||||
{%- for message in loop_messages %}
|
||||
{%- if message.role == "assistant" %}
|
||||
{# Add reasoning content in to content field for unified processing below. #}
|
||||
{%- if message.reasoning_content is defined and message.reasoning_content is string and message.reasoning_content | trim | length > 0 %}
|
||||
{%- set content = "<think>\n" ~ message.reasoning_content ~ "\n</think>\n" ~ (message.content | default('', true)) %}
|
||||
{%- else %}
|
||||
{%- set content = message.content | default('', true) %}
|
||||
{%- if content is string -%}
|
||||
{# Allow downstream logic to to take care of broken thought, only handle coherent reasoning here. #}
|
||||
{%- if '<think>' not in content and '</think>' not in content -%}
|
||||
{%- set content = "<think></think>" ~ content -%}
|
||||
{%- endif -%}
|
||||
{%- else -%}
|
||||
{%- set content = content -%}
|
||||
{%- endif -%}
|
||||
{%- endif %}
|
||||
{%- if message.tool_calls is defined and message.tool_calls is iterable and message.tool_calls | length > 0 %}
|
||||
{# Assistant message has tool calls. #}
|
||||
{{- '<|im_start|>assistant\n' }}
|
||||
{%- set include_content = not (truncate_history_thinking and loop.index0 < ns.last_user_idx) %}
|
||||
{%- if content is string and content | trim | length > 0 %}
|
||||
{%- if include_content %}
|
||||
{{- (content | trim) ~ '\n' -}}
|
||||
{%- else %}
|
||||
{%- set c = (content | string) %}
|
||||
{%- if '</think>' in c %}
|
||||
{# Keep only content after the last closing think. Also generation prompt causes this. #}
|
||||
{%- set c = c.split('</think>')[-1] %}
|
||||
{%- elif '<think>' in c %}
|
||||
{# If <think> was opened but never closed, drop the trailing think segment #}
|
||||
{%- set c = c.split('<think>')[0] %}
|
||||
{%- endif %}
|
||||
{%- set c = "<think></think>" ~ c | trim %}
|
||||
{%- if c | length > 0 %}
|
||||
{{- c ~ '\n' -}}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- else %}
|
||||
{{- "<think></think>" -}}
|
||||
{%- endif %}
|
||||
{%- for tool_call in message.tool_calls %}
|
||||
{%- if tool_call.function is defined %}
|
||||
{%- set tool_call = tool_call.function %}
|
||||
{%- endif %}
|
||||
{{- '<tool_call>\n<function=' ~ tool_call.name ~ '>\n' -}}
|
||||
{%- if tool_call.arguments is defined %}
|
||||
{%- for args_name, args_value in tool_call.arguments|items %}
|
||||
{{- '<parameter=' ~ args_name ~ '>\n' -}}
|
||||
{%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
|
||||
{{- args_value ~ '\n</parameter>\n' -}}
|
||||
{%- endfor %}
|
||||
{%- endif %}
|
||||
{{- '</function>\n</tool_call>\n' -}}
|
||||
{%- endfor %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- else %}
|
||||
{# Assistant message doesn't have tool calls. #}
|
||||
{%- if not (truncate_history_thinking and loop.index0 < ns.last_user_idx) %}
|
||||
{{- '<|im_start|>assistant\n' ~ (content | default('', true) | string | trim) ~ '<|im_end|>\n' }}
|
||||
{%- else %}
|
||||
{%- set c = (content | default('', true) | string) %}
|
||||
{%- if '<think>' in c and '</think>' in c %}
|
||||
{%- set c = "<think></think>" ~ c.split('</think>')[-1] %}
|
||||
{%- endif %}
|
||||
{%- set c = c | trim %}
|
||||
{%- if c | length > 0 %}
|
||||
{{- '<|im_start|>assistant\n' ~ c ~ '<|im_end|>\n' }}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>assistant\n<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- elif message.role == "user" or message.role == "system" %}
|
||||
{{- '<|im_start|>' + message.role + '\n' }}
|
||||
{%- set content = message.content | string %}
|
||||
{%- if message.role == "user" and loop.index0 == ns.last_user_idx and low_effort %}
|
||||
{{- content + '\n\n{reasoning effort: low}' }}
|
||||
{%- else %}
|
||||
{{- content }}
|
||||
{%- endif %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- elif message.role == "tool" %}
|
||||
{%- if loop.previtem and loop.previtem.role != "tool" %}
|
||||
{{- '<|im_start|>user\n' }}
|
||||
{%- endif %}
|
||||
{{- '<tool_response>\n' }}
|
||||
{{- message.content }}
|
||||
{{- '\n</tool_response>\n' }}
|
||||
{%- if not loop.last and loop.nextitem.role != "tool" %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- elif loop.last %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
|
||||
{%- if add_generation_prompt %}
|
||||
{%- if enable_thinking %}
|
||||
{{- '<|im_start|>assistant\n<think>\n' }}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>assistant\n<think></think>' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
37
config.json
Normal file
37
config.json
Normal file
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"architectures": [
|
||||
"GraniteForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"attention_multiplier": 0.015625,
|
||||
"bos_token_id": 100283,
|
||||
"dtype": "bfloat16",
|
||||
"embedding_multiplier": 1.0,
|
||||
"eos_token_id": 100257,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 2560,
|
||||
"initializer_range": 0.1,
|
||||
"intermediate_size": 8192,
|
||||
"logits_scaling": 1.0,
|
||||
"max_position_embeddings": 131072,
|
||||
"mlp_bias": false,
|
||||
"model_type": "granite",
|
||||
"num_attention_heads": 40,
|
||||
"num_hidden_layers": 40,
|
||||
"num_key_value_heads": 8,
|
||||
"pad_token_id": 100257,
|
||||
"residual_multiplier": 1.0,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_parameters": {
|
||||
"rope_theta": 10000000,
|
||||
"rope_type": "default"
|
||||
},
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 10000000,
|
||||
"tie_word_embeddings": false,
|
||||
"transformers_version": "4.57.1",
|
||||
"use_cache": true,
|
||||
"vocab_size": 100352,
|
||||
"torch_dtype": "bfloat16"
|
||||
}
|
||||
10
generation_config.json
Normal file
10
generation_config.json
Normal file
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"do_sample": true,
|
||||
"bos_token_id": 100283,
|
||||
"eos_token_id": 100257,
|
||||
"pad_token_id": 100257,
|
||||
"temperature": 1.0,
|
||||
"top_p": 0.95,
|
||||
"transformers_version": "4.57.1"
|
||||
}
|
||||
24
granite_thinking_parser.py
Normal file
24
granite_thinking_parser.py
Normal file
@@ -0,0 +1,24 @@
|
||||
from vllm.reasoning.abs_reasoning_parsers import ReasoningParserManager
|
||||
from vllm.reasoning.deepseek_r1_reasoning_parser import DeepSeekR1ReasoningParser
|
||||
|
||||
|
||||
@ReasoningParserManager.register_module("granite_thinking_parser")
|
||||
class GraniteThinkingReasoningParser(DeepSeekR1ReasoningParser):
|
||||
|
||||
def extract_reasoning(self, model_output, request):
|
||||
reasoning_content, final_content = super().extract_reasoning(
|
||||
model_output, request
|
||||
)
|
||||
|
||||
if final_content is not None:
|
||||
final_content = final_content.lstrip("\n")
|
||||
|
||||
if (
|
||||
hasattr(request, "chat_template_kwargs")
|
||||
and request.chat_template_kwargs
|
||||
and request.chat_template_kwargs.get("enable_thinking") is False
|
||||
and final_content is None
|
||||
):
|
||||
reasoning_content, final_content = None, reasoning_content
|
||||
|
||||
return reasoning_content, final_content
|
||||
100001
merges.txt
Normal file
100001
merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
model-00001-of-00002.safetensors
Normal file
3
model-00001-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:db39eb2776b5404dbfd0696cf7fe5ec671f0dad0e0918c62c5aea9e2eb5825d7
|
||||
size 5358533712
|
||||
3
model-00002-of-00002.safetensors
Normal file
3
model-00002-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8fba6ea17551d5d129eac519dbd70237139221c7fd0b89623ef54d9711062505
|
||||
size 1960983408
|
||||
370
model.safetensors.index.json
Normal file
370
model.safetensors.index.json
Normal file
@@ -0,0 +1,370 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 6805672960
|
||||
},
|
||||
"weight_map": {
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"lm_head.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.36.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.39.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.37.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.36.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.38.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.36.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.32.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.37.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.39.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.37.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.36.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.38.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.34.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.33.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.35.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.29.mlp.down_proj.weight": "model-00002-of-00002.safetensors"
|
||||
}
|
||||
}
|
||||
30
special_tokens_map.json
Normal file
30
special_tokens_map.json
Normal file
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"unk_token": {
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
501276
tokenizer.json
Normal file
501276
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
785
tokenizer_config.json
Normal file
785
tokenizer_config.json
Normal file
@@ -0,0 +1,785 @@
|
||||
{
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"100256": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100257": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100258": {
|
||||
"content": "<|fim_prefix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100259": {
|
||||
"content": "<|fim_middle|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100260": {
|
||||
"content": "<|fim_suffix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100261": {
|
||||
"content": "<|fim_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100262": {
|
||||
"content": "<|filename|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100263": {
|
||||
"content": "<|reponame|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100264": {
|
||||
"content": "<|unused_11|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100265": {
|
||||
"content": "<|unused_12|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100266": {
|
||||
"content": "<|unused_13|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100267": {
|
||||
"content": "<|unused_14|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100268": {
|
||||
"content": "<|unused_15|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100269": {
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100270": {
|
||||
"content": "<tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100271": {
|
||||
"content": "</tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100272": {
|
||||
"content": "<tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100273": {
|
||||
"content": "</tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100274": {
|
||||
"content": "<think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100275": {
|
||||
"content": "</think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"100276": {
|
||||
"content": "[INST]",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100277": {
|
||||
"content": "[/INST]",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100278": {
|
||||
"content": "[AVAILABLE_TOOLS]",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100279": {
|
||||
"content": "[/AVAILABLE_TOOLS]",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100280": {
|
||||
"content": "[TOOL_RESULTS]",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100281": {
|
||||
"content": "[/TOOL_RESULTS]",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100282": {
|
||||
"content": "[TOOL_CALLS]",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100283": {
|
||||
"content": "<s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100284": {
|
||||
"content": "</s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100285": {
|
||||
"content": "<|unused_16|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100286": {
|
||||
"content": "<|unused_17|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100287": {
|
||||
"content": "<|unused_18|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100288": {
|
||||
"content": "<|unused_19|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100289": {
|
||||
"content": "<|unused_20|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100290": {
|
||||
"content": "<|unused_21|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100291": {
|
||||
"content": "<|unused_22|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100292": {
|
||||
"content": "<|unused_23|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100293": {
|
||||
"content": "<|unused_24|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100294": {
|
||||
"content": "<|unused_25|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100295": {
|
||||
"content": "<|unused_26|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100296": {
|
||||
"content": "<|unused_27|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100297": {
|
||||
"content": "<|unused_28|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100298": {
|
||||
"content": "<|unused_29|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100299": {
|
||||
"content": "<|unused_30|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100300": {
|
||||
"content": "<|unused_31|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100301": {
|
||||
"content": "<|unused_32|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100302": {
|
||||
"content": "<|unused_33|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100303": {
|
||||
"content": "<|unused_34|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100304": {
|
||||
"content": "<|unused_35|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100305": {
|
||||
"content": "<|unused_36|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100306": {
|
||||
"content": "<|unused_37|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100307": {
|
||||
"content": "<|unused_38|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100308": {
|
||||
"content": "<|unused_39|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100309": {
|
||||
"content": "<|unused_40|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100310": {
|
||||
"content": "<|unused_41|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100311": {
|
||||
"content": "<|unused_42|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100312": {
|
||||
"content": "<|unused_43|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100313": {
|
||||
"content": "<|unused_44|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100314": {
|
||||
"content": "<|unused_45|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100315": {
|
||||
"content": "<|unused_46|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100316": {
|
||||
"content": "<|unused_47|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100317": {
|
||||
"content": "<|unused_48|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100318": {
|
||||
"content": "<|unused_49|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100319": {
|
||||
"content": "<|unused_50|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100320": {
|
||||
"content": "<|unused_51|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100321": {
|
||||
"content": "<|unused_52|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100322": {
|
||||
"content": "<|unused_53|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100323": {
|
||||
"content": "<|unused_54|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100324": {
|
||||
"content": "<|unused_55|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100325": {
|
||||
"content": "<|unused_56|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100326": {
|
||||
"content": "<|unused_57|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100327": {
|
||||
"content": "<|unused_58|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100328": {
|
||||
"content": "<|unused_59|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100329": {
|
||||
"content": "<|unused_60|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100330": {
|
||||
"content": "<|unused_61|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100331": {
|
||||
"content": "<|unused_62|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100332": {
|
||||
"content": "<|unused_63|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100333": {
|
||||
"content": "<|unused_64|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100334": {
|
||||
"content": "<|unused_65|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100335": {
|
||||
"content": "<|unused_66|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100336": {
|
||||
"content": "<|unused_67|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100337": {
|
||||
"content": "<|unused_68|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100338": {
|
||||
"content": "<|unused_69|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100339": {
|
||||
"content": "<|unused_70|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100340": {
|
||||
"content": "<|unused_71|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100341": {
|
||||
"content": "<|unused_72|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100342": {
|
||||
"content": "<|unused_73|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100343": {
|
||||
"content": "<|unused_74|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100344": {
|
||||
"content": "<|unused_75|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100345": {
|
||||
"content": "<|unused_76|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100346": {
|
||||
"content": "<|unused_77|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100347": {
|
||||
"content": "<|unused_78|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100348": {
|
||||
"content": "<|unused_79|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100349": {
|
||||
"content": "<|unused_80|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100350": {
|
||||
"content": "<|unused_81|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"100351": {
|
||||
"content": "<|unused_82|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
}
|
||||
},
|
||||
"backend": "tokenizers",
|
||||
"bos_token": "<s>",
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|im_end|>",
|
||||
"errors": "replace",
|
||||
"extra_special_tokens": {},
|
||||
"is_local": true,
|
||||
"model_max_length": 1000000000000000019884624838656,
|
||||
"pad_token": "<|im_end|>",
|
||||
"padding_side": "left",
|
||||
"tokenizer_class": "GPT2Tokenizer",
|
||||
"unk_token": "<unk>"
|
||||
}
|
||||
1
vocab.json
Normal file
1
vocab.json
Normal file
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user