初始化项目,由ModelHub XC社区提供模型

Model: unsloth/granite-4.1-8b-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-17 02:10:24 +08:00
commit f03ee46538
24 changed files with 743 additions and 0 deletions

57
.gitattributes vendored Normal file
View File

@@ -0,0 +1,57 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-UD-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-UD-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-UD-Q2_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-UD-Q3_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-UD-Q4_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-UD-Q5_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-UD-Q6_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-UD-Q8_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
imatrix_unsloth.gguf_file filter=lfs diff=lfs merge=lfs -text
granite-4.1-8b-BF16.gguf filter=lfs diff=lfs merge=lfs -text

620
README.md Normal file
View File

@@ -0,0 +1,620 @@
---
base_model:
- ibm-granite/granite-4.1-8b
license: apache-2.0
library_name: transformers
tags:
- language
- unsloth
- granite-4.1
---
> [!NOTE]
> Includes Unsloth **chat template fixes**! <br> For `llama.cpp`, use `--jinja`
>
<div>
<p style="margin-top: 0;margin-bottom: 0;">
<em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
</p>
<div style="display: flex; gap: 5px; align-items: center; ">
<a href="https://github.com/unslothai/unsloth/">
<img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
</a>
<a href="https://discord.gg/unsloth">
<img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
</a>
<a href="https://docs.unsloth.ai/">
<img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
</a>
</div>
</div>
[![mof-class3-qualified](https://mot.isitopen.ai/modules/mof/assets/badge_class3_qualified.png)](https://mot.isitopen.ai/model/1160)
# Granite-4.1-8B
**Model Summary:**
Granite-4.1-8B is a 8B parameter long-context instruct model finetuned from *Granite-4.1-8B-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. Granite 4.1 models have gone through an improved post-training pipeline, including supervised finetuning and reinforcement learning alignment, resulting in enhanced tool calling, instruction following, and chat capabilities.
- **Developers:** Granite Team, IBM
- **HF Collection:** [Granite 4.1 Language Models HF Collection](https://huggingface.co/collections/ibm-granite/granite-41-language-models)
- **Technical Blog:** [Granite-4.1 Blog](https://huggingface.co/blog/ibm-granite/granite-4-1)
- **GitHub Repository:** [ibm-granite/granite-4.1-language-models](https://github.com/ibm-granite/granite-4.1-language-models)
- **Website**: [Granite Docs](https://www.ibm.com/granite/docs/)
- **Release Date**: April 29th, 2026
- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
**Supported Languages:**
English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese. Users may finetune Granite 4.1 models for languages beyond these languages.
**Intended use:**
The model is designed to follow general instructions and can serve as the foundation for AI assistants across diverse domains, including business applications, as well as for LLM agents equipped with tool-use capabilities.
*Capabilities*
* Summarization
* Text classification
* Text extraction
* Question-answering
* Retrieval Augmented Generation (RAG)
* Code related tasks
* Function-calling tasks
* Multilingual dialog use cases
* Fill-In-the-Middle (FIM) code completions
<!-- <todo>Need to test the examples. (especially the tool calling and RAG ones)</todo>
-->
**Generation:**
This is a simple example of how to use Granite-4.1-8B model.
Install the following libraries:
```shell
pip install torch torchvision torchaudio
pip install accelerate
pip install transformers
```
Then, copy the snippet from the section that is relevant for your use case.
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda"
model_path = "ibm-granite/granite-4.1-8b"
tokenizer = AutoTokenizer.from_pretrained(model_path)
# drop device_map if running on CPU
model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
model.eval()
# change input text as desired
chat = [
{ "role": "user", "content": "Please list one IBM Research laboratory located in the United States. You should only output its name and location." },
]
chat = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
# tokenize the text
input_tokens = tokenizer(chat, return_tensors="pt").to(device)
# generate output tokens
output = model.generate(**input_tokens,
max_new_tokens=100)
# decode output tokens into text
output = tokenizer.batch_decode(output)
# print output
print(output[0])
```
Expected output:
```shell
<|start_of_role|>user<|end_of_role|>Please list one IBM Research laboratory located in the United States. You should only output its name and location.<|end_of_text|>
<|start_of_role|>assistant<|end_of_role|>IBM Almaden Research Laboratory, San Jose, California, United States.<|end_of_text|>
```
<!-- 📣 **Update [2025-10-07]:** Added a *default system prompt* to the chat template to guide the model towards more *professional, accurate, and safe* responses. -->
**Tool-calling:**
Granite-4.1-8B comes with enhanced tool calling capabilities, enabling seamless integration with external functions and APIs. To define a list of tools please follow OpenAI's function [definition schema](https://platform.openai.com/docs/guides/function-calling?api-mode=responses#defining-functions).
This is an example of how to use Granite-4.1-8B model tool-calling ability:
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda"
model_path = "ibm-granite/granite-4.1-8b"
tokenizer = AutoTokenizer.from_pretrained(model_path)
# drop device_map if running on CPU
model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
model.eval()
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather for a specified city.",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "Name of the city"
}
},
"required": ["city"]
}
}
}
]
# change input text as desired
chat = [
{ "role": "user", "content": "What's the weather like in Boston right now?" },
]
chat = tokenizer.apply_chat_template(chat, \
tokenize=False, \
tools=tools, \
add_generation_prompt=True)
# tokenize the text
input_tokens = tokenizer(chat, return_tensors="pt").to(device)
# generate output tokens
output = model.generate(**input_tokens,
max_new_tokens=100)
# decode output tokens into text
output = tokenizer.batch_decode(output)
# print output
print(output[0])
```
Expected output:
```shell
<|start_of_role|>system<|end_of_role|>You are a helpful assistant with access to the following tools. You may call one or more tools to assist with the user query.
You are provided with function signatures within <tools></tools> XML tags:
- <tools>
- unsloth
{"type": "function", "function": {"name": "get_current_weather", "description": "Get the current weather for a specified city.", "parameters": {"type": "object", "properties": {"city": {"type": "string", "description": "Name of the city"}}, "required": ["city"]}}}
</tools>
For each tool call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
- <tool_call>
- unsloth
{"name": <function-name>, "arguments": <args-json-object>}
</tool_call>. If a tool does not exist in the provided list of tools, notify the user that you do not have the ability to fulfill the request.<|end_of_text|>
<|start_of_role|>user<|end_of_role|>What's the weather like in Boston right now?<|end_of_text|>
<|start_of_role|>assistant<|end_of_role|><tool_call>
{"name": "get_current_weather", "arguments": {"city": "Boston"}}
</tool_call><|end_of_text|>
```
<!-- **Retrieval Augmented Generation:**
*Coming soon* -->
**Evaluation Results:**
<table>
<!-- <caption><b> All Results</b></caption> -->
<thead>
<tr>
<th style="text-align:left; background-color: #001d6c; color: white;">Benchmarks</th>
<th style="text-align:left; background-color: #001d6c; color: white;">Metric</th>
<th style="text-align:center; background-color: #001d6c; color: white;">3B Dense</th>
<th style="text-align:center; background-color: #001d6c; color: white;">8B Dense</th>
<th style="text-align:center; background-color: #001d6c; color: white;">30B Dense</th>
</tr>
</thead>
<tbody>
<tr>
<td colspan="5" style="text-align:center; background-color: #FFFFFF; color: #2D2D2D; font-style:italic;">
General Tasks
</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MMLU</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">5-shot</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">67.02</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">73.84</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">80.16</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MMLU-Pro</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">5-shot, CoT</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">49.83</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">55.99</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">64.09</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">BBH</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">3-shot, CoT</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">75.83</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">80.51</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">83.74</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">AGI EVAL</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">0-shot, CoT</td>
<td style="text-align:right; background-color:#FFFFFF; color: #2D2D2D;">65.16</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">72.43</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">77.80</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">GPQA</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">0-shot, CoT</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">31.70</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">41.96</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">45.76</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">SimpleQA</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;"></td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">3.68</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">4.82</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">6.81</td>
</tr>
<tr>
<td colspan="5" style="text-align:center; background-color: #FFFFFF; color: #2D2D2D; font-style:italic;">
Alignment Tasks
</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">AlpacaEval 2.0</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;"></td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">38.57</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">50.08</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">56.16</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">IFEval Avg</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;"></td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">82.30</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">87.06</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">89.65</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">ArenaHard</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;"></td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">37.80</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">68.98</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">71.02</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MTBench Avg</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;"></td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">7.57</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">8.61</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">8.61</td>
</tr>
<tr>
<td colspan="5" style="text-align:center; background-color: #FFFFFF; color: #2D2D2D; font-style:italic;">
Math Tasks
</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">GSM8K</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">8-shot</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">86.88</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">92.49</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">94.16</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">GSM Symbolic</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">8-shot</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">81.32</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">83.70</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">75.70</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">Minerva Math</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">0-shot, CoT</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">67.94</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">80.10</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">81.32</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">DeepMind Math</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">0-shot, CoT</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">64.64</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">80.07</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">81.93</td>
</tr>
<tr>
<td colspan="5" style="text-align:center; background-color: #FFFFFF; color: #2D2D2D; font-style:italic;">
Code Tasks
</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">HumanEval</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">pass@1</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">81.71</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">85.37</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">88.41</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">HumanEval+</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">pass@1</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">76.83</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">79.88</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">85.37</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MBPP</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">pass@1</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">71.16</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">87.30</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">85.45</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MBPP+</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">pass@1</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">62.17</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">73.81</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">73.54</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">CRUXEval-O</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">pass@1</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">40.75</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">47.63</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">55.75</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">BigCodeBench</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">pass@1</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">32.19</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">35.00</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">38.77</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MULTIPLE</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">pass@1</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">52.54</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">60.26</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">62.31</td>
</tr>
<tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">Eval+ Avg</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">pass@1</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">67.05</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">80.21</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">82.66</td>
</tr>
<tr>
<td colspan="5" style="text-align:center; background-color: #FFFFFF; color: #2D2D2D; font-style:italic;">
Tool Calling Tasks
</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">BFCL v3</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;"></td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">60.80</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">68.27</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">73.68</td>
</tr>
<tr>
<td colspan="5" style="text-align:center; background-color: #FFFFFF; color: #2D2D2D; font-style:italic;">
Multilingual Tasks
</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MMMLU</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">5-shot</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">57.61</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">64.84</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">73.71</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">INCLUDE</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">5-shot</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">52.05</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">58.89</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">67.26</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MGSM</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">8-shot</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">70.00</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">82.32</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">71.12</td>
</tr>
<tr>
<td colspan="6" style="text-align:center; background-color: #FFFFFF; color: #2D2D2D; font-style:italic;">
Safety
</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">SALAD-Bench</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;"></td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">93.95</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">95.80</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">96.41</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">AttaQ</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;"></td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">81.88</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">81.19</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">85.76</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">Tulu3 Safety Eval Avg</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;"></td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">66.84</td>
<td style="text-align:right; background-color: #DAE8FF; color: #2D2D2D;">75.57</td>
<td style="text-align:right; background-color: #FFFFFF; color: #2D2D2D;">78.19</td>
</tr>
</tbody></table>
<table>
<caption><b>Multilingual Benchmarks and the included languages:</b></caption>
<thead>
<tr>
<th style="text-align:left; background-color: #001d6c; color: white;">Benchmarks</th>
<th style="text-align:left; background-color: #001d6c; color: white;"># Langs</th>
<th style="text-align:center; background-color: #001d6c; color: white;">Languages</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MMMLU</td>
<td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">11</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">ar, de, en, es, fr, ja, ko, pt, zh, bn, hi</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">INCLUDE</td>
<td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">14</td>
<!-- <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">hindi, bengali, tamil, telugu, arabic, german, spanish, french, italian, japanese, korean, dutch, portuguese, chinese</td> -->
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">hi, bn, ta, te, ar, de, es, fr, it, ja, ko, nl, pt, zh</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">MGSM</td>
<td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">5</td>
<td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">en, es, fr, ja, zh</td>
</tr>
</tbody>
</table>
**Model Architecture:**
Granite-4.1-8B baseline is built on a decoder-only dense transformer architecture. Core components of this architecture are: GQA, RoPE, MLP with SwiGLU, RMSNorm, and shared input/output embeddings.
<table>
<thead>
<tr>
<th style="text-align:left; background-color: #001d6c; color: white;">Model</th>
<th style="text-align:center; background-color: #001d6c; color: white;">3B Dense</th>
<th style="text-align:center; background-color: #001d6c; color: white;">8B Dense</th>
<th style="text-align:center; background-color: #001d6c; color: white;">30B Dense</th>
</tr></thead>
<tbody>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;">Embedding size</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">2560</td>
<td style="text-align:center; background-color: #DAE8FF; color: black;">4096</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">4096</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;">Number of layers</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">40</td>
<td style="text-align:center; background-color: #DAE8FF; color: black;">40</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">64</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;">Attention head size</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">64</td>
<td style="text-align:center; background-color: #DAE8FF; color: black;">128</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">128</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;">Number of attention heads</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">40</td>
<td style="text-align:center; background-color: #DAE8FF; color: black;">32</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">32</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;">Number of KV heads</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">8</td>
<td style="text-align:center; background-color: #DAE8FF; color: black;">8</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">8</td>
</tr>
<!--<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;">Mamba2 state size</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">-</td>
<td style="text-align:center; background-color: #DAE8FF; color: black;"></td>
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;">Number of Mamba2 heads</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
<td style="text-align:center; background-color: #DAE8FF; color: black;"></td>
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
</tr>-->
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;">MLP / Shared expert hidden size</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">8192</td>
<td style="text-align:center; background-color: #DAE8FF; color: black;">12800</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">32768</td>
</tr>
<!--<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;">Num. Experts</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
<td style="text-align:center; background-color: #DAE8FF; color: black;"></td>
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;">Num. active Experts</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
<td style="text-align:center; background-color: #DAE8FF; color: black;"></td>
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;">Expert hidden size</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
<td style="text-align:center; background-color: #DAE8FF; color: black;"></td>
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
</tr>-->
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;">MLP activation</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">SwiGLU</td>
<td style="text-align:center; background-color: #DAE8FF; color: black;">SwiGLU</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">SwiGLU</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;">Sequence length</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">131072</td>
<td style="text-align:center; background-color: #DAE8FF; color: black;">131072</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">131072</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;">Position embedding</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">RoPE</td>
<td style="text-align:center; background-color: #DAE8FF; color: black;">RoPE</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">RoPE</td>
</tr>
<tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;"># Parameters</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">3B</td>
<td style="text-align:center; background-color: #DAE8FF; color: black;">8B</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;">30B</td>
</tr>
<!-- <tr>
<td style="text-align:left; background-color: #FFFFFF; color: black;"># Active parameters</td>
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
<td style="text-align:center; background-color: #DAE8FF; color: black;"></td>
<td style="text-align:center; background-color: #FFFFFF; color: black;"></td>
</tr>-->
</tbody></table>
**Training Data:**
Overall, our SFT data is largely comprised of three key sources: (1) publicly available datasets with permissive license, (2) internal synthetic data targeting specific capabilities, and (3) a select set of human-curated data.
**Supervised Fine-Tuning and Reinforcement Learning:**
Instruct model has been fine tuned with significantly improved SFT-pipeline and Reinforcement learning pipelines with high quality mix of various datasets as mentioned above. With rigorous SFT-RL cycles we have improved Granite-4.1 model's tool calling, instruction following and chat capabilities. For further details please check our [Granite-4.1 Blog]((https://huggingface.co/blog/ibm-granite/granite-4-1)).
**Infrastructure:**
We trained the Granite 4.1 Language Models utilizing an NVIDIA GB200 NVL72 cluster hosted in CoreWeave. Intra-rack communication occurs via the 72-GPU NVLink domain, and a non-blocking, full Fat-Tree NDR 400 Gb/s InfiniBand network provides inter-rack communication. This cluster provides a scalable and efficient infrastructure for training our models over thousands of GPUs.
**Ethical Considerations and Limitations:**
Granite 4.1 Instruction Models are primarily finetuned using instruction-response pairs mostly in English, but also multilingual data covering multiple languages. Although this model can handle multilingual dialog use cases, its performance might not be similar to English tasks. In such cases, introducing a small number of examples (few-shot) can help the model in generating more accurate outputs. While this model has been aligned by keeping safety in consideration, the model may in some cases produce inaccurate, biased, or unsafe responses to user prompts. We urge the community to use this model with proper safety testing and tuning tailored for their specific tasks. To enhance safety in enterprise deployments, we recommend using Granite 4.1 Language models alongside [Granite Guardian](https://huggingface.co/ibm-granite/granite-guardian-4.1-8b), a model designed to detect and flag risks in inputs and outputs across key dimensions outlined in the IBM AI Risk Atlas.
**Resources**
- ⭐️ Learn about the latest updates with Granite: https://www.ibm.com/granite
- 📄 Get started with tutorials, best practices, and prompt engineering advice: https://www.ibm.com/granite/docs/
- 💡 Learn about the latest Granite learning resources: https://ibm.biz/granite-learning-resources
<!-- ## Citation
```
@misc{granite-models,
author = {author 1, author2, ...},
title = {},
journal = {},
volume = {},
year = {2024},
url = {https://arxiv.org/abs/0000.00000},
}
``` -->

3
granite-4.1-8b-BF16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f9089b3ff9e675f4da976c7b7d0e5418776f6950476894a9b9d4d43a672dfc01
size 17587418240

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:27420a7bcc444670fc3904dff2febc95787cead611e8a241cec3fc2d31e6d54f
size 5076923776

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:afb7f7b971d6ef310325fc93b157ba90bb602f5c58f6cf2304f54598c4a0ab4c
size 4833129856

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:170d197f021db40ac208a1327fa305f7b299fff00138d7dc0a6e98a630bee88b
size 4347049344

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:22081fbcf392ecc563ce637dca526e450e1fbc82184520bbbda2448d767a730a
size 3942954368

3
granite-4.1-8b-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c2814c4863583a856254041e502b37720c58b18752c3f4be137d368f6777cc55
size 5072336256

3
granite-4.1-8b-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:11554d284e06ecc3be8aebc1f2d91b87907be8d6593ed01d86d7aec30beca032
size 5579715968

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0f45c1af986e9900bb3b6ba46a25937e1bb80426935bc242d88c9ca90e9f5c88
size 5347915136

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:26550945ded0b8ff5faa73823e4922ea5327fd8feff1e8d9eed231ee29f6638c
size 5090817408

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4449463ec311c289a0d084aa9f95b8b7289bce7aa2959726e755c71614834cd4
size 6253884800

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c064f2e75bd4dc8e688842bbca72cd3233dcd2dd4c4096cae3cd9fcdfd2afe54
size 6103479680

3
granite-4.1-8b-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:17faed942e1363e0deda704068b698c6ab411899b4756cd96471168db272ec63
size 7216477568

3
granite-4.1-8b-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ac2ad3e017fa98aecac34ac8b5d6996faca82919030cea4fcd6bb1444e82edda
size 9345611136

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8587f6f436308125eb38643362892ce4b3aaae4518d3fb484bcce2f80d0cced7
size 3272078720

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:13d47e05d432acd518860d6835fd2b86437a6866d67fbacba8f6522b341e9dd0
size 3606148480

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f36313315fc3e7b480345ac1f21f80557f690be977893be61e4cae63b9edc924
size 3625055616

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4983ca9a12f8dad726afa423d2f69558dfb0cef6d34aa197a0270c25e19c50ed
size 4581389696

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4025a8b2d13fcc8474a9101d90bab4a11c1074faddd929e96b7a65c7418b720b
size 5494551936

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d84beb950ecc4be6c7a014f89ae6d83c792e234c596df11cdaa421371274d461
size 6290830720

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f4e2584150d14bc7b73b555afb627af33c4ef58785664503cd1184918cf63d13
size 7913321856

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ad27932f41b51b9948b76d8d532204b75bc6166b73b25bbbb0aca21aae43e535
size 11246810496

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d4a4217e3f80f6a18bbda0186af8adbe9b9916351a40e976ac4bd30a957a45b0
size 6023264