初始化项目，由ModelHub XC社区提供模型
Model: Mungert/granite-7b-instruct-GGUF Source: Original Platform
This commit is contained in:
ModelHub XC
76
.gitattributes
vendored
Normal file
76
.gitattributes
vendored
Normal file
@@ -0,0 +1,76 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zstandard filter=lfs diff=lfs merge=lfs -text
|
||||
*.tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
*.db* filter=lfs diff=lfs merge=lfs -text
|
||||
*.ark* filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
*.ggml filter=lfs diff=lfs merge=lfs -text
|
||||
*.llamafile* filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
granite-7b-instruct-q3_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-iq3_s.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-iq2_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-q2_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-q3_k_s.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-iq3_xxs.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-iq3_xs.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-iq2_s.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-iq2_xs.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-q2_k_s.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-iq4_xs.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-f16_q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-iq3_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-iq4_nl.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-iq2_xxs.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-bf16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-bf16_q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct.imatrix filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-q5_1.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-q5_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-q5_k_s.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-q4_k_s.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-q5_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
granite-7b-instruct-q6_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
196
README.md
Normal file
196
README.md
Normal file
@@ -0,0 +1,196 @@
|
||||
---
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- granite
|
||||
- ibm
|
||||
- lab
|
||||
- labrador
|
||||
- labradorite
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
base_model: ibm/granite-7b-base
|
||||
new_version: ibm-granite/granite-3.1-8b-instruct
|
||||
---
|
||||
|
||||
# <span style="color: #7FFF7F;">granite-7b-instruct GGUF Models</span>
|
||||
|
||||
|
||||
## <span style="color: #7F7FFF;">Model Generation Details</span>
|
||||
|
||||
This model was generated using [llama.cpp](https://github.com/ggerganov/llama.cpp) at commit [`0a5a3b5c`](https://github.com/ggerganov/llama.cpp/commit/0a5a3b5cdfd887cf0f8e09d9ff89dee130cfcdde).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## <span style="color: #7FFF7F;">Quantization Beyond the IMatrix</span>
|
||||
|
||||
I've been experimenting with a new quantization approach that selectively elevates the precision of key layers beyond what the default IMatrix configuration provides.
|
||||
|
||||
In my testing, standard IMatrix quantization underperforms at lower bit depths, especially with Mixture of Experts (MoE) models. To address this, I'm using the `--tensor-type` option in `llama.cpp` to manually "bump" important layers to higher precision. You can see the implementation here:
|
||||
👉 [Layer bumping with llama.cpp](https://github.com/Mungert69/GGUFModelBuilder/blob/main/model-converter/tensor_list_builder.py)
|
||||
|
||||
While this does increase model file size, it significantly improves precision for a given quantization level.
|
||||
|
||||
### **I'd love your feedback—have you tried this? How does it perform for you?**
|
||||
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
<a href="https://readyforquantum.com/huggingface_gguf_selection_guide.html" style="color: #7FFF7F;">
|
||||
Click here to get info on choosing the right GGUF model format
|
||||
</a>
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
<!--Begin Original Model Card-->
|
||||
|
||||
# Model Card for Granite-7b-lab [Paper](https://arxiv.org/abs/2403.01081)
|
||||
|
||||
### Overview
|
||||
|
||||
|
||||
|
||||
### Performance
|
||||
|
||||
| Model | Alignment | Base | Teacher | MTBench (Avg) * | MMLU(5-shot) |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| [Llama-2-13b-chat-hf](https://huggingface.co/meta-llama/Llama-2-13b-chat-hf) | RLHF | Llama-2-13b | Human Annotators | 6.65 |54.58 |
|
||||
| [Orca-2-13b](https://huggingface.co/microsoft/Orca-2-13b) | Progressive Training | Llama-2-13b | GPT-4 | 6.15 | 60.37 * |
|
||||
| [WizardLM-13B-V1.2](https://huggingface.co/WizardLM/WizardLM-13B-V1.2) | Evol-Instruct | Llama-2-13b | GPT-4 | 7.20 | 54.83 |
|
||||
| [Labradorite-13b](https://huggingface.co/ibm/labradorite-13b) | Large-scale Alignment for chatBots (LAB) | Llama-2-13b | Mixtral-8x7B-Instruct | 7.23 | 58.89 |
|
||||
| [Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1) | SFT | Mistral-7B-v0.1 | - | 6.84 | 60.37 |
|
||||
| [zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) | SFT/DPO | Mistral-7B-v0.1 | GPT-4 | 7.34 | 61.07 |
|
||||
| [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) | SFT | Mistral-7B-v0.1 | - | 7.6** | 60.78 |
|
||||
| [Merlinite-7b-lab](https://huggingface.co/instructlab/merlinite-7b-lab) | Large-scale Alignment for chatBots (LAB) | Mistral-7B-v0.1 | Mixtral-8x7B-Instruct | 7.66 |64.88 |
|
||||
| Granite-7b-lab | Large-scale Alignment for chatBots (LAB) | Granite-7b-base| Mixtral-8x7B-Instruct | 6.69 | 51.91 |
|
||||
|
||||
[*] Numbers for models other than Merlinite-7b-lab, Granite-7b-lab and [Labradorite-13b](https://huggingface.co/ibm/labradorite-13b) are taken from [lmsys/chatbot-arena-leaderboard](https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard)
|
||||
|
||||
[**] Numbers taken from [MistralAI Release Blog](https://mistral.ai/news/la-plateforme/)
|
||||
|
||||
### Method
|
||||
|
||||
LAB: **L**arge-scale **A**lignment for chat**B**ots is a novel synthetic data-based alignment tuning method for LLMs from IBM Research. Granite-7b-lab is a Granite-7b-base derivative model trained with the LAB methodology, using Mixtral-8x7b-Instruct as a teacher model.
|
||||
|
||||
LAB consists of three key components:
|
||||
|
||||
1. Taxonomy-driven data curation process
|
||||
2. Large-scale synthetic data generator
|
||||
3. Two-phased-training with replay buffers
|
||||
|
||||
|
||||
|
||||
LAB approach allows for adding new knowledge and skills, in an incremental fashion, to an already pre-trained model without suffering from catastrophic forgetting.
|
||||
|
||||
Taxonomy is a tree of seed examples that are used to prompt a teacher model to generate synthetic data. Taxonomy allows the data curator or the model designer to easily specify a diverse set of the knowledge-domains and skills that they would like to include in their LLM. At a high level, these can be categorized into three high-level bins - knowledge, foundational skills, and compositional skills. The leaf nodes of the taxonomy are tasks associated with one or more seed examples.
|
||||
|
||||
|
||||
|
||||
During the synthetic data generation, **unlike previous approaches where seed examples are uniformly drawn from the entire pool (i.e. self-instruct), we use the taxonomy to drive the sampling process**: For each knowledge/skill, we only use the local examples within the leaf node as seeds to prompt the teacher model.
|
||||
This makes the teacher model better exploit the task distributions defined by the local examples of each node and the diversity in the taxonomy itself ensures the entire generation covers a wide range of tasks, as illustrated below. In turns, this allows for using Mixtral 8x7B as the teacher model for generation while performing very competitively with models such as ORCA-2, WizardLM, and Zephyr Beta that rely on synthetic data generated by much larger and capable models like GPT-4.
|
||||
|
||||
|
||||
|
||||
For adding new domain-specific knowledge, we provide an external knowledge source (document) and prompt the model to generate questions and answers based on the document.
|
||||
Foundational skills such as reasoning and compositional skills such as creative writing are generated through in-context learning using the seed examples from the taxonomy.
|
||||
|
||||
Additionally, to ensure the data is high-quality and safe, we employ steps to check the questions and answers to ensure that they are grounded and safe. This is done using the same teacher model that generated the data.
|
||||
|
||||
Our training consists of two major phases: knowledge tuning and skills tuning.
|
||||
There are two steps in knowledge tuning where the first step learns simple knowledge (short samples) and the second step learns complicated knowledge (longer samples).
|
||||
The second step uses replay a replay buffer with data from the first step.
|
||||
Both foundational skills and compositional skills are learned during the skills tuning phases, where a replay buffer of data from the knowledge phase is used.
|
||||
Importantly, we use a set of hyper-parameters for training that are very different from standard small-scale supervised fine-training: larger batch size and carefully optimized learning rate and scheduler.
|
||||
|
||||
|
||||
|
||||
## Model description
|
||||
- **Model Name**: Granite-7b-lab
|
||||
- **Language(s):** Primarily English
|
||||
- **License:** Apache 2.0
|
||||
- **Base model:** [ibm/granite-7b-base](https://huggingface.co/ibm/granite-7b-base)
|
||||
- **Teacher Model:** [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)
|
||||
|
||||
## Prompt Template
|
||||
|
||||
```python
|
||||
sys_prompt = "You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior."
|
||||
|
||||
prompt = f'<|system|>\n{sys_prompt}\n<|user|>\n{inputs}\n<|assistant|>\n'
|
||||
stop_token = '<|endoftext|>'
|
||||
```
|
||||
|
||||
We advise utilizing the system prompt employed during the model's training for optimal inference performance, as there could be performance variations based on the provided instructions.
|
||||
|
||||
|
||||
|
||||
**Bias, Risks, and Limitations**
|
||||
|
||||
Granite-7b-lab is a base model and has not undergone any safety alignment, there it may produce problematic outputs. In the absence of adequate safeguards and RLHF, there exists a risk of malicious utilization of these models for generating disinformation or harmful content. Caution is urged against complete reliance on a specific language model for crucial decisions or impactful information, as preventing these models from fabricating content is not straightforward. Additionally, it remains uncertain whether smaller models might exhibit increased susceptibility to hallucination in ungrounded generation scenarios due to their reduced sizes and memorization capacities. This aspect is currently an active area of research, and we anticipate more rigorous exploration, comprehension, and mitigations in this domain.
|
||||
|
||||
<!--End Original Model Card-->
|
||||
|
||||
---
|
||||
|
||||
# <span id="testllm" style="color: #7F7FFF;">🚀 If you find these models useful</span>
|
||||
|
||||
Help me test my **AI-Powered Quantum Network Monitor Assistant** with **quantum-ready security checks**:
|
||||
|
||||
👉 [Quantum Network Monitor](https://readyforquantum.com/?assistant=open&utm_source=huggingface&utm_medium=referral&utm_campaign=huggingface_repo_readme)
|
||||
|
||||
|
||||
The full Open Source Code for the Quantum Network Monitor Service available at my github repos ( repos with NetworkMonitor in the name) : [Source Code Quantum Network Monitor](https://github.com/Mungert69). You will also find the code I use to quantize the models if you want to do it yourself [GGUFModelBuilder](https://github.com/Mungert69/GGUFModelBuilder)
|
||||
|
||||
💬 **How to test**:
|
||||
Choose an **AI assistant type**:
|
||||
- `TurboLLM` (GPT-4.1-mini)
|
||||
- `HugLLM` (Hugginface Open-source models)
|
||||
- `TestLLM` (Experimental CPU-only)
|
||||
|
||||
### **What I’m Testing**
|
||||
I’m pushing the limits of **small open-source models for AI network monitoring**, specifically:
|
||||
- **Function calling** against live network services
|
||||
- **How small can a model go** while still handling:
|
||||
- Automated **Nmap security scans**
|
||||
- **Quantum-readiness checks**
|
||||
- **Network Monitoring tasks**
|
||||
|
||||
🟡 **TestLLM** – Current experimental model (llama.cpp on 2 CPU threads on huggingface docker space):
|
||||
- ✅ **Zero-configuration setup**
|
||||
- ⏳ 30s load time (slow inference but **no API costs**) . No token limited as the cost is low.
|
||||
- 🔧 **Help wanted!** If you’re into **edge-device AI**, let’s collaborate!
|
||||
|
||||
### **Other Assistants**
|
||||
🟢 **TurboLLM** – Uses **gpt-4.1-mini** :
|
||||
- **It performs very well but unfortunatly OpenAI charges per token. For this reason tokens usage is limited.
|
||||
- **Create custom cmd processors to run .net code on Quantum Network Monitor Agents**
|
||||
- **Real-time network diagnostics and monitoring**
|
||||
- **Security Audits**
|
||||
- **Penetration testing** (Nmap/Metasploit)
|
||||
|
||||
🔵 **HugLLM** – Latest Open-source models:
|
||||
- 🌐 Runs on Hugging Face Inference API. Performs pretty well using the lastest models hosted on Novita.
|
||||
|
||||
### 💡 **Example commands you could test**:
|
||||
1. `"Give me info on my websites SSL certificate"`
|
||||
2. `"Check if my server is using quantum safe encyption for communication"`
|
||||
3. `"Run a comprehensive security audit on my server"`
|
||||
4. '"Create a cmd processor to .. (what ever you want)" Note you need to install a [Quantum Network Monitor Agent](https://readyforquantum.com/Download/?utm_source=huggingface&utm_medium=referral&utm_campaign=huggingface_repo_readme) to run the .net code on. This is a very flexible and powerful feature. Use with caution!
|
||||
|
||||
### Final Word
|
||||
|
||||
I fund the servers used to create these model files, run the Quantum Network Monitor service, and pay for inference from Novita and OpenAI—all out of my own pocket. All the code behind the model creation and the Quantum Network Monitor project is [open source](https://github.com/Mungert69). Feel free to use whatever you find helpful.
|
||||
|
||||
If you appreciate the work, please consider [buying me a coffee](https://www.buymeacoffee.com/mahadeva) ☕. Your support helps cover service costs and allows me to raise token limits for everyone.
|
||||
|
||||
I'm also open to job opportunities or sponsorship.
|
||||
|
||||
Thank you! 😊
|
||||
1
configuration.json
Normal file
1
configuration.json
Normal file
@@ -0,0 +1 @@
|
||||
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}
|
||||
3
granite-7b-instruct-bf16.gguf
Normal file
3
granite-7b-instruct-bf16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a43035072436928a4e05adaffca90eb9a4d7579363d313cbeacdd655692f8fd7
|
||||
size 13478237600
|
||||
3
granite-7b-instruct-bf16_q8_0.gguf
Normal file
3
granite-7b-instruct-bf16_q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d0a3906ef88f93a819f3c12a4530743b9c4a124e934158812b7bf06b221a4f79
|
||||
size 9766278560
|
||||
3
granite-7b-instruct-f16_q8_0.gguf
Normal file
3
granite-7b-instruct-f16_q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:2a5754e2b7cffe5190ceb5a2175560a416910c928c70105dd99facfa3555aebf
|
||||
size 9766278560
|
||||
3
granite-7b-instruct-iq2_m.gguf
Normal file
3
granite-7b-instruct-iq2_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9299eb07e617f96a67b2c0133bfd12b83c2988f159dca0c911e7c28d01607fec
|
||||
size 2468988640
|
||||
3
granite-7b-instruct-iq2_s.gguf
Normal file
3
granite-7b-instruct-iq2_s.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d23a21ba7de832edab6b9af2263286e04aff4a3269e97a76312231ec610a61c9
|
||||
size 2359936736
|
||||
3
granite-7b-instruct-iq2_xs.gguf
Normal file
3
granite-7b-instruct-iq2_xs.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e86f4a1d42370ab34d8da94f2312ab1f74204659a983530d08face6886038660
|
||||
size 2278852320
|
||||
3
granite-7b-instruct-iq2_xxs.gguf
Normal file
3
granite-7b-instruct-iq2_xxs.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:cacbae28a09f5d0df3b7254d02e763ddf938b3bd746535bf7959bc6dd637e319
|
||||
size 2092451552
|
||||
3
granite-7b-instruct-iq3_m.gguf
Normal file
3
granite-7b-instruct-iq3_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8b09be967fd8560bd0cfcd50efbc8ca69c38fccb30a4aa548bd029fad0969ce5
|
||||
size 3194898144
|
||||
3
granite-7b-instruct-iq3_s.gguf
Normal file
3
granite-7b-instruct-iq3_s.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:bb6639fb1176380b94b52c6ed37449adecc79ed629b562d589ad9f5eba8d020a
|
||||
size 3194898144
|
||||
3
granite-7b-instruct-iq3_xs.gguf
Normal file
3
granite-7b-instruct-iq3_xs.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4d7e9368645de1b24369ed3900740f383594065a947bf062661c48aa63e9e514
|
||||
size 2863220448
|
||||
3
granite-7b-instruct-iq3_xxs.gguf
Normal file
3
granite-7b-instruct-iq3_xxs.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:14f97250cb2f2dcdad659c434cd0b75006cfa30f1fbab93876630c8eb626ecaf
|
||||
size 2762295008
|
||||
3
granite-7b-instruct-iq4_nl.gguf
Normal file
3
granite-7b-instruct-iq4_nl.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:224ad5b63ef3ce5d43819f3e164527dd3413718780339c1bec889832360bbe7f
|
||||
size 3825854432
|
||||
3
granite-7b-instruct-iq4_xs.gguf
Normal file
3
granite-7b-instruct-iq4_xs.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5f55da453c55c07ed0946df4d8f6626342704842f479015c13efc247d6f3eb41
|
||||
size 3619382240
|
||||
3
granite-7b-instruct-q2_k_m.gguf
Normal file
3
granite-7b-instruct-q2_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1e2a3f363448c76afc309c9085551f9983273d7addaf3e62da758dce2c6bd39d
|
||||
size 2578557152
|
||||
3
granite-7b-instruct-q2_k_s.gguf
Normal file
3
granite-7b-instruct-q2_k_s.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4747f1245f9cf284c9a2f1bbe54f897f39a8cbce0a3e5ab755c847c1d04194c8
|
||||
size 2543732448
|
||||
3
granite-7b-instruct-q3_k_m.gguf
Normal file
3
granite-7b-instruct-q3_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b53a1f37514fc081c08dac8f4993747fa2f6af41bc4cc122c5a2e338ec8c6610
|
||||
size 3321211104
|
||||
3
granite-7b-instruct-q3_k_s.gguf
Normal file
3
granite-7b-instruct-q3_k_s.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f3b34de1c93a9a5b5b0e660aa54437b19e70f16a22722300fb7eb2119bca5398
|
||||
size 3286386400
|
||||
3
granite-7b-instruct-q4_0.gguf
Normal file
3
granite-7b-instruct-q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:63040e9957bc895ce26806a6212343d93ec800077bdd322b160ed7cabf0ccc3d
|
||||
size 3792053984
|
||||
3
granite-7b-instruct-q4_1.gguf
Normal file
3
granite-7b-instruct-q4_1.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:20401fb9446c00522e20129b4b9a50dccd5fe55020d27f4997985541cb0e9d8d
|
||||
size 4213192416
|
||||
3
granite-7b-instruct-q4_k_m.gguf
Normal file
3
granite-7b-instruct-q4_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f609a558abbbfd5e0a00b91809dcc2e0991041e12ff2649e1f91fe6f2686c1b8
|
||||
size 4065896672
|
||||
3
granite-7b-instruct-q4_k_s.gguf
Normal file
3
granite-7b-instruct-q4_k_s.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:357733ee1546f52251589c607658462335a0fa60bbb9fcfe2622e8e4d92118f6
|
||||
size 3938298080
|
||||
3
granite-7b-instruct-q5_0.gguf
Normal file
3
granite-7b-instruct-q5_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4aa8046b73e60dd99960d49e6c7bce855184da3c2f2ef4f59b6f34b906961ae2
|
||||
size 4634330848
|
||||
3
granite-7b-instruct-q5_1.gguf
Normal file
3
granite-7b-instruct-q5_1.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a76c42735ab7af98a7d14e4d4d5f4b8bc9eca779f29f8b59365ba64700bfe614
|
||||
size 5055469280
|
||||
3
granite-7b-instruct-q5_k_m.gguf
Normal file
3
granite-7b-instruct-q5_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7644396973ffd0505040e2b1aa0497b641f83b4c0f43bccc31d28903d10136e5
|
||||
size 4818446560
|
||||
3
granite-7b-instruct-q5_k_s.gguf
Normal file
3
granite-7b-instruct-q5_k_s.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5005eba4a25d24c61494940f2fc65efee6c53fe3c69816aacb4e2ef0ca17d7f9
|
||||
size 4752713952
|
||||
3
granite-7b-instruct-q6_k_m.gguf
Normal file
3
granite-7b-instruct-q6_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7a5cd83cf156d11deaa71ef5ec370dc3ffed014bf092c0d8eb46dcf71c23b63d
|
||||
size 5529250016
|
||||
3
granite-7b-instruct-q8_0.gguf
Normal file
3
granite-7b-instruct-q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3fd3917b7fcba745046d40f2e35fefe25ef65db87f97d72beee37c5367fbc2f7
|
||||
size 7161161120
|
||||
3
granite-7b-instruct.imatrix
Normal file
3
granite-7b-instruct.imatrix
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c4e1fb5fa618c7f2137eae26e015790ca6f6286dcc607b969f5387fa5509736b
|
||||
size 4562220
|
||||
Reference in New Issue
Block a user