mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-08 13:56:14 +08:00
commit 434c5bcc4d
24 changed files with 205 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
.gitattributes vendored Normal file
View File

@@ -0,0 +1,57 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
TowerInstruct-Mistral-7B-v0.2.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text

82
README.md Normal file
View File

@@ -0,0 +1,82 @@
---
base_model: Unbabel/TowerInstruct-Mistral-7B-v0.2
language:
- en
- de
- fr
- zh
- pt
- nl
- ru
- ko
- it
- es
library_name: transformers
license: cc-by-nc-4.0
quantized_by: mradermacher
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
weighted/imatrix quants of https://huggingface.co/Unbabel/TowerInstruct-Mistral-7B-v0.2
<!-- provided-files -->
static quants are available at https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-IQ1_S.gguf) | i1-IQ1_S | 1.7 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-IQ1_M.gguf) | i1-IQ1_M | 1.9 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.1 | |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.3 | |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-IQ2_S.gguf) | i1-IQ2_S | 2.4 | |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-IQ2_M.gguf) | i1-IQ2_M | 2.6 | |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-Q2_K.gguf) | i1-Q2_K | 2.8 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 2.9 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.1 | |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.3 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-IQ3_S.gguf) | i1-IQ3_S | 3.3 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-IQ3_M.gguf) | i1-IQ3_M | 3.4 | |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-Q3_K_M.gguf) | i1-Q3_K_M | 3.6 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-Q3_K_L.gguf) | i1-Q3_K_L | 3.9 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.0 | |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-Q4_0.gguf) | i1-Q4_0 | 4.2 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.2 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-Q4_K_M.gguf) | i1-Q4_K_M | 4.5 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.1 | |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.2 | |
| [GGUF](https://huggingface.co/mradermacher/TowerInstruct-Mistral-7B-v0.2-i1-GGUF/resolve/main/TowerInstruct-Mistral-7B-v0.2.i1-Q6_K.gguf) | i1-Q6_K | 6.0 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
TowerInstruct-Mistral-7B-v0.2.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cf2111b2d977071e8ec0d153188ef3118e42d2e6aa6903fddcaaec44a453cee2
size 1754455424

3
TowerInstruct-Mistral-7B-v0.2.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:057fe5798209269ee90f42815c99493dca0643f83530d943975b67bfee110698
size 1612111232

3
TowerInstruct-Mistral-7B-v0.2.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:006217d1027abde62b23a436561be64069803759a6ae3d3268580b43ab0142be
size 2500722880

3
TowerInstruct-Mistral-7B-v0.2.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:24d08fc8afbb13b73fce81a36fe6886f5e1d2228af751e3a3a0fbb85d0085abc
size 2310930624

3
TowerInstruct-Mistral-7B-v0.2.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bdbad1734ffa0726bf3c258225a61777d502c943800f493e81f38ea3b7ffb527
size 2198265216

3
TowerInstruct-Mistral-7B-v0.2.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a9a9434ae7f0e5e8e4eb57ce1252204193a77dabba820a70c2c6c359571c7283
size 1991695744

3
TowerInstruct-Mistral-7B-v0.2.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f34eb5c7ae69cc63c9480afd42ca0c51543895b924530c7216ed83de4790a5d0
size 3284903168

3
TowerInstruct-Mistral-7B-v0.2.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:76d754356499d2687b3a5690d1c0f362ee609d612f17794dfcdb359d025e907d
size 3182404864

3
TowerInstruct-Mistral-7B-v0.2.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1daf56ecc29cbf9e71959e20fe2056eb318cf99465cb28e6afca3c9a9bb24fea
size 3018827008

3
TowerInstruct-Mistral-7B-v0.2.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:560c940d109d191ffa18c8dfb9fceb62fe3cfa98fbd53f92903d978be00f2856
size 2827354304

3
TowerInstruct-Mistral-7B-v0.2.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ca9267c3023aabdcbb38fc5101cbcbff2d2b376ae74ac286eb2f2d4b6b49434d
size 3907700800

3
TowerInstruct-Mistral-7B-v0.2.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4443a4e597361ac780a5e300ad75a6ef29baea9734996d258f0c7c110c2fa2c0
size 2719252928

3
TowerInstruct-Mistral-7B-v0.2.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f7e1fb60b34adfc6cee8f6f112848386c958cc56b383ba2079ebdb9d943cf19a
size 3822036224

3
TowerInstruct-Mistral-7B-v0.2.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ed49e01317c389a70b873a8244db7ccd1b7a0042eacb6e646e4413c51c297e14
size 3518997760

3
TowerInstruct-Mistral-7B-v0.2.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c81b297b573e466e7936e35f1a53585360d5892aa76feda480d340214ebcb4af
size 3164579072

3
TowerInstruct-Mistral-7B-v0.2.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:28deb9bc9d1d707d2d4c144ca5177e6ec3e50dce24fadbe61ec9a7fa464cb080
size 4123609408

3
TowerInstruct-Mistral-7B-v0.2.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b35a89879f304e18eb701ccabe9b74ef7be4b13efad848ac98c77150af95da39
size 4368451904

3
TowerInstruct-Mistral-7B-v0.2.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:51685cba949b23c57e909c93c484686c31f44c652a08a8f35f5c9fbe810ebe60
size 4140386624

3
TowerInstruct-Mistral-7B-v0.2.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b772b588b9fc440555a61b64985868f3d75f74fe134278601472b1741bc93250
size 5131423040

3
TowerInstruct-Mistral-7B-v0.2.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cbf54a2a72930617d50b45a4e71904d1a23f54975e2375d1d0e285a4adf1a4bd
size 4997729600

3
TowerInstruct-Mistral-7B-v0.2.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c93e55b7a2054b2535ad058239135066c02b7093d6af700ee49e45d53f55420f
size 5942079872

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3bd6671aa27bbe1a66dbcca8413d2ebeabdff31fa774decbae7a898e8a05cbbd
size 4988157