mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-12 05:09:28 +08:00
commit 80aa0bc8b1
24 changed files with 207 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
.gitattributes vendored Normal file
View File

@@ -0,0 +1,57 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-8x7b-v0.1.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text

3
Qwen1.5-8x7b-v0.1.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:94b9856195eaa61f6b783bcd304def57f07fb995ee5d53bcc08e664da54852e0
size 9466504736

3
Qwen1.5-8x7b-v0.1.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8abd85478778454ed7c57c81d201bc51747c907a84870681993e0b44756d11b2
size 8676140576

3
Qwen1.5-8x7b-v0.1.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9a84791a9241d2b115671b8d0aff96785095b44ac88781b5aaf6d5621a6b9fa3
size 13101159968

3
Qwen1.5-8x7b-v0.1.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5bf3945bc8142c0a432a140b3033a65474ec123e341f823bc97255366e6e0a63
size 12047341088

3
Qwen1.5-8x7b-v0.1.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:83eb4622daf094efeffae7201f35ffe7d60a9ebcb584497575bae4ac5788f021
size 11837597216

3
Qwen1.5-8x7b-v0.1.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2b6b0b532a0169dcc3ac094884bb522f01dc5b8fcf5ccea889f55950ffb87d56
size 10783778336

3
Qwen1.5-8x7b-v0.1.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6ffd6c4d10a13ba04ed985ce258fecce4bc8079be3e993f09ff24086abc5328d
size 18175559200

3
Qwen1.5-8x7b-v0.1.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:103ea939ab17bb95c2462ceb0cedd1c577978e06744d01794163fb81ad8852de
size 17409050144

3
Qwen1.5-8x7b-v0.1.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7053a32d6cd519ca339a9f47cd587c53e1141df7a395eac6637b8d8e946eb19d
size 16572286496

3
Qwen1.5-8x7b-v0.1.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1fdfa5643c43912217d49494ce9894c2f705c6efd22054bd379b0bcf5b49cc30
size 15644481056

3
Qwen1.5-8x7b-v0.1.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a5b51d1e3de517a47c2718541b45b3a02d340e524c39c778ee3bcd2984cded1f
size 21043705376

3
Qwen1.5-8x7b-v0.1.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:02ebcae0601dc64958772c54997a51ceea5dc82c545ca1536c9358283bce3ef1
size 14946702880

3
Qwen1.5-8x7b-v0.1.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c0e5fab8afec10e048b14e0a32fa8566dab9eafa28a604ee9481180c1042adfb
size 20246496800

3
Qwen1.5-8x7b-v0.1.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8191f96cb0b8b7f2f5d4fba9e936f8193ad89a574f2788c76ade8b463791aa66
size 19032245792

3
Qwen1.5-8x7b-v0.1.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:08d4cf15d3beed9e253882e7d1b8156d39d60423ee7496e9d8023cb66bb0cbc5
size 17409050144

3
Qwen1.5-8x7b-v0.1.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1dc9d40c9556bac0349b2801bb56eec8cd6d8bca04b4a6f5dbb217e949377ac1
size 22185129504

3
Qwen1.5-8x7b-v0.1.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c7b2ac946b3b7ad4355ab0ccf2ed7f000c9275aaaa6a748b4c55571db151cb1d
size 23649990176

3
Qwen1.5-8x7b-v0.1.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:831ed3f0883a43484b7545abd2d8479df5928cfbe17b2eec8630991c7b484f00
size 22342415904

3
Qwen1.5-8x7b-v0.1.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:84e03447bbd5c60b1c21358d834a392a8ff0404da227f96c6ed917b490957a2c
size 27401991712

3
Qwen1.5-8x7b-v0.1.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:722a4efdc9ae1596a9a8bf155eb77585c256efa8786959e0ae5790b4df5fb2df
size 26635482656

3
Qwen1.5-8x7b-v0.1.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dd2f5384d4322bf346dc82adf7400396db076c1dd62d9bdaa8c70c614bebd2a8
size 31459796512

84
README.md Normal file
View File

@@ -0,0 +1,84 @@
---
base_model: MaziyarPanahi/Qwen1.5-8x7b-v0.1
datasets:
- Crystalcareai/MoD-150k
language:
- en
library_name: transformers
license: other
license_link: https://huggingface.co/Qwen/Qwen1.5-72B-Chat/blob/main/LICENSE
license_name: tongyi-qianwen
quantized_by: mradermacher
tags:
- axolotl
- generated_from_trainer
- moe
- qwen
- mixtral
- text-generation-inference
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
weighted/imatrix quants of https://huggingface.co/MaziyarPanahi/Qwen1.5-8x7b-v0.1
<!-- provided-files -->
static quants are available at https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-IQ1_S.gguf) | i1-IQ1_S | 8.8 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-IQ1_M.gguf) | i1-IQ1_M | 9.6 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 10.9 | |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-IQ2_XS.gguf) | i1-IQ2_XS | 11.9 | |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-IQ2_S.gguf) | i1-IQ2_S | 12.1 | |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-IQ2_M.gguf) | i1-IQ2_M | 13.2 | |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-Q2_K.gguf) | i1-Q2_K | 15.0 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 15.7 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-IQ3_XS.gguf) | i1-IQ3_XS | 16.7 | |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-IQ3_S.gguf) | i1-IQ3_S | 17.5 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-Q3_K_S.gguf) | i1-Q3_K_S | 17.5 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-IQ3_M.gguf) | i1-IQ3_M | 18.3 | |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-Q3_K_M.gguf) | i1-Q3_K_M | 19.1 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-Q3_K_L.gguf) | i1-Q3_K_L | 20.3 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-IQ4_XS.gguf) | i1-IQ4_XS | 21.1 | |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-Q4_0.gguf) | i1-Q4_0 | 22.3 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-Q4_K_S.gguf) | i1-Q4_K_S | 22.4 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-Q4_K_M.gguf) | i1-Q4_K_M | 23.7 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-Q5_K_S.gguf) | i1-Q5_K_S | 26.7 | |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-Q5_K_M.gguf) | i1-Q5_K_M | 27.5 | |
| [GGUF](https://huggingface.co/mradermacher/Qwen1.5-8x7b-v0.1-i1-GGUF/resolve/main/Qwen1.5-8x7b-v0.1.i1-Q6_K.gguf) | i1-Q6_K | 31.6 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:551c6021fdeacd77676ebf5f4beef74ca15f0971028799a680cdd9a8618592ad
size 22291347