liodon-ai/Qwen2.5-14B-Instruct-imatrix-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: liodon-ai/Qwen2.5-14B-Instruct-imatrix-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-08 00:57:18 +08:00
commit 2b3069120d
9 changed files with 128 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

42
.gitattributes vendored Normal file
View File

@@ -0,0 +1,42 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
Qwen2.5-14B-Instruct-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-14B-Instruct-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-14B-Instruct-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-14B-Instruct-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-14B-Instruct-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-14B-Instruct-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-14B-Instruct-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

3
Qwen2.5-14B-Instruct-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:04af879d78fc42b62f06854647021c0e7f2d474d2e38f0be464573e7a1e17ec3
size 5356147008

3
Qwen2.5-14B-Instruct-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6aacb37697e0370f1cfd447f653c02921b17f833b75e3dc000c42079d09c4713
size 6916538688

3
Qwen2.5-14B-Instruct-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4739d07b80d03108419ca74b45767b635452a3278deb53658f603f2b227a8a6a
size 8119841088

3
Qwen2.5-14B-Instruct-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:08a8f61d9e6dcb4bba08514ba2b01592371fc0ae7908142046afabe700c161c3
size 8988111168

3
Qwen2.5-14B-Instruct-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0b6fe2eb537be156685d1e432ad3065042cf0ac101644f192ea76e383fa6ff84
size 10508874048

3
Qwen2.5-14B-Instruct-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0c6fa1e1401edc061dff92ebf078ac40c1043ce907df7fd8718a9256529d9575
size 12124684608

3
Qwen2.5-14B-Instruct-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:86ed159e481445c6cd6e7bcf537164a49d698fef5c712e6e9ecaaf62e0215f75
size 15701598528

65
README.md Normal file
View File

@@ -0,0 +1,65 @@
---
license: other
base_model: unsloth/Qwen2.5-14B-Instruct
pipeline_tag: text-generation
tags:
- gguf
- local-llm
- llama.cpp
- lm-studio
- quantized
- imatrix
- sub-4-bit
- qwen2
- base_model:Qwen/Qwen2.5-14B-Instruct
---
# Qwen2.5-14B-Instruct — iMatrix GGUF
GGUF imatrix quantizations of [unsloth/Qwen2.5-14B-Instruct](https://huggingface.co/unsloth/Qwen2.5-14B-Instruct), published by [Liodon AI](https://huggingface.co/liodon-ai).
## Quick Start
**llama.cpp**
```bash
llama-cli -hf liodon-ai/Qwen2.5-14B-Instruct-imatrix-GGUF:Q4_K_M
```
**Ollama**
```bash
ollama run hf.co/liodon-ai/Qwen2.5-14B-Instruct-imatrix-GGUF:Q4_K_M
```
**LM Studio / Jan** — search `liodon-ai/Qwen2.5-14B-Instruct-imatrix-GGUF` and pick your quant.
## Quants
| Quant | Size | VRAM est. | Notes |
|-------|------|-----------|-------|
| `IQ2_M` | 5.36 GB | ~6 GB | 2-bit, iMatrix — smallest usable |
| `IQ3_M` | 6.92 GB | ~8 GB | 3-bit, iMatrix — great quality/size tradeoff |
| `IQ4_XS` | 8.12 GB | ~9 GB | 4-bit extra-small, iMatrix |
| `Q4_K_M` | 8.99 GB | ~10 GB | 4-bit, iMatrix-calibrated (recommended) |
| `Q5_K_M` | 10.51 GB | ~12 GB | 5-bit, iMatrix-calibrated |
| `Q6_K` | 12.12 GB | ~14 GB | 6-bit, iMatrix-calibrated, near-lossless |
| `Q8_0` | 15.70 GB | ~18 GB | 8-bit, essentially lossless |
## What is iMatrix?
Standard quantization treats all weights equally. iMatrix runs 128 calibration chunks through
the full-precision model to find which weights matter most, then allocates more precision where
it counts. At Q2/Q3/Q4 this means noticeably better coherence and instruction-following —
**same file size, better output**.
Calibration: 2M tokens of [WikiText-103](https://huggingface.co/datasets/wikitext).
> Also see plain (non-iMatrix) quants: `liodon-ai/Qwen2.5-14B-Instruct-GGUF`
## Source
- **Model**: [unsloth/Qwen2.5-14B-Instruct](https://huggingface.co/unsloth/Qwen2.5-14B-Instruct)
- **License**: other
---
*Quantized by [Liodon AI](https://huggingface.co/liodon-ai)*