RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-15 18:40:16 +08:00
commit ef25c36ce3
24 changed files with 190 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
.gitattributes vendored Normal file
View File

@@ -0,0 +1,57 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_K.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_K.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

67
README.md Normal file
View File

@@ -0,0 +1,67 @@
Quantization made by Richard Erkhov.
[Github](https://github.com/RichardErkhov)
[Discord](https://discord.gg/pvy7H8DZMG)
[Request more models](https://github.com/RichardErkhov/quant_request)
qwen2.5-7B-Instruct-v0.2.9-v0.3 - GGUF
- Model creator: https://huggingface.co/RTX3090x2/
- Original model: https://huggingface.co/RTX3090x2/qwen2.5-7B-Instruct-v0.2.9-v0.3/
| Name | Quant method | Size |
| ---- | ---- | ---- |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q2_K.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q2_K.gguf) | Q2_K | 2.81GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ3_XS.gguf) | IQ3_XS | 3.12GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ3_S.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ3_S.gguf) | IQ3_S | 3.26GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K_S.gguf) | Q3_K_S | 3.25GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ3_M.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ3_M.gguf) | IQ3_M | 3.33GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K.gguf) | Q3_K | 3.55GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K_M.gguf) | Q3_K_M | 3.55GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K_L.gguf) | Q3_K_L | 3.81GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ4_XS.gguf) | IQ4_XS | 3.96GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_0.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_0.gguf) | Q4_0 | 4.13GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ4_NL.gguf) | IQ4_NL | 4.16GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_K_S.gguf) | Q4_K_S | 4.15GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_K.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_K.gguf) | Q4_K | 4.36GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_K_M.gguf) | Q4_K_M | 4.36GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_1.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_1.gguf) | Q4_1 | 4.54GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_0.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_0.gguf) | Q5_0 | 4.95GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_K_S.gguf) | Q5_K_S | 4.95GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_K.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_K.gguf) | Q5_K | 5.07GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_K_M.gguf) | Q5_K_M | 5.07GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_1.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_1.gguf) | Q5_1 | 5.36GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q6_K.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q6_K.gguf) | Q6_K | 5.82GB |
| [qwen2.5-7B-Instruct-v0.2.9-v0.3.Q8_0.gguf](https://huggingface.co/RichardErkhov/RTX3090x2_-_qwen2.5-7B-Instruct-v0.2.9-v0.3-gguf/blob/main/qwen2.5-7B-Instruct-v0.2.9-v0.3.Q8_0.gguf) | Q8_0 | 7.54GB |
Original model description:
---
base_model: unsloth/qwen2.5-7b-instruct-bnb-4bit
language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- qwen2
- trl
---
# Uploaded model
- **Developed by:** RTX3090x2
- **License:** apache-2.0
- **Finetuned from model :** unsloth/qwen2.5-7b-instruct-bnb-4bit
This qwen2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:95efbc0fe866dc2a6e031b50cbdf99824b3f4e0cd7ba4a08b5849f572f4544c6
size 3574012416

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bed469d2d0bacecbcd1ba0a14dd3bf6e82499db01eebae91faff4f1e9355caff
size 3499192832

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9cba8c51ea1cbee059385ad7431437b6b4bba7dd95e344995c49100b57f6a89b
size 3346256384

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ab111b9f9efc9dd7e0efd9ddaafdca341ef9d7c75bf906aadbf3d40cdbaff939
size 4463274496

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d449908982f7f033b9100e9f71febc05085cbffdbb168ede568ec6461b40cedf
size 4250298880

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e0eda85d1eb11d1a56dbd18a2e655a1725af64ffd8b9476e9d54fa3859e5360f
size 3015940608

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:875d4d0d6d51b749330d8c74d2cafe0bcaaf02b94d141cdf3c32ef3aa9ed1ee0
size 3808391680

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:29f06b1d01b94b66d6f6b8e552a7cda89dcf8522acd10b0bc763f783fad02d89
size 4088459776

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:875d4d0d6d51b749330d8c74d2cafe0bcaaf02b94d141cdf3c32ef3aa9ed1ee0
size 3808391680

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:62f913f2dcbb198e499669442a0c24646f9924087beccab79735d82d0f717659
size 3492368896

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d8bb61447f4c8412f3e8395f8df7eeaf89a762943569b4d76a0fb468fb56e4e6
size 4431391232

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0661bc6e2f9a73b96b270e5b3f5b5561c035dfa927bce7ddfebd0ca0f43434fa
size 4873284096

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a299ebdca4bae8c8f5525688f0340cc9063d7ab9bc7daa173e65f940f0912f9e
size 4683074048

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a299ebdca4bae8c8f5525688f0340cc9063d7ab9bc7daa173e65f940f0912f9e
size 4683074048

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2075e5bd9cb75e353394466212ade9711fb2ea7d89caa7cd5c81fd06956cedaf
size 4457769472

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:adeabf120c959b3fb1126db4e575219f6f1a12906160c036b63b543cb1433582
size 5315176960

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:463bb60d1c183a3177cb5c07cc5154c7fd38291ae3c29c8a4d603521b9bf02bb
size 5757069824

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a0d4832e7e86ea85fd23bc9e2de33bb3a808363f4375f4368f5e3103b2fd8950
size 5444831744

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a0d4832e7e86ea85fd23bc9e2de33bb3a808363f4375f4368f5e3103b2fd8950
size 5444831744

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3a32fc62b4ab31a916079b66634cf0a0212a0d2de18713dc4df096843fc57bca
size 5315176960

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5cd87e88a1c5218041031fbba575f203fcb06efc17c0d186dcf87850ec8e5e2b
size 6254199296

3
qwen2.5-7B-Instruct-v0.2.9-v0.3.Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:604f956a1cee92d623087b6b9b962e6c5dee77b256af9b3e46aa7750c56d6db5
size 8098525696