RichardErkhov/Diksha2001_-_VLLM-new-gguf
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: RichardErkhov/Diksha2001_-_VLLM-new-gguf
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-03 23:50:40 +08:00
commit b096438cd3
24 changed files with 191 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
.gitattributes vendored Normal file
View File

@@ -0,0 +1,57 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q3_K.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q4_K.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q5_K.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
VLLM-new.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

68
README.md Normal file
View File

@@ -0,0 +1,68 @@
Quantization made by Richard Erkhov.
[Github](https://github.com/RichardErkhov)
[Discord](https://discord.gg/pvy7H8DZMG)
[Request more models](https://github.com/RichardErkhov/quant_request)
VLLM-new - GGUF
- Model creator: https://huggingface.co/Diksha2001/
- Original model: https://huggingface.co/Diksha2001/VLLM-new/
| Name | Quant method | Size |
| ---- | ---- | ---- |
| [VLLM-new.Q2_K.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q2_K.gguf) | Q2_K | 2.96GB |
| [VLLM-new.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.IQ3_XS.gguf) | IQ3_XS | 3.28GB |
| [VLLM-new.IQ3_S.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.IQ3_S.gguf) | IQ3_S | 3.43GB |
| [VLLM-new.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q3_K_S.gguf) | Q3_K_S | 3.41GB |
| [VLLM-new.IQ3_M.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.IQ3_M.gguf) | IQ3_M | 3.52GB |
| [VLLM-new.Q3_K.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q3_K.gguf) | Q3_K | 3.74GB |
| [VLLM-new.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q3_K_M.gguf) | Q3_K_M | 3.74GB |
| [VLLM-new.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q3_K_L.gguf) | Q3_K_L | 4.03GB |
| [VLLM-new.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.IQ4_XS.gguf) | IQ4_XS | 4.18GB |
| [VLLM-new.Q4_0.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q4_0.gguf) | Q4_0 | 4.34GB |
| [VLLM-new.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.IQ4_NL.gguf) | IQ4_NL | 4.38GB |
| [VLLM-new.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q4_K_S.gguf) | Q4_K_S | 4.37GB |
| [VLLM-new.Q4_K.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q4_K.gguf) | Q4_K | 4.58GB |
| [VLLM-new.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q4_K_M.gguf) | Q4_K_M | 4.58GB |
| [VLLM-new.Q4_1.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q4_1.gguf) | Q4_1 | 4.78GB |
| [VLLM-new.Q5_0.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q5_0.gguf) | Q5_0 | 5.21GB |
| [VLLM-new.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q5_K_S.gguf) | Q5_K_S | 5.21GB |
| [VLLM-new.Q5_K.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q5_K.gguf) | Q5_K | 5.34GB |
| [VLLM-new.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q5_K_M.gguf) | Q5_K_M | 5.34GB |
| [VLLM-new.Q5_1.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q5_1.gguf) | Q5_1 | 5.65GB |
| [VLLM-new.Q6_K.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q6_K.gguf) | Q6_K | 6.14GB |
| [VLLM-new.Q8_0.gguf](https://huggingface.co/RichardErkhov/Diksha2001_-_VLLM-new-gguf/blob/main/VLLM-new.Q8_0.gguf) | Q8_0 | 7.95GB |
Original model description:
---
base_model: unsloth/Meta-Llama-3.1-8B-bnb-4bit
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- sft
license: apache-2.0
language:
- en
---
# Uploaded model
- **Developed by:** Diksha2001
- **License:** apache-2.0
- **Finetuned from model :** unsloth/Meta-Llama-3.1-8B-bnb-4bit
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)

3
VLLM-new.IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:86873f9080eb68cd7dba52612680389735b2bd114b1d7e642ab7481a1894c94c
size 3784824736

3
VLLM-new.IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8c6f295f0203d30f748cfa87c7d3221059c4d2572e1e6c4c2b77fb990ecfe0b4
size 3682326432

3
VLLM-new.IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0648f2dec952f2cdfcc50691147958d4b0392349f42046e1cd5ca4e5a4d73f18
size 3518748576

3
VLLM-new.IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5271124a53abcf711389b49cfc0c5081cdc67f5e07b546b92ff4890670ba1756
size 4707350432

3
VLLM-new.IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1d14dc6decc5abee4ccfb88a87fd48db2b3a5fe11d7e71f69a3dea2ec46de858
size 4484364192

3
VLLM-new.Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:99ecac93f8e2ce454a20d6b275a46b90f645eb2067a0690e4b4bc7eadbba2aa6
size 3179132832

3
VLLM-new.Q3_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b8238bfb010b346da2b42f33b15cf7e661aef1f4a997894138fd503d2f079342
size 4018919328

3
VLLM-new.Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8411f83551f82774bd2f23b60452d5a291fc04d65ab0d847f5cf7969bfd70c54
size 4321957792

3
VLLM-new.Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b8238bfb010b346da2b42f33b15cf7e661aef1f4a997894138fd503d2f079342
size 4018919328

3
VLLM-new.Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9f981ce3965c35a33444ae28f0365d5d065bf17ef73dbfd710a5d21637a43835
size 3664500640

3
VLLM-new.Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:71ef9a0852a2ff2a52b223b48744362e897c760070372eadb2c9733d0922dc7c
size 4661213088

3
VLLM-new.Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:66e123410969466277d5f9cc5c1d94f8ca6c36166c43c94154f010836d4aba90
size 5130254240

3
VLLM-new.Q4_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b6688dc288227b8f715aabb2dec6dbbb3932c501c083d6405d1dc65a0777d72a
size 4920735648

3
VLLM-new.Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b6688dc288227b8f715aabb2dec6dbbb3932c501c083d6405d1dc65a0777d72a
size 4920735648

3
VLLM-new.Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2a3b8f6faec5292e69d28fd90da62457ba9136db68243e71e9afcce5e307c0af
size 4692670368

3
VLLM-new.Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ce13ba136786316b9d53dfc31c0424db7fc1751d78269d6dca6587f2c2900d04
size 5599295392

3
VLLM-new.Q5_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3190d93c96e6cb8b96a87db9b882e5f57c8842d882198a89c12e4aa0f6f6504e
size 6068336544

3
VLLM-new.Q5_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5f2d04be386483acf6635c71d7a483c961eccbf7020999e0138ecd11a9fea7fe
size 5732988832

3
VLLM-new.Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5f2d04be386483acf6635c71d7a483c961eccbf7020999e0138ecd11a9fea7fe
size 5732988832

3
VLLM-new.Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3db604468376e0bf9ad76274ddf87363bf844877be6e2e187ae7f039d49e80ab
size 5599295392

3
VLLM-new.Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e6c5b11ffa9704d393a6fb10f48b4169d13bc10e4171c5466f5d8ef390793b95
size 6596007840

3
VLLM-new.Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:87067662ecdf9d5806555494f7b7ea0a02433aadfef6c053aa8be1437d487522
size 8540772256