shaowenchen/vicuna-7b-v1.5-16k-gguf
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: shaowenchen/vicuna-7b-v1.5-16k-gguf
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-27 23:29:07 +08:00
commit 6f3761a983
18 changed files with 154 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
*.gguf filter=lfs diff=lfs merge=lfs -text

63
README.md Normal file
View File

@@ -0,0 +1,63 @@
---
inference: false
language:
- zh
- en
license: other
model_creator: lmsys
model_link: https://huggingface.co/lmsys/vicuna-7b-v1.5-16k
model_name: chinese-llama-2-7b-16k
model_type: vicuna
pipeline_tag: text-generation
quantized_by: shaowenchen
tasks:
- text2text-generation
tags:
- gguf
- vicuna
- chinese
---
## Provided files
| Name | Quant method | Size |
| ------------------------------ | ------------ | ------ |
| vicuna-7b-v1.5-16k.Q2_K.gguf | Q2_K | 2.6 GB |
| vicuna-7b-v1.5-16k.Q3_K.gguf | Q3_K | 3.1 GB |
| vicuna-7b-v1.5-16k.Q3_K_L.gguf | Q3_K_L | 3.3 GB |
| vicuna-7b-v1.5-16k.Q3_K_S.gguf | Q3_K_S | 2.7 GB |
| vicuna-7b-v1.5-16k.Q4_0.gguf | Q4_0 | 3.6 GB |
| vicuna-7b-v1.5-16k.Q4_1.gguf | Q4_1 | 3.9 GB |
| vicuna-7b-v1.5-16k.Q4_K.gguf | Q4_K | 3.8 GB |
| vicuna-7b-v1.5-16k.Q4_K_S.gguf | Q4_K_S | 3.6 GB |
| vicuna-7b-v1.5-16k.Q5_0.gguf | Q5_0 | 4.3 GB |
| vicuna-7b-v1.5-16k.Q5_1.gguf | Q5_1 | 4.7 GB |
| vicuna-7b-v1.5-16k.Q5_K.gguf | Q5_K | 4.5 GB |
| vicuna-7b-v1.5-16k.Q5_K_S.gguf | Q5_K_S | 4.3 GB |
| vicuna-7b-v1.5-16k.Q6_K.gguf | Q6_K | 5.1 GB |
| vicuna-7b-v1.5-16k.Q8_0.gguf | Q8_0 | 6.7 GB |
| vicuna-7b-v1.5-16k.gguf | full | 13 GB |
Usage:
```
docker run --rm -it -p 8000:8000 -v /path/to/models:/models -e MODEL=/models/gguf-model-name.gguf hubimage/llama-cpp-python:latest
```
and you can view http://localhost:8000/docs to see the swagger UI.
## Provided images
| Name | Quant method | Compressed Size |
| ------------------------------------------ | ------------ | --------------- |
| `shaowenchen/vicuna-7b-v1.5-16k-gguf:Q2_K` | Q2_K | 2.88 GB |
| `shaowenchen/vicuna-7b-v1.5-16k-gguf:Q3_K` | Q3_K | 3.3 GB |
| `shaowenchen/vicuna-7b-v1.5-16k-gguf:Q4_K` | Q4_K | 4 GB |
Usage:
```
docker run --rm -p 8000:8000 shaowenchen/vicuna-7b-v1.5-16k-gguf:Q2_K
```
and you can view http://localhost:8000/docs to see the swagger UI.

10
configuration.json Normal file
View File

@@ -0,0 +1,10 @@
{
"framework": "pytorch",
"task": "text-generation",
"model": {
"type": "vicuna"
},
"pipeline": {
"type": "vicuna-7b-v1.5-16k-text-generation-pipe"
}
}

3
vicuna-7b-v1.5-16k.Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8b13c04f8095c6c9fe945ac1ec3e4a4f605b0c98bcd7c9dc0e997036b2cd6073
size 2825940704

3
vicuna-7b-v1.5-16k.Q3_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f2bdb2e06aaf2635c1f7a0457f2f99aede734d6ddcd2506df6e9fc15f78dfff4
size 3298004704

3
vicuna-7b-v1.5-16k.Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0dc20551372ee90f42442a0604cb766799bd3af5b3199adfb5080b7496db4947
size 3597111008

3
vicuna-7b-v1.5-16k.Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bc4a5f3f8277fd7933b41140cd4519cdc4b7248fcaeccef0cfb0e1c6373b327c
size 2948304608

3
vicuna-7b-v1.5-16k.Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:337720869e48526ec8118313e9aceb6e7fa23f629455c189c1268a4945cbf69b
size 3825807072

3
vicuna-7b-v1.5-16k.Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:789e4d522ed33e7a0139ff2e6baaedf03c8203be9fa3631c27bb044fa35a58e5
size 4238749408

3
vicuna-7b-v1.5-16k.Q4_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:706312090f71d94c2a04f2d7d2a238f2b3562618283090e26208bd20645ced84
size 4081004256

3
vicuna-7b-v1.5-16k.Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:01599b4a39884e1c567c799333c13c659d4f413f3644f1c6273704b30c7647fa
size 3856740064

3
vicuna-7b-v1.5-16k.Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f89de37f851ccbf526acb0d38208ffe4a224b13f9d8619af7930fdbc88617594
size 4651691744

3
vicuna-7b-v1.5-16k.Q5_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5e514d3cc341f2d7e61be7ede104124be28fbd8ec73dd9d6f67c1e58fc296713
size 5064634080

3
vicuna-7b-v1.5-16k.Q5_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:13a887af2dc34a7c5d202259a34e331d2b1940153d90730d70f7f42fcf5d2e13
size 4783156960

3
vicuna-7b-v1.5-16k.Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8dcff54f03bd16281eab102d2f61574ada8c3b2571dd5d55790f611e8277ae21
size 4651691744

3
vicuna-7b-v1.5-16k.Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7aca36b256139106610406ce10ef19f89a1fc9cfc8130a2913f07417ce81a7db
size 5529194208

3
vicuna-7b-v1.5-16k.Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:36d9c3a5b69f66e62c3352e76da7c05b54dbed6201736183f1042e51d103fa02
size 7161089760

3
vicuna-7b-v1.5-16k.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b3bfe54b24d4014acaabfa58c3b443a0c265a78a3bffeb49a829b2f04b4a718f
size 13478104768