shaowenchen/longchat-13b-16k-gguf
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: shaowenchen/longchat-13b-16k-gguf
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-16 01:02:38 +08:00
commit 80e9c175f6
18 changed files with 155 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
*.gguf filter=lfs diff=lfs merge=lfs -text

64
README.md Normal file
View File

@@ -0,0 +1,64 @@
---
inference: true
language:
- zh
- en
license: other
model_creator: lmsys
model_link: https://huggingface.co/lmsys/longchat-13b-16k
model_name: longchat-13b-16k
model_type: longchat
pipeline_tag: text-generation
quantized_by: shaowenchen
tasks:
- text2text-generation
tags:
- gguf
- longchat
- 13b
- 16k
---
## Provided files
| Name | Quant method | Size |
| ---------------------------- | ------------ | ------ |
| longchat-13b-16k.Q2_K.gguf | Q2_K | 5.1 GB |
| longchat-13b-16k.Q3_K.gguf | Q3_K | 5.9 GB |
| longchat-13b-16k.Q3_K_L.gguf | Q3_K_L | 6.5 GB |
| longchat-13b-16k.Q3_K_S.gguf | Q3_K_S | 5.3 GB |
| longchat-13b-16k.Q4_0.gguf | Q4_0 | 6.9 GB |
| longchat-13b-16k.Q4_1.gguf | Q4_1 | 7.6 GB |
| longchat-13b-16k.Q4_K.gguf | Q4_K | 7.3 GB |
| longchat-13b-16k.Q4_K_S.gguf | Q4_K_S | 6.9 GB |
| longchat-13b-16k.Q5_0.gguf | Q5_0 | 8.4 GB |
| longchat-13b-16k.Q5_1.gguf | Q5_1 | 9.1 GB |
| longchat-13b-16k.Q5_K.gguf | Q5_K | 8.6 GB |
| longchat-13b-16k.Q5_K_S.gguf | Q5_K_S | 8.4 GB |
| longchat-13b-16k.Q6_K.gguf | Q6_K | 9.9 GB |
| longchat-13b-16k.Q8_0.gguf | Q8_0 | 13 GB |
| longchat-13b-16k.gguf | full | 24 GB |
Usage:
```
docker run --rm -it -p 8000:8000 -v /path/to/models:/models -e MODEL=/models/gguf-model-name.gguf hubimage/llama-cpp-python:latest
```
and you can view http://localhost:8000/docs to see the swagger UI.
## Provided images
| Name | Quant method | Compressed Size |
| ---------------------------------------- | ------------ | --------------- |
| `shaowenchen/longchat-13b-16k-gguf:Q2_K` | Q2_K | 7.47 GB |
| `shaowenchen/longchat-13b-16k-gguf:Q3_K` | Q3_K | 6.11 GB |
| `shaowenchen/longchat-13b-16k-gguf:Q4_K` | Q4_K | 5.29 GB |
Usage:
```
docker run --rm -p 8000:8000 shaowenchen/longchat-13b-16k-gguf:Q2_K
```
and you can view http://localhost:8000/docs to see the swagger UI.

10
configuration.json Normal file
View File

@@ -0,0 +1,10 @@
{
"framework": "pytorch",
"task": "text-generation",
"model": {
"type": "longchat"
},
"pipeline": {
"type": "longchat-13b-16k-text-generation-pipe"
}
}

3
longchat-13b-16k.Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3779d4febe1fd4e5121f5430529d6169537a9dee1596d2dcae34e5e5a3427c96
size 5429348256

3
longchat-13b-16k.Q3_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ee53e00448d89752aa49148bd57b858eff261ed434118ca7727ca51a68a773e7
size 6337769376

3
longchat-13b-16k.Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:04a08d751fea3204d5c6cfecd723044f2075a8744614f4d16135006417b433d4
size 6929559456

3
longchat-13b-16k.Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7e4b48f519062148ace72931413ee42718ff0e68a582091158607459a6cc77fb
size 5658980256

3
longchat-13b-16k.Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:962582bdae575684c7582f4a09b56aea457d35311951a6d3b87d921857f39610
size 7365834656

3
longchat-13b-16k.Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:948ae04d504e19f404e5546700a6201b991c365a6d7f9d902338fc174ba52859
size 8169060256

3
longchat-13b-16k.Q4_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5a24f1285c68523d46d7294f3195fb6822df6b7f711e91416b4efe7ed5edb168
size 7865956256

3
longchat-13b-16k.Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:13c6f9abdf28a325cf6847defb2cf9a9f3861b1c543f2436bc88fdac8d3865f3
size 7414331296

3
longchat-13b-16k.Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:75ed3167dba3650de6e674f93a5a7eb967deaff12876aaf4f2d714e614809114
size 8972285856

3
longchat-13b-16k.Q5_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:415b20beb5f892ae5693c60d860bf1ab0135fa8c7b13aeaa0e22b07d3d576b97
size 9775511456

3
longchat-13b-16k.Q5_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a9f08ddee15f174e6c4b6a23c3a7ea5b0b30cff96f589ae48e289b04575f9d3f
size 9229924256

3
longchat-13b-16k.Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3305af3855cd22764bb8e7762efc347d2c370441d9bcb2695fc219363def6eff
size 8972285856

3
longchat-13b-16k.Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8886457d379326cd9f0d2a061dcaba360ea7869a2ee78564985f004e28799c5d
size 10679140256

3
longchat-13b-16k.Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ddb66e400abdf588d08d16c259de526008b763311b11e8feb65b3d3d5e2ac1e5
size 13831319456

3
longchat-13b-16k.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3aa1b6a5d9da12433aadc247d92d112319af8f12c4df6ad0b6f439a969ed8e3a
size 26033303424