mradermacher/RM-NLHF-Qwen-32B-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/RM-NLHF-Qwen-32B-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-29 09:08:17 +08:00
commit 97cde2a4f1
26 changed files with 215 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

59
.gitattributes vendored Normal file
View File

@@ -0,0 +1,59 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
RM-NLHF-Qwen-32B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text

84
README.md Normal file
View File

@@ -0,0 +1,84 @@
---
base_model: Tongyi-ConvAI/RM-NLHF-Qwen-32B
language:
- en
library_name: transformers
license: mit
mradermacher:
readme_rev: 1
quantized_by: mradermacher
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
<!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S -->
<!-- ### quants_skip: -->
<!-- ### skip_mmproj: -->
weighted/imatrix quants of https://huggingface.co/Tongyi-ConvAI/RM-NLHF-Qwen-32B
<!-- provided-files -->
***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#RM-NLHF-Qwen-32B-i1-GGUF).***
static quants are available at https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-IQ1_S.gguf) | i1-IQ1_S | 7.4 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-IQ1_M.gguf) | i1-IQ1_M | 8.0 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 9.1 | |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 10.1 | |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-IQ2_S.gguf) | i1-IQ2_S | 10.5 | |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-IQ2_M.gguf) | i1-IQ2_M | 11.4 | |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 11.6 | very low quality |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-Q2_K.gguf) | i1-Q2_K | 12.4 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 12.9 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 13.8 | |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 14.5 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-IQ3_S.gguf) | i1-IQ3_S | 14.5 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-IQ3_M.gguf) | i1-IQ3_M | 14.9 | |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 16.0 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 17.3 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 17.8 | |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-Q4_0.gguf) | i1-Q4_0 | 18.8 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 18.9 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 20.0 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-Q4_1.gguf) | i1-Q4_1 | 20.7 | |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 22.7 | |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 23.4 | |
| [GGUF](https://huggingface.co/mradermacher/RM-NLHF-Qwen-32B-i1-GGUF/resolve/main/RM-NLHF-Qwen-32B.i1-Q6_K.gguf) | i1-Q6_K | 27.0 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
RM-NLHF-Qwen-32B.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4db3dabe7eb7115655907911e540b3046cd34852be71a1c58a278e8af8d9a5e7
size 7932160704

3
RM-NLHF-Qwen-32B.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:31d040e11543a2268e5a8ad80684d63a7ff5cd06f8eab12892363d40b9c21189
size 7274506944

3
RM-NLHF-Qwen-32B.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b183ea5c907a30cf36b9526d0856e21aad6e8db755ca06b62c9a5c1629543fa2
size 11264441024

3
RM-NLHF-Qwen-32B.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dbdd268cf7619befaf0460b4b7c9215f06ec95fa8bd931eb250d670384a07386
size 10387569344

3
RM-NLHF-Qwen-32B.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1b4e5fa985dd3d414e24c066393a3907f07c6dc7299c4b6f340ce88f98cbb73c
size 9957550784

3
RM-NLHF-Qwen-32B.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:62948226a16a729e599ff38012f12a79e3ed3fe35ac82418ed011eebc2874227
size 9028250304

3
RM-NLHF-Qwen-32B.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:720549f2339a441f11cf5adba59ff6c3323179eaf1e313b424b2f63df9c7e6eb
size 14810122944

3
RM-NLHF-Qwen-32B.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:97afefcea990c1e32d416d4db56b2db19cee7134d5712e96369ebdfb17962a32
size 14436895424

3
RM-NLHF-Qwen-32B.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8c06d48e63b83db8660d5d43d4cca3f362019710b1a1819d4f895b6373b8c144
size 13705513664

3
RM-NLHF-Qwen-32B.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1284c089e64432e9f6c301130d0ca2b84df8f0ff34acf79abf1c094113aae470
size 12839271104

3
RM-NLHF-Qwen-32B.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dca919fa6027b0a778a0ecad17945f40d69b64ee5dbaa91e76af053734e61f5c
size 17693153984

3
RM-NLHF-Qwen-32B.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:35ada61d4da01fec1bc2bf7bff28a5ee25a14e68ffeaf7c5501944357415ef23
size 12313098944

3
RM-NLHF-Qwen-32B.i1-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4e1c8e676f67afd7adee9c1c1aa618b3be16e9902f9dedc25768d0b66c842ac7
size 11488000704

3
RM-NLHF-Qwen-32B.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a2b1f7e95867fdb112dc87b443b5c020c8f0429d6e3282bd97f4ef9a746123b7
size 17247079104

3
RM-NLHF-Qwen-32B.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:73fcb5a4430f81fda84e727e58dc7230893ccf4202fdb2b891551b8f8de0a7bf
size 15935048384

3
RM-NLHF-Qwen-32B.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:33ecb2aac9dc1acfe252ac19bbae68ca29726436c601472d703b54f55dca2163
size 14392330944

3
RM-NLHF-Qwen-32B.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8ec2b2bd1c14bc14d59e9679ff5002f2d4a40c35925c5ea266ac909d26eb5323
size 18711009984

3
RM-NLHF-Qwen-32B.i1-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f7f597998e070883425dff5842c17bc9ac3cb6f61041a14d54a7ab6f2e61f472
size 20639242944

3
RM-NLHF-Qwen-32B.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:71a01e3739fae995eed8235540d1cf54f2382ae74adb09aaa6d427c5fc22aeaf
size 19851336384

3
RM-NLHF-Qwen-32B.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8a7a77bebd50b79938a48f1a3b81b20c2809e803352c81bbb9d100a424227347
size 18784410304

3
RM-NLHF-Qwen-32B.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4d4ce58c225b2a94f03b320d27981323ae24743bc1369b35082232bee7a50673
size 23262157504

3
RM-NLHF-Qwen-32B.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ee2ff89996f9b3495ffa81c0f25d8a22682dd09318526a30fc41219a8cb4d418
size 22638254784

3
RM-NLHF-Qwen-32B.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4b737d1a124267ef56a7532dd00e3983e14b80853859c311544b9bf37f70a664
size 26886154944

3
RM-NLHF-Qwen-32B.imatrix.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fd4bda819d055d29ac39ceb2bd28caf649855d69bb517b29ef52c5a7b28a13f6
size 15011072