commit d64feff92ae9829246ff2f872de2b714c7e44503 Author: ModelHub XC Date: Sun Sep 27 13:47:20 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..0e202e3 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,50 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.f16.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama3.1-8b-instruct-LLMPC-Red-Team.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_M.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_M.gguf new file mode 100644 index 0000000..5ad6f1b --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:50a2328b9de1d72c4be27ea0b646f110064276231c357346248d6feb2bffa848 +size 3784828000 diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_S.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_S.gguf new file mode 100644 index 0000000..0950611 --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:433e89eac970a62b6d98527ab3364785e46c80d639f279e38e951abb850bc64c +size 3682329696 diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_XS.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_XS.gguf new file mode 100644 index 0000000..7ee21b5 --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ecfbe4779bb407b30d6f6283eaafb0dea7e56b5507b21331861ae47620db984b +size 3518751840 diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ4_XS.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ4_XS.gguf new file mode 100644 index 0000000..6c23a62 --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b6ffe52810aefebdd4a1a71c32c98393a8f8f203f45fbe29f5519e6fb4ddb0bf +size 4484367456 diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.Q2_K.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q2_K.gguf new file mode 100644 index 0000000..ebc86aa --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5e92b6b53641bbe6a7e0024b0809363b297fb531b328a7faf45e51be18a36c77 +size 3179136096 diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_L.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_L.gguf new file mode 100644 index 0000000..509e54b --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5bafb7d91c18820b60016f7a90840ac4786a9eb16a0ed6c74cc7f0bcda47a1fe +size 4321961056 diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_M.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_M.gguf new file mode 100644 index 0000000..a32b13e --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:63706cd933f9701bd02cc09daf9f4d2f3054c83ca27d467aca3255b0375c89a7 +size 4018922592 diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_S.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_S.gguf new file mode 100644 index 0000000..af8a323 --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0446ba65327d69e183f491fc43d9e52cf2fc109fe20224fad0e183c00797b8d3 +size 3664503904 diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.Q4_K_M.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q4_K_M.gguf new file mode 100644 index 0000000..777a2e7 --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f4ef0e3585530ffc06264fd09801830d0f9fa1f2febd17a2ad5566b23a7ef7dd +size 4920738912 diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.Q4_K_S.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q4_K_S.gguf new file mode 100644 index 0000000..9c51187 --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b04349f24b018be7bd070904bbe1313f6a77de21343a8a619276227d9a7e2f99 +size 4692673632 diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.Q5_K_M.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q5_K_M.gguf new file mode 100644 index 0000000..eacb8ab --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9499ff9d2be479e800dc3341fd97d88b24c4ff706e9db41c90535825d6f28159 +size 5732992096 diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.Q5_K_S.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q5_K_S.gguf new file mode 100644 index 0000000..0549acf --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b88c01ef7d2a3596c55a9fa105fc7119d0716a8dd07949ab0014f2651d13877d +size 5599298656 diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.Q6_K.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q6_K.gguf new file mode 100644 index 0000000..afab6d9 --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b9536f8b3fbcaf6e7f1bf4d19c735f85906c6624bfd68ae9100e3b9293c3d768 +size 6596011104 diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.Q8_0.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q8_0.gguf new file mode 100644 index 0000000..a74137f --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f609ba81babd6734ea2788d64f0c01ba1d57d9f2e519cd040b26a7af4eb9e711 +size 8540775520 diff --git a/Llama3.1-8b-instruct-LLMPC-Red-Team.f16.gguf b/Llama3.1-8b-instruct-LLMPC-Red-Team.f16.gguf new file mode 100644 index 0000000..e81f392 --- /dev/null +++ b/Llama3.1-8b-instruct-LLMPC-Red-Team.f16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:57a81d381d2bcfc02d8d174d15bae6a036188bc61c43124ced46a6748d02c088 +size 16068895840 diff --git a/README.md b/README.md new file mode 100644 index 0000000..ac7a5ce --- /dev/null +++ b/README.md @@ -0,0 +1,67 @@ +--- +base_model: LLM-PBE/Llama3.1-8b-instruct-LLMPC-Red-Team +language: +- en +library_name: transformers +license: llama3.1 +quantized_by: mradermacher +--- +## About + + + + + + +static quants of https://huggingface.co/LLM-PBE/Llama3.1-8b-instruct-LLMPC-Red-Team + + +weighted/imatrix quants are available at https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-i1-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.Q2_K.gguf) | Q2_K | 3.3 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_XS.gguf) | IQ3_XS | 3.6 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_S.gguf) | Q3_K_S | 3.8 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_S.gguf) | IQ3_S | 3.8 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ3_M.gguf) | IQ3_M | 3.9 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_M.gguf) | Q3_K_M | 4.1 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.Q3_K_L.gguf) | Q3_K_L | 4.4 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.IQ4_XS.gguf) | IQ4_XS | 4.6 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.Q4_K_S.gguf) | Q4_K_S | 4.8 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.Q4_K_M.gguf) | Q4_K_M | 5.0 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.Q5_K_S.gguf) | Q5_K_S | 5.7 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.Q5_K_M.gguf) | Q5_K_M | 5.8 | | +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.Q6_K.gguf) | Q6_K | 6.7 | very good quality | +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.Q8_0.gguf) | Q8_0 | 8.6 | fast, best quality | +| [GGUF](https://huggingface.co/mradermacher/Llama3.1-8b-instruct-LLMPC-Red-Team-GGUF/resolve/main/Llama3.1-8b-instruct-LLMPC-Red-Team.f16.gguf) | f16 | 16.2 | 16 bpw, overkill | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. + +