commit 4912561242fede85023184268c5601b0763f100e Author: ModelHub XC Date: Tue May 26 12:59:19 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Llama-3.2-3B-thinking-8k-v1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..f9bea64 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,47 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-thinking-8k-v1.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-thinking-8k-v1.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-thinking-8k-v1.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-thinking-8k-v1.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-thinking-8k-v1.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-thinking-8k-v1.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-thinking-8k-v1.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-thinking-8k-v1.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-thinking-8k-v1.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-thinking-8k-v1.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-thinking-8k-v1.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-thinking-8k-v1.f16.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Llama-3.2-3B-thinking-8k-v1.IQ4_XS.gguf b/Llama-3.2-3B-thinking-8k-v1.IQ4_XS.gguf new file mode 100644 index 0000000..0548f71 --- /dev/null +++ b/Llama-3.2-3B-thinking-8k-v1.IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:18ddf7099858fa73fca972a7b0ca36b53aec213cd2f7b31a81ca772102f42c78 +size 1840907488 diff --git a/Llama-3.2-3B-thinking-8k-v1.Q2_K.gguf b/Llama-3.2-3B-thinking-8k-v1.Q2_K.gguf new file mode 100644 index 0000000..939290d --- /dev/null +++ b/Llama-3.2-3B-thinking-8k-v1.Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a94274661a0c69e27a059a4dbf868728b8001ed6af392dd64f1d1c4b851b63c1 +size 1363936480 diff --git a/Llama-3.2-3B-thinking-8k-v1.Q3_K_L.gguf b/Llama-3.2-3B-thinking-8k-v1.Q3_K_L.gguf new file mode 100644 index 0000000..634e614 --- /dev/null +++ b/Llama-3.2-3B-thinking-8k-v1.Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1095cc95b358638be9550af94f0abec99ab610446993ec8e32e1be7fa0d703e +size 1815348448 diff --git a/Llama-3.2-3B-thinking-8k-v1.Q3_K_M.gguf b/Llama-3.2-3B-thinking-8k-v1.Q3_K_M.gguf new file mode 100644 index 0000000..552edb8 --- /dev/null +++ b/Llama-3.2-3B-thinking-8k-v1.Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4e467ffee7d857fbcef06d5936c1e725c9e3662f428e94a0104a0eec497d2bc4 +size 1687160032 diff --git a/Llama-3.2-3B-thinking-8k-v1.Q3_K_S.gguf b/Llama-3.2-3B-thinking-8k-v1.Q3_K_S.gguf new file mode 100644 index 0000000..efda50d --- /dev/null +++ b/Llama-3.2-3B-thinking-8k-v1.Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c0a59e65488635998dbc7e17877eb57e9f84f9282f8d86f3aa0b6abaf8a877c4 +size 1542849760 diff --git a/Llama-3.2-3B-thinking-8k-v1.Q4_K_M.gguf b/Llama-3.2-3B-thinking-8k-v1.Q4_K_M.gguf new file mode 100644 index 0000000..88c9d73 --- /dev/null +++ b/Llama-3.2-3B-thinking-8k-v1.Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44237862c3cea9a53b071053ed7bf77f2806b1cd3f437f41187c2d33ba635f3c +size 2019378400 diff --git a/Llama-3.2-3B-thinking-8k-v1.Q4_K_S.gguf b/Llama-3.2-3B-thinking-8k-v1.Q4_K_S.gguf new file mode 100644 index 0000000..3b382c0 --- /dev/null +++ b/Llama-3.2-3B-thinking-8k-v1.Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4941810c4dfde74f5403a4a8bf6070c88a86bc23216d31f050fd24ec6341aff0 +size 1928201440 diff --git a/Llama-3.2-3B-thinking-8k-v1.Q5_K_M.gguf b/Llama-3.2-3B-thinking-8k-v1.Q5_K_M.gguf new file mode 100644 index 0000000..cd5c743 --- /dev/null +++ b/Llama-3.2-3B-thinking-8k-v1.Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dda851aba36d27b087719aa9cd98b9edde301ef4b8119362fb6be0e7a8c2229c +size 2322154720 diff --git a/Llama-3.2-3B-thinking-8k-v1.Q5_K_S.gguf b/Llama-3.2-3B-thinking-8k-v1.Q5_K_S.gguf new file mode 100644 index 0000000..d3edeb2 --- /dev/null +++ b/Llama-3.2-3B-thinking-8k-v1.Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8bf8ba8933d6343c3d1bcc08f22364b2376e6b26c0a8c767d34f225b9856a46b +size 2269512928 diff --git a/Llama-3.2-3B-thinking-8k-v1.Q6_K.gguf b/Llama-3.2-3B-thinking-8k-v1.Q6_K.gguf new file mode 100644 index 0000000..92a44c5 --- /dev/null +++ b/Llama-3.2-3B-thinking-8k-v1.Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9dbcf6ccaa835ff861fc875e25300517de954fc4701347dfa82832da6d22cf80 +size 2643854560 diff --git a/Llama-3.2-3B-thinking-8k-v1.Q8_0.gguf b/Llama-3.2-3B-thinking-8k-v1.Q8_0.gguf new file mode 100644 index 0000000..7636473 --- /dev/null +++ b/Llama-3.2-3B-thinking-8k-v1.Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0c2314e2719cba722871c10056e821569ca2a4f47c2da7a4d7312ffc8b23718a +size 3421900000 diff --git a/Llama-3.2-3B-thinking-8k-v1.f16.gguf b/Llama-3.2-3B-thinking-8k-v1.f16.gguf new file mode 100644 index 0000000..c156002 --- /dev/null +++ b/Llama-3.2-3B-thinking-8k-v1.f16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0e43abc72b5f1b9c013f0dd776f9062394cb2725288e270479f1f3b804bcc58a +size 6433688800 diff --git a/README.md b/README.md new file mode 100644 index 0000000..82214ce --- /dev/null +++ b/README.md @@ -0,0 +1,74 @@ +--- +base_model: x1nx3r/Llama-3.2-3B-thinking-8k-v1 +datasets: +- a-m-team/AM-DeepSeek-R1-Distilled-1.4M +language: +- en +library_name: transformers +license: llama3.2 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +--- +## About + + + + + + + + + +static quants of https://huggingface.co/x1nx3r/Llama-3.2-3B-thinking-8k-v1 + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Llama-3.2-3B-thinking-8k-v1-GGUF).*** + +weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion. +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Llama-3.2-3B-thinking-8k-v1-GGUF/resolve/main/Llama-3.2-3B-thinking-8k-v1.Q2_K.gguf) | Q2_K | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.2-3B-thinking-8k-v1-GGUF/resolve/main/Llama-3.2-3B-thinking-8k-v1.Q3_K_S.gguf) | Q3_K_S | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.2-3B-thinking-8k-v1-GGUF/resolve/main/Llama-3.2-3B-thinking-8k-v1.Q3_K_M.gguf) | Q3_K_M | 1.8 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.2-3B-thinking-8k-v1-GGUF/resolve/main/Llama-3.2-3B-thinking-8k-v1.Q3_K_L.gguf) | Q3_K_L | 1.9 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.2-3B-thinking-8k-v1-GGUF/resolve/main/Llama-3.2-3B-thinking-8k-v1.IQ4_XS.gguf) | IQ4_XS | 1.9 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.2-3B-thinking-8k-v1-GGUF/resolve/main/Llama-3.2-3B-thinking-8k-v1.Q4_K_S.gguf) | Q4_K_S | 2.0 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.2-3B-thinking-8k-v1-GGUF/resolve/main/Llama-3.2-3B-thinking-8k-v1.Q4_K_M.gguf) | Q4_K_M | 2.1 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.2-3B-thinking-8k-v1-GGUF/resolve/main/Llama-3.2-3B-thinking-8k-v1.Q5_K_S.gguf) | Q5_K_S | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.2-3B-thinking-8k-v1-GGUF/resolve/main/Llama-3.2-3B-thinking-8k-v1.Q5_K_M.gguf) | Q5_K_M | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.2-3B-thinking-8k-v1-GGUF/resolve/main/Llama-3.2-3B-thinking-8k-v1.Q6_K.gguf) | Q6_K | 2.7 | very good quality | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.2-3B-thinking-8k-v1-GGUF/resolve/main/Llama-3.2-3B-thinking-8k-v1.Q8_0.gguf) | Q8_0 | 3.5 | fast, best quality | +| [GGUF](https://huggingface.co/mradermacher/Llama-3.2-3B-thinking-8k-v1-GGUF/resolve/main/Llama-3.2-3B-thinking-8k-v1.f16.gguf) | f16 | 6.5 | 16 bpw, overkill | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. + +