commit 6f07690efeb2bc692824f18ad4dcda369cf6cd1f Author: ModelHub XC Date: Mon Sep 14 14:18:15 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: MaziyarPanahi/Llama-3.2-3B-Overthinker-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..854fcd0 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,52 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker-GGUF_imatrix.dat filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.fp16.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3.2-3B-Overthinker.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Llama-3.2-3B-Overthinker-GGUF_imatrix.dat b/Llama-3.2-3B-Overthinker-GGUF_imatrix.dat new file mode 100644 index 0000000..9a2533a --- /dev/null +++ b/Llama-3.2-3B-Overthinker-GGUF_imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de240eaca971f2926e3bf64113bb91b2c2c901fadd32eeb2cf7cf6845101f335 +size 2988366 diff --git a/Llama-3.2-3B-Overthinker.IQ1_M.gguf b/Llama-3.2-3B-Overthinker.IQ1_M.gguf new file mode 100644 index 0000000..156a12b --- /dev/null +++ b/Llama-3.2-3B-Overthinker.IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a47d11f6db7b18efbc0405fbb42c1519571f2f77847ec57ec7c5c40266257362 +size 924188416 diff --git a/Llama-3.2-3B-Overthinker.IQ1_S.gguf b/Llama-3.2-3B-Overthinker.IQ1_S.gguf new file mode 100644 index 0000000..e5941b1 --- /dev/null +++ b/Llama-3.2-3B-Overthinker.IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8e4791bacf323f84920fcdb95bcc4cebdb5e76b1d908a49bd06fde447e468d3a +size 868155136 diff --git a/Llama-3.2-3B-Overthinker.IQ2_XS.gguf b/Llama-3.2-3B-Overthinker.IQ2_XS.gguf new file mode 100644 index 0000000..2b7f443 --- /dev/null +++ b/Llama-3.2-3B-Overthinker.IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0649829b7df19b1cdb48e3d07f83bda1b2a98215369c10fa213e4cebb28136b9 +size 1100545792 diff --git a/Llama-3.2-3B-Overthinker.IQ3_XS.gguf b/Llama-3.2-3B-Overthinker.IQ3_XS.gguf new file mode 100644 index 0000000..0f7c26d --- /dev/null +++ b/Llama-3.2-3B-Overthinker.IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5c84e2c32f06fa212d9aba65c505cb94121cf02f9a9d6238f878fa55707a057d +size 1476785920 diff --git a/Llama-3.2-3B-Overthinker.IQ4_XS.gguf b/Llama-3.2-3B-Overthinker.IQ4_XS.gguf new file mode 100644 index 0000000..c06d125 --- /dev/null +++ b/Llama-3.2-3B-Overthinker.IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:74e70bae0fe3c72151a5aece13598d435d9a7f300921a551277ff6bc36f84157 +size 1829107456 diff --git a/Llama-3.2-3B-Overthinker.Q2_K.gguf b/Llama-3.2-3B-Overthinker.Q2_K.gguf new file mode 100644 index 0000000..8d0b3b1 --- /dev/null +++ b/Llama-3.2-3B-Overthinker.Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:07cf4225195a282e3569ac5a377b41eee8e4da6228291f43987cb288b522c10f +size 1363932928 diff --git a/Llama-3.2-3B-Overthinker.Q3_K_L.gguf b/Llama-3.2-3B-Overthinker.Q3_K_L.gguf new file mode 100644 index 0000000..eccd82d --- /dev/null +++ b/Llama-3.2-3B-Overthinker.Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:320cb51ee77627399f87681cbc377c2ef72b52620a97a835ff27b2955d8d7df5 +size 1815344896 diff --git a/Llama-3.2-3B-Overthinker.Q3_K_M.gguf b/Llama-3.2-3B-Overthinker.Q3_K_M.gguf new file mode 100644 index 0000000..c0357c0 --- /dev/null +++ b/Llama-3.2-3B-Overthinker.Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4543033ff23d2dc883e270c174763ba8077b88263602a789182df981f382fffa +size 1687156480 diff --git a/Llama-3.2-3B-Overthinker.Q3_K_S.gguf b/Llama-3.2-3B-Overthinker.Q3_K_S.gguf new file mode 100644 index 0000000..4f32a5a --- /dev/null +++ b/Llama-3.2-3B-Overthinker.Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:08a6c380c3a38f699abfb63e3ca28fd519c596a0624379c27750f9c288951254 +size 1542846208 diff --git a/Llama-3.2-3B-Overthinker.Q4_K_M.gguf b/Llama-3.2-3B-Overthinker.Q4_K_M.gguf new file mode 100644 index 0000000..7ea22dc --- /dev/null +++ b/Llama-3.2-3B-Overthinker.Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2c4990152a91778149ee0c63a603ba410e3fd362ccf34a109035c102c2fc18d0 +size 2019374848 diff --git a/Llama-3.2-3B-Overthinker.Q4_K_S.gguf b/Llama-3.2-3B-Overthinker.Q4_K_S.gguf new file mode 100644 index 0000000..7709243 --- /dev/null +++ b/Llama-3.2-3B-Overthinker.Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c5313dd56a96a087e7f91c96a8b8517db0949874b192eea323f5cc3826be3977 +size 1928197888 diff --git a/Llama-3.2-3B-Overthinker.Q5_K_M.gguf b/Llama-3.2-3B-Overthinker.Q5_K_M.gguf new file mode 100644 index 0000000..d1265c5 --- /dev/null +++ b/Llama-3.2-3B-Overthinker.Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e9620938313d48195fb01bc1f3ef012329b7d004d0f46b89c918d526eaa6ab4d +size 2322151168 diff --git a/Llama-3.2-3B-Overthinker.Q5_K_S.gguf b/Llama-3.2-3B-Overthinker.Q5_K_S.gguf new file mode 100644 index 0000000..f2021ae --- /dev/null +++ b/Llama-3.2-3B-Overthinker.Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5cb268cf1cc4277423e874c357ac96c1598d385da1c919f5b29ec265323e728b +size 2269509376 diff --git a/Llama-3.2-3B-Overthinker.Q6_K.gguf b/Llama-3.2-3B-Overthinker.Q6_K.gguf new file mode 100644 index 0000000..ee005f5 --- /dev/null +++ b/Llama-3.2-3B-Overthinker.Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:25fab04793097611616d9e37cbf0b1ab7fbb562dd2e562afc8d497efee31aebd +size 2643851008 diff --git a/Llama-3.2-3B-Overthinker.Q8_0.gguf b/Llama-3.2-3B-Overthinker.Q8_0.gguf new file mode 100644 index 0000000..a84e154 --- /dev/null +++ b/Llama-3.2-3B-Overthinker.Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2cebb22264692be17b9b74fe35b8efe44422a9ea096195a90384f28a1ad77a8e +size 3421896448 diff --git a/Llama-3.2-3B-Overthinker.fp16.gguf b/Llama-3.2-3B-Overthinker.fp16.gguf new file mode 100644 index 0000000..59c485f --- /dev/null +++ b/Llama-3.2-3B-Overthinker.fp16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:310d8a5902ddab47f83d06dc98252795b161100fbe58fe322a078b897a84a74b +size 6433685024 diff --git a/README.md b/README.md new file mode 100644 index 0000000..8f39bee --- /dev/null +++ b/README.md @@ -0,0 +1,46 @@ +--- +tags: +- quantized +- 2-bit +- 3-bit +- 4-bit +- 5-bit +- 6-bit +- 8-bit +- GGUF +- text-generation +- text-generation +model_name: Llama-3.2-3B-Overthinker-GGUF +base_model: Lyte/Llama-3.2-3B-Overthinker +inference: false +model_creator: Lyte +pipeline_tag: text-generation +quantized_by: MaziyarPanahi +--- +# [MaziyarPanahi/Llama-3.2-3B-Overthinker-GGUF](https://huggingface.co/MaziyarPanahi/Llama-3.2-3B-Overthinker-GGUF) +- Model creator: [Lyte](https://huggingface.co/Lyte) +- Original model: [Lyte/Llama-3.2-3B-Overthinker](https://huggingface.co/Lyte/Llama-3.2-3B-Overthinker) + +## Description +[MaziyarPanahi/Llama-3.2-3B-Overthinker-GGUF](https://huggingface.co/MaziyarPanahi/Llama-3.2-3B-Overthinker-GGUF) contains GGUF format model files for [Lyte/Llama-3.2-3B-Overthinker](https://huggingface.co/Lyte/Llama-3.2-3B-Overthinker). + +### About GGUF + +GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. + +Here is an incomplete list of clients and libraries that are known to support GGUF: + +* [llama.cpp](https://github.com/ggerganov/llama.cpp). The source project for GGUF. Offers a CLI and a server option. +* [llama-cpp-python](https://github.com/abetlen/llama-cpp-python), a Python library with GPU accel, LangChain support, and OpenAI-compatible API server. +* [LM Studio](https://lmstudio.ai/), an easy-to-use and powerful local GUI for Windows and macOS (Silicon), with GPU acceleration. Linux available, in beta as of 27/11/2023. +* [text-generation-webui](https://github.com/oobabooga/text-generation-webui), the most widely used web UI, with many features and powerful extensions. Supports GPU acceleration. +* [KoboldCpp](https://github.com/LostRuins/koboldcpp), a fully featured web UI, with GPU accel across all platforms and GPU architectures. Especially good for story telling. +* [GPT4All](https://gpt4all.io/index.html), a free and open source local running GUI, supporting Windows, Linux and macOS with full GPU accel. +* [LoLLMS Web UI](https://github.com/ParisNeo/lollms-webui), a great web UI with many interesting and unique features, including a full model library for easy model selection. +* [Faraday.dev](https://faraday.dev/), an attractive and easy to use character-based chat GUI for Windows and macOS (both Silicon and Intel), with GPU acceleration. +* [candle](https://github.com/huggingface/candle), a Rust ML framework with a focus on performance, including GPU support, and ease of use. +* [ctransformers](https://github.com/marella/ctransformers), a Python library with GPU accel, LangChain support, and OpenAI-compatible AI server. Note, as of time of writing (November 27th 2023), ctransformers has not been updated in a long time and does not support many recent models. + +## Special thanks + +🙏 Special thanks to [Georgi Gerganov](https://github.com/ggerganov) and the whole team working on [llama.cpp](https://github.com/ggerganov/llama.cpp/) for making all of this possible. \ No newline at end of file diff --git a/config.json b/config.json new file mode 100644 index 0000000..9f0f76f --- /dev/null +++ b/config.json @@ -0,0 +1,3 @@ +{ + "model_type": "mistral" +} \ No newline at end of file