From dbbbdc32c6777dbdfe7cc8272db9e51db6958f49 Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Sun, 21 Jun 2026 07:28:12 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: QuantFactory/Llama-Sentient-3.2-3B-Instruct-GGUF Source: Original Platform --- .gitattributes | 49 ++++++++++++++++ Llama-Sentient-3.2-3B-Instruct.Q2_K.gguf | 3 + Llama-Sentient-3.2-3B-Instruct.Q3_K_L.gguf | 3 + Llama-Sentient-3.2-3B-Instruct.Q3_K_M.gguf | 3 + Llama-Sentient-3.2-3B-Instruct.Q3_K_S.gguf | 3 + Llama-Sentient-3.2-3B-Instruct.Q4_0.gguf | 3 + Llama-Sentient-3.2-3B-Instruct.Q4_1.gguf | 3 + Llama-Sentient-3.2-3B-Instruct.Q4_K_M.gguf | 3 + Llama-Sentient-3.2-3B-Instruct.Q4_K_S.gguf | 3 + Llama-Sentient-3.2-3B-Instruct.Q5_0.gguf | 3 + Llama-Sentient-3.2-3B-Instruct.Q5_1.gguf | 3 + Llama-Sentient-3.2-3B-Instruct.Q5_K_M.gguf | 3 + Llama-Sentient-3.2-3B-Instruct.Q5_K_S.gguf | 3 + Llama-Sentient-3.2-3B-Instruct.Q6_K.gguf | 3 + Llama-Sentient-3.2-3B-Instruct.Q8_0.gguf | 3 + README.md | 66 ++++++++++++++++++++++ configuration.json | 1 + 17 files changed, 158 insertions(+) create mode 100644 .gitattributes create mode 100644 Llama-Sentient-3.2-3B-Instruct.Q2_K.gguf create mode 100644 Llama-Sentient-3.2-3B-Instruct.Q3_K_L.gguf create mode 100644 Llama-Sentient-3.2-3B-Instruct.Q3_K_M.gguf create mode 100644 Llama-Sentient-3.2-3B-Instruct.Q3_K_S.gguf create mode 100644 Llama-Sentient-3.2-3B-Instruct.Q4_0.gguf create mode 100644 Llama-Sentient-3.2-3B-Instruct.Q4_1.gguf create mode 100644 Llama-Sentient-3.2-3B-Instruct.Q4_K_M.gguf create mode 100644 Llama-Sentient-3.2-3B-Instruct.Q4_K_S.gguf create mode 100644 Llama-Sentient-3.2-3B-Instruct.Q5_0.gguf create mode 100644 Llama-Sentient-3.2-3B-Instruct.Q5_1.gguf create mode 100644 Llama-Sentient-3.2-3B-Instruct.Q5_K_M.gguf create mode 100644 Llama-Sentient-3.2-3B-Instruct.Q5_K_S.gguf create mode 100644 Llama-Sentient-3.2-3B-Instruct.Q6_K.gguf create mode 100644 Llama-Sentient-3.2-3B-Instruct.Q8_0.gguf create mode 100644 README.md create mode 100644 configuration.json diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..dfd0af1 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,49 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Llama-Sentient-3.2-3B-Instruct.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-Sentient-3.2-3B-Instruct.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Llama-Sentient-3.2-3B-Instruct.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Llama-Sentient-3.2-3B-Instruct.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +Llama-Sentient-3.2-3B-Instruct.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama-Sentient-3.2-3B-Instruct.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-Sentient-3.2-3B-Instruct.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-Sentient-3.2-3B-Instruct.Q5_0.gguf filter=lfs diff=lfs merge=lfs -text +Llama-Sentient-3.2-3B-Instruct.Q5_1.gguf filter=lfs diff=lfs merge=lfs -text +Llama-Sentient-3.2-3B-Instruct.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama-Sentient-3.2-3B-Instruct.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Llama-Sentient-3.2-3B-Instruct.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-Sentient-3.2-3B-Instruct.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-Sentient-3.2-3B-Instruct.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Llama-Sentient-3.2-3B-Instruct.Q2_K.gguf b/Llama-Sentient-3.2-3B-Instruct.Q2_K.gguf new file mode 100644 index 0000000..f602a06 --- /dev/null +++ b/Llama-Sentient-3.2-3B-Instruct.Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:499b0c733ba3dfc6dbad27ddc0ee8f6e7dd40c3a206f73c278f2cf011ae3a983 +size 1363936896 diff --git a/Llama-Sentient-3.2-3B-Instruct.Q3_K_L.gguf b/Llama-Sentient-3.2-3B-Instruct.Q3_K_L.gguf new file mode 100644 index 0000000..76832a5 --- /dev/null +++ b/Llama-Sentient-3.2-3B-Instruct.Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7cfb2d001232fd885ad72cb84a5c69bf881e0feb2769b701a606016105a2fad9 +size 1815348864 diff --git a/Llama-Sentient-3.2-3B-Instruct.Q3_K_M.gguf b/Llama-Sentient-3.2-3B-Instruct.Q3_K_M.gguf new file mode 100644 index 0000000..5c2d553 --- /dev/null +++ b/Llama-Sentient-3.2-3B-Instruct.Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:613716929a3a89c0b16c09cbcd630c46ec387b8adb272abb0036ae942ec6d6bf +size 1687160448 diff --git a/Llama-Sentient-3.2-3B-Instruct.Q3_K_S.gguf b/Llama-Sentient-3.2-3B-Instruct.Q3_K_S.gguf new file mode 100644 index 0000000..c3724d5 --- /dev/null +++ b/Llama-Sentient-3.2-3B-Instruct.Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9a380b1d50b93295c390672ff29e6cf506cac029176e16de8a7b7e8f704a501b +size 1542850176 diff --git a/Llama-Sentient-3.2-3B-Instruct.Q4_0.gguf b/Llama-Sentient-3.2-3B-Instruct.Q4_0.gguf new file mode 100644 index 0000000..d5aef83 --- /dev/null +++ b/Llama-Sentient-3.2-3B-Instruct.Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eeb1282c942ef9e14f47ed97f6f21548b55ca5f73b2b0f2fe465167f81454998 +size 1917191808 diff --git a/Llama-Sentient-3.2-3B-Instruct.Q4_1.gguf b/Llama-Sentient-3.2-3B-Instruct.Q4_1.gguf new file mode 100644 index 0000000..0f004b6 --- /dev/null +++ b/Llama-Sentient-3.2-3B-Instruct.Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e6001ad9ca928fb9d4aab226b15912795ee7143343f3a625c21180f0af648f99 +size 2093352576 diff --git a/Llama-Sentient-3.2-3B-Instruct.Q4_K_M.gguf b/Llama-Sentient-3.2-3B-Instruct.Q4_K_M.gguf new file mode 100644 index 0000000..955126e --- /dev/null +++ b/Llama-Sentient-3.2-3B-Instruct.Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3f855ce0522bfdc39fc826162ba6d89f15cc3740c5207da10e70baa3348b7812 +size 2019378816 diff --git a/Llama-Sentient-3.2-3B-Instruct.Q4_K_S.gguf b/Llama-Sentient-3.2-3B-Instruct.Q4_K_S.gguf new file mode 100644 index 0000000..3f6e192 --- /dev/null +++ b/Llama-Sentient-3.2-3B-Instruct.Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cd6aa336943ed367f0203e3100df9b0c68ded6342f811fb758c9ff2ed0139955 +size 1928201856 diff --git a/Llama-Sentient-3.2-3B-Instruct.Q5_0.gguf b/Llama-Sentient-3.2-3B-Instruct.Q5_0.gguf new file mode 100644 index 0000000..a534f95 --- /dev/null +++ b/Llama-Sentient-3.2-3B-Instruct.Q5_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e30472768b78051a1227461790954427408a8d93d6964bebeb4b9a18dd727ab1 +size 2269513344 diff --git a/Llama-Sentient-3.2-3B-Instruct.Q5_1.gguf b/Llama-Sentient-3.2-3B-Instruct.Q5_1.gguf new file mode 100644 index 0000000..772954c --- /dev/null +++ b/Llama-Sentient-3.2-3B-Instruct.Q5_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8a0d3205bca129cb1875a2e2763ef82950fc879320608d865e5fc36ceeca5c7c +size 2445674112 diff --git a/Llama-Sentient-3.2-3B-Instruct.Q5_K_M.gguf b/Llama-Sentient-3.2-3B-Instruct.Q5_K_M.gguf new file mode 100644 index 0000000..6e5e82b --- /dev/null +++ b/Llama-Sentient-3.2-3B-Instruct.Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:08ff49f9c538499a1d3afbd1a324b695c61013c1abe03d8710f54ba2116fde4c +size 2322155136 diff --git a/Llama-Sentient-3.2-3B-Instruct.Q5_K_S.gguf b/Llama-Sentient-3.2-3B-Instruct.Q5_K_S.gguf new file mode 100644 index 0000000..8f27fe1 --- /dev/null +++ b/Llama-Sentient-3.2-3B-Instruct.Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:125b7f98b450acc3ce0916f4c3474c1105fb9eccb5044450a00ede2c30c4814f +size 2269513344 diff --git a/Llama-Sentient-3.2-3B-Instruct.Q6_K.gguf b/Llama-Sentient-3.2-3B-Instruct.Q6_K.gguf new file mode 100644 index 0000000..3578317 --- /dev/null +++ b/Llama-Sentient-3.2-3B-Instruct.Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6620e7fa0463861e78780327887cc375f1ab5c433086befde99517f5b1d3d61a +size 2643854976 diff --git a/Llama-Sentient-3.2-3B-Instruct.Q8_0.gguf b/Llama-Sentient-3.2-3B-Instruct.Q8_0.gguf new file mode 100644 index 0000000..aa74485 --- /dev/null +++ b/Llama-Sentient-3.2-3B-Instruct.Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0f8b1e71f6c2f6dae8c1b47f4191ef31d5b6295e03acc9785560c76817f90d70 +size 3421900416 diff --git a/README.md b/README.md new file mode 100644 index 0000000..d1925b3 --- /dev/null +++ b/README.md @@ -0,0 +1,66 @@ + +--- + +license: creativeml-openrail-m +datasets: +- mlabonne/lmsys-arena-human-preference-55k-sharegpt +language: +- en +base_model: +- meta-llama/Llama-3.2-3B-Instruct +pipeline_tag: text-generation +library_name: transformers +tags: +- Llama +- Llama-Cpp +- Llama3.2 +- Instruct +- 3B +- bin +- Sentient + +--- + +[![QuantFactory Banner](https://lh7-rt.googleusercontent.com/docsz/AD_4nXeiuCm7c8lEwEJuRey9kiVZsRn2W-b4pWlu3-X534V3YmVuVc2ZL-NXg2RkzSOOS2JXGHutDuyyNAUtdJI65jGTo8jT9Y99tMi4H4MqL44Uc5QKG77B0d6-JfIkZHFaUA71-RtjyYZWVIhqsNZcx8-OMaA?key=xt3VSDoCbmTY7o-cwwOFwQ)](https://hf.co/QuantFactory) + + +# QuantFactory/Llama-Sentient-3.2-3B-Instruct-GGUF +This is quantized version of [prithivMLmods/Llama-Sentient-3.2-3B-Instruct](https://huggingface.co/prithivMLmods/Llama-Sentient-3.2-3B-Instruct) created using llama.cpp + +# Original Model Card + +## Llama-Sentient-3.2-3B-Instruct Modelfile + +| File Name | Size | Description | Upload Status | +|-----------------------------------------|--------------|-----------------------------------------|----------------| +| `.gitattributes` | 1.57 kB | Git attributes configuration file | Uploaded | +| `README.md` | 42 Bytes | Initial commit README | Uploaded | +| `config.json` | 1.04 kB | Configuration file | Uploaded | +| `generation_config.json` | 248 Bytes | Generation configuration file | Uploaded | +| `pytorch_model-00001-of-00002.bin` | 4.97 GB | PyTorch model file (part 1) | Uploaded (LFS) | +| `pytorch_model-00002-of-00002.bin` | 1.46 GB | PyTorch model file (part 2) | Uploaded (LFS) | +| `pytorch_model.bin.index.json` | 21.2 kB | Model index file | Uploaded | +| `special_tokens_map.json` | 477 Bytes | Special tokens mapping | Uploaded | +| `tokenizer.json` | 17.2 MB | Tokenizer JSON file | Uploaded (LFS) | +| `tokenizer_config.json` | 57.4 kB | Tokenizer configuration file | Uploaded | + +| Model Type | Size | Context Length | Link | +|------------|------|----------------|------| +| GGUF | 3B | - | [🤗 Llama-Sentient-3.2-3B-Instruct-GGUF](https://huggingface.co/prithivMLmods/Llama-Sentient-3.2-3B-Instruct-GGUF) | + +The **Llama-Sentient-3.2-3B-Instruct** model is a fine-tuned version of the **Llama-3.2-3B-Instruct** model, optimized for **text generation** tasks, particularly where instruction-following abilities are critical. This model is trained on the **mlabonne/lmsys-arena-human-preference-55k-sharegpt** dataset, which enhances its performance in conversational and advisory contexts, making it suitable for a wide range of applications. + +### Key Use Cases: +1. **Conversational AI**: Engage in intelligent dialogue, offering coherent responses and following instructions, useful for customer support and virtual assistants. +2. **Text Generation**: Generate high-quality, contextually appropriate content such as articles, summaries, explanations, and other forms of written communication based on user prompts. +3. **Instruction Following**: Follow specific instructions with accuracy, making it ideal for tasks that require structured guidance, such as technical troubleshooting or educational assistance. + +The model uses a **PyTorch-based architecture** and includes a range of necessary files such as configuration files, tokenizer files, and model weight files for deployment. + +### Intended Applications: +- **Chatbots** for virtual assistance, customer support, or as personal digital assistants. +- **Content Creation Tools**, aiding in the generation of written materials, blog posts, or automated responses based on user inputs. +- **Educational and Training Systems**, providing explanations and guided learning experiences in various domains. +- **Human-AI Interaction** platforms, where the model can follow user instructions to provide personalized assistance or perform specific tasks. + +With its strong foundation in instruction-following and conversational contexts, the **Llama-Sentient-3.2-3B-Instruct** model offers versatile applications for both general and specialized domains. diff --git a/configuration.json b/configuration.json new file mode 100644 index 0000000..bbeeda1 --- /dev/null +++ b/configuration.json @@ -0,0 +1 @@ +{"framework": "pytorch", "task": "text-generation", "allow_remote": true} \ No newline at end of file