From 73b0149019fd9b5d0b34fbff62ec7c046f42fb49 Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Fri, 10 Apr 2026 21:39:03 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: Bloce3an/qwen2.5-0.5B-entities-relationship-gguf Source: Original Platform --- .gitattributes | 38 +++++++++++++++++ README.md | 69 +++++++++++++++++++++++++++++++ qwen2.5-0.5b-instruct.Q4_K_M.gguf | 3 ++ qwen2.5-0.5b-instruct.Q6_K.gguf | 3 ++ qwen2.5-0.5b-instruct.Q8_0.gguf | 3 ++ 5 files changed, 116 insertions(+) create mode 100644 .gitattributes create mode 100644 README.md create mode 100644 qwen2.5-0.5b-instruct.Q4_K_M.gguf create mode 100644 qwen2.5-0.5b-instruct.Q6_K.gguf create mode 100644 qwen2.5-0.5b-instruct.Q8_0.gguf diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..d2667e6 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,38 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +qwen2.5-0.5b-instruct.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +qwen2.5-0.5b-instruct.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +qwen2.5-0.5b-instruct.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..40d9bfd --- /dev/null +++ b/README.md @@ -0,0 +1,69 @@ +--- +base_model: Qwen/Qwen2.5-0.5B-Instruct +library_name: gguf +pipeline_tag: text-generation +tags: +- gguf +- qwen +- llama.cpp +- knowledge-graph +- information-extraction +- unsloth +license: apache-2.0 +language: +- en +--- + +# Qwen2.5-0.5B-Instruct Knowledge Graph Extractor (GGUF) + +This is a GGUF quantized version of the `qwen2.5-0.5B-kg-lora2` model. The base model (`Qwen2.5-0.5B-Instruct`) was fine-tuned with Unsloth to strictly extract Knowledge Graph triples from unstructured text. + +The weights have been merged and quantized to allow for fast, lightweight inference on CPU and edge devices using tools like [llama.cpp](https://github.com/ggerganov/llama.cpp), [Ollama](https://ollama.com/), or [LM Studio](https://lmstudio.ai/). + +## Available Quantizations +This repository contains the following quantized files: +- **`Q8_0`**: 8-bit quantization. Highest quality, nearly identical to the original fp16 model. Good if you have enough RAM. +- **`Q6_K`**: 6-bit quantization. Excellent balance between size and quality. +- **`Q4_K_M`**: 4-bit quantization. Recommended for most users. Fast inference and very low memory footprint with minimal quality loss. + +## Prompt Format (CRITICAL) +This model was strictly fine-tuned on a specific ChatML system prompt. **You MUST use this exact system prompt** or the model will hallucinate or output the wrong format. + +```text +<|im_start|>system +You are an expert at extracting clean, accurate knowledge graph triples from text. + +Your task is to carefully read the input text and extract **all** meaningful triples in this exact format: +(subject | relation | object) + +Strict rules you must follow: +- Subject and object must be specific named entities or concrete concepts explicitly mentioned in the text (people, organizations, locations, events, products, years, etc.) +- Relation should be a short, clear predicate in base form or simple present tense (examples: "is", "has", "works at", "located in", "born in", "capital of", "founded in") +- Only extract triples that are **directly supported** by the text — do **not** infer, assume, hallucinate or add information that is not clearly stated +- If uncertain about a triple → do **not** include it +- Each triple must be written on its **own separate line** +- Do **not** add any explanations, headings, numbering, bullet points, comments, or extra text of any kind +- If no valid triples can be extracted → return exactly one line: "No triples found"<|im_end|> +<|im_start|>user +Text: +{your_input_text}<|im_end|> +<|im_start|>assistant +``` + +## Usage with llama.cpp + +Once you have downloaded the `.gguf` file of your choice (e.g. `model-unsloth.Q4_K_M.gguf`), you can run it via `llama.cpp` using the CLI. Since Qwen2.5 uses ChatML, ensure that you pass the exact system instruction. + +```bash +./main -m model-unsloth.Q4_K_M.gguf \ + --color \ + -c 2048 \ + -temp 0.1 \ + --repeat_penalty 1.15 \ + -p "<|im_start|>system\nYou are an expert at extracting clean, accurate knowledge graph triples from text.\n\nYour task is to carefully read the input text and extract **all** meaningful triples in this exact format:\n(subject | relation | object)\n\nStrict rules you must follow:\n- Subject and object must be specific named entities or concrete concepts explicitly mentioned in the text (people, organizations, locations, events, products, years, etc.)\n- Relation should be a short, clear predicate in base form or simple present tense (examples: \"is\", \"has\", \"works at\", \"located in\", \"born in\", \"capital of\", \"founded in\")\n- Only extract triples that are **directly supported** by the text — do **not** infer, assume, hallucinate or add information that is not clearly stated\n- If uncertain about a triple → do **not** include it\n- Each triple must be written on its **own separate line**\n- Do **not** add any explanations, headings, numbering, bullet points, comments, or extra text of any kind\n- If no valid triples can be extracted → return exactly one line: \"No triples found\"<|im_end|>\n<|im_start|>user\nText:\nThe Tasmanian Devil is a carnivorous marsupial of the family Dasyuridae.<|im_end|>\n<|im_start|>assistant\n" +``` + +## Intended Use +- Local processing of sensitive documents. +- Rapid edge-device extraction of explicit Entity-Relation-Entity relationships. +- Pipeline integration for RAG (Retrieval-Augmented Generation) graph curation. \ No newline at end of file diff --git a/qwen2.5-0.5b-instruct.Q4_K_M.gguf b/qwen2.5-0.5b-instruct.Q4_K_M.gguf new file mode 100644 index 0000000..11d32b9 --- /dev/null +++ b/qwen2.5-0.5b-instruct.Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:96e00af0db93a58a22397e9451a7869f9b883068ac8c8f5cebab23d182e0c77f +size 397807456 diff --git a/qwen2.5-0.5b-instruct.Q6_K.gguf b/qwen2.5-0.5b-instruct.Q6_K.gguf new file mode 100644 index 0000000..b14ac00 --- /dev/null +++ b/qwen2.5-0.5b-instruct.Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f29a47e32993d3302756fcdc473a3e12ee6648318a33a004d830f552fe7a7222 +size 505736032 diff --git a/qwen2.5-0.5b-instruct.Q8_0.gguf b/qwen2.5-0.5b-instruct.Q8_0.gguf new file mode 100644 index 0000000..3936aec --- /dev/null +++ b/qwen2.5-0.5b-instruct.Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ae565c315ccb2c0d12cf57c62ea11696cbf0bff2f03b182a9014392f44fc619b +size 531067744