初始化项目，由ModelHub XC社区提供模型

Model: DhruvalLabs/qwen3-8b-claude-agentic-fable5-GGUF Source: Original Platform
2026-08-08 00:39:19 +08:00
commit e235d06b36
12 changed files with 240 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,45 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+qwen3-8b-claude-agentic-fable5-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-8b-claude-agentic-fable5-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-8b-claude-agentic-fable5-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-8b-claude-agentic-fable5-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-8b-claude-agentic-fable5-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-8b-claude-agentic-fable5-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-8b-claude-agentic-fable5-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-8b-claude-agentic-fable5-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-8b-claude-agentic-fable5-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-8b-claude-agentic-fable5-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,165 @@
+---
+license: apache-2.0
+base_model: DhruvalLabs/qwen3-8b-claude-agentic-fable5
+pipeline_tag: text-generation
+tags:
+  - base_model:adapter:Qwen/Qwen3-8B
+  - text-generation-inference
+  - quantized
+  - qwen3
+  - safetensors
+  - gguf
+  - conversational
+  - region:us
+  - unsloth
+  - tool-use
+  - transformers
+  - lora
+  - finetuned
+  - claude
+  - agentic
+  - license:apache-2.0
+  - en
+  - coding-agent
+  - base_model:Qwen/Qwen3-8B
+  - text-generation
+  - fable5
+language:
+  - en
+---
+
+<div align="center">
+
+# qwen3-8b-claude-agentic-fable5 — GGUF Quantizations
+
+[![Model on HF](https://img.shields.io/badge/🤗-Model_on_HuggingFace-yellow)](https://huggingface.co/DhruvalLabs/qwen3-8b-claude-agentic-fable5-GGUF)
+[![Original Model](https://img.shields.io/badge/Original-DhruvalLabs_qwen3-8b-claude-agentic-fable5-blue)](https://huggingface.co/DhruvalLabs/qwen3-8b-claude-agentic-fable5)
+[![quant-kit](https://img.shields.io/badge/Made_with-quant--kit-green)](https://github.com/DhruvalPtl/quant-kit)
+
+**Quantized GGUF versions of [DhruvalLabs/qwen3-8b-claude-agentic-fable5](https://huggingface.co/DhruvalLabs/qwen3-8b-claude-agentic-fable5)**
+
+Works with **[llama.cpp](https://github.com/ggerganov/llama.cpp)** · **[Ollama](https://ollama.ai)** · **[LM Studio](https://lmstudio.ai)** · **[Open WebUI](https://openwebui.com)** · **[Jan](https://jan.ai)**
+
+*Quantized by **[DhruvalLabs](https://huggingface.co/DhruvalLabs)** on July 07, 2026 using [quant-kit](https://github.com/DhruvalPtl/quant-kit)*
+
+</div>
+
+---
+
+## ⚖️ The Pareto Frontier — Efficiency vs Intelligence
+
+> Can you run a powerful model on a laptop without losing its intelligence?
+
+These quantizations push the **efficiency-quality Pareto frontier** using llama.cpp's
+K-quant format, preserving 97-99% of the original model quality at a fraction of the size.
+
+| Benchmark | Original (FP16) | Q4_K_M | Quality Retained |
+|---|---|---|---|
+| **MMLU Pro** | *See [original card](https://huggingface.co/DhruvalLabs/qwen3-8b-claude-agentic-fable5)* | *Run benchmarks* | ~97-99% |
+| **HellaSwag** | *See [original card](https://huggingface.co/DhruvalLabs/qwen3-8b-claude-agentic-fable5)* | *Run benchmarks* | ~97-99% |
+| **ARC Challenge** | *See [original card](https://huggingface.co/DhruvalLabs/qwen3-8b-claude-agentic-fable5)* | *Run benchmarks* | ~97-99% |
+| **TruthfulQA** | *See [original card](https://huggingface.co/DhruvalLabs/qwen3-8b-claude-agentic-fable5)* | *Run benchmarks* | ~97-99% |
+| **GSM8K** | *See [original card](https://huggingface.co/DhruvalLabs/qwen3-8b-claude-agentic-fable5)* | *Run benchmarks* | ~97-99% |
+
+
+---
+
+## 📦 Available Files
+
+| Filename | Size | RAM Required | Quant | Quality | Best For |
+|---|---|---|---|---|---|
+| `qwen3-8b-claude-agentic-fable5-Q2_K.gguf` | 3.06 GB | ~4.6 GB | `Q2_K` | ⭐ | Extreme compression, significant quality loss. |
+| `qwen3-8b-claude-agentic-fable5-Q3_K_L.gguf` | 4.13 GB | ~5.6 GB | `Q3_K_L` | ⭐⭐⭐ | Slightly better than Q3_K_M, still a compromise. |
+| `qwen3-8b-claude-agentic-fable5-Q3_K_M.gguf` | 3.84 GB | ~5.3 GB | `Q3_K_M` | ⭐⭐⭐ | Very small file. Quality drop noticeable. |
+| `qwen3-8b-claude-agentic-fable5-Q3_K_S.gguf` | 3.51 GB | ~5.0 GB | `Q3_K_S` | ⭐⭐ | Very high compression, high quality loss. |
+| `qwen3-8b-claude-agentic-fable5-Q4_K_M.gguf` | 4.68 GB | ~6.2 GB | `Q4_K_M` ✅ **Recommended** | ⭐⭐⭐⭐ | Best balance of size and quality. Recommended for most users. |
+| `qwen3-8b-claude-agentic-fable5-Q4_K_S.gguf` | 4.47 GB | ~6.0 GB | `Q4_K_S` | ⭐⭐⭐½ | Good speed/size balance, slight quality loss. |
+| `qwen3-8b-claude-agentic-fable5-Q5_K_M.gguf` | 5.45 GB | ~6.9 GB | `Q5_K_M` | ⭐⭐⭐⭐½ | Better quality than Q4, slightly larger. Great if you have the RAM. |
+| `qwen3-8b-claude-agentic-fable5-Q5_K_S.gguf` | 5.33 GB | ~6.8 GB | `Q5_K_S` | ⭐⭐⭐⭐ | Large but accurate. |
+| `qwen3-8b-claude-agentic-fable5-Q6_K.gguf` | 6.26 GB | ~7.8 GB | `Q6_K` | ⭐⭐⭐⭐⭐ | Near-perfect quality, very large. |
+| `qwen3-8b-claude-agentic-fable5-Q8_0.gguf` | 8.11 GB | ~9.6 GB | `Q8_0` | ⭐⭐⭐⭐⭐ | Closest to original quality. Use when RAM is not a concern. |
+
+
+### 💡 Which file should I download?
+
+- **Most users:** `qwen3-8b-claude-agentic-fable5-Q4_K_M.gguf` — best balance of size and quality
+- **High RAM (32GB+):** `qwen3-8b-claude-agentic-fable5-Q8_0.gguf` — near-original quality
+- **Low RAM (8GB):** `qwen3-8b-claude-agentic-fable5-Q3_K_M.gguf` — fits in 8GB with room to spare
+
+---
+
+## ⚡ Speed Benchmarks
+
+*Run `python benchmark.py --model qwen3-8b-claude-agentic-fable5` to generate speed results.*
+
+
+---
+
+## 🧠 Quality Benchmarks
+
+*Run `kaggle_bench.ipynb` on Kaggle to benchmark this model.*
+
+
+---
+
+## 🚀 How to Use
+
+### Ollama
+```bash
+ollama run dhruvallabs/qwen3-8b-claude-agentic-fable5
+```
+
+### LM Studio / Jan / Open WebUI
+Search for `DhruvalLabs/qwen3-8b-claude-agentic-fable5` in the model browser.
+
+### llama.cpp CLI
+```bash
+# Download the binary from https://github.com/ggerganov/llama.cpp/releases
+./llama-cli \
+  -m qwen3-8b-claude-agentic-fable5-Q4_K_M.gguf \
+  -p "You are a helpful assistant." \
+  --conversation \
+  -n 512
+```
+
+### Python — llama-cpp-python
+```python
+from llama_cpp import Llama
+
+llm = Llama(
+    model_path="./qwen3-8b-claude-agentic-fable5-Q4_K_M.gguf",
+    n_gpu_layers=-1,   # -1 = offload everything to GPU
+    n_ctx=4096,
+)
+
+response = llm.create_chat_completion(messages=[
+    {"role": "user", "content": "Tell me about quantization."}
+])
+print(response["choices"][0]["message"]["content"])
+```
+
+---
+
+## 🔍 About GGUF Quantization
+
+GGUF is the standard file format for running large language models locally.
+Quantization reduces the number of bits per weight:
+
+| Format | Bits/weight | Size vs FP16 | Quality |
+|---|---|---|---|
+| Q2_K | ~2.6 | 16% | ⭐ |
+| Q3_K_M | ~3.3 | 21% | ⭐⭐⭐ |
+| Q4_K_M | ~4.5 | 28% | ⭐⭐⭐⭐ ← sweet spot |
+| Q5_K_M | ~5.6 | 35% | ⭐⭐⭐⭐½ |
+| Q8_0 | ~8.5 | 53% | ⭐⭐⭐⭐⭐ |
+
+---
+
+## 💬 Community & Feedback
+
+Found an issue? Have a question? Open a **Discussion** in the Community tab above.
+
+If these quantizations were useful, please consider:
+- ⭐ Starring [quant-kit](https://github.com/DhruvalPtl/quant-kit) on GitHub
+- 👍 Liking this model on HuggingFace
+- 💬 Leaving feedback in the Community tab
--- a/qwen3-8b-claude-agentic-fable5-Q2_K.gguf
+++ b/qwen3-8b-claude-agentic-fable5-Q2_K.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66962c6852515ea5a52640acf03df8816b8c4e0f777a3b67a8497556ed48addc
+size 3281733088
--- a/qwen3-8b-claude-agentic-fable5-Q3_K_L.gguf
+++ b/qwen3-8b-claude-agentic-fable5-Q3_K_L.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0cfe538420ff900aa00097ac944f17c4277c79355199e126b384fdc6e967018
+size 4431394272
--- a/qwen3-8b-claude-agentic-fable5-Q3_K_M.gguf
+++ b/qwen3-8b-claude-agentic-fable5-Q3_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:14c0757d49cf7104ce6648c5562502cad7c9fc10390934515b5aa2a22e2ccf3d
+size 4124161504
--- a/qwen3-8b-claude-agentic-fable5-Q3_K_S.gguf
+++ b/qwen3-8b-claude-agentic-fable5-Q3_K_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d19a09b759b1c90ff1d50f4b107cd4900232f57b9bec748c6810263b10c7001c
+size 3769611744
--- a/qwen3-8b-claude-agentic-fable5-Q4_K_M.gguf
+++ b/qwen3-8b-claude-agentic-fable5-Q4_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c08e00c50e731c53aa2b74528675ab06a5b873ebdb36ef04dcfbe26326814340
+size 5027784160
--- a/qwen3-8b-claude-agentic-fable5-Q4_K_S.gguf
+++ b/qwen3-8b-claude-agentic-fable5-Q4_K_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33eb94bb8fca60a0684ce4047ad89eef95a769c970e933399ee5287863d94df6
+size 4802012640
--- a/qwen3-8b-claude-agentic-fable5-Q5_K_M.gguf
+++ b/qwen3-8b-claude-agentic-fable5-Q5_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75b25d7273f645d57f5b8141cc3c767e4a742aa90ce52adcb84e4d0030ddf9e0
+size 5851112928
--- a/qwen3-8b-claude-agentic-fable5-Q5_K_S.gguf
+++ b/qwen3-8b-claude-agentic-fable5-Q5_K_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:399e08917f9ae57da12659d962a164bc74eb54158fd928a58d02fb874e292e13
+size 5720761824
--- a/qwen3-8b-claude-agentic-fable5-Q6_K.gguf
+++ b/qwen3-8b-claude-agentic-fable5-Q6_K.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f3e7e7ec69c1c2ac90dfda29801ebdecf14d09ec219c598ae518cd6cd4990ea
+size 6725899744
--- a/qwen3-8b-claude-agentic-fable5-Q8_0.gguf
+++ b/qwen3-8b-claude-agentic-fable5-Q8_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa13c8cb80e2de28a4617216f40eabb6d17148fe53d2c04b2991457d776557e2
+size 8709518816