commit 27b012b5dd016938b2369da8bdbcef62bddf34a1 Author: ModelHub XC Date: Sat Jul 25 13:05:10 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: RefinedNeuro/VibeThinker-3B-Hermes-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..0c614c4 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,47 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +VibeThinker-3B-Hermes-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +VibeThinker-3B-Hermes-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +VibeThinker-3B-Hermes-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +VibeThinker-3B-Hermes-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +VibeThinker-3B-Hermes-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +VibeThinker-3B-Hermes-f16.gguf filter=lfs diff=lfs merge=lfs -text +VibeThinker-3B-Hermes-v03-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +VibeThinker-3B-Hermes-v03-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +VibeThinker-3B-Hermes-v03-f16.gguf filter=lfs diff=lfs merge=lfs -text +VibeThinker-3B-Hermes-v04-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +VibeThinker-3B-Hermes-v04-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +VibeThinker-3B-Hermes-v04-f16.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..c619519 --- /dev/null +++ b/README.md @@ -0,0 +1,82 @@ +--- +license: apache-2.0 +base_model: WeiboAI/VibeThinker-3B +datasets: + - lambda/hermes-agent-reasoning-traces +language: + - en +pipeline_tag: text-generation +tags: + - tool-calling + - function-calling + - hermes + - reasoning + - gguf + - llama-cpp + - research-preview +--- + +> 👉 **Newer model:** [RefinedToolCall-V5-3B](https://huggingface.co/RefinedNeuro/RefinedToolCallV5-3b) — multi-turn agentic ~3.7×, single-turn 0.707, recovery 0.896, reasoning intact. `ollama run refinedneuro/refinedtoolcallv5-3b` + +# VibeThinker-3B-Hermes — GGUF (Research Preview) + +⚠️ **Research preview / experimental.** Read **Limitations** before use. Known issues: +repetition loops on out-of-distribution multi-turn input, and over-eagerness to call tools. + +GGUF quantizations of a LoRA fine-tune of +[`WeiboAI/VibeThinker-3B`](https://huggingface.co/WeiboAI/VibeThinker-3B) that adds +Hermes-style function calling (`…` + `…`) while +preserving reasoning. See the full (transformers) model card for training details and +benchmarks. + +## Files + +| File | Quant | Size | +|---|---|---| +| `…-Q6_K.gguf` | **Q6_K** | 2.4 GB | min recommended | +| `…-Q8_0.gguf` | Q8_0 | 3.1 GB | +| `…-f16.gguf` | F16 | 5.8 GB | best | + +> **Why only Q6_K and up?** We measured tool-call fidelity on a multi-step agentic task: +> **Q6_K, Q8_0 and F16 pass; Q3/Q4/Q5 fail** (they emit malformed/incomplete tool calls and +> can loop). Since this is a tool-calling model, the lower quants were removed to avoid +> shipping a broken experience. Use **Q6_K** for the best size/quality balance. + +## Usage (llama.cpp) + +```bash +# use Q6_K+ for tool-calling +llama-cli -m VibeThinker-3B-Hermes-Q6_K.gguf \ + --temp 0.6 --top-p 0.95 --repeat-penalty 1.1 \ + -p "" +``` + +## Recommended settings + +- **Use a stop on ``** for single-call settings — essential to avoid the model + appending duplicate/hallucinated calls. +- temperature 0.6, top_p 0.95, repeat-penalty ≈ 1.1. +- Stop tokens: `<|im_end|>` (151645) and `<|endoftext|>` (151643). +- Hermes system prompt with a `` block; the model emits `…` then + `\n{"name": …, "arguments": …}\n`. + +## Benchmarks (summary) + +- **Reasoning (AIME 2024):** base avg@4 0.842 → this model 0.783; pass@4 0.867 unchanged. +- **Tool-calling (BFCL single-turn, with stop fix):** live_simple ~60 %, live_multiple ~31 %, + live_relevance 87.5 %, live_irrelevance ~34 % (over-eager). + +## Limitations + +- Repetition loops on OOD multi-turn input (emits the same call many times; avg ~25 calls/turn + vs ~1.3 expected). Mitigate with `stop=[""]` + repeat-penalty. +- Over-eager to call tools (does not always decline when it should). +- Single-turn exact-match is poor without the stop fix. +- ~6-pt avg@4 reasoning dip vs base; trained 1 epoch on one dataset config. + +Not recommended for production agents as-is. + +## License + +Apache-2.0. Built on `WeiboAI/VibeThinker-3B` and `lambda/hermes-agent-reasoning-traces` +(both Apache-2.0). diff --git a/VibeThinker-3B-Hermes-Q6_K.gguf b/VibeThinker-3B-Hermes-Q6_K.gguf new file mode 100644 index 0000000..254f71e --- /dev/null +++ b/VibeThinker-3B-Hermes-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:827b84c206d17a16e292125fe00db871376d8d814059b8a367064056242f0441 +size 2538158304 diff --git a/VibeThinker-3B-Hermes-Q8_0.gguf b/VibeThinker-3B-Hermes-Q8_0.gguf new file mode 100644 index 0000000..30e679f --- /dev/null +++ b/VibeThinker-3B-Hermes-Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a5b16583889c12b737beb25f6af128eb8c4a19a2c57838b130e839e573a264cc +size 3285475552 diff --git a/VibeThinker-3B-Hermes-f16.gguf b/VibeThinker-3B-Hermes-f16.gguf new file mode 100644 index 0000000..39eda43 --- /dev/null +++ b/VibeThinker-3B-Hermes-f16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ab59796d106b286ea2cc2076564c067a18c54b92e07aff98fc50e1f36e21b31b +size 6178316512 diff --git a/VibeThinker-3B-Hermes-v03-Q6_K.gguf b/VibeThinker-3B-Hermes-v03-Q6_K.gguf new file mode 100644 index 0000000..75a0c4f --- /dev/null +++ b/VibeThinker-3B-Hermes-v03-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:356e393d4ba4bd7758f46c68c3a90bb43e38cdc9e6a6694e16479f90fcd30f60 +size 2538158304 diff --git a/VibeThinker-3B-Hermes-v03-Q8_0.gguf b/VibeThinker-3B-Hermes-v03-Q8_0.gguf new file mode 100644 index 0000000..2e1a763 --- /dev/null +++ b/VibeThinker-3B-Hermes-v03-Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2cb25a9b3d47fdf0f4caaefebaf0f2287088a81a8240edebb18715df9f92bfd2 +size 3285475552 diff --git a/VibeThinker-3B-Hermes-v03-f16.gguf b/VibeThinker-3B-Hermes-v03-f16.gguf new file mode 100644 index 0000000..9ffe4fd --- /dev/null +++ b/VibeThinker-3B-Hermes-v03-f16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de8c753e9618d005c3d91fbd1448714e63cf38813fa7a6abc9eb41808aea2d9d +size 6178316512 diff --git a/VibeThinker-3B-Hermes-v04-Q6_K.gguf b/VibeThinker-3B-Hermes-v04-Q6_K.gguf new file mode 100644 index 0000000..3c539eb --- /dev/null +++ b/VibeThinker-3B-Hermes-v04-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c21bdfdaecc90f546b440557d0988246ed2373578d5045c3cbfa973f609be489 +size 2538158304 diff --git a/VibeThinker-3B-Hermes-v04-Q8_0.gguf b/VibeThinker-3B-Hermes-v04-Q8_0.gguf new file mode 100644 index 0000000..7fb1b38 --- /dev/null +++ b/VibeThinker-3B-Hermes-v04-Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:72d8000f1fa997ab6d41e6d38dd0d07520408215ffed7d610a02b6631ebb36c9 +size 3285476352 diff --git a/VibeThinker-3B-Hermes-v04-f16.gguf b/VibeThinker-3B-Hermes-v04-f16.gguf new file mode 100644 index 0000000..771c823 --- /dev/null +++ b/VibeThinker-3B-Hermes-v04-f16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ecd87e2ec45861149401b491e2bcd1f5148fc308f113632146c2bd208377275a +size 6178317312