commit a06329703e97aa6d9043770f2b999f4602b45f4b Author: ModelHub XC Date: Sun Jun 21 02:50:18 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: fesalfayed/gpt-oss-20b-hermes_agent-tool-finetune_gguf Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..0fdcd8e --- /dev/null +++ b/.gitattributes @@ -0,0 +1,42 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +gpt-oss-20b-hermes.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +gpt-oss-20b-hermes.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +gpt-oss-20b-hermes.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +gpt-oss-20b-hermes.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +gpt-oss-20b-hermes.f16.gguf filter=lfs diff=lfs merge=lfs -text +assets/banner_gguf.png filter=lfs diff=lfs merge=lfs -text +assets/cover_gguf.png filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..2f8f688 --- /dev/null +++ b/README.md @@ -0,0 +1,165 @@ +--- +license: apache-2.0 +language: [en] +library_name: gguf +pipeline_tag: text-generation +base_model: openai/gpt-oss-20b +base_model_relation: quantized +tags: + - gpt-oss + - moe + - agent + - hermes-agent + - tool-use + - function-calling + - harmony + - reasoning + - gguf + - quantized + - llama-cpp + - llama.cpp + - ollama + - lm-studio +--- + +

+ gpt-oss-20b · Hermes-Agent tool finetune · GGUF +

+ +# gpt-oss-20b · Hermes-Agent tool finetune · GGUF + +GGUF quants for llama.cpp, Ollama, and LM Studio. Five quants shipped — pick by RAM budget. + +- **Format** — GGUF +- **Quants shipped** — Q3_K_M, Q4_K_M, Q5_K_M, Q8_0, F16 +- **Recommended** — Q4_K_M for 16 GB RAM, Q8_0 for quality +- **Runtime** — llama.cpp, Ollama, LM Studio, koboldcpp + +## What this is + +A tool-use finetune of OpenAI's `gpt-oss-20b` for [Hermes-Agent](https://github.com/fesalfayed/agent-bridge), +a local agent framework that needs models which call tools reliably, follow +multi-turn instructions, and don't argue with system prompts. + +The base model is the 21B-parameter (3.6B active) Mixture-of-Experts release +from OpenAI. This finetune preserves the Harmony chat template and the +reasoning-effort knob, and improves: + +- Function-calling adherence (correct JSON, no commentary mid-call) +- Long agent loops (10+ turns of tool → observe → plan) +- System-prompt fidelity (respects role boundaries and refusal/allow-list rules) + +It is **not** affiliated with NousResearch's Hermes model series. "Hermes-Agent" +here refers to the local agent framework only. + +## Files + +| Quant | Size | Use case | +|---------|-----------|-------------------------------------------| +| Q3_K_M | ~10.0 GB | Tight RAM, lowest acceptable quality | +| Q4_K_M | ~12.5 GB | Best quality / size trade-off (default) | +| Q5_K_M | ~14.5 GB | Higher quality, modest size bump | +| Q8_0 | ~22 GB | Near-lossless | +| F16 | ~41 GB | Reference, no quantization | + +(Sizes are approximate; check the file list for exact bytes.) + +## Quickstart + +### llama.cpp + +```bash +./llama-server \ + -hf fesalfayed/gpt-oss-20b-hermes_agent-tool-finetune_gguf:Q4_K_M \ + --port 1234 \ + -c 8192 \ + --jinja +``` + +### Ollama + +```bash +ollama run hf.co/fesalfayed/gpt-oss-20b-hermes_agent-tool-finetune_gguf:Q4_K_M +``` + +### LM Studio + +Search for `fesalfayed/gpt-oss-20b-hermes_agent-tool-finetune_gguf` in the Discover tab and pick a quant. Enable +"Use Jinja chat template" in the model settings so Harmony renders correctly. + +## Hermes-Agent integration + +Add a profile in `~/.hermes/config.yaml`: + +```yaml +profiles: + gpt-oss-20b-tools: + provider: openai + base_url: http://127.0.0.1:1234/v1 # LM Studio / vLLM / mlx_lm.server + model: fesalfayed/gpt-oss-20b-hermes_agent-tool-finetune_gguf + temperature: 0.7 + top_p: 0.95 + min_p: 0.1 # important for MoE stability + max_tokens: 8192 + tool_choice: auto +``` + +Then `hermes profile use gpt-oss-20b-tools` and the agent loop will route +tool calls through this model. + +## Sampling + +| Param | Value | Why | +|---|---|---| +| temperature | 0.7 | balanced; drop to 0.2 for strict tool calls | +| top_p | 0.95 | standard nucleus | +| min_p | 0.1 | required for MoE — prevents dead-expert tokens | +| repetition_penalty | 1.0 | the model handles repetition itself | + +Harmony reasoning effort: set the system message to `Reasoning: low|medium|high`. +`high` is roughly 3-4x more output tokens but noticeably better on multi-step +tool plans. + +## Training + +- Base: `openai/gpt-oss-20b` +- Method: LoRA SFT (rank 64, alpha 16) merged back into BF16 +- Frame: Unsloth + TRL on a single H100 (80 GB) +- Data: ~42k tool-use traces from Hermes-Agent sessions, filtered for + successful tool calls and clean JSON. No synthetic distillation. +- Length: 8192 tokens, packing on +- Loss: assistant-only, mask user/system/tool + +The `_16bit` repo holds the merged BF16 weights. The `_4bit`, `_mlx`, and +`_gguf` repos are quantizations of that checkpoint. + +## Limitations + +- Math and code-generation are unchanged from the base — this finetune + optimizes the agent loop, not raw reasoning. +- The model can over-call tools when given vague instructions. Add a + "if you can answer directly, do so" line to the system prompt. +- English only. Other languages were not in the training mix. +- Not safety-tuned beyond what `gpt-oss-20b` already provides. + +## Other formats + +- [BF16 reference](https://huggingface.co/fesalfayed/gpt-oss-20b-hermes_agent-tool-finetune_16bit) — full precision, vLLM / Transformers +- [MXFP4 4-bit](https://huggingface.co/fesalfayed/gpt-oss-20b-hermes_agent-tool-finetune_4bit) — fits a 16 GB GPU +- [MLX](https://huggingface.co/fesalfayed/gpt-oss-20b-hermes_agent-tool-finetune_mlx) — Apple Silicon native +- [GGUF](https://huggingface.co/fesalfayed/gpt-oss-20b-hermes_agent-tool-finetune_gguf) — llama.cpp / Ollama / LM Studio + +## License + +Apache-2.0, inherited from the base model. No additional restrictions. + +## Citation + +```bibtex +@misc{fesalfayed_gptoss20b_hermesagent_2025, + author = {Fayed, Fesal}, + title = {gpt-oss-20b Hermes-Agent tool finetune (gguf)}, + year = {2025}, + url = {https://huggingface.co/fesalfayed/gpt-oss-20b-hermes_agent-tool-finetune_gguf}, +} +``` diff --git a/assets/banner_gguf.png b/assets/banner_gguf.png new file mode 100644 index 0000000..d5efe27 --- /dev/null +++ b/assets/banner_gguf.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cd91fa5781ccaeeb324e154e643f4e5760ab04627946e899111cd689cbf7bc3b +size 150107 diff --git a/assets/cover_gguf.png b/assets/cover_gguf.png new file mode 100644 index 0000000..723aec7 --- /dev/null +++ b/assets/cover_gguf.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5d369a1de9551917793be6b566ae6bd0531f72719ad7c6d95ed42acd08b26e42 +size 137781 diff --git a/gpt-oss-20b-hermes.Q3_K_M.gguf b/gpt-oss-20b-hermes.Q3_K_M.gguf new file mode 100644 index 0000000..1364f92 --- /dev/null +++ b/gpt-oss-20b-hermes.Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e9e33160e0fa9cbb9c6b21f230bb44702305cc32d6669e97d354649ca2be18d1 +size 12916149760 diff --git a/gpt-oss-20b-hermes.Q4_K_M.gguf b/gpt-oss-20b-hermes.Q4_K_M.gguf new file mode 100644 index 0000000..ac2c01f --- /dev/null +++ b/gpt-oss-20b-hermes.Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e7827781cc95655615d97fa824ffd5373d86915ae6c2fb0d144a3b78476a361d +size 15805135360 diff --git a/gpt-oss-20b-hermes.Q5_K_M.gguf b/gpt-oss-20b-hermes.Q5_K_M.gguf new file mode 100644 index 0000000..a4b6aec --- /dev/null +++ b/gpt-oss-20b-hermes.Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1079f90fa051aa577d0b63627a031d12cef3da3911c0d9972608f35416ed17d3 +size 16893061120 diff --git a/gpt-oss-20b-hermes.Q8_0.gguf b/gpt-oss-20b-hermes.Q8_0.gguf new file mode 100644 index 0000000..2e47d4c --- /dev/null +++ b/gpt-oss-20b-hermes.Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:57e6dd6b4435156b48d34cdbf7903ec6566e395676ebc92d66e78da155f48ae1 +size 22261911040 diff --git a/gpt-oss-20b-hermes.f16.gguf b/gpt-oss-20b-hermes.f16.gguf new file mode 100644 index 0000000..4225715 --- /dev/null +++ b/gpt-oss-20b-hermes.f16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a95be5e9b6506ae91b6c247c1532e34fc88e22ec564da377e44f4764967eac90 +size 41860887040