commit 814da401f5cfc73cc8528041e1588650178cea6a Author: ModelHub XC Date: Tue Sep 1 15:14:16 2026 +0800 初始化项目,由ModelHub XC社区提供模型 Model: ledgergap/Pollux-4B-Judge-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..11fd5be --- /dev/null +++ b/.gitattributes @@ -0,0 +1,37 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Pollux-4B-Judge.BF16.gguf filter=lfs diff=lfs merge=lfs -text +Pollux-4B-Judge.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Pollux-4B-Judge.BF16.gguf b/Pollux-4B-Judge.BF16.gguf new file mode 100644 index 0000000..9469f77 --- /dev/null +++ b/Pollux-4B-Judge.BF16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f2a3ede8f4bd29033dadc82c642be26449b39fa48979a05ac816eeb7468981af +size 8051285216 diff --git a/Pollux-4B-Judge.Q8_0.gguf b/Pollux-4B-Judge.Q8_0.gguf new file mode 100644 index 0000000..69fbef2 --- /dev/null +++ b/Pollux-4B-Judge.Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:31257c7fd02626c0ea33e9bf48d32caa193b7a8c676c00975dd2867f616a83bb +size 4280405216 diff --git a/README.md b/README.md new file mode 100644 index 0000000..da17c22 --- /dev/null +++ b/README.md @@ -0,0 +1,114 @@ +--- +license: mit + +language: +- ru + +base_model: ai-forever/Pollux-4B-Judge + +base_model_relation: quantized + +library_name: gguf + +pipeline_tag: text-generation + +tags: + +- gguf + +- llama.cpp + +- lmstudio + +- qwen3 + +- pollux + +- russian + +- llm-as-a-judge + +- quantized +quantized_by: ledgergap +--- + +# Pollux-4B-Judge GGUF + +This repository contains GGUF versions of [`ai-forever/Pollux-4B-Judge`](https://huggingface.co/ai-forever/Pollux-4B-Judge) for local inference with llama.cpp, LM Studio, and other GGUF-compatible runtimes. + +Pollux-4B-Judge is a Russian-oriented LLM-as-a-judge model based on Qwen3-4B. It is intended for evaluating model answers against a specific criterion and scoring rubric. + +## Files + +| File | Type | Quantized | Notes | +|---|---:|---:|---| +| `Pollux-4B-Judge.BF16.gguf` | BF16 GGUF conversion | No | High-precision reference version | +| `Pollux-4B-Judge.Q8_0.gguf` | Q8_0 GGUF quantization | Yes | High-quality quantized version | + +## Which file should I use? + +Use `Pollux-4B-Judge.BF16.gguf` if you want the highest-quality reference version. + +Use `Pollux-4B-Judge.Q8_0.gguf` if you want a practical local version with lower memory usage and minimal expected quality loss. + +## Recommended inference settings + +For judge-style usage, the original model card uses: + +| Setting | Value | +|---|---:| +| Temperature | `0.0` | +| Max tokens | `512` | + +For local GGUF inference, choose a context length large enough to fit the full evaluation prompt: instruction, reference answer, evaluated answer, criterion, and rubric. A practical starting point is `8192`, but this is a local runtime recommendation rather than an official value from the original model card. + +The model is intended to evaluate one criterion per request. + +## Prompt format + +Recommended prompt structure: + +```text +### Задание для оценки: +{instruction} + +### Эталонный ответ: +{reference_answer} + +### Ответ для оценки: +{answer} + +### Критерий оценки: +{criterion} + +### Шкала оценивания по критерию: +{rubric} +``` + +## Use with llama.cpp + +BF16: + +```bash +llama-server -hf ledgergap/Pollux-4B-Judge-GGUF:BF16 -c 8192 -ngl 99 +``` + +Q8_0: + +```bash +llama-server -hf ledgergap/Pollux-4B-Judge-GGUF:Q8_0 -c 8192 -ngl 99 +``` + +## Use with LM Studio + +Open LM Studio and paste this repository URL into the model search/download field: + +```text +https://huggingface.co/ledgergap/Pollux-4B-Judge-GGUF +``` + +Then select either the BF16 or Q8_0 GGUF file. + +## Original model + +Original model: [`ai-forever/Pollux-4B-Judge`](https://huggingface.co/ai-forever/Pollux-4B-Judge) \ No newline at end of file