Pollux-4B-Judge-GGUF/README.md

license, language, base_model, base_model_relation, library_name, pipeline_tag, tags, quantized_by

license	language	base_model	base_model_relation	library_name	pipeline_tag	tags	quantized_by
mit	ru	ai-forever/Pollux-4B-Judge	quantized	gguf	text-generation	gguf llama.cpp lmstudio qwen3 pollux russian llm-as-a-judge quantized	ledgergap

Pollux-4B-Judge GGUF

This repository contains GGUF versions of ai-forever/Pollux-4B-Judge for local inference with llama.cpp, LM Studio, and other GGUF-compatible runtimes.

Pollux-4B-Judge is a Russian-oriented LLM-as-a-judge model based on Qwen3-4B. It is intended for evaluating model answers against a specific criterion and scoring rubric.

Files

File	Type	Quantized	Notes
Pollux-4B-Judge.BF16.gguf	BF16 GGUF conversion	No	High-precision reference version
Pollux-4B-Judge.Q8_0.gguf	Q8_0 GGUF quantization	Yes	High-quality quantized version

Which file should I use?

Use Pollux-4B-Judge.BF16.gguf if you want the highest-quality reference version.

Use Pollux-4B-Judge.Q8_0.gguf if you want a practical local version with lower memory usage and minimal expected quality loss.

Recommended inference settings

For judge-style usage, the original model card uses:

Setting	Value
Temperature	0.0
Max tokens	512

For local GGUF inference, choose a context length large enough to fit the full evaluation prompt: instruction, reference answer, evaluated answer, criterion, and rubric. A practical starting point is 8192, but this is a local runtime recommendation rather than an official value from the original model card.

The model is intended to evaluate one criterion per request.

Prompt format

Recommended prompt structure:

### Задание для оценки:
{instruction}

### Эталонный ответ:
{reference_answer}

### Ответ для оценки:
{answer}

### Критерий оценки:
{criterion}

### Шкала оценивания по критерию:
{rubric}

Use with llama.cpp

BF16:

llama-server -hf ledgergap/Pollux-4B-Judge-GGUF:BF16 -c 8192 -ngl 99

Q8_0:

llama-server -hf ledgergap/Pollux-4B-Judge-GGUF:Q8_0 -c 8192 -ngl 99

Use with LM Studio

Open LM Studio and paste this repository URL into the model search/download field:

https://huggingface.co/ledgergap/Pollux-4B-Judge-GGUF

Then select either the BF16 or Q8_0 GGUF file.

Original model

Original model: ai-forever/Pollux-4B-Judge