初始化项目，由ModelHub XC社区提供模型
Model: ledgergap/Pollux-4B-Judge-GGUF Source: Original Platform
This commit is contained in:
ModelHub XC
37
.gitattributes
vendored
Normal file
37
.gitattributes
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
Pollux-4B-Judge.BF16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Pollux-4B-Judge.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
3
Pollux-4B-Judge.BF16.gguf
Normal file
3
Pollux-4B-Judge.BF16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f2a3ede8f4bd29033dadc82c642be26449b39fa48979a05ac816eeb7468981af
|
||||
size 8051285216
|
||||
3
Pollux-4B-Judge.Q8_0.gguf
Normal file
3
Pollux-4B-Judge.Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:31257c7fd02626c0ea33e9bf48d32caa193b7a8c676c00975dd2867f616a83bb
|
||||
size 4280405216
|
||||
114
README.md
Normal file
114
README.md
Normal file
@@ -0,0 +1,114 @@
|
||||
---
|
||||
license: mit
|
||||
|
||||
language:
|
||||
- ru
|
||||
|
||||
base_model: ai-forever/Pollux-4B-Judge
|
||||
|
||||
base_model_relation: quantized
|
||||
|
||||
library_name: gguf
|
||||
|
||||
pipeline_tag: text-generation
|
||||
|
||||
tags:
|
||||
|
||||
- gguf
|
||||
|
||||
- llama.cpp
|
||||
|
||||
- lmstudio
|
||||
|
||||
- qwen3
|
||||
|
||||
- pollux
|
||||
|
||||
- russian
|
||||
|
||||
- llm-as-a-judge
|
||||
|
||||
- quantized
|
||||
quantized_by: ledgergap
|
||||
---
|
||||
|
||||
# Pollux-4B-Judge GGUF
|
||||
|
||||
This repository contains GGUF versions of [`ai-forever/Pollux-4B-Judge`](https://huggingface.co/ai-forever/Pollux-4B-Judge) for local inference with llama.cpp, LM Studio, and other GGUF-compatible runtimes.
|
||||
|
||||
Pollux-4B-Judge is a Russian-oriented LLM-as-a-judge model based on Qwen3-4B. It is intended for evaluating model answers against a specific criterion and scoring rubric.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Type | Quantized | Notes |
|
||||
|---|---:|---:|---|
|
||||
| `Pollux-4B-Judge.BF16.gguf` | BF16 GGUF conversion | No | High-precision reference version |
|
||||
| `Pollux-4B-Judge.Q8_0.gguf` | Q8_0 GGUF quantization | Yes | High-quality quantized version |
|
||||
|
||||
## Which file should I use?
|
||||
|
||||
Use `Pollux-4B-Judge.BF16.gguf` if you want the highest-quality reference version.
|
||||
|
||||
Use `Pollux-4B-Judge.Q8_0.gguf` if you want a practical local version with lower memory usage and minimal expected quality loss.
|
||||
|
||||
## Recommended inference settings
|
||||
|
||||
For judge-style usage, the original model card uses:
|
||||
|
||||
| Setting | Value |
|
||||
|---|---:|
|
||||
| Temperature | `0.0` |
|
||||
| Max tokens | `512` |
|
||||
|
||||
For local GGUF inference, choose a context length large enough to fit the full evaluation prompt: instruction, reference answer, evaluated answer, criterion, and rubric. A practical starting point is `8192`, but this is a local runtime recommendation rather than an official value from the original model card.
|
||||
|
||||
The model is intended to evaluate one criterion per request.
|
||||
|
||||
## Prompt format
|
||||
|
||||
Recommended prompt structure:
|
||||
|
||||
```text
|
||||
### Задание для оценки:
|
||||
{instruction}
|
||||
|
||||
### Эталонный ответ:
|
||||
{reference_answer}
|
||||
|
||||
### Ответ для оценки:
|
||||
{answer}
|
||||
|
||||
### Критерий оценки:
|
||||
{criterion}
|
||||
|
||||
### Шкала оценивания по критерию:
|
||||
{rubric}
|
||||
```
|
||||
|
||||
## Use with llama.cpp
|
||||
|
||||
BF16:
|
||||
|
||||
```bash
|
||||
llama-server -hf ledgergap/Pollux-4B-Judge-GGUF:BF16 -c 8192 -ngl 99
|
||||
```
|
||||
|
||||
Q8_0:
|
||||
|
||||
```bash
|
||||
llama-server -hf ledgergap/Pollux-4B-Judge-GGUF:Q8_0 -c 8192 -ngl 99
|
||||
```
|
||||
|
||||
## Use with LM Studio
|
||||
|
||||
Open LM Studio and paste this repository URL into the model search/download field:
|
||||
|
||||
```text
|
||||
https://huggingface.co/ledgergap/Pollux-4B-Judge-GGUF
|
||||
```
|
||||
|
||||
Then select either the BF16 or Q8_0 GGUF file.
|
||||
|
||||
## Original model
|
||||
|
||||
Original model: [`ai-forever/Pollux-4B-Judge`](https://huggingface.co/ai-forever/Pollux-4B-Judge)
|
||||
Reference in New Issue
Block a user