初始化项目,由ModelHub XC社区提供模型
Model: ledgergap/Pollux-4B-Judge-GGUF Source: Original Platform
This commit is contained in:
37
.gitattributes
vendored
Normal file
37
.gitattributes
vendored
Normal file
@@ -0,0 +1,37 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Pollux-4B-Judge.BF16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Pollux-4B-Judge.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
3
Pollux-4B-Judge.BF16.gguf
Normal file
3
Pollux-4B-Judge.BF16.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:f2a3ede8f4bd29033dadc82c642be26449b39fa48979a05ac816eeb7468981af
|
||||||
|
size 8051285216
|
||||||
3
Pollux-4B-Judge.Q8_0.gguf
Normal file
3
Pollux-4B-Judge.Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:31257c7fd02626c0ea33e9bf48d32caa193b7a8c676c00975dd2867f616a83bb
|
||||||
|
size 4280405216
|
||||||
114
README.md
Normal file
114
README.md
Normal file
@@ -0,0 +1,114 @@
|
|||||||
|
---
|
||||||
|
license: mit
|
||||||
|
|
||||||
|
language:
|
||||||
|
- ru
|
||||||
|
|
||||||
|
base_model: ai-forever/Pollux-4B-Judge
|
||||||
|
|
||||||
|
base_model_relation: quantized
|
||||||
|
|
||||||
|
library_name: gguf
|
||||||
|
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
|
||||||
|
tags:
|
||||||
|
|
||||||
|
- gguf
|
||||||
|
|
||||||
|
- llama.cpp
|
||||||
|
|
||||||
|
- lmstudio
|
||||||
|
|
||||||
|
- qwen3
|
||||||
|
|
||||||
|
- pollux
|
||||||
|
|
||||||
|
- russian
|
||||||
|
|
||||||
|
- llm-as-a-judge
|
||||||
|
|
||||||
|
- quantized
|
||||||
|
quantized_by: ledgergap
|
||||||
|
---
|
||||||
|
|
||||||
|
# Pollux-4B-Judge GGUF
|
||||||
|
|
||||||
|
This repository contains GGUF versions of [`ai-forever/Pollux-4B-Judge`](https://huggingface.co/ai-forever/Pollux-4B-Judge) for local inference with llama.cpp, LM Studio, and other GGUF-compatible runtimes.
|
||||||
|
|
||||||
|
Pollux-4B-Judge is a Russian-oriented LLM-as-a-judge model based on Qwen3-4B. It is intended for evaluating model answers against a specific criterion and scoring rubric.
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
| File | Type | Quantized | Notes |
|
||||||
|
|---|---:|---:|---|
|
||||||
|
| `Pollux-4B-Judge.BF16.gguf` | BF16 GGUF conversion | No | High-precision reference version |
|
||||||
|
| `Pollux-4B-Judge.Q8_0.gguf` | Q8_0 GGUF quantization | Yes | High-quality quantized version |
|
||||||
|
|
||||||
|
## Which file should I use?
|
||||||
|
|
||||||
|
Use `Pollux-4B-Judge.BF16.gguf` if you want the highest-quality reference version.
|
||||||
|
|
||||||
|
Use `Pollux-4B-Judge.Q8_0.gguf` if you want a practical local version with lower memory usage and minimal expected quality loss.
|
||||||
|
|
||||||
|
## Recommended inference settings
|
||||||
|
|
||||||
|
For judge-style usage, the original model card uses:
|
||||||
|
|
||||||
|
| Setting | Value |
|
||||||
|
|---|---:|
|
||||||
|
| Temperature | `0.0` |
|
||||||
|
| Max tokens | `512` |
|
||||||
|
|
||||||
|
For local GGUF inference, choose a context length large enough to fit the full evaluation prompt: instruction, reference answer, evaluated answer, criterion, and rubric. A practical starting point is `8192`, but this is a local runtime recommendation rather than an official value from the original model card.
|
||||||
|
|
||||||
|
The model is intended to evaluate one criterion per request.
|
||||||
|
|
||||||
|
## Prompt format
|
||||||
|
|
||||||
|
Recommended prompt structure:
|
||||||
|
|
||||||
|
```text
|
||||||
|
### Задание для оценки:
|
||||||
|
{instruction}
|
||||||
|
|
||||||
|
### Эталонный ответ:
|
||||||
|
{reference_answer}
|
||||||
|
|
||||||
|
### Ответ для оценки:
|
||||||
|
{answer}
|
||||||
|
|
||||||
|
### Критерий оценки:
|
||||||
|
{criterion}
|
||||||
|
|
||||||
|
### Шкала оценивания по критерию:
|
||||||
|
{rubric}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Use with llama.cpp
|
||||||
|
|
||||||
|
BF16:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
llama-server -hf ledgergap/Pollux-4B-Judge-GGUF:BF16 -c 8192 -ngl 99
|
||||||
|
```
|
||||||
|
|
||||||
|
Q8_0:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
llama-server -hf ledgergap/Pollux-4B-Judge-GGUF:Q8_0 -c 8192 -ngl 99
|
||||||
|
```
|
||||||
|
|
||||||
|
## Use with LM Studio
|
||||||
|
|
||||||
|
Open LM Studio and paste this repository URL into the model search/download field:
|
||||||
|
|
||||||
|
```text
|
||||||
|
https://huggingface.co/ledgergap/Pollux-4B-Judge-GGUF
|
||||||
|
```
|
||||||
|
|
||||||
|
Then select either the BF16 or Q8_0 GGUF file.
|
||||||
|
|
||||||
|
## Original model
|
||||||
|
|
||||||
|
Original model: [`ai-forever/Pollux-4B-Judge`](https://huggingface.co/ai-forever/Pollux-4B-Judge)
|
||||||
Reference in New Issue
Block a user