Tonic/GemmaX2-28-2B-gguf
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: Tonic/GemmaX2-28-2B-gguf
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-04-20 13:50:06 +08:00
commit 91d1d96ccf
9 changed files with 455 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

40
.gitattributes vendored Normal file
View File

@@ -0,0 +1,40 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
gemmax2-28-2b-f16.gguf filter=lfs diff=lfs merge=lfs -text
gemmax2-28-2b-bf16.gguf filter=lfs diff=lfs merge=lfs -text
gemmax2-28-2b-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
gemmax2-28-2b-tq1_0.gguf filter=lfs diff=lfs merge=lfs -text
gemmax2-28-2b-tq2_0.gguf filter=lfs diff=lfs merge=lfs -text

186
README.md Normal file
View File

@@ -0,0 +1,186 @@
---
license: apache-2.0
language:
- ar
- bn
- cs
- de
- en
- es
- fa
- fr
- he
- hi
- id
- it
- ja
- km
- ko
- lo
- ms
- my
- nl
- pl
- pt
- ru
- th
- tl
- tk
- ur
- vi
- zh
base_model:
- ModelSpace/GemmaX2-28-2B-v0.1
pipeline_tag: translation
library_name: transformers
tags:
- gemma
- translation
- multilingual
- quantized
metrics:
- bleu
- rouge
model-index:
- name: GemmaX2-28-2B-gguf
results:
- task:
type: translation
dataset:
name: google/smol
type: translation
metrics:
- name: ROUGE
type: translate
value: 0.6230
source:
name: self-eval
url: https://huggingface.co/Tonic/GemmaX2-28-2B-gguf/blob/main/results_en_es_1740935448.json
- name: GemmaX2-28-2B-gguf
results:
- task:
type: translation
dataset:
name: google/smol
type: translation
metrics:
- name: BLEU
type: translate
value: 0.6358
source:
name: self-eval
url: https://huggingface.co/Tonic/GemmaX2-28-2B-gguf/blob/main/results_en_es_1740935448.json
---
# Model Card for GemmaX2-28-2B GGUF Quantizations
## Model Overview
**GemmaX2-28-2B GGUF Quantizations** are a set of quantized variants of `GemmaX2-28-2B-v0.1`, an LLM-based translation model developed by Xiaomi. The original model was finetuned from `GemmaX2-28-2B-Pretrain`, which itself is a continually pretrained version of `Gemma2-2B` using a diverse dataset of 56 billion tokens across 28 languages. These GGUF versions (`f16`, `bf16`, `q8_0`, `tq1_0`, `tq2_0`) were created to optimize the model for efficient inference on resource-constrained environments while preserving translation capabilities.
- **Developed by**: Xiaomi (original model); quantized by Tonic
- **Model Type**: Transformer-based language model, finetuned for translation, quantized to GGUF format
- **Quantization Formats**: `f16` (16-bit float), `bf16` (bfloat16), `q8_0` (8-bit quantization), `tq1_0` (ternary quantization 1), `tq2_0` (ternary quantization 2)
- **Languages**: Arabic, Bengali, Czech, German, English, Spanish, Persian, French, Hebrew, Hindi, Indonesian, Italian, Japanese, Khmer, Korean, Lao, Malay, Burmese, Dutch, Polish, Portuguese, Russian, Thai, Tagalog, Turkish, Urdu, Vietnamese, Chinese
- **License**: [Apache 2.0]
- **Repository**: [Tonic/GemmaX2-28-2B-gguf](https://huggingface.co/Tonic/GemmaX2-28-2B-gguf)
## Model Description
`GemmaX2-28-2B-v0.1` is designed for multilingual machine translation, built on `GemmaX2-28-2B-Pretrain`, which was pretrained on a mix of monolingual and parallel data (56 billion tokens) across 28 languages. The finetuning process used a small, high-quality set of translation instruction data to enhance its performance. These GGUF quantizations were generated using `convert_hf_to_gguf.py`, converting the original Hugging Face model into formats compatible with tools like `llama.cpp` for efficient deployment.
### Quantization Details
- **Source Model**: `ModelSpace/GemmaX2-28-2B-v0.1`
- **Conversion Tool**: `convert_hf_to_gguf.py`
- **Quantization Types**:
- `f16`: 16-bit floating-point, minimal precision loss, larger file size (~5-7GB).
- `bf16`: Brain floating-point 16-bit, optimized for certain hardware (e.g., TPUs), similar size to `f16`.
- `q8_0`: 8-bit quantization, reduced size (~3-4GB), slight precision trade-off.
- `tq1_0`: Ternary quantization (1-bit), smallest size (~1-2GB), higher precision loss.
- `tq2_0`: Ternary quantization (2-bit variant), slightly larger than `tq1_0`, balanced size vs. quality.
## Intended Use
These quantized models are intended for:
- **Multilingual Translation**: Translating text across the 28 supported languages.
- **Efficient Inference**: Deployment on edge devices, low-memory systems, or environments with limited compute resources using GGUF-compatible frameworks (e.g., `llama.cpp`).
- **Research**: Studying the trade-offs between quantization levels and translation performance.
### Use Cases
- Real-time translation applications.
- Offline translation on mobile or embedded devices.
- Benchmarking quantized LLM performance in multilingual settings.
## Model Performance
The original `GemmaX2-28-2B-v0.1` model’s performance is detailed in the paper [Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study](https://arxiv.org/abs/2502.02481). Quantization introduces varying degrees of performance trade-offs:
- **`f16` and `bf16`**: Near-identical to the original model’s accuracy, with minimal degradation.
- **`q8_0`**: Slight reduction in translation quality, still suitable for most practical applications.
- **`tq1_0` and `tq2_0`**: Noticeable quality loss, best for scenarios prioritizing speed and size over precision.
Exact metrics depend on the downstream task and dataset; users are encouraged to evaluate performance for their specific use case.
## How to Use
### With Transformers (Original Model)
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "ModelSpace/GemmaX2-28-2B-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
text = "Translate this from Chinese to English:\nChinese: 我爱机器翻译\nEnglish:"
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
### With GGUF (Quantized Models)
Download a GGUF file from `Tonic/GemmaX2-28-2B-gguf` and use it with a GGUF-compatible inference tool like `llama.cpp`:
```bash
# Example with llama.cpp
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
make
# Run inference with q8_0 model
./main -m gemmax2-28-2b-q8_0.gguf -p "Translate from Chinese to English: 我爱机器翻译\nEnglish:""
```
Available files:
- `gemmax2-28-2b-f16.gguf`
- `gemmax2-28-2b-bf16.gguf`
- `gemmax2-28-2b-q8_0.gguf`
- `gemmax2-28-2b-tq1_0.gguf`
- `gemmax2-28-2b-tq2_0.gguf`
## Limitations
- **Language Support**: Only supports the 28 languages listed above; performance on unsupported languages is not guaranteed.
- **Quantization Trade-offs**: Lower-bit quantizations (`tq1_0`, `tq2_0`) may degrade translation quality, especially for complex sentences or rare language pairs.
- **Hardware Compatibility**: `bf16` benefits from specific hardware support (e.g., NVIDIA Ampere GPUs, TPUs); performance may vary otherwise.
- **Future Improvements**: The original authors plan to enhance `GemmaX2-28-2B`’s translation capabilities, which may not be reflected in these quantized versions until updated.
## Citation
For the original model:
```bibtex
@misc{cui2025multilingualmachinetranslationopen,
title={Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study},
author={Menglong Cui and Pengzhi Gao and Wei Liu and Jian Luan and Bin Wang},
year={2025},
eprint={2502.02481},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2502.02481},
}
```
For these quantized versions, please also credit:
- **Quantization by**: [Tonic](https://huggingface.co/Tonic)
- **Repository**: [Tonic/GemmaX2-28-2B-gguf](https://huggingface.co/Tonic/GemmaX2-28-2B-gguf)
## Contact
For questions about the original model, refer to Xiaomi’s publication. For issues with the GGUF quantizations, contact Tonic via Hugging Face discussions at `Tonic/GemmaX2-28-2B-gguf`.

107
evaluate.json Normal file
View File

@@ -0,0 +1,107 @@
{
"rouge": {
"rouge1": 0.6255101764812026,
"rouge2": 0.4212945982663935,
"rougeL": 0.5973657536570702,
"rougeLsum": 0.5985905113024548
},
"bleu": {
"bleu": 0.31871536850070264,
"precisions": [
0.6347305389221557,
0.39490445859872614,
0.2789115646258503,
0.19708029197080293
],
"brevity_penalty": 0.9302618655343314,
"length_ratio": 0.9325842696629213,
"translation_length": 166.0,
"reference_length": 178.0
},
"metadata": {
"model": "hf.co/Tonic/GemmaX2-28-2B-gguf:BF16",
"run_id": "1740853522",
"timestamp": 1740853573.095875,
"datetime": "2025-03-01T13:26:13.095875",
"source_language": "English",
"source_code": "en",
"target_language": "Spanish",
"target_code": "es",
"source_texts": [
"It allows me to work by following my vibes and molding my teaching style to the learning style of the audience.",
"The unrelenting soul-force of those who would hold us accountable blows that all away.",
"When taken regularly in therapeutic doses, ginger root can effectively cure nausea;",
"They accomplished their goals and created a legacy of self-determination to which others may aspire.",
"But faster economic activity could also translate into some degree of inflation.",
"The main characteristics a stone should possess are uniform surface, grain, and density.",
"Lettuce, no matter the variety, is considered a cool-weather crop.",
"It was a trick to lure foolish humans to an ogre who would teach them a very valuable lesson.",
"Just to clarify, though, assertive is not the same as aggressive or arrogant.",
"The bus, normally crowded, was stunningly nearly empty at this hour."
],
"prompts": [
"Translate this from English to Spanish:\nEnglish: It allows me to work by following my vibes and molding my teaching style to the learning style of the audience.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: The unrelenting soul-force of those who would hold us accountable blows that all away.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: When taken regularly in therapeutic doses, ginger root can effectively cure nausea;\nSpanish:",
"Translate this from English to Spanish:\nEnglish: They accomplished their goals and created a legacy of self-determination to which others may aspire.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: But faster economic activity could also translate into some degree of inflation.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: The main characteristics a stone should possess are uniform surface, grain, and density.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: Lettuce, no matter the variety, is considered a cool-weather crop.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: It was a trick to lure foolish humans to an ogre who would teach them a very valuable lesson.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: Just to clarify, though, assertive is not the same as aggressive or arrogant.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: The bus, normally crowded, was stunningly nearly empty at this hour.\nSpanish:"
],
"references": [
[
"Me permite trabajar seg\u00fan mis impresiones y ajustar mis m\u00e9todos de ense\u00f1anza al modelo de aprendizaje del p\u00fablico."
],
[
"La implacable fuerza del alma de quienes nos piden cuentas echa por tierra todo eso."
],
[
"Si se toma de forma regular en dosis terap\u00e9uticas, la ra\u00edz de jengibre puede ser un remedio efectivo contra las n\u00e1useas;"
],
[
"Han cumplido sus objetivos y creado un precedente de autodeterminaci\u00f3n al que otros es posible que aspiren."
],
[
"Sin embargo, la r\u00e1pida actividad econ\u00f3mica tambi\u00e9n podr\u00eda conllevar un cierto grado de inflaci\u00f3n."
],
[
"Las principales caracter\u00edsticas que deber\u00eda poseer una piedra son una superficie, un granulado y una densidad uniformes."
],
[
"La lechuga, independientemente de la variedad, se considera un cultivo de clima fr\u00edo."
],
[
"Fue una estrategia para llevar a ingenuos humanos hasta un ogro que les ense\u00f1ar\u00eda una muy valiosa lecci\u00f3n."
],
[
"Aunque, como mera aclaraci\u00f3n, asertivo no es lo mismo que agresivo o arrogante."
],
[
"El autob\u00fas, normalmente lleno de gente, estaba sorprendentemente casi vac\u00edo a esta hora."
]
],
"predictions": [
"Esto me permite trabajar siguiendo mis instintos y modelar mi estilo de ense\u00f1anza en funci\u00f3n del estilo de aprendizaje del p\u00fablico.",
"La implacable fuerza de los que nos responsabilizar\u00edan dispersa todo eso.",
"Si se toma regularmente en dosis terap\u00e9uticas, la ra\u00edz de jengibre puede curar eficazmente las n\u00e1useas.",
"Lograron sus objetivos y dejaron un legado de autodeterminaci\u00f3n al que otros pueden aspirar.",
"Pero una mayor actividad econ\u00f3mica podr\u00eda traducirse tambi\u00e9n en un cierto grado de inflaci\u00f3n.",
"Las principales caracter\u00edsticas que deber\u00eda tener una piedra son su superficie, grano y densidad uniformes.",
"La lechuga, independientemente de la variedad, se considera un cultivo para climas fr\u00edos.",
"Era un truco para atrapar a unos humanos tontos con el fin de ense\u00f1\u00e1rseles una valiosa lecci\u00f3n.",
"Pero, a fin de dejarlo claro, el concepto no es lo mismo que la agresividad o el orgullo.",
"El autob\u00fas estaba incre\u00edblemente vac\u00edo en esta hora normalmente tan concurrida."
],
"samples": 10
},
"telemetry": {
"runtime": 47.645625591278076,
"avg_eval_time": 0.07864224910736084,
"avg_pred_time": 4.541392850875854,
"eval_rate": 2.518594278295445,
"pred_rate": 13.211805710318231
}
}

3
gemmax2-28-2b-bf16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b47a0ecab1c83e5a8519ad97268251901e08c3f02a97743f62727f0eca471c39
size 5235213664

3
gemmax2-28-2b-f16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5ae5b246b208abb034765050d3ac160fd187bb943d30e12b2af352960237c3f2
size 5235213664

3
gemmax2-28-2b-q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a83f6f4fdac4edf9e0c954e0c516df965f4e5cf35a3c77d027be4e8673e5e7a5
size 2784494944

3
gemmax2-28-2b-tq1_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:44fa403aec99706e4ac9a6e40f21afae1a4fb8827069756048754d59eeeddda6
size 1613657440

3
gemmax2-28-2b-tq2_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a65c10a0155a846794acfb57d2d81747963fd10006f7405a7fb710e1aabae3c8
size 1708545376

107
results_en_es_1740935448.json Normal file
View File

@@ -0,0 +1,107 @@
{
"rouge": {
"rouge1": 0.6230424412060338,
"rouge2": 0.41322763891410624,
"rougeL": 0.5914651417057188,
"rougeLsum": 0.5937938122982229
},
"bleu": {
"bleu": 0.3150735589106241,
"precisions": [
0.6358381502890174,
0.3803680981595092,
0.2679738562091503,
0.17482517482517482
],
"brevity_penalty": 0.9657177024852738,
"length_ratio": 0.9662921348314607,
"translation_length": 172.0,
"reference_length": 178.0
},
"metadata": {
"model": "hf.co/Tonic/GemmaX2-28-2B-gguf:BF16",
"run_id": "1740935448",
"timestamp": 1740935505.1009474,
"datetime": "2025-03-02T18:11:45.100947",
"source_language": "English",
"source_code": "en",
"target_language": "Spanish",
"target_code": "es",
"source_texts": [
"It allows me to work by following my vibes and molding my teaching style to the learning style of the audience.",
"The unrelenting soul-force of those who would hold us accountable blows that all away.",
"When taken regularly in therapeutic doses, ginger root can effectively cure nausea;",
"They accomplished their goals and created a legacy of self-determination to which others may aspire.",
"But faster economic activity could also translate into some degree of inflation.",
"The main characteristics a stone should possess are uniform surface, grain, and density.",
"Lettuce, no matter the variety, is considered a cool-weather crop.",
"It was a trick to lure foolish humans to an ogre who would teach them a very valuable lesson.",
"Just to clarify, though, assertive is not the same as aggressive or arrogant.",
"The bus, normally crowded, was stunningly nearly empty at this hour."
],
"prompts": [
"Translate this from English to Spanish:\nEnglish: It allows me to work by following my vibes and molding my teaching style to the learning style of the audience.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: The unrelenting soul-force of those who would hold us accountable blows that all away.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: When taken regularly in therapeutic doses, ginger root can effectively cure nausea;\nSpanish:",
"Translate this from English to Spanish:\nEnglish: They accomplished their goals and created a legacy of self-determination to which others may aspire.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: But faster economic activity could also translate into some degree of inflation.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: The main characteristics a stone should possess are uniform surface, grain, and density.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: Lettuce, no matter the variety, is considered a cool-weather crop.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: It was a trick to lure foolish humans to an ogre who would teach them a very valuable lesson.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: Just to clarify, though, assertive is not the same as aggressive or arrogant.\nSpanish:",
"Translate this from English to Spanish:\nEnglish: The bus, normally crowded, was stunningly nearly empty at this hour.\nSpanish:"
],
"references": [
[
"Me permite trabajar seg\u00fan mis impresiones y ajustar mis m\u00e9todos de ense\u00f1anza al modelo de aprendizaje del p\u00fablico."
],
[
"La implacable fuerza del alma de quienes nos piden cuentas echa por tierra todo eso."
],
[
"Si se toma de forma regular en dosis terap\u00e9uticas, la ra\u00edz de jengibre puede ser un remedio efectivo contra las n\u00e1useas;"
],
[
"Han cumplido sus objetivos y creado un precedente de autodeterminaci\u00f3n al que otros es posible que aspiren."
],
[
"Sin embargo, la r\u00e1pida actividad econ\u00f3mica tambi\u00e9n podr\u00eda conllevar un cierto grado de inflaci\u00f3n."
],
[
"Las principales caracter\u00edsticas que deber\u00eda poseer una piedra son una superficie, un granulado y una densidad uniformes."
],
[
"La lechuga, independientemente de la variedad, se considera un cultivo de clima fr\u00edo."
],
[
"Fue una estrategia para llevar a ingenuos humanos hasta un ogro que les ense\u00f1ar\u00eda una muy valiosa lecci\u00f3n."
],
[
"Aunque, como mera aclaraci\u00f3n, asertivo no es lo mismo que agresivo o arrogante."
],
[
"El autob\u00fas, normalmente lleno de gente, estaba sorprendentemente casi vac\u00edo a esta hora."
]
],
"predictions": [
"Me permite trabajar siguiendo mis propias vibraciones y adaptar mi estilo de ense\u00f1anza al estilo de aprendizaje del p\u00fablico asistente.",
"El implacable poder de aquellos que nos quieren hacer responsables nos derriba a todos.",
"Si se toma regularmente en dosis terap\u00e9uticas, la ra\u00edz de jengibre puede curar eficazmente el mareo.",
"Lograron sus objetivos y dejaron un legado de autodeterminaci\u00f3n al que otros pueden aspirar.",
"Pero la mayor actividad econ\u00f3mica podr\u00eda traducirse tambi\u00e9n en un cierto grado de inflaci\u00f3n.",
"Las principales caracter\u00edsticas que debe tener una piedra son la superficie, el grano y la densidad uniformes.",
"La lechuga, independientemente de su variedad, se considera un cultivo para climas fr\u00edos.",
"Fue un truco para atraer a los humanos tontos hacia un ogro que les ense\u00f1ar\u00eda una lecci\u00f3n muy valiosa.",
"Sin embargo, aclarado esto, tener asertividad no es lo mismo que ser agresivo o arrogante.",
"El autob\u00fas estaba incre\u00edblemente vac\u00edo en esta hora de la ma\u00f1ana normalmente tan concurrida."
],
"samples": 10
},
"telemetry": {
"runtime": 15.239701747894287,
"avg_eval_time": 15.239701747894287,
"avg_pred_time": 1.1688670873641969,
"eval_rate": 3.937084924138385,
"pred_rate": 51.3317558930506
}
}