初始化项目，由ModelHub XC社区提供模型

Model: davron04/gemma-3-270m-dueta Source: Original Platform
2026-07-24 06:09:11 +08:00
commit 5cb898efb5
9 changed files with 260 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,85 @@
+---
+library_name: transformers
+license: mit
+datasets:
+- MLDataScientist/SlimOrca-Dedup-English-Uzbek
+- ML-Jonibek/English-Uzbek-Translation-1
+- Jonibek21/English-Uzbek-Translation
+- davron04/wikimedia_v20230407_en_uz
+language:
+- uz
+- en
+base_model:
+- google/gemma-3-270m
+pipeline_tag: translation
+---
+
+# gemma-3-270m-dueta
+
+**DUETA** — a **D**ecoder-only, **U**zbek–**E**nglish **T**ransformer-based model for machine tr**A**nslation.
+
+This model is a fine-tuned version of [google/gemma-3-270m](https://huggingface.co/google/gemma-3-270m), adapted for bidirectional English ↔ Uzbek translation using a decoder-only architecture. The approach follows the methodology described in *DIETA: A Decoder-only transformer-based model for Italian–English machine TrAnslation*, applying the same decoder-only translation paradigm to the English–Uzbek language pair.
+
+## Model Details
+
+- **Base model:** [google/gemma-3-270m](https://huggingface.co/google/gemma-3-270m)
+- **Architecture:** Decoder-only transformer
+- **Languages:** English (`en`), Uzbek (`uz`)
+- **Task:** Machine translation (En→Uz and Uz→En)
+- **Reference paper:** DIETA: A Decoder-only transformer-based model for Italian–English machine TrAnslation
+
+## Training Data
+
+The model was fine-tuned on a combination of the following datasets:
+
+| Dataset | Source |
+|---|---|
+| `wikimedia-v20230407` | [OPUS](https://opus.nlpl.eu/) |
+| `MLDataScientist/SlimOrca-Dedup-English-Uzbek` | Hugging Face |
+| `ML-Jonibek/English-Uzbek-Translation-1` | Hugging Face |
+| `Jonibek21/English-Uzbek-Translation` | Hugging Face |
+
+## How to Use
+
+The model uses a simple prompt format with language tags (`<english>` / `<uzbek>`) to indicate the source and target languages, and generates a translation in a decoder-only, causal-LM fashion.
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+MODEL = "davron04/gemma-3-270m-dueta"
+ENGLISH_TAG = "<english>"
+UZBEK_TAG = "<uzbek>"
+
+def load_tokenizer_and_model(model_name: str):
+    tokenizer = AutoTokenizer.from_pretrained(model_name)
+    model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
+    return tokenizer, model
+
+def translate_text(source_text: str, source_lang: str, target_lang: str, tokenizer, model) -> str:
+    model_input = f"{source_lang}: {source_text}\n{target_lang}:"
+    token_ids = tokenizer.encode(model_input, return_tensors="pt").to(model.device)
+    input_token_count = token_ids.shape[1]
+    output = model.generate(token_ids)
+    generated_token_ids = output[0][input_token_count:]
+    translated_text = tokenizer.decode(generated_token_ids, skip_special_tokens=True)
+    return translated_text
+
+tokenizer, model = load_tokenizer_and_model(MODEL)
+source_text = "Hello, world! What can I do for you today?"
+translated_text = translate_text(source_text, ENGLISH_TAG, UZBEK_TAG, tokenizer, model)
+print(f"Translated text: {translated_text}")
+
+"""Salom, dunyo! Bugun siz uchun nima qilishim mumkin?"""
+```
+
+To translate from Uzbek to English, simply swap the `source_lang` and `target_lang` arguments:
+
+```python
+translated_text = translate_text(source_text, UZBEK_TAG, ENGLISH_TAG, tokenizer, model)
+```
+
+## Limitations
+
+- **Context length:** Avoid feeding the model very long context. Performance degrades with long inputs; it is recommended to **split long text into individual sentences** before translation and process them one at a time (or in short chunks) rather than passing entire paragraphs or documents at once.
+- As with any machine translation model, performance may vary across domains, informal/colloquial text, and low-resource constructs not well represented in the training data.
+- The model has not been evaluated for factual accuracy preservation in translation; it should not be used for translating sensitive or critical content without human review.
--- a/added_tokens.json
+++ b/added_tokens.json
@@ -0,0 +1,3 @@
+{
+  "<image_soft_token>": 262144
+}
--- a/config.json
+++ b/config.json
@@ -0,0 +1,62 @@
+{
+  "_sliding_window_pattern": 6,
+  "architectures": [
+    "Gemma3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attn_logit_softcapping": null,
+  "bos_token_id": 2,
+  "dtype": "bfloat16",
+  "eos_token_id": 1,
+  "final_logit_softcapping": null,
+  "head_dim": 256,
+  "hidden_activation": "gelu_pytorch_tanh",
+  "hidden_size": 640,
+  "initializer_range": 0.02,
+  "intermediate_size": 2048,
+  "layer_types": [
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "model_type": "gemma3_text",
+  "num_attention_heads": 4,
+  "num_hidden_layers": 18,
+  "num_key_value_heads": 1,
+  "pad_token_id": 0,
+  "query_pre_attn_scalar": 256,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "full_attention": {
+      "rope_theta": 1000000.0,
+      "rope_type": "default"
+    },
+    "sliding_attention": {
+      "rope_theta": 10000.0,
+      "rope_type": "default"
+    }
+  },
+  "sliding_window": 512,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.9.0",
+  "use_bidirectional_attention": false,
+  "use_cache": true,
+  "vocab_size": 262144
+}
--- a/generation_config.json
+++ b/generation_config.json
@@ -0,0 +1,10 @@
+{
+  "do_sample": false,
+  "early_stopping": true,
+  "eos_token_id": 1,
+  "max_length": 32768,
+  "no_repeat_ngram_size": 5,
+  "num_beams": 4,
+  "pad_token_id": 0,
+  "transformers_version": "5.9.0"
+}
--- a/model.safetensors
+++ b/model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d75543bfc5bab0bbd5211ad03f4703f15c8dd46eea100f3004f441dfae7b5035
+size 536223056
--- a/special_tokens_map.json
+++ b/special_tokens_map.json
@@ -0,0 +1,33 @@
+{
+  "boi_token": "<start_of_image>",
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eoi_token": "<end_of_image>",
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image_soft_token>",
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}
--- a/tokenizer.json
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:daab2354f8a74e70d70b4d1f804939b68a8c9624dd06cb7858e52dd8970e9726
+size 33384567
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
@@ -0,0 +1,25 @@
+{
+  "backend": "tokenizers",
+  "boi_token": "<start_of_image>",
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "eoi_token": "<end_of_image>",
+  "eos_token": "<eos>",
+  "image_token": "<image_soft_token>",
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "boi_token": "<start_of_image>",
+    "eoi_token": "<end_of_image>",
+    "image_token": "<image_soft_token>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "left",
+  "sp_model_kwargs": null,
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}