From 4df0da552c107ec12c3a391ec0166c0bc8da4b8b Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Mon, 17 Aug 2026 17:01:16 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: kth8/gemma-3-270m-it-Conversation-GGUF Source: Original Platform --- .gitattributes | 36 ++++++++++ README.md | 99 ++++++++++++++++++++++++++ gemma-3-270m-it-Conversation-bf16.gguf | 3 + 3 files changed, 138 insertions(+) create mode 100644 .gitattributes create mode 100644 README.md create mode 100644 gemma-3-270m-it-Conversation-bf16.gguf diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..0ad9e1c --- /dev/null +++ b/.gitattributes @@ -0,0 +1,36 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +gemma-3-270m-it-Conversation-bf16.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..c98723a --- /dev/null +++ b/README.md @@ -0,0 +1,99 @@ +--- +license: gemma +language: +- en +base_model: kth8/gemma-3-270m-it-Conversation +datasets: +- kth8/multi-turn-conversation-50000x +pipeline_tag: text-generation +library_name: transformers +tags: +- sft +- trl +- unsloth +- gemma +- gemma3 +- gemma3_text +--- +![logo](https://storage.googleapis.com/gweb-developer-goog-blog-assets/images/gemma-3_2.original.png) +A supervised fine-tune of [unsloth/gemma-3-270m-it](https://huggingface.co/unsloth/gemma-3-270m-it) on the [kth8/multi-turn-conversation-50000x](https://huggingface.co/datasets/kth8/multi-turn-conversation-50000x) dataset. + +## Usage example + +**System prompt** +``` +You are a helpful assistant. +``` +**User prompt** +``` +Hey there! How's it going? +``` +**Assistant response** +``` +Hey! I'm doing great, thanks for asking! I'm here and ready to help with whatever you need. What's on your mind today? +``` +## Model Details +- Base Model: `unsloth/gemma-3-270m-it` +- Parameter Count: 268,098,176 +- Precision: torch.bfloat16 + +## Training Settings + +### PEFT +- Rank: 32 +- LoRA alpha: 64 +- Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj +- Gradient checkpointing: unsloth + +### SFT +- Epoch: 1 +- Batch size: 8 +- Gradient Accumulation steps: 2 +- Learning rate: 0.0002 +- Optimizer: adamw_torch_fused +- Learning rate scheduler: cosine +- Warmup steps: 100 +- Weight decay: 0.01 + +## Training stats +- Date: 2026-06-16T15:15:17.789826 +- GPU: NVIDIA L4 +- Peak VRAM usage: 16.455 GB +- Global step: 3120 +- Training runtime (seconds): 18261.5241 +- Best validation loss: 1.666245937347412 + +| Step | Training Loss | Validation Loss | +|------|---------------|-----------------| +| 0 | No log | 2.784440 | +| 155 | 1.882700 | 1.881819 | +| 310 | 1.805000 | 1.832387 | +| 465 | 1.803100 | 1.804098 | +| 620 | 1.781600 | 1.782886 | +| 775 | 1.785700 | 1.765646 | +| 930 | 1.776400 | 1.749293 | +| 1085 | 1.753500 | 1.736082 | +| 1240 | 1.732600 | 1.725711 | +| 1395 | 1.703100 | 1.715472 | +| 1550 | 1.730700 | 1.705917 | +| 1705 | 1.713500 | 1.697924 | +| 1860 | 1.725500 | 1.690107 | +| 2015 | 1.707200 | 1.684427 | +| 2170 | 1.687700 | 1.678853 | +| 2325 | 1.675800 | 1.674952 | +| 2480 | 1.723100 | 1.671108 | +| 2635 | 1.684300 | 1.668909 | +| 2790 | 1.692800 | 1.667304 | +| 2945 | 1.663200 | 1.666461 | +| 3100 | 1.676500 | 1.666246 | + +## Framework versions +- Unsloth: 2026.6.7 +- TRL: 0.22.2 +- Transformers: 4.56.2 +- Pytorch: 2.11.0+cu128 +- Datasets: 5.0.0 +- Tokenizers: 0.22.2 + +## License +This model is released under the Gemma license. See the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) and [Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy) regarding the use of Gemma-generated content. \ No newline at end of file diff --git a/gemma-3-270m-it-Conversation-bf16.gguf b/gemma-3-270m-it-Conversation-bf16.gguf new file mode 100644 index 0000000..e615ec7 --- /dev/null +++ b/gemma-3-270m-it-Conversation-bf16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5f2b38b39ce67a5ec99140f61ba81a5c4091ca21878283e1189f06fb6e4619f6 +size 542835744