初始化项目,由ModelHub XC社区提供模型
Model: kth8/gemma-3-270m-it-Conversation-GGUF Source: Original Platform
This commit is contained in:
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
gemma-3-270m-it-Conversation-bf16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
99
README.md
Normal file
99
README.md
Normal file
@@ -0,0 +1,99 @@
|
|||||||
|
---
|
||||||
|
license: gemma
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
base_model: kth8/gemma-3-270m-it-Conversation
|
||||||
|
datasets:
|
||||||
|
- kth8/multi-turn-conversation-50000x
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
library_name: transformers
|
||||||
|
tags:
|
||||||
|
- sft
|
||||||
|
- trl
|
||||||
|
- unsloth
|
||||||
|
- gemma
|
||||||
|
- gemma3
|
||||||
|
- gemma3_text
|
||||||
|
---
|
||||||
|

|
||||||
|
A supervised fine-tune of [unsloth/gemma-3-270m-it](https://huggingface.co/unsloth/gemma-3-270m-it) on the [kth8/multi-turn-conversation-50000x](https://huggingface.co/datasets/kth8/multi-turn-conversation-50000x) dataset.
|
||||||
|
|
||||||
|
## Usage example
|
||||||
|
|
||||||
|
**System prompt**
|
||||||
|
```
|
||||||
|
You are a helpful assistant.
|
||||||
|
```
|
||||||
|
**User prompt**
|
||||||
|
```
|
||||||
|
Hey there! How's it going?
|
||||||
|
```
|
||||||
|
**Assistant response**
|
||||||
|
```
|
||||||
|
Hey! I'm doing great, thanks for asking! I'm here and ready to help with whatever you need. What's on your mind today?
|
||||||
|
```
|
||||||
|
## Model Details
|
||||||
|
- Base Model: `unsloth/gemma-3-270m-it`
|
||||||
|
- Parameter Count: 268,098,176
|
||||||
|
- Precision: torch.bfloat16
|
||||||
|
|
||||||
|
## Training Settings
|
||||||
|
|
||||||
|
### PEFT
|
||||||
|
- Rank: 32
|
||||||
|
- LoRA alpha: 64
|
||||||
|
- Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
|
||||||
|
- Gradient checkpointing: unsloth
|
||||||
|
|
||||||
|
### SFT
|
||||||
|
- Epoch: 1
|
||||||
|
- Batch size: 8
|
||||||
|
- Gradient Accumulation steps: 2
|
||||||
|
- Learning rate: 0.0002
|
||||||
|
- Optimizer: adamw_torch_fused
|
||||||
|
- Learning rate scheduler: cosine
|
||||||
|
- Warmup steps: 100
|
||||||
|
- Weight decay: 0.01
|
||||||
|
|
||||||
|
## Training stats
|
||||||
|
- Date: 2026-06-16T15:15:17.789826
|
||||||
|
- GPU: NVIDIA L4
|
||||||
|
- Peak VRAM usage: 16.455 GB
|
||||||
|
- Global step: 3120
|
||||||
|
- Training runtime (seconds): 18261.5241
|
||||||
|
- Best validation loss: 1.666245937347412
|
||||||
|
|
||||||
|
| Step | Training Loss | Validation Loss |
|
||||||
|
|------|---------------|-----------------|
|
||||||
|
| 0 | No log | 2.784440 |
|
||||||
|
| 155 | 1.882700 | 1.881819 |
|
||||||
|
| 310 | 1.805000 | 1.832387 |
|
||||||
|
| 465 | 1.803100 | 1.804098 |
|
||||||
|
| 620 | 1.781600 | 1.782886 |
|
||||||
|
| 775 | 1.785700 | 1.765646 |
|
||||||
|
| 930 | 1.776400 | 1.749293 |
|
||||||
|
| 1085 | 1.753500 | 1.736082 |
|
||||||
|
| 1240 | 1.732600 | 1.725711 |
|
||||||
|
| 1395 | 1.703100 | 1.715472 |
|
||||||
|
| 1550 | 1.730700 | 1.705917 |
|
||||||
|
| 1705 | 1.713500 | 1.697924 |
|
||||||
|
| 1860 | 1.725500 | 1.690107 |
|
||||||
|
| 2015 | 1.707200 | 1.684427 |
|
||||||
|
| 2170 | 1.687700 | 1.678853 |
|
||||||
|
| 2325 | 1.675800 | 1.674952 |
|
||||||
|
| 2480 | 1.723100 | 1.671108 |
|
||||||
|
| 2635 | 1.684300 | 1.668909 |
|
||||||
|
| 2790 | 1.692800 | 1.667304 |
|
||||||
|
| 2945 | 1.663200 | 1.666461 |
|
||||||
|
| 3100 | 1.676500 | 1.666246 |
|
||||||
|
|
||||||
|
## Framework versions
|
||||||
|
- Unsloth: 2026.6.7
|
||||||
|
- TRL: 0.22.2
|
||||||
|
- Transformers: 4.56.2
|
||||||
|
- Pytorch: 2.11.0+cu128
|
||||||
|
- Datasets: 5.0.0
|
||||||
|
- Tokenizers: 0.22.2
|
||||||
|
|
||||||
|
## License
|
||||||
|
This model is released under the Gemma license. See the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) and [Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy) regarding the use of Gemma-generated content.
|
||||||
3
gemma-3-270m-it-Conversation-bf16.gguf
Normal file
3
gemma-3-270m-it-Conversation-bf16.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:5f2b38b39ce67a5ec99140f61ba81a5c4091ca21878283e1189f06fb6e4619f6
|
||||||
|
size 542835744
|
||||||
Reference in New Issue
Block a user