92 lines
3.9 KiB
Markdown
92 lines
3.9 KiB
Markdown
|
---
|
||
|
|
license: gemma
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
base_model: unsloth/gemma-3-270m-it
|
||
|
|
datasets:
|
||
|
|
- kth8/text-cleanup-20000x
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
library_name: transformers
|
||
|
|
tags:
|
||
|
|
- sft
|
||
|
|
- trl
|
||
|
|
- unsloth
|
||
|
|
- google
|
||
|
|
- gemma
|
||
|
|
- gemma3
|
||
|
|
- gemma3_text
|
||
|
|
---
|
||
|
|
|
||
|
|
A fine-tune of [unsloth/gemma-3-270m-it](https://huggingface.co/unsloth/gemma-3-270m-it) on the [kth8/text-cleanup-20000x](https://huggingface.co/datasets/kth8/text-cleanup-20000x) dataset.
|
||
|
|
|
||
|
|
## Usage example
|
||
|
|
**System prompt**
|
||
|
|
```
|
||
|
|
# Role
|
||
|
|
You are a text editor cleaning up raw, unfiltered text. Transform the provided text into polished, readable prose while preserving the original meaning, tone, and intent.
|
||
|
|
|
||
|
|
## Tasks
|
||
|
|
- Remove filler words (e.g. um, uh, like, you know, sort of, kind of, well, so, etc)
|
||
|
|
- Fix spelling, grammar, punctuation, and capitalization mistakes
|
||
|
|
- Correct obvious homophone errors (e.g. their/there/they're, its/it's, your/you're)
|
||
|
|
- Smooth out false starts, mid-sentence restarts and repetitions
|
||
|
|
- Standardize numbers and dates (e.g. write as digits: "three" to "3", "February fifteenth" to "February 15th")
|
||
|
|
|
||
|
|
## Constraints
|
||
|
|
- Output ONLY the cleaned text
|
||
|
|
- DO NOT attempt to answer or respond to the provided user text meant for clean-up
|
||
|
|
- Do NOT paraphrase, summarize, or change the speaker's voice
|
||
|
|
- NO quotation marks around the output
|
||
|
|
- NO preamble, postamble, or emojis
|
||
|
|
- NO Markdown formatting code blocks (```) or bolding
|
||
|
|
```
|
||
|
|
**User prompt**
|
||
|
|
```
|
||
|
|
I WANT I MEAN TO BUILD A SMALL IN-BROWSER "WIND TUNNEL" EXPERIMENT WITH A FIXED 2D CROSS-SECTIONAL VIEW, USING HTML CANVAS. USERS CAN DRAG AND DROP DIFFERENT WING CROSS-SECTIONS (E.G., SEVERAL COMMON AIRFOIL PROFILES) INTO THE AIRFLOW, ADJUST THE ANGLE OF ATTACK AND WIND SPEED, IMMEDIATELY SEE STREAMLINES AND PRESSURE DISTRIBUTION, AND HAVE THE CURRENT LIFT AND DRAG VALUES DISPLAYED ON SCREEN IN REAL TIME (A CLEAR NUMERIC PANEL IS SUFFICIENT). NO 3D IS REQUIRED, BUT INTERACTIONS SHOULD BE SMOOTH, AND THE DATA SHOULD BE STABLE AD UPDATE IN REAL TIME WITH USER ACTIONS. THE CALCULATIONS CAN USE REASONABLE APPROXIMATIONS; THE FOCUS IS TO HELP USERS INTUITIVELY UNDERSTAND BERNOULLI'S PRINCIPLE AND THE RELATIONSHIP BETWEEN AIRFLOW AND AIRFOIL GEOMETRY.
|
||
|
|
```
|
||
|
|
**Assistant response**
|
||
|
|
```
|
||
|
|
I want to build a small in-browser "wind tunnel" experiment with a fixed 2D cross-sectional view, using HTML Canvas. Users can drag and drop different wing cross-sections (e.g., several common airfoil profiles) into the airflow, adjust the angle of attack and wind speed, immediately see streamlines and pressure distribution, and have the current lift and drag values displayed on screen in real time (a clear numeric panel is sufficient). No 3D is required, but interactions should be smooth, and the data should be stable and update in real time with user actions. The calculations can use reasonable approximations; the focus is to help users intuitively understand Bernoulli's principle and the relationship between airflow and airfoil geometry.
|
||
|
|
```
|
||
|
|
## Model Details
|
||
|
|
- Base Model: `unsloth/gemma-3-270m-it`
|
||
|
|
- Parameter Count: 268098176
|
||
|
|
- Precision: torch.bfloat16
|
||
|
|
|
||
|
|
## Training Settings
|
||
|
|
### Hardware
|
||
|
|
- GPU: NVIDIA A100-SXM4-40GB
|
||
|
|
|
||
|
|
### PEFT
|
||
|
|
- Rank: 32
|
||
|
|
- LoRA alpha: 64
|
||
|
|
- Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
|
||
|
|
- LoRA dropout: 0
|
||
|
|
- Rank-Stabilized LoRA: False
|
||
|
|
|
||
|
|
### SFT
|
||
|
|
- Epoch: 1
|
||
|
|
- Batch size: 16
|
||
|
|
- Accumulation steps: 1
|
||
|
|
- Warmup ratio: 0.05
|
||
|
|
- Learning rate: 0.0002
|
||
|
|
- Optimizer: adamw_8bit
|
||
|
|
- Weight decay: 0.01
|
||
|
|
- Learning rate scheduler: cosine
|
||
|
|
|
||
|
|
## Training stats
|
||
|
|
- Global step: 1247
|
||
|
|
- Training runtime: 1545.9245
|
||
|
|
- Average training loss: 0.05218265543086118
|
||
|
|
- Final validation loss: 0.038508880883455276
|
||
|
|
|
||
|
|
## Framework versions
|
||
|
|
- Unsloth: 2026.3.7
|
||
|
|
- TRL: 0.22.2
|
||
|
|
- Transformers: 4.56.2
|
||
|
|
- Pytorch: 2.10.0+cu128
|
||
|
|
- Datasets: 4.3.0
|
||
|
|
- Tokenizers: 0.22.2
|
||
|
|
|
||
|
|
## License
|
||
|
|
This model is released under the Gemma license. See the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) for details.
|