42 lines
1.7 KiB
Markdown
42 lines
1.7 KiB
Markdown
|
---
|
||
|
|
base_model: unsloth/Qwen3-0.6B-Base-unsloth-bnb-4bit
|
||
|
|
tags:
|
||
|
|
- text-generation-inference
|
||
|
|
- transformers
|
||
|
|
- unsloth
|
||
|
|
- qwen3
|
||
|
|
license: apache-2.0
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
datasets:
|
||
|
|
- Svngoku/black_mirror_scripts
|
||
|
|
---
|
||
|
|
|
||
|
|
## Model
|
||
|
|
|
||
|
|
| Metric | Value |
|
||
|
|
|--------|-------|
|
||
|
|
| Model | Svngoku/qwen3-black-mirror (https://huggingface.co/Svngoku/qwen3-black-mirror) |
|
||
|
|
| Base | Qwen3-0.6B-Base (4-bit) |
|
||
|
|
| Data | 33 episodes across 7 seasons, auto-merged |
|
||
|
|
| Epochs | 3 |
|
||
|
|
| Steps | 15 |
|
||
|
|
| Training time | 25 seconds |
|
||
|
|
| Loss | 3.08 -> 2.09 (epoch 2) -> 2.54 (epoch 3) |
|
||
|
|
| Upload | Merged 16-bit model (1.19 GB) |
|
||
|
|
The model was merged (LoRA weights folded into the base model) and pushed as a standalone model to Svngoku/qwen3-black-mirror. The inference test shows it's generating coherent text with some thematic influence (surveillance, privacy, ratings -- very Black Mirror).
|
||
|
|
Artifacts created:
|
||
|
|
- scripts/continued-pretraining.py -- custom script with multi-split support
|
||
|
|
- Svngoku/black-mirror-training (https://huggingface.co/datasets/Svngoku/black-mirror-training) -- HF dataset repo hosting the script
|
||
|
|
- Svngoku/qwen3-black-mirror (https://huggingface.co/Svngoku/qwen3-black-mirror) -- the trained model
|
||
|
|
- Svngoku/qwen3-black-mirror-test (https://huggingface.co/Svngoku/qwen3-black-mirror-test) -- dry-run LoRA adapter
|
||
|
|
|
||
|
|
# Uploaded finetuned model
|
||
|
|
|
||
|
|
- **Developed by:** Svngoku
|
||
|
|
- **License:** apache-2.0
|
||
|
|
- **Finetuned from model :** unsloth/Qwen3-0.6B-Base-unsloth-bnb-4bit
|
||
|
|
|
||
|
|
This qwen3 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
||
|
|
|
||
|
|
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|