77 lines
2.4 KiB
Markdown
77 lines
2.4 KiB
Markdown
---
|
||
language:
|
||
- en
|
||
license: apache-2.0
|
||
base_model: Qwen/Qwen3-0.6B
|
||
library_name: transformers
|
||
tags:
|
||
- unsloth
|
||
- qwen3
|
||
- lora
|
||
- rewriting
|
||
- style-transfer
|
||
- unslop
|
||
pipeline_tag: text-generation
|
||
---
|
||
|
||
# qwen3-0.6b-unslop-good-lora-v1
|
||
|
||
A small pilot fine-tune of Qwen3-0.6B for unslop rewriting: taking AI-sounding passages and attempting to rewrite them into cleaner, more natural prose while preserving meaning.
|
||
|
||
This is the weakest model in the current unslop pilot family and should be treated as a proof-of-behavior artifact, not a production-ready standalone model.
|
||
|
||
## How it was trained
|
||
|
||
- Base model: `Qwen/Qwen3-0.6B`
|
||
- Training path: Unsloth 4-bit LoRA fine-tuning on Hugging Face Jobs
|
||
- Dataset: `N8Programs/unslop-good`
|
||
- Rows used: 1000 (full training split)
|
||
- Objective: conversational rewrite / style cleanup
|
||
|
||
## Intended use
|
||
|
||
Use this model as a pipeline stage for:
|
||
- rewriting AI-sounding prose into more natural text
|
||
- reducing cliché-heavy or overblown style
|
||
- experimenting with a compact unslopper before scaling to larger models
|
||
|
||
## Limitations
|
||
|
||
- pilot-sized dataset
|
||
- very small model size (0.6B)
|
||
- may introduce local coherence issues on longer passages
|
||
- may overcompress content and drop details
|
||
- should be reviewed by a human or used as one stage in a larger editing pipeline
|
||
|
||
## Example usage
|
||
|
||
```python
|
||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||
|
||
repo = "Oysiyl/qwen3-0.6b-unslop-good-lora-v1"
|
||
messages = [
|
||
{"role": "user", "content": "Polish this AI passage to feel more human while preserving meaning:\n[TEXT HERE]"}
|
||
]
|
||
|
||
tokenizer = AutoTokenizer.from_pretrained(repo)
|
||
model = AutoModelForCausalLM.from_pretrained(repo, device_map="auto")
|
||
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
||
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.8, repetition_penalty=1.1, do_sample=True)
|
||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
```
|
||
|
||
## Training loss vs training progress
|
||
|
||
|
||
|
||
Training loss is shown on normalized training progress (0–100%) for optimization visibility only. It helps show whether the run trained smoothly, but it is not by itself evidence of generalization or overfitting. Held-out rewrite fidelity remains the real decision metric.
|
||
|
||
## Judgment
|
||
|
||
Too bad output.
|
||
|
||
## Conclusion
|
||
|
||
Too bad output.
|