---
license: mit
datasets:
- lunahr/normalization-data-mixed
language:
- en
base_model:
- Qwen/Qwen3-0.6B-Base
pipeline_tag: text-generation
library_name: transformers
tags:
- text-transformation
- text-normalization
new_version: lunahr/CeluneNorm-0.6B-v1.2
---

# Model Card for CeluneNorm-0.6B-v1.1

## Model Details

### Model Description
CeluneNorm is a lightweight text normalization model designed for TTS and general preprocessing pipelines.

It converts poorly formatted input into clean, readable text while preserving the original meaning.

Example:

- Input: `this is a badly formed sentence`
- Output: `This is a badly formed sentence.`

The model is conservative by design:
- It does not rewrite sentences
- It avoids changing meaning
- It preserves domain-specific tokens (e.g. URLs, commands, names)

### Usage

The model expects input in the following format:
```
YOUR INPUT<NORM>
```

It will generate the normalized version of the input.

Inference example:
```py
from transformers import pipeline, AutoTokenizer

model_id = "lunahr/CeluneNorm-0.6B-v1.1"

tokenizer = AutoTokenizer.from_pretrained(model_id)
pipe = pipeline(
    "text-generation",
    model=model_id,
    tokenizer=model_id,
    device="cuda:0",  # "cpu" for CPU-only, slower
)

def normalize(text: str) -> str:
    history = [
        {"role": "user", "content": text}
    ]
    prompt = tokenizer.apply_chat_template(history, tokenize=False)

    out = pipe(
        prompt,
        max_new_tokens=512,
        do_sample=False,
        return_full_text=False,
    )

    return out[0]["generated_text"].strip()

# example
print(normalize("if i type something more complicated into celune it will fix it"))
```

Caution: CeluneNorm only works reliably on sequences below 128 tokens. Longer inputs may cause problems.

### Key Characteristics

- Deterministic (no sampling required)
- Preserves structure and intent
- Handles mixed text (natural language + technical content)
- Conservative punctuation (prefers `.` over `!` unless explicit)
- Supports multi-sentence normalization when boundaries are clear

---

- **Developed by:** https://huggingface.co/lunahr  
- **Model type:** Causal Language Model  
- **Language(s):** English  
- **License:** MIT  
- **Base model:** Qwen/Qwen3-0.6B-Base  

---

## Limitations

This model is not intended to be a full grammar correction system.

Possible limitations include:

- May miss some punctuation or casing corrections
- May be conservative with contractions (e.g. `there s` → unchanged)
- May preserve ambiguous casing when intent is unclear
- Does not expand slang or rewrite informal language

The model prioritizes safety and meaning preservation over aggressive correction.

---

## Training Details

### Dataset

Trained on: https://huggingface.co/datasets/lunahr/normalization-data-mixed

The dataset includes a mix of:

- Formal text (Wikipedia-style)
- Conversational text (PersonaChat)
- Synthetic edge cases
- Quoted text handling

This combination helps the model generalize across both clean and noisy inputs.

---

### Training Procedure

- Fine-tuned from Qwen3-0.6B-Base
- Hardware: Kaggle dual NVIDIA T4 (FP16)
- Training time: ~1.5 hours
- Epochs: 3

Training configuration highlights:

- Learning rate: 8e-5
- Gradient clipping: 1.0
- Warmup: 200 steps (~10%)

---

### Metrics

- Final training loss: 0.08841  
- Mean token accuracy: 97.53%  

These metrics reflect token-level accuracy; real-world normalization quality is slightly lower but more representative (~90–95% human-level correctness).

---