supra-title-50M-pre-gguf/README.md

---
license: apache-2.0
datasets:
- SupraLabs/chat-titles-filtered-115K
- SupraLabs/supra-wild-titles-130k
language:
- en
base_model:
- SupraLabs/supra-title-50m-pre
pipeline_tag: text-generation
tags:
- chat
- title
- chat-title
- title-generation
- title_generator
- topics
- topic-maker
---

<h1 align="center">Supra Title 50M · GGUF format</h1>


![Supra Title 50M pre](https://cdn-uploads.huggingface.co/production/uploads/68a5d0966d33a07f8aad2e51/IDkrEH4HRv0UkrMmFWu3f.png)

<h3 align="center">Introduction</h3>

Introducing Supra Title 50M, our latest and most compact chat title generation model. Built on our newest 50M-parameter foundation model, it features an extended context length for its size, enabling stronger input understanding and more accurate title generation. Designed for efficiency, Supra Title 50M delivers fast inference while advancing the capabilities of small language models. 

---

## About Model...

Supra Title uses SupraLab's latest foundation model (`SupraLabs/Supra-1.5-50M-Base-exp`) to power title generation with SupraLab's title dataset (which currently is the world's biggest dataset for chat titles).

The model features an extended 4k context length, making this one of only models that can run seamlessly on a wide range of hardware, ranging from very low-end CPUs to server-grade GPU clusters.

The amazing team at SupraLabs has open-sourced the chat title dataset and the models for finetuning & inference!

| model | repository |
|--------|------------|
| SupraLabs/supra-title-50m-pre | [Supra Title 50M PREV](https://huggingface.co/SupraLabs/supra-title-50m-pre) |
| SupraLabs/supra-title-50M-pre-gguf | (this model) |


We recommend using the [Supra Title 50M PREV](https://huggingface.co/SupraLabs/supra-title-50m-pre) model for finetuning/RL, and [Supra Title 50M GGUF](https://huggingface.co/SupraLabs/supra-title-50M-pre-gguf) (at Q8) for inference.

---

## Model Generation Samples...

Our model outputs the chat title as the response, reducing complexity!

``` text
User Message > How does AI work?
Generated Title > AI Working Explained
------------------------------------------------------------
User Message > Who was Albert Einstein?
Generated Title > Albert Einstein's Origin
------------------------------------------------------------
User Message > How to make a discord server?
Generated Title > Discord Server Creation
------------------------------------------------------------
User Message > Why does my WiFi keeps disconnecting the whole time??
Generated Title > WiFi Disconnect Time
------------------------------------------------------------
User Message > How to bake a cake?
Generated Title > Baking A Cake
------------------------------------------------------------
User Message > Why is the sky blue?
Generated Title > Why Is The Sky Blue
------------------------------------------------------------
User Message > How does a car work?
Generated Title > Car Work Mechanics
------------------------------------------------------------
User Message > What is Google?
Generated Title > Google Identity Explained
------------------------------------------------------------
User Message > How does Google Maps work?
Generated Title > Google Maps Explained
------------------------------------------------------------
User Message > Who is Elon Musk?
Generated Title > Elon Musk Identity
------------------------------------------------------------
User Message > Name the top three products of Microsoft
Generated Title > Top Three Microsoft Products
------------------------------------------------------------
```

Devs can use the raw output as the title, which reduces the complexity of JSON or other formats!

---

## Model Inference...

_To run our model, you can use the provided code, which automatically downloads the model from HuggingFace._

``` Python
import os
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

def main():
    # Path to your final trained model directory
    model_path = "./title-model-final-FINAL"
    
    if not os.path.exists(model_path):
        print(f"[!] Error: Model path '{model_path}' does not exist.")
        print("Please check your OUTPUT_DIR or copy your trained model here.")
        return

    print(f"[*] Loading tokenizer from {model_path}...")
    tokenizer = AutoTokenizer.from_pretrained(model_path)

    print(f"[*] Loading model onto available device...")
    model = AutoModelForCausalLM.from_pretrained(
        model_path,
        torch_dtype=torch.bfloat16,
        device_map="auto"
    )
    model.eval()
    print("[+] Model and tokenizer loaded successfully.\n")
    print("=" * 60)
    print("Chat Title Generator Interactive Loop")
    print("Type your message/article below. Type 'exit' or 'quit' to stop.")
    print("=" * 60 + "\n")

    while True:
        try:
            user_input = input("User Message > ").strip()
        except (KeyboardInterrupt, EOFError):
            print("\n[*} Exiting...")
            break

        if not user_input:
            continue
        
        if user_input.lower() in ["exit", "quit"]:
            print("[*] Goodbye!")
            break

        # Format the prompt exactly like during training
        prompt = f"User: {user_input}\nTitle: "
        
        # Tokenize inputs
        inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
        
        # Generate with optimized hyperparameters for concise title generation
        with torch.no_grad():
            outputs = model.generate(
                **inputs,
                max_new_tokens=24,        # Titles should be very short
                do_sample=True,           # Enables dynamic sampling
                temperature=0.4,          # Lower temperature prevents random word drift
                top_p=0.85,               # Nucleus sampling for high-probability tokens
                top_k=40,                 # Limits vocabulary choices to top 40 candidates
                repetition_penalty=1.2,   # Hard safety rail against repeating words (like Emanuel Emanuel...)
                pad_token_id=tokenizer.pad_token_id,
                eos_token_id=tokenizer.eos_token_id
            )

        # Extract only the newly generated tokens
        generated_ids = outputs[0][inputs["input_ids"].shape[1]:]
        predicted_title = tokenizer.decode(generated_ids, skip_special_tokens=True).strip()

        print(f"Generated Title > {predicted_title}")
        print("-" * 60)

if __name__ == "__main__":
    main()
```


With this code, you can run `supra-title-50M-pre-gguf` in your terminal. After loading in, type your prompt to get chat titles!
Our model works by analyzing the user input that a user sends to an LLM, then generating a 2-4 word title!

We recommend using the current settings, or:

``` text
- temp=0.55
- top_K=15
- top_P=0.85
- repitition_penalty=1.35
- max_tok=10
```

---

## Citation...

_If you use this model, kindly cite SupraLabs_ 🤗

```bibtex
@misc{supralabs2026,
  author       = {{SupraLabs}},
  title        = {SupraLabs Open Research},
  year         = {2026},
  publisher    = {Hugging Face},
  organization = {SupraLabs},
  url          = {https://huggingface.co/SupraLabs}
}
```