---
pipeline_tag: text-generation
library_name: transformers
tags: []
---

### Model Description


BharatGPT mini is a Transformer-based language model pretrained on a large corpus of publicly available text data using a self-supervised learning approach. This means the model was trained without any human-labeled annotations—learning directly from raw text using an automatic mechanism to generate training signals.

During pretraining, BharatGPT mini was optimized for the causal language modeling task: given a sequence of tokens, the model learns to predict the next token in the sequence. More specifically, it takes a sequence of continuous text as input and is trained to predict the next word or subword by shifting the target sequence one position to the right. A masking mechanism ensures that predictions for token i are based only on tokens from positions 1 to i, without peeking at future tokens. This preserves the autoregressive nature of language modeling.

Through this training process, BharatGPT mini develops a deep internal understanding of language patterns, grammar, and semantics. While it can be fine-tuned for various downstream tasks such as classification, summarization, or question answering, it performs best in text generation tasks, which align with its original training objective.


```python
import torch
from transformers import GPT2Tokenizer, GPT2LMHeadModel

# Load tokenizer and model
tokenizer = GPT2Tokenizer.from_pretrained("CoRover/BharatGPT-mini")
model = GPT2LMHeadModel.from_pretrained("CoRover/BharatGPT-mini")

model.eval()

# Input text
text = "Future of AI"

# Tokenize
inputs = tokenizer(
    text,
    return_tensors="pt"
)

# Generate text
with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        max_length=100,
        do_sample=True,
        top_p=0.95,
        top_k=50,
        temperature=0.8,
        repetition_penalty=1.1,
        eos_token_id=tokenizer.eos_token_id
    )

# Decode output
generated_text = tokenizer.decode(output_ids[0], skip_special_tokens=True)
print(generated_text)
```

It is best suited for S-RAG (Secure Retrieval-Augmented Generation) or fine-tuning with your own data. For enhanced performance, integration with Conversational Agentic AI platform is recommended (though not mandatory). This platform enables the creation of multi-modal and multi-lingual AI Agents, Co-Pilots, and Virtual Assistants (such as ChatBots, VoiceBots, and VideoBots) using a sovereign AI and composite AI approach. It leverages classic NLP, grounded generative AI with BharatGPT, and Generally Available LLMs to deliver powerful, versatile AI solutions.

## Usage and Limitations

- **License:** Non-Commercial. For academic and research purposes only. For commercial use, please visit [Conversational Gen AI platform](https://builder.corover.ai) or [Contact Us](https://corover.ai/contact/).

- **Terms of Use:**  [Terms and Conditions](https://corover.ai/terms-conditions/)

- **Responsible AI Framework**: [CoRover's Responsible AI Framework](https://corover.ai/responsible-generative-ai-key-factors-for-ai-safety-and-trust/)

- **Developed by:** CoRover.ai