64 lines
3.1 KiB
Markdown
64 lines
3.1 KiB
Markdown
|
---
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
library_name: transformers
|
||
|
|
tags: []
|
||
|
|
---
|
||
|
|
|
||
|
|
### Model Description
|
||
|
|
|
||
|
|
|
||
|
|
BharatGPT mini is a Transformer-based language model pretrained on a large corpus of publicly available text data using a self-supervised learning approach. This means the model was trained without any human-labeled annotations—learning directly from raw text using an automatic mechanism to generate training signals.
|
||
|
|
|
||
|
|
During pretraining, BharatGPT mini was optimized for the causal language modeling task: given a sequence of tokens, the model learns to predict the next token in the sequence. More specifically, it takes a sequence of continuous text as input and is trained to predict the next word or subword by shifting the target sequence one position to the right. A masking mechanism ensures that predictions for token i are based only on tokens from positions 1 to i, without peeking at future tokens. This preserves the autoregressive nature of language modeling.
|
||
|
|
|
||
|
|
Through this training process, BharatGPT mini develops a deep internal understanding of language patterns, grammar, and semantics. While it can be fine-tuned for various downstream tasks such as classification, summarization, or question answering, it performs best in text generation tasks, which align with its original training objective.
|
||
|
|
|
||
|
|
|
||
|
|
```python
|
||
|
|
import torch
|
||
|
|
from transformers import GPT2Tokenizer, GPT2LMHeadModel
|
||
|
|
|
||
|
|
# Load tokenizer and model
|
||
|
|
tokenizer = GPT2Tokenizer.from_pretrained("CoRover/BharatGPT-mini")
|
||
|
|
model = GPT2LMHeadModel.from_pretrained("CoRover/BharatGPT-mini")
|
||
|
|
|
||
|
|
model.eval()
|
||
|
|
|
||
|
|
# Input text
|
||
|
|
text = "Future of AI"
|
||
|
|
|
||
|
|
# Tokenize
|
||
|
|
inputs = tokenizer(
|
||
|
|
text,
|
||
|
|
return_tensors="pt"
|
||
|
|
)
|
||
|
|
|
||
|
|
# Generate text
|
||
|
|
with torch.no_grad():
|
||
|
|
output_ids = model.generate(
|
||
|
|
**inputs,
|
||
|
|
max_length=100,
|
||
|
|
do_sample=True,
|
||
|
|
top_p=0.95,
|
||
|
|
top_k=50,
|
||
|
|
temperature=0.8,
|
||
|
|
repetition_penalty=1.1,
|
||
|
|
eos_token_id=tokenizer.eos_token_id
|
||
|
|
)
|
||
|
|
|
||
|
|
# Decode output
|
||
|
|
generated_text = tokenizer.decode(output_ids[0], skip_special_tokens=True)
|
||
|
|
print(generated_text)
|
||
|
|
```
|
||
|
|
|
||
|
|
It is best suited for S-RAG (Secure Retrieval-Augmented Generation) or fine-tuning with your own data. For enhanced performance, integration with Conversational Agentic AI platform is recommended (though not mandatory). This platform enables the creation of multi-modal and multi-lingual AI Agents, Co-Pilots, and Virtual Assistants (such as ChatBots, VoiceBots, and VideoBots) using a sovereign AI and composite AI approach. It leverages classic NLP, grounded generative AI with BharatGPT, and Generally Available LLMs to deliver powerful, versatile AI solutions.
|
||
|
|
|
||
|
|
## Usage and Limitations
|
||
|
|
|
||
|
|
- **License:** Non-Commercial. For academic and research purposes only. For commercial use, please visit [Conversational Gen AI platform](https://builder.corover.ai) or [Contact Us](https://corover.ai/contact/).
|
||
|
|
|
||
|
|
- **Terms of Use:** [Terms and Conditions](https://corover.ai/terms-conditions/)
|
||
|
|
|
||
|
|
- **Responsible AI Framework**: [CoRover's Responsible AI Framework](https://corover.ai/responsible-generative-ai-key-factors-for-ai-safety-and-trust/)
|
||
|
|
|
||
|
|
- **Developed by:** CoRover.ai
|