116 lines
2.6 KiB
Markdown
116 lines
2.6 KiB
Markdown
---
|
||
license: apache-2.0
|
||
pipeline_tag: text-generation
|
||
library_name: transformers
|
||
tags:
|
||
- mistral
|
||
- chatbot
|
||
---
|
||
# Roy-v1
|
||
|
||
**Roy** is a personal AI assistant model created and fine-tuned by **Souvik Pramanick**.
|
||
Designed to be helpful, conversational, and practical for everyday tasks such as learning, coding, problem solving, and general assistance.
|
||
|
||
---
|
||
|
||
## Creator
|
||
|
||
**Founder & Trainer:**
|
||
**Souvik Pramanick**
|
||
GitHub: https://github.com/Souvik18p
|
||
HuggingFace: https://huggingface.co/souvik18
|
||
|
||
Roy is an independent project built with the vision of creating a smart, friendly, and customizable AI assistant.
|
||
|
||
---
|
||
|
||
## What Roy Can Do
|
||
|
||
Roy is capable of:
|
||
|
||
- Natural conversation and assistance
|
||
- Answering general knowledge questions
|
||
- Solving math and logical problems
|
||
- Helping with coding and debugging
|
||
- Writing emails, stories, and content
|
||
- Explaining concepts in simple language
|
||
- Brainstorming ideas and learning support
|
||
|
||
---
|
||
|
||
## Model Details
|
||
|
||
- **Model Name:** Roy-v1
|
||
- **Parameters:** 7B
|
||
- **Architecture:** LLaMA-based
|
||
- **Tensor Type:** F16
|
||
- **Format:** Safetensors
|
||
- **License:** Open for community usage
|
||
|
||
Base Model: **souvik18/Roy-v1**
|
||
|
||
## Quantized Versions (Community)
|
||
|
||
Thanks to [@mradermacher](https://huggingface.co/mradermacher) for providing GGUF quants of Roy-v1:
|
||
|
||
**https://huggingface.co/mradermacher/Roy-v1-GGUF**
|
||
|
||
These versions allow Roy to run on:
|
||
- CPU only systems
|
||
- Low VRAM GPUs
|
||
- Mobile / local apps via llama.cpp, ollama, koboldcpp
|
||
|
||
|
||
## Quick Usage
|
||
|
||
### Using HuggingFace Transformers
|
||
|
||
```python
|
||
!pip install -U transformers datasets accelerate bitsandbytes peft huggingface_hub
|
||
|
||
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
|
||
import torch
|
||
|
||
MODEL_ID = "souvik18/Roy-v1"
|
||
|
||
# 4bit config – works best on Kaggle
|
||
bnb_config = BitsAndBytesConfig(
|
||
load_in_4bit=True,
|
||
bnb_4bit_compute_dtype=torch.float16,
|
||
bnb_4bit_use_double_quant=True,
|
||
)
|
||
|
||
print(" Loading tokenizer...")
|
||
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
|
||
tokenizer.pad_token = tokenizer.eos_token
|
||
|
||
print(" Loading model (4bit)...")
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
MODEL_ID,
|
||
quantization_config=bnb_config,
|
||
device_map="auto"
|
||
)
|
||
|
||
print("\n Roy-v1 Loaded Successfully!")
|
||
|
||
while True:
|
||
text = input("You: ")
|
||
if text.lower() in ["exit","quit"]:
|
||
break
|
||
|
||
prompt = f"[INST] {text} [/INST]"
|
||
|
||
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
||
|
||
with torch.no_grad():
|
||
out = model.generate(
|
||
**inputs,
|
||
max_new_tokens=200,
|
||
temperature=0.7,
|
||
top_p=0.9,
|
||
do_sample=True
|
||
)
|
||
|
||
|
||
|