yuj-v1/README.md

license, tags, model-index

license	tags	model-index
apache-2.0	merge hindi english Llama2 ai4bharat/Airavata BhabhaAI/Gajendra-v0.1	name results yuj-v1 task dataset metrics source type name text-generation Text Generation name type config split args AI2 Reasoning Challenge (25-Shot) ai2_arc ARC-Challenge test num_few_shot 25 type value name acc_norm 45.65 normalized accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=shuvom/yuj-v1 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args HellaSwag (10-Shot) hellaswag validation num_few_shot 10 type value name acc_norm 70.1 normalized accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=shuvom/yuj-v1 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args MMLU (5-Shot) cais/mmlu all test num_few_shot 5 type value name acc 43.78 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=shuvom/yuj-v1 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args TruthfulQA (0-shot) truthful_qa multiple_choice validation num_few_shot 0 type value mc2 41.69 url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=shuvom/yuj-v1 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args Winogrande (5-shot) winogrande winogrande_xl validation num_few_shot 5 type value name acc 69.85 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=shuvom/yuj-v1 Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args GSM8k (5-shot) gsm8k main test num_few_shot 5 type value name acc 4.78 accuracy url name https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=shuvom/yuj-v1 Open LLM Leaderboard

The Model yuj-v1:

The yuj-v1 model is a blend of advanced models strategically crafted to enhance Hindi Language Models (LLMs) effectively and democratically. Its primary goals include catalyzing the development of Hindi and its communities, making significant contributions to linguistic knowledge. The term "yuj," from Sanskrit, signifies fundamental unity, highlighting the integration of sophisticated technologies to improve the language experience for users in the Hindi-speaking community.

Official GGUF version: shuvom/yuj-v1-GGUF

Below are the model which are leverage to build this yuj-v1:

• ai4bharat/Airavata
• BhabhaAI/Gajendra-v0.1

☄️Space to use it (yuj-v1 tryO):

💻 Usage:

First, you need to install some of below packages:

1. Bits and bytes

!pip install bitsandbytes

2. Accelerate (to install the latest version)

!pip install git+https://github.com/huggingface/accelerate.git

3. Usage

# Usage
import torch

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

# load the model in 4-bit quantization
tokenizer = AutoTokenizer.from_pretrained("shuvom/yuj-v1")
model = AutoModelForCausalLM.from_pretrained("shuvom/yuj-v1",torch_dtype=torch.bfloat16,load_in_4bit=True)

prompt = "युज शीर्ष द्विभाषी मॉडल में से एक है"
inputs = tokenizer(prompt, return_tensors="pt")

# Generate
generate_ids = model.generate(inputs.input_ids, max_length=65)
tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]

4. Output

युज शीर्ष द्विभाषी मॉडल में से एक है। यह एक उत्पादक मॉडल है जो एक साथ एक ट्रांसफॉर्मर और एक आत्म-ध्यान तंत्रिका नेटवर्क को जोड़ता है। यह एक ट्रांसफॉर्मर वास्तुकला का उपयोग करता है जो एक ट्रांसफॉर्मर मॉडल की तुलना में बहुत अधिक जटिल है।

🧩 Configuration

models:
  - model: sarvamai/OpenHathi-7B-Hi-v0.1-Base
    # no parameters necessary for base model
  - model: ai4bharat/Airavata
    parameters:
      density: 0.5
      weight: 0.5
  - model: BhabhaAI/Gajendra-v0.1
    parameters:
      density: 0.5
      weight: 0.3
merge_method: ties
base_model: sarvamai/OpenHathi-7B-Hi-v0.1-Base
parameters:
  normalize: true
dtype: float16

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	45.97
AI2 Reasoning Challenge (25-Shot)	45.65
HellaSwag (10-Shot)	70.10
MMLU (5-Shot)	43.78
TruthfulQA (0-shot)	41.69
Winogrande (5-shot)	69.85
GSM8k (5-shot)	4.78