SKT-ST-X-0-3B/README.md

license, language, tags, library_name, pipeline_tag, datasets, model-index

license	language	tags	library_name	pipeline_tag	datasets	model-index
apache-2.0	en hi	moe slm skt-ai-labs 3b-model mixture-of-experts bilingual	pytorch	text-generation	SKT-NRS/SKT-OMNI-CORPUS-2T	name results SKT-ST-X-0-3B-V1 task dataset metrics type Classification type name mteb/mtop_domain MTEB MTOPDomainClassification (en) type value accuracy 70.95 task dataset metrics type Classification type name mteb/amazon_polarity MTEB AmazonPolarityClassification type value accuracy 46.88 task dataset metrics type STS type name mteb/biosses-sts MTEB BIOSSES type value cos_sim_pearson 47.19 task dataset metrics type Reranking type name mteb/scidocs-reranking MTEB SciDocsRR type value mrr 28.33 task dataset metrics type Classification type name mteb/tweet_sentiment Tweet Sentiment type value f1 26.51 task dataset metrics type Clustering type name mteb/stackexchange_clustering StackExchange Clustering type value v_measure 35.55

SKT AI LABS

SKT-ST-X-0-3B-V1

COMPACT MOE POWERHOUSE

3B Total Params • 1.1B Active • English & Hindi

A highly efficient Small Language Model (SLM) built on Mixtral MoE architecture for stability. Delivers intelligent responses with a tiny footprint.

🏗️ Model Architecture

Total Parameters	~3 Billion
Active Parameters	~1.1 Billion (2 Experts/Token)
Architecture	Mixture of Experts (MoE)
Number of Experts	4
Context Length	8K Tokens
Training Data	40B Tokens (SKT-OMNI-CORPUS-2T)

✨ Key Capabilities

• Bilingual Mastery: Fluent in both English and Hindi.
• Efficient Reasoning: Logical thinking and problem solving despite small size.
• Basic Coding: Python scripts, algorithms, and logic debugging.
• Creative Writing: Stories, poems, and roleplay with personality.
• Knowledge QA: Accurate general knowledge retrieval.

---

🛠️ Quick Start Guide

Installation

pip install transformers accelerate torch peft bitsandbytes

Basic Inference

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "sKT-Ai-Labs/SKT-ST-X-0-3B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.float16
)

prompt = "What is Quantum Physics?"
formatted = f"<|user|>\n{prompt}\n<|assistant|>\n"
inputs = tokenizer(formatted, return_tensors="pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens=100)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response.split("<|assistant|>")[-1].strip())

⚡ 4-bit Quantization (Low VRAM)

from transformers import BitsAndBytesConfig

quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=quant_config,
    device_map="auto"
)

---

📊 MTEB Benchmark Highlights

Task	Dataset	Metric	Score
Classification	MTOP Domain (en)	Accuracy	70.95
Classification	Amazon Polarity	Accuracy	46.88
STS	BIOSSES	Cosine Pearson	47.19
Reranking	SciDocs RR	MRR	28.33
Classification	Tweet Sentiment	F1	26.51
Clustering	StackExchange	V-Measure	35.55

Full benchmark results available in the model metadata.

---

❤️ Support Our Mission

🙏 Drop a Heart ❤ For Our Hard Work!

If you believe in the vision of Sovereign Indian AI, please show your support by dropping a heart below. Your encouragement fuels our journey!

❤️ Like This Dataset ➕ Follow SKT AI LABS

Kindly follow us for more updates and contribute to our open-source journey!

📜 License & Citation

This model is released under the Apache-2.0 License.

• View Full License
• Third Party Notices

@misc{SKT-ST-X-0-3B,
  author = {SKT AI LABS, India},
  title = {SKT-ST-X-0-3B: A Compact Mixture of Experts Model},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/sKT-Ai-Labs/SKT-ST-X-0-3B}
}

Made with ❤️ by SKT AI LABS

Support: support@sktailabs.in