初始化项目，由ModelHub XC社区提供模型

Model: sKT-Ai-Labs/SKT-ST-X-0-3B Source: Original Platform
2026-07-07 20:21:19 +08:00
commit 0895d6fb70
57 changed files with 278126 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,222 @@
+---
+license: apache-2.0
+language:
+- en
+- hi
+tags:
+- moe
+- slm
+- skt-ai-labs
+- 3b-model
+- mixture-of-experts
+- bilingual
+library_name: pytorch
+pipeline_tag: text-generation
+datasets:
+- SKT-NRS/SKT-OMNI-CORPUS-2T
+model-index:
+- name: SKT-ST-X-0-3B-V1
+  results:
+  - task: {type: Classification}
+    dataset: {type: mteb/mtop_domain, name: MTEB MTOPDomainClassification (en)}
+    metrics: [{type: accuracy, value: 70.95}]
+  - task: {type: Classification}
+    dataset: {type: mteb/amazon_polarity, name: MTEB AmazonPolarityClassification}
+    metrics: [{type: accuracy, value: 46.88}]
+  - task: {type: STS}
+    dataset: {type: mteb/biosses-sts, name: MTEB BIOSSES}
+    metrics: [{type: cos_sim_pearson, value: 47.19}]
+  - task: {type: Reranking}
+    dataset: {type: mteb/scidocs-reranking, name: MTEB SciDocsRR}
+    metrics: [{type: mrr, value: 28.33}]
+  - task: {type: Classification}
+    dataset: {type: mteb/tweet_sentiment, name: Tweet Sentiment}
+    metrics: [{type: f1, value: 26.51}]
+  - task: {type: Clustering}
+    dataset: {type: mteb/stackexchange_clustering, name: StackExchange Clustering}
+    metrics: [{type: v_measure, value: 35.55}]
+---
+
+<div style="max-width: 900px; margin: 0 auto; font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;">
+
+### <span style="color:#B8860B; font-size:24px; font-weight:bold;">SKT</span> <span style="color:#333; font-size:24px; font-weight:bold;">AI LABS </span>
+
+<div style="background: linear-gradient(90deg, #FFD700 0%, #FFAA00 50%, #B8860B 100%); height: 8px; margin: 10px 0 20px 0; border-radius: 4px;"></div>
+
+# <span style="color:#000; font-size:48px; font-weight:800; letter-spacing:-1px;">SKT-ST-X-0-3B-V1</span>
+
+<!-- ==================== EPIC GOLDEN BANNER ==================== -->
+<div style="background: linear-gradient(135deg, #FFD700 0%, #FFCC00 35%, #B8860B 70%, #FFD700 100%); 
+            padding: 40px 20px; 
+            border-radius: 20px; 
+            text-align: center; 
+            margin: 30px 0;             border: 4px solid #FFFFFF; 
+            box-shadow: 0 15px 30px rgba(184, 134, 11, 0.4);">
+
+  <h2 style="color:#000; font-size:36px; font-weight:800; margin:0 0 15px 0; text-transform: uppercase; letter-spacing: 1px;">
+   COMPACT MOE POWERHOUSE
+  </h2>
+  
+  <p style="color:#000; font-size:22px; font-weight:600; margin:15px 0;">
+    3B Total Params • 1.1B Active • English & Hindi
+  </p>
+  
+  <p style="color:#222; font-size:18px; max-width:800px; margin:0 auto; line-height:1.6;">
+    A highly efficient Small Language Model (SLM) built on Mixtral MoE architecture for stability. 
+    Delivers intelligent responses with a tiny footprint.
+  </p>
+</div>
+
+<p align="center">
+  <img src="Webp/img.png" alt="SKT AI LABS Model Card" width="100%" style="max-width:600px; border-radius: 16px; border: 4px solid #FFD700; box-shadow: 0 10px 25px rgba(0,0,0,0.2);">
+</p>
+
+<!-- Badges -->
+<p align="center">
+  <a href="https://huggingface.co/sKT-Ai-Labs"><img src="https://img.shields.io/badge/🤗--HuggingFace-yellow?style=for-the-badge&logo=huggingface" alt="HF"></a>
+  <a href="https://www.sktailabs.in"><img src="https://img.shields.io/badge/SKT_AI_LABS-green?style=for-the-badge" alt="Website"></a>
+  <a href="https://opensource.org/licenses/Apache-2.0"><img src="https://img.shields.io/badge/License-Apache%202.0-blue?style=for-the-badge" alt="License"></a>
+</p>
+
+<!-- Main Content -->
+<div style="background: linear-gradient(135deg, #FFF9E6 0%, #FFF0C2 100%); padding: 30px; border-radius: 16px; border-left: 6px solid #B8860B; margin: 25px 0;">
+  <h2 style="color:#B8860B; font-size:32px; margin-bottom:15px;">🏗️ Model Architecture</h2>
+  
+  <table style="width:100%; border-collapse: collapse; color:#333;">
+    <tr style="border-bottom: 1px solid #E6C200;"><td style="padding:10px;"><strong>Total Parameters</strong></td><td style="padding:10px;">~3 Billion</td></tr>
+    <tr style="border-bottom: 1px solid #E6C200;"><td style="padding:10px;"><strong>Active Parameters</strong></td><td style="padding:10px;">~1.1 Billion (2 Experts/Token)</td></tr>
+    <tr style="border-bottom: 1px solid #E6C200;"><td style="padding:10px;"><strong>Architecture</strong></td><td style="padding:10px;">Mixture of Experts (MoE)</td></tr>
+    <tr style="border-bottom: 1px solid #E6C200;"><td style="padding:10px;"><strong>Number of Experts</strong></td><td style="padding:10px;">4</td></tr>
+    <tr style="border-bottom: 1px solid #E6C200;"><td style="padding:10px;"><strong>Context Length</strong></td><td style="padding:10px;">8K Tokens</td></tr>
+    <tr><td style="padding:10px;"><strong>Training Data</strong></td><td style="padding:10px;">40B Tokens (SKT-OMNI-CORPUS-2T)</td></tr>
+  </table>
+</div>
+
+<div style="background: #fff; padding: 25px; border-radius: 16px; border: 2px solid #FFD700; margin: 20px 0;">
+  <h3 style="color:#B8860B; font-size:24px; margin-top:0;">✨ Key Capabilities</h3>
+  <ul style="color:#333; font-size:16px; line-height:2.0;">
+    <li><strong>Bilingual Mastery:</strong> Fluent in both English and Hindi.</li>
+    <li><strong>Efficient Reasoning:</strong> Logical thinking and problem solving despite small size.</li>
+    <li><strong>Basic Coding:</strong> Python scripts, algorithms, and logic debugging.</li>
+    <li><strong>Creative Writing:</strong> Stories, poems, and roleplay with personality.</li>
+    <li><strong>Knowledge QA:</strong> Accurate general knowledge retrieval.</li>  </ul>
+</div>
+
+<hr style="border: 0; height: 1px; background: #FFD700; margin: 40px 0;">
+
+## 🛠️ Quick Start Guide
+
+### Installation
+<div style="background: linear-gradient(135deg, #FFFDF0 0%, #FFF8DC 100%); padding: 15px; border-radius: 8px; border: 1px solid #E6C200; margin-bottom: 15px;">
+
+```bash
+pip install transformers accelerate torch peft bitsandbytes
+```
+</div>
+
+### Basic Inference
+<div style="background: linear-gradient(135deg, #FFFDF0 0%, #FFF8DC 100%); padding: 15px; border-radius: 8px; border: 1px solid #E6C200;">
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+model_id = "sKT-Ai-Labs/SKT-ST-X-0-3B"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    device_map="auto",
+    torch_dtype=torch.float16
+)
+
+prompt = "What is Quantum Physics?"
+formatted = f"<|user|>\n{prompt}\n<|assistant|>\n"
+inputs = tokenizer(formatted, return_tensors="pt").to("cuda")
+
+outputs = model.generate(**inputs, max_new_tokens=100)
+response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(response.split("<|assistant|>")[-1].strip())
+```
+</div>
+
+### ⚡ 4-bit Quantization (Low VRAM)
+<div style="background: linear-gradient(135deg, #FFFDF0 0%, #FFF8DC 100%); padding: 15px; border-radius: 8px; border: 1px solid #E6C200;">
+
+```python
+from transformers import BitsAndBytesConfig
+
+quant_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_compute_dtype=torch.float16)
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    quantization_config=quant_config,
+    device_map="auto"
+)
+```
+</div>
+
+---
+
+## 📊 MTEB Benchmark Highlights
+
+| Task | Dataset | Metric | Score |
+| :--- | :--- | :--- | :--- |
+| **Classification** | MTOP Domain (en) | Accuracy | **70.95** |
+| **Classification** | Amazon Polarity | Accuracy | **46.88** |
+| **STS** | BIOSSES | Cosine Pearson | **47.19** |
+| **Reranking** | SciDocs RR | MRR | **28.33** |
+| **Classification** | Tweet Sentiment | F1 | **26.51** |
+| **Clustering** | StackExchange | V-Measure | **35.55** |
+
+*Full benchmark results available in the model metadata.*
+
+---
+## ❤️ Support Our Mission
+
+<div style="background: linear-gradient(135deg, #FFF9E6 0%, #FFF0C2 100%); padding: 30px; border-radius: 16px; border: 2px solid #FFD700; text-align: center; margin: 40px 0; box-shadow: 0 5px 15px rgba(0,0,0,0.1);">
+  <h3 style="color:#B8860B; font-size:26px; margin-bottom:15px;">🙏 Drop a Heart ❤ For Our Hard Work!</h3>
+  <p style="color:#333; font-size:18px; margin-bottom:20px;">
+    If you believe in the vision of Sovereign Indian AI, please show your support by dropping a heart below. Your encouragement fuels our journey!
+  </p>
+  
+  <div style="display: flex; justify-content: center; gap: 20px; flex-wrap: wrap;">
+    <a href="https://huggingface.co/sKT-Ai-Labs" target="_blank" style="background: linear-gradient(#FFD700, #FFAA00); color: #000; padding: 12px 25px; border-radius: 50px; text-decoration: none; font-weight: bold; font-size: 16px; box-shadow: 0 4px 10px rgba(0,0,0,0.2); transition: transform 0.2s;">
+      ❤️ Like This Dataset
+    </a>
+    <a href="https://huggingface.co/sKT-Ai-Labs" target="_blank" style="background: linear-gradient(#B8860B, #8B4513); color: #fff; padding: 12px 25px; border-radius: 50px; text-decoration: none; font-weight: bold; font-size: 16px; box-shadow: 0 4px 10px rgba(0,0,0,0.2); transition: transform 0.2s;">
+      ➕ Follow SKT AI LABS    </a>
+  </div>
+  
+  <p style="color:#666; font-size:15px; margin-top:20px; font-style: italic;">
+    Kindly follow us for more updates and contribute to our open-source journey!
+  </p>
+</div>
+
+
+
+## 📜 License & Citation
+
+This model is released under the **Apache-2.0 License**.
+
+-   [View Full License](Index/USED.MD)
+-   [Third Party Notices](Index/THIRD_PARTY_NOTICES.MD)
+
+```bibtex
+@misc{SKT-ST-X-0-3B,
+  author = {SKT AI LABS, India},
+  title = {SKT-ST-X-0-3B: A Compact Mixture of Experts Model},
+  year = {2026},
+  publisher = {Hugging Face},
+  url = {https://huggingface.co/sKT-Ai-Labs/SKT-ST-X-0-3B}
+}
+```
+
+<div style="text-align:center; margin-top:50px; padding: 20px; background: #FFF9E6; border-radius: 10px;">
+  <h3 style="color:#B8860B; margin:0;">Made with ❤️ by SKT AI LABS</h3>
+  <p style="color:#555; margin-top:5px;">Support: support@sktailabs.in</p>
+</div>
+
+</div>