73 lines
4.0 KiB
Markdown
73 lines
4.0 KiB
Markdown
|
---
|
|||
|
|
license: apache-2.0
|
|||
|
|
base_model: lofofora/smollm2-135m-instruction-tuning-v0
|
|||
|
|
tags:
|
|||
|
|
- text-generation-inference
|
|||
|
|
- transformers
|
|||
|
|
- smollm2
|
|||
|
|
- conversational
|
|||
|
|
- studyverse
|
|||
|
|
- instruction-tuned
|
|||
|
|
languages:
|
|||
|
|
- en
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# 🚀 CosmiQ AI — The Intelligent Assistant
|
|||
|
|
|
|||
|
|
**CosmiQ AI** is a highly-optimized, lightweight Small Language Model (SLM) designed for swift, multi-turn assistant-style conversations. Built on top of the **SmolLM2 (135M)** architecture, this model is fine-tuned to enable efficient, low-latency text generation directly on edge devices such as mobile phones and standard laptops without requiring heavy GPU infrastructure.
|
|||
|
|
|
|||
|
|
## 🏗️ Model Architecture
|
|||
|
|
CosmiQ AI leverages a compact yet powerful architecture heavily inspired by the structural refinements found in foundational models like Llama-3:
|
|||
|
|
- **Architecture Type:** Autoregressive Decoder-Only Transformer
|
|||
|
|
- **Parameters:** 135 Million (13.5 Crore) — Perfect for local deployment with an extremely low memory footprint.
|
|||
|
|
- **Attention Mechanism:** Grouped-Query Attention (GQA) to accelerate text streaming speed and optimize Key-Value (KV) cache utilization.
|
|||
|
|
- **Position Embeddings:** Rotary Position Embedding (RoPE) for stable long-context processing.
|
|||
|
|
- **Vocabulary Size:** 49,152 tokens.
|
|||
|
|
|
|||
|
|
## 🧠 Training Methodology & Algorithms
|
|||
|
|
The raw foundational knowledge was aligned into a conversational, assistant-ready state using two core paradigms:
|
|||
|
|
1. **Supervised Fine-Tuning (SFT):** The model underwent strict optimization to understand multi-turn conversation syntaxes, explicitly learning to track boundaries using special chat markers like `<|im_start|>user` and `<|im_start|>assistant`.
|
|||
|
|
2. **Direct Preference Optimization (DPO):** Aligned through preference optimization to output concise, factual, and helpful multi-turn responses while actively penalizing repetitive or unhelpful generations.
|
|||
|
|
|
|||
|
|
## 📊 Dataset Insights
|
|||
|
|
The stellar accuracy of this 135M parameter model is a direct result of being trained on highly curated, high-quality synthetic datasets rather than unverified scraped web text:
|
|||
|
|
- **Cosmopedia v2:** A massive synthetic dataset containing high-quality textbooks, structured academic tutorials, and educational narratives scaled using larger foundational models (like Mixtral). This enables CosmiQ AI to excel exceptionally well at science and academic queries (e.g., explaining photosynthesis, stomata, etc.).
|
|||
|
|
- **UltraChat & UltraFeedback:** Highly polished conversational data used during instruction fine-tuning to perfect the model’s persona, social interaction, and question-answering conversational flow.
|
|||
|
|
|
|||
|
|
## 🚀 Deployment & Integration
|
|||
|
|
This model is actively hosted and integrated into production via **Hugging Face Spaces** using a robust streaming architecture:
|
|||
|
|
- **Frontend UI Framework:** Gradio 6
|
|||
|
|
- **Streaming Backend:** PyTorch `TextIteratorStreamer` running with advanced structural history filters to prevent null or misaligned conversational tensors from interrupting runtime processing.
|
|||
|
|
|
|||
|
|
### Quick Usage via Transformers Pipeline
|
|||
|
|
You can test and run this model locally using the following Python script:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|||
|
|
import torch
|
|||
|
|
|
|||
|
|
model_id = "JNX25/studyverse-smollm2-instruct"
|
|||
|
|
|
|||
|
|
print("Loading Tokenizer & Model...")
|
|||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
|||
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32)
|
|||
|
|
|
|||
|
|
# Structuring a conversational thread
|
|||
|
|
messages = [{"role": "user", "content": "Explain photosynthesis in one clear sentence."}]
|
|||
|
|
rendered_chat = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
|||
|
|
|
|||
|
|
# Processing Tensors
|
|||
|
|
inputs = tokenizer(rendered_chat, return_tensors="pt")
|
|||
|
|
outputs = model.generate(
|
|||
|
|
input_ids=inputs["input_ids"],
|
|||
|
|
attention_mask=inputs["attention_mask"],
|
|||
|
|
max_new_tokens=150,
|
|||
|
|
do_sample=True,
|
|||
|
|
temperature=0.7,
|
|||
|
|
top_p=0.9,
|
|||
|
|
eos_token_id=tokenizer.eos_token_id
|
|||
|
|
)
|
|||
|
|
|
|||
|
|
print("\n--- Response ---")
|
|||
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|