初始化项目，由ModelHub XC社区提供模型

Model: Abiray/Kshana-170M-Base
Source: Original Platform

README.md

@@ -0,0 +1,112 @@
+---
+language:
+- en
+license: apache-2.0
+tags:
+- text-generation
+- causal-lm
+- llama
+- custom-architecture
+- pytorch
+- gqa
+datasets:
+- fineweb-edu
+- cosmopedia
+---
+## 🌌 Kshana-170M Base
+
+Kshana-170M-Base is a compact 170M-parameter foundational causal language model built by Abiray. Moving along the architectural lineage of its predecessor (*Sutra*), *Kshana* is trained from scratch using a highly optimized Llama-style architecture with Grouped-Query Attention (GQA) for blazing inference velocity. 
+
+Despite its compact size, it achieves highly competitive results on key reasoning benchmarks, making it an optimal base for downstream fine-tuning workflows or resource-constrained edge deployment.
+
+*Note: As a raw base model, it requires downstream instruction tuning to perform as a conversational chat agent.*
+
+## 🏆 Benchmarks
+
+The base weights were evaluated head-to-head against sub-500M architectures using `lm-evaluation-harness` within an identical runtime environment. To align with open-source presentation standards, scores reflect peak performance metric selection targets (`acc` for science and single-token knowledge choice selections, `acc_norm` for length-penalized situational context completions).
+
+| Benchmark | 🌌 Kshana-170M (Ours) | 🪵 SmolLM2-135M | 🌾 Nandi-Mini-150M | 📐 Pythia-160m | 🔹 OPT-125m | 🧮 Cerebras-256M | ⚙️ Pythia-410m |
+| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+| **Parameters** | **169.9M** | 135M | 150M | 160M | 125M | 256M | 410M |
+| **SciQ** *(Sci)* | **81.90%** | 84.10% | 89.10% | 55.70% | 78.20% | 75.70% | 80.40% |
+| **PIQA** *(Logic)* | **66.81%** | 68.34% | 65.13% | 59.19% | 62.62% | 61.10% | 66.70% |
+| **ARC-Easy** *(Know)* | **57.07%** | 64.39% | 54.67% | 37.58% | 42.76% | 40.99% | 51.98% |
+| **HellaSwag** *(Ctx)*| **39.84%** | 43.17% | 37.11% | 30.49% | 31.62% | 28.60% | 40.02% |
+
+## 🧠 Model Architecture
+
+Kshana-170M is based on the `LlamaForCausalLM` architecture with a native Grouped-Query Attention (GQA) layout to compress hardware footprint:
+
+| Parameter | Value |
+| :--- | :--- |
+| Parameters | 169,906,752 |
+| Hidden size | 576 |
+| Layers | 32 |
+| Attention heads | 9 |
+| KV heads (GQA) | 3 |
+| Head dimension | 64 |
+| Intermediate size | 1,536 |
+| Activation | SwiGLU (`silu`) |
+| Max Context | 8,192 tokens |
+| Vocabulary size | 49,152 |
+
+## ⚙️ Training Configuration
+
+| Parameter | Value |
+| :--- | :--- |
+| Optimizer | AdamW |
+| Learning rate | 3e-4 |
+| LR scheduler | Cosine Decay |
+| Precision | `bfloat16` / `float16` hybrid |
+
+## 📚 Training Data
+
+Trained on a volume of **65 Billion tokens**. The corpus characteristics include high-quality deduplicated web extracts, structured synthetic reasoning texts, and educational literature subsets (focusing on FineWeb-Edu, Wikipedia, and Cosmopedia). Data was rigorously filtered using MinHash LSH deduplication and language filtering matrices.
+
+## 🎯 Operational Scope & Intended Use
+
+### ✅ Targeted Applications
+* **Downstream Fine-Tuning (SFT/DPO):** Acts as a clean, lightweight base for training specialized assistants, custom chat agents, or task-specific models.
+* **Local & Edge Deployment:** Designed with Grouped-Query Attention (GQA) for efficient quantization (via `llama.cpp` / GGUF), making it ideal for low-power hardware like consumer CPUs, laptops, and mobile devices.
+* **Text Completion & Routing:** Well-suited for low-latency text continuation, basic autocomplete features, or classification tasks like routing user queries quickly before passing them to larger models.
+
+### ❌ Out-of-Scope Limits
+* **Coding & Mathematics:** The model's training data consists strictly of natural language text (FineWeb-Edu and Cosmopedia). Because it was *never* exposed to structured math datasets or code repositories during training, it cannot write code scripts, debug software, or calculate mathematical formulas.
+* **Factual Knowledge Retrieval:** Trained on a strict budget of 65 Billion tokens with a sub-200M parameter boundary, the model lacks the capacity to serve as an open-domain factual encyclopedia. It will hallucinate facts if asked about niche topics without being provided reference text directly in the prompt (e.g., via RAG).
+* **Interactive Chat (Out of the box):** As a raw base model, it will naturally attempt to autocomplete text rather than hold a conversational dialogue. It requires standard instruction fine-tuning before it can be used as a traditional chatbot.
+
+## 🚀 Inference & Edge Deployment
+
+The model can be initialized within minutes using standard workflows via the Hugging Face `transformers` environment. Its native GQA layout makes it highly compatible with quantization layers (via `llama.cpp` / GGUF) to run on consumer CPUs or embedded devices at extreme tokens-per-second.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "Abiray/Kshana-170M-Base"
+
+# Initialize matching vocabulary tokenizer
+tokenizer = AutoTokenizer.from_pretrained(model_id, token=True)
+
+# Pull weights matching verified float16 layout
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.float16,
+    device_map="auto",
+    token=True
+)
+
+prompt = "The basic physical principle behind gravitational collapse is"
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+with torch.no_grad():
+    outputs = model.generate(
+        **inputs, 
+        max_new_tokens=64,
+        temperature=0.6,
+        top_p=0.85,
+        do_sample=True,
+        pad_token_id=tokenizer.eos_token_id
+    )
+
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))