初始化项目，由ModelHub XC社区提供模型

Model: OvercastLab/Quark-50m
Source: Original Platform

README.md

@@ -0,0 +1,124 @@
+---
+language:
+- en
+- code
+license: apache-2.0
+tags:
+- smol
+- pretraining
+- instruct
+- 50M
+- causal-lm
+- gqa
+- swiglu
+- rmsnorm
+datasets:
+- HuggingFaceTB/smollm-corpus
+metrics:
+- perplexity
+model-index:
+- name: Quark-50m-Instruct
+  results: []
+pipeline_tag: text-generation
+---
+
+# Quark-50m-Instruct
+
+**Quark-50m-Instruct** is a small (≈56M parameters) decoder-only language model, fine-tuned for instruction following.
+It is built on the same architecture of “SmolLM” family and was fully pretrained on 5 billion tokens from
+[HuggingFaceTB/smollm‑corpus](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus).
+
+- **Model type:** Causal Language Model (LLaMA‑style decoder)
+- **Architecture:** GQA · SwiGLU · RMSNorm · RoPE · Weight‑tying
+- **Pretraining tokens:** 5 B
+- **Fine‑tuning:** Instruction‑tuned (details below)
+- **Creators:** [OvercastLab](https://huggingface.co/OvercastLab) (research & development lab for ML/AI)
+- **Release date:** 22 April 2026
+
+## Model Summary
+
+Quark-50m-Instruct is designed to be an efficient assistant that can run on consumer GPUs (e.g., RTX 3070 with 8 GB VRAM)
+and even on CPU for light workloads. It is **not** competitive with large models on knowledge‑intensive tasks,
+but it excels at:
+
+- Simple conversational tasks
+- Code generation and explanation (Python)
+- Short text rewriting and summarisation
+- On‑device / edge inference
+
+The architecture closely follows the efficient‑small‑LM blueprint popularised by SmolLM:
+
+| Component   | Details                       |
+|-------------|-------------------------------|
+| Vocab size  | 49,152                        |
+| Hidden size | 384                           |
+| Layers      | 24                            |
+| Attention   | Grouped Query (6 Q heads, 2 KV heads) |
+| FFN         | SwiGLU with 1,024 intermediate |
+| Position    | RoPE (θ = 10,000)             |
+| Normalisation | RMSNorm (pre‑block)         |
+
+Total trainable parameters: **≈48 M** (with weight tying).
+
+### Benchmark Evaluation Metrics
+
+| Category | Benchmark | Metric | Score / Value | Status |
+| :--- | :--- | :--- | :---: | :---: |
+| **Linguistics & Grammar** | BLiMP | Accuracy | 68.12% | Success |
+| **Commonsense & Reasoning** | PIQA | Normalized Accuracy | 57.83% | Success |
+| | COPA | Accuracy | 57.00% | Success |
+| | BoolQ | Accuracy | 52.17% | Success |
+| | WinoGrande | Accuracy | 47.36% | Success |
+| | HellaSwag | Normalized Accuracy | 28.49% | Success |
+| | RACE | Accuracy | 26.41% | Success |
+| | CommonsenseQA | Accuracy | 20.31% | Success |
+| **Academic & Knowledge** | SciQ | Normalized Accuracy | 49.00% | Success |
+| | ARC-Easy | Normalized Accuracy | 36.49% | Success |
+| | MMLU | Accuracy | 25.64% | Success |
+| | ARC-Challenge | Normalized Accuracy | 25.17% | Success |
+| | OpenBookQA | Normalized Accuracy | 25.40% | Success |
+| **Language Modeling** | LAMBADA | Accuracy | 15.87% | Success |
+| | WikiText-2 | Word Perplexity | 251.76 | Success |
+
+*Note: The Arithmetic benchmark failed due to outdated script support (`arithmetic.py`), and SocialIQA failed due to a registration tag error (`siqa`). Total baseline execution completed successfully for all other 15 tasks.*
+
+
+## Uses
+
+### Direct Use
+The model can be used via the 🤗 Transformers library for standard text generation.
+It expects chat‑formatted input (see example below).
+
+### Downstream Use
+Because of the open Apache‑2.0 license, you may fine‑tune Quark-50m‑Instruct on your own data for
+domain‑specific tasks – for instance, a customer‑support bot, a code reviewer, or a story writer.
+
+### Limitations
+- Limited world knowledge (stopped at mid‑2025 pretraining data).
+- Short context window (2,048 tokens).
+- Small size means it can make more factual mistakes than larger models.
+
+## How to Get Started
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_name = "ThingAI/Quark-50m-Instruct"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
+
+messages = [
+    {"role": "system", "content": "You are Quark, a helpful assistant."},
+    {"role": "user", "content": "Explain group query attention in one sentence."}
+]
+
+inputs = tokenizer.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
+
+outputs = model.generate(inputs, max_new_tokens=128)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))