初始化项目，由ModelHub XC社区提供模型

Model: Sepolian/qwen2.5-0.5b-sft-openorca
Source: Original Platform

README.md

@@ -0,0 +1,115 @@
+---
+library_name: transformers
+tags:
+  - qwen2.5
+  - sft
+  - openorca
+  - generated_from_trainer
+base_model: Qwen/Qwen2.5-0.5B
+datasets:
+  - Open-Orca/OpenOrca
+language:
+  - en
+license: apache-2.0
+---
+
+# Model Card for Qwen2.5-0.5B-SFT-OpenOrca
+
+This model is a fine-tuned version of [Qwen/Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B) on a cleaned subset of the [Open-Orca/OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca) dataset. It was trained to follow instructions and reason across various tasks.
+
+## Model Details
+
+### Model Description
+
+- **Model type:** Causal Language Model
+- **Language(s) (NLP):** English
+- **License:** Apache 2.0
+- **Finetuned from model:** [Qwen/Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B)
+
+### Model Sources
+
+- **Repository:** [Sepolian/qwen2.5-0.5b-sft-openorca](https://huggingface.co/Sepolian/qwen2.5-0.5b-sft-openorca)
+- **Dataset:** [Open-Orca/OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca)
+
+
+## Training Details
+
+### Training Data
+
+The model was trained on a **300k sample subset** of the OpenOrca dataset. We applied a rigorous **Data Cleaning Pipeline** to ensure high quality:
+1.  **Length & Refusal Filtering**: Removed samples with very short questions (<20 chars) or responses (<120 chars), and filtered out "I cannot answer..." refusals.
+2.  **Language Filtering**: Kept only English samples using a fast ASCII-ratio heuristic (>0.95) and `langdetect` fallback.
+3.  **Repetition Filtering**: Removed samples with excessive word repetition (consecutive words >5) or high n-gram frequency.
+4.  **MinHash Deduplication**: Applied 3-gram MinHash LSH (threshold 0.85) to remove near-duplicates.
+5.  **System Prompt Standardization**: Unified system prompts to a standard "You are a helpful assistant..." format.
+
+### Training Procedure
+
+#### Training Hyperparameters
+
+- **Training regime:** bf16 mixed precision
+- **Optimizer:** AdamW (`paged_adamw_32bit`)
+- **Learning Rate:** 8e-6 with cosine schedule (warmup ratio 0.03)
+- **Epochs:** 1
+- **Packing:** Enabled (packed short sequences for efficiency)
+
+#### Speeds, Sizes, Times
+
+- **Hardware:** trained on NVIDIA L4 (24GB) via Google Colab
+- **Training Time:** ~5 hours (approx)
+
+## Evaluation
+
+### Benchmarks
+
+evaluated the model on standard benchmarks using `lm-evaluation-harness`.
+
+
+![full_eval](https://cdn-uploads.huggingface.co/production/uploads/67c90770fab614ccfdc7a6f5/wpDlF09G1rY75NtLdysyn.png)
+
+## How to Get Started with the Model
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "Sepolian/qwen2.5-0.5b-sft-openorca"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
+
+messages = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user", "content": "Explain quantum computing in simple terms."},
+]
+
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+generated_ids = model.generate(
+    model_inputs.input_ids,
+    max_new_tokens=512
+)
+
+generated_ids = [
+    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
+
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+print(response)
+```
+
+## Citation
+
+```bibtex
+@misc{openorca,
+  title = {OpenOrca: An Open Dataset of GPT Augmented FLAN Reasoning Traces},
+  author = {Mukherjee, Subhabrata and Mitra, Arindam and Skjellum, Ganesh and Catalyurek, Umit and Tielelman, Thomas and Monroy-Hernandez, Andres},
+  year = {2023},
+  publisher = {HuggingFace}
+}
+```