初始化项目，由ModelHub XC社区提供模型

Model: SykoSLM/SykoLLM-V6.0-Test
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,57 @@
+---
+license: apache-2.0
+language:
+- tr
+- en
+base_model: SykoSLM/SykoLLM-V5.9-Mini
+pipeline_tag: text-generation
+library_name: transformers
+tags:
+- nlp
+- code
+- phi3
+- depth-up-scaling
+- untrained
+---
+
+# SykoLLM-V6.0-Test
+
+## Model Overview
+**SykoLLM-V6.0-Test** is an up-scaled and structurally expanded version of the previous SykoLLM models. Developed by **SykoSLM**, this model is currently in the experimental/testing phase. 
+
+The primary objective of this release is to provide a structurally larger foundation model by expanding both the depth (number of layers) and the width (intermediate size / MLP capacity) of the previous architecture, without losing the pre-trained knowledge.
+
+## Architectural Expansion (Up-Scaling)
+In order to overcome the "Knowledge Interference" (capacity bottleneck) observed in previous iterations, significant architectural changes have been applied to this model:
+
+* **Depth Up-Scaling (DUS):** The number of hidden layers has been increased to **24**. This was achieved by carefully duplicating and mapping the existing layers to preserve the logical and syntactic capabilities of the model.
+* **Width Expansion (MLP Scaling):** The `intermediate_size` has been expanded to **3072**. To prevent catastrophic forgetting, the newly added weights in the feed-forward networks were initialized with exact zero (`0.0`). This ensures that the newly added parameters act as identity functions during the initial forward pass.
+
+## ⚠️ Important Notice: Status of the Model
+**This model is currently UNTRAINED on the newly added parameters.** 
+
+It has been expanded solely to save pre-training time and preserve existing knowledge. While the model retains the capabilities of its predecessor, the newly added parameters (~100M+ new parameters) are currently dormant (zeroed out). 
+
+To fully utilize the expanded capacity and activate the new parameters, **fine-tuning is required**. If used in its current state, the model will function similarly to the previous smaller version, as the new structural capacity has not yet been fine-tuned on new or existing datasets.
+
+## Why This Approach?
+Training a Large Language Model from scratch requires immense computational resources and time. By utilizing Net2Net (Knowledge Distillation) principles:
+1. We preserve the billions of tokens worth of knowledge already embedded in the model.
+2. We provide the model with a much larger "encyclopedic" memory (MLP expansion) to prevent data overlapping and hallucination.
+3. We drastically reduce the time required to achieve a higher parameter count.
+
+## Usage
+You can load the model using the `transformers` library, but please keep in mind that it requires further fine-tuning for optimal performance.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "SykoSLM/SykoLLM-V6.0-Test"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto")
+```
+
+---
+
+Developed by SykoSLM

config.json

@@ -0,0 +1,35 @@
+{
+  "architectures": [
+    "Phi3ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "dtype": "float16",
+  "embd_pdrop": 0.0,
+  "eos_token_id": 0,
+  "hidden_act": "silu",
+  "hidden_size": 1024,
+  "ignore_keys_at_rope_validation": null,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "max_position_embeddings": 1024,
+  "model_type": "phi3",
+  "num_attention_heads": 8,
+  "num_hidden_layers": 24,
+  "num_key_value_heads": 2,
+  "original_max_position_embeddings": 4096,
+  "pad_token_id": 4,
+  "partial_rotary_factor": 1.0,
+  "resid_pdrop": 0.0,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "partial_rotary_factor": 1.0,
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.0.0",
+  "use_cache": false,
+  "vocab_size": 50000
+}

generation_config.json

@@ -0,0 +1,10 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "eos_token_id": 0,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 4,
+  "transformers_version": "5.0.0",
+  "use_cache": false
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7dd52b095af4c1ba8b9c625898ced3d9cb8740453ec36495b937cdd9f18bcc90
+size 783731024

tokenizer_config.json

@@ -0,0 +1,10 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "is_local": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<|endoftext|>"
+}