初始化项目，由ModelHub XC社区提供模型

Model: SykoSLM/SykoLLM-v6.9
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,154 @@
+---
+language:
+- en
+license: apache-2.0
+tags:
+- causal-lm
+- text-generation
+- pretrained
+- tpu
+- sykollm
+base_model: SykoSLM/SykoLLM-V6.8
+---
+
+# SykoLLM-V6.9
+
+**The most powerful model in the SykoLLM family — trained on 8 billion tokens.**
+
+SykoLLM-V6.9 is a 391M parameter causal language model, trained from scratch on a carefully curated mixture of high-quality English datasets. It is the latest and most capable model in the SykoLLM series, surpassing all previous versions in both token count and training quality.
+
+---
+
+## Model Details
+
+| Property | Value |
+|---|---|
+| **Architecture** | Causal Language Model (Phi-3 based) |
+| **Parameters** | 391,857,152 |
+| **Context Length** | 1,024 tokens |
+| **Vocabulary Size** | 50,000 |
+| **Hidden Size** | 1,024 |
+| **Intermediate Size** | 3,072 |
+| **Layers** | 24 |
+| **Attention Heads** | 8 (GQA: 2 KV heads) |
+| **Precision** | bfloat16 |
+| **Language** | English only |
+
+---
+
+## Training Details
+
+| Property | Value |
+|---|---|
+| **Total Tokens** | ~8 Billion |
+| **Training Steps** | 30,000 |
+| **Effective Batch Size** | 256 (16 × 2 × 8 cores) |
+| **Learning Rate** | 4e-4 (cosine decay) |
+| **Optimizer** | Adafactor |
+| **Hardware** | Google TPU v5e-8 |
+| **Precision** | bfloat16 (XLA native) |
+| **Weight Decay** | 0.05 |
+| **Warmup Steps** | 200 |
+
+---
+
+## Training Data
+
+SykoLLM-V6.9 was trained on a curated mixture of 4 high-quality datasets, interleaved with carefully tuned sampling probabilities:
+
+| Dataset | Sampling | Description |
+|---|---|---|
+| [openbmb/Ultra-FineWeb](https://huggingface.co/datasets/openbmb/Ultra-FineWeb) | 25% | High-quality web text, scored and filtered |
+| [openbmb/Ultra-FineWeb-L3](https://huggingface.co/datasets/openbmb/Ultra-FineWeb-L3) | 40% | Multi-style synthetic English pretraining data |
+| [openbmb/UltraData-Math](https://huggingface.co/datasets/openbmb/UltraData-Math) | 20% | High-quality mathematical reasoning data |
+| [openbmb/UltraChat](https://huggingface.co/datasets/openbmb/UltraChat) | 15% | Multi-turn conversational data |
+
+All datasets were filtered with a quality score threshold of ≥ 0.85 and additional heuristic filters to remove low-quality, noisy, or excessively long samples.
+
+---
+
+## Chat Format
+
+SykoLLM-V6.9 uses the following chat template:
+
+```
+<|user|>
+Your message here<|end|>
+<|assistant|>
+Model response here<|end|>
+```
+
+For multi-turn conversations:
+
+```
+<|user|>
+Hello, how are you?<|end|>
+<|assistant|>
+I'm doing great, thank you for asking!<|end|>
+<|user|>
+Can you help me with a math problem?<|end|>
+<|assistant|>
+Of course! What's the problem?<|end|>
+```
+
+---
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+model_id = "SykoSLM/SykoLLM-V6.9"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+    trust_remote_code=True,
+)
+
+prompt = "<|user|>\nWhat is the capital of France?<|end|>\n<|assistant|>\n"
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+with torch.no_grad():
+    outputs = model.generate(
+        **inputs,
+        max_new_tokens=256,
+        temperature=0.7,
+        top_p=0.9,
+        do_sample=True,
+    )
+
+print(tokenizer.decode(outputs[0], skip_special_tokens=False))
+```
+
+---
+
+## SykoLLM Family
+
+| Model | Tokens | Notes |
+|---|---|---|
+| SykoLLM-V6.9 | **~8B** | **Most powerful — current** |
+| SykoLLM-V6.8 | <8B | Previous version |
+| SykoLLM-V6.6 | <8B | Earlier version |
+
+---
+
+## Limitations
+
+- **English only** — the model was trained exclusively on English data and does not support other languages.
+- **Context length** is limited to 1,024 tokens.
+- As a base pretrained model, it may produce outputs that are inaccurate, biased, or inappropriate. Use with appropriate safety measures.
+- Not instruction-tuned — for best results, use the chat format described above.
+
+---
+
+## License
+
+This model is released under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
+
+---
+
+*Trained with ❤️ by [SykoSLM](https://huggingface.co/SykoSLM)*

config.json

@@ -0,0 +1,34 @@
+{
+  "architectures": [
+    "Phi3ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 50000,
+  "dtype": "bfloat16",
+  "embd_pdrop": 0.0,
+  "eos_token_id": 3,
+  "hidden_act": "silu",
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "max_position_embeddings": 1024,
+  "model_type": "phi3",
+  "num_attention_heads": 8,
+  "num_hidden_layers": 24,
+  "num_key_value_heads": 2,
+  "original_max_position_embeddings": 4096,
+  "pad_token_id": 3,
+  "partial_rotary_factor": 1.0,
+  "resid_pdrop": 0.0,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "partial_rotary_factor": 1.0,
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.9.0",
+  "use_cache": false,
+  "vocab_size": 50000
+}

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50000,
+  "eos_token_id": [
+    3,
+    0
+  ],
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 3,
+  "transformers_version": "5.9.0",
+  "use_cache": false
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eea4d8ca0dca05a31aedbdf22411939e6129a8b6b057daeac2cafebcb69e9689
+size 783731168

tokenizer_config.json

@@ -0,0 +1,15 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "eos_token": "<|end|>",
+  "extra_special_tokens": [
+    "<|user|>",
+    "<|assistant|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|end|>",
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<|endoftext|>"
+}