初始化项目，由ModelHub XC社区提供模型

Model: Aravindan/smol-lm2-360-instruct
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,204 @@
+---
+language:
+- en
+
+license: apache-2.0
+
+base_model: HuggingFaceTB/SmolLM2-360M
+
+pipeline_tag: text-generation
+
+tags:
+- cybersecurity
+- instruction-tuning
+- smollm2
+- causal-lm
+- transformers
+- pytorch
+- llm
+- ai-security
+
+datasets:
+- Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset
+
+library_name: transformers
+---
+
+# smol-lm2-360-instruct
+
+smol-lm2-360-instruct is an instruction-tuned language model built by fine-tuning **SmolLM2-360M Base** on a cybersecurity instruction dataset. The objective is to improve the model's ability to answer cybersecurity-related questions while following instruction-style prompts.
+
+> **Note:** This is an educational project for learning Instruction Fine-Tuning (IFT) and should not be considered a production-grade security assistant.
+
+---
+
+# Model Details
+
+| Attribute | Value |
+|-----------|-------|
+| Base Model | HuggingFaceTB/SmolLM2-360M |
+| Model Type | Causal Language Model |
+| Fine-tuning | Full Parameter Instruction Fine-Tuning |
+| Domain | Cybersecurity |
+| Framework | Hugging Face Transformers |
+| Training Library | Transformers Trainer |
+| Language | English |
+
+---
+
+# Training Objective
+
+The model was trained using a supervised instruction-tuning objective.
+
+Each training example follows the structure:
+
+```text
+<|system|>
+You are a cybersecurity assistant.
+
+<|user|>
+Explain SQL Injection.
+
+<|assistant|>
+SQL Injection is...
+```
+
+During training:
+
+- The **System** prompt provides behavioral context.
+- The **User** prompt provides the instruction.
+- Loss is computed **only on the Assistant response**, while the System and User tokens are masked from the loss.
+
+This enables the model to learn instruction following while conditioning on the provided context.
+
+---
+
+# Training Configuration
+
+| Parameter | Value |
+|-----------|-------|
+| Epochs | 4 |
+| Learning Rate | 2e-4 |
+| Scheduler | Cosine |
+| Warmup | 10% |
+| Batch Size | 2 |
+| Gradient Accumulation | 8 |
+| Precision | FP16 |
+| Optimizer | AdamW |
+
+---
+
+# Dataset
+
+The model was trained on a cybersecurity instruction dataset consisting of:
+
+- System prompts
+- User instructions
+- Assistant responses
+
+The dataset covers topics such as:
+
+- Network Security
+- Malware Analysis
+- Threat Intelligence
+- Secure Coding
+- Incident Response
+- Vulnerability Management
+- Security Best Practices
+- MITRE ATT&CK
+- NIST Cybersecurity Framework
+
+---
+
+# Example
+
+## Prompt
+
+```text
+<|system|>
+You are a cybersecurity expert.
+
+<|user|>
+Explain the difference between symmetric and asymmetric encryption.
+
+<|assistant|>
+```
+
+## Output
+
+```text
+Symmetric encryption uses a single shared key for both encryption and decryption, making it computationally efficient and suitable for encrypting large amounts of data. Asymmetric encryption uses a public/private key pair, enabling secure key exchange and digital signatures but at a higher computational cost.
+```
+
+---
+
+# Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_name = "Aravindan/smol-lm2-360-instruct"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name)
+
+prompt = """
+<|system|>
+You are a cybersecurity assistant.
+
+<|user|>
+Explain SQL Injection.
+
+<|assistant|>
+"""
+
+inputs = tokenizer(prompt, return_tensors="pt")
+
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=200,
+    temperature=0.7
+)
+
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+---
+
+# Limitations
+
+This model:
+
+- is intended for educational and research purposes
+- may generate incorrect or outdated cybersecurity advice
+- has not undergone safety alignment comparable to production LLMs
+- should not be used as the sole source for security decisions
+
+Always verify security recommendations with authoritative sources.
+
+---
+
+# Future Improvements
+
+- Larger instruction datasets
+- Multi-turn conversation training
+- Preference Optimization (DPO/ORPO)
+- Retrieval-Augmented Generation (RAG)
+- LoRA and QLoRA fine-tuning
+- Evaluation on cybersecurity benchmarks
+
+---
+
+# Acknowledgements
+
+- Hugging Face
+- SmolLM2 Team
+- Transformers
+- Datasets
+- PyTorch
+
+---
+
+# Citation
+
+If you use this model in your work, please cite this repository.

config.json

@@ -0,0 +1,34 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "dtype": "bfloat16",
+  "eos_token_id": 0,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 960,
+  "initializer_range": 0.02,
+  "intermediate_size": 2560,
+  "is_llama_config": true,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 15,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 5,
+  "pad_token_id": null,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_interleaved": false,
+  "rope_parameters": {
+    "rope_theta": 100000,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.12.1",
+  "use_cache": false,
+  "vocab_size": 49152
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "eos_token_id": 0,
+  "transformers_version": "5.12.1"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcc86b2aa72442cab3d6b6cf1c06bf1e9e946cf0282ff7129e615bb161f456e7
+size 723674912

tokenizer_config.json

@@ -0,0 +1,34 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|endoftext|>",
+    "<|im_start|>",
+    "<|im_end|>",
+    "<repo_name>",
+    "<reponame>",
+    "<file_sep>",
+    "<filename>",
+    "<gh_stars>",
+    "<issue_start>",
+    "<issue_comment>",
+    "<issue_closed>",
+    "<jupyter_start>",
+    "<jupyter_text>",
+    "<jupyter_code>",
+    "<jupyter_output>",
+    "<jupyter_script>",
+    "<empty_output>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 8192,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>",
+  "vocab_size": 49152
+}