初始化项目，由ModelHub XC社区提供模型

Model: IamMcCoy/siwon-mini-instruct-0626 Source: Original Platform
2026-09-10 08:36:18 +08:00
commit 5a3fb5793f
16 changed files with 201850 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,131 @@
+---
+language:
+- ko
+- en
+library_name: transformers
+tags:
+- instruction-tuning
+- korean
+- phi-4
+- causal-lm
+model_creator: microsoft
+base_model: microsoft/Phi-4-mini-instruct
+model_name: siwon-mini-instruct-0626
+pipeline_tag: text-generation
+---
+
+<p align="center">
+  <img src="https://cdn-uploads.huggingface.co/production/uploads/665818433a098887a5b95015/IlaWEzz4UrH7d14DkXGgQ.png" width="300" height="300">
+</p>
+
+# siwon-mini-instruct-0626
+
+This model is a fine-tuned version of [`microsoft/Phi-4-mini-instruct`](https://huggingface.co/microsoft/Phi-4-mini-instruct), adapted for Korean instruction-based tasks. The tuning was focused on enhancing Korean performance through supervised fine-tuning with Korean instruction datasets.
+
+---
+
+## 🔧 Token Adjustments
+
+The original model used the same token ID (199999) for multiple special tokens such as BOS, EOS, PAD, and UNK. This caused confusion in instruction-following tasks. We fixed this by remapping the token IDs as follows:
+
+| Token Type | Original ID | Fixed ID | 
+|------------|-------------|----------|
+| BOS        | 199999      | 199999   | 
+| EOS        | 199999      | 200020   |
+| PAD        | 199999      | 200029   |
+| UNK        | 199999      | 200030   | 
+
+These changes ensure proper differentiation and functioning of special tokens during generation and training.
+
+---
+
+## 🗨️ Chat Template
+
+The chat template was updated accordingly to support multi-turn conversation formatting in the Korean context:
+
+```jinja2
+{% for message in messages %}
+{% if message['role'] == 'system' and 'tools' in message and message['tools'] is not none %}
+{{ '<|' + message['role'] + '|>' + message['content'] + '<|tool|>' + message['tools'] + '<|/tool|>' + '<|end|>' }}
+{% else %}
+{{ '<|' + message['role'] + '|>' + message['content'] + '<|end|>' }}
+{% endif %}
+{% endfor %}
+{% if add_generation_prompt %}{{ '<|assistant|>' }}{% endif %}
+```
+
+## 🧪 Inference with Transformers
+Below is an example of how to load and use the model with the adjusted tokenizer, token IDs, and custom prompt template. 
+
+> **Note**: This model uses a custom `chat_template` and updated special token IDs:
+> - `<|end|>` → 200020 (EOS)
+> - `<|dummy_85|>` → 200029 (PAD)
+> - `ï¿½` → 200030 (UNK)
+>
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_path = "IamMcCoy/siwon-mini-instruct-0626"
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_path,
+    device_map="auto",
+    torch_dtype=torch.bfloat16,
+    trust_remote_code=True
+)
+
+tokenizer = AutoTokenizer.from_pretrained(
+    model_path,
+    trust_remote_code=True,
+)
+
+messages = [
+        {"role": "system", "content": "You are a helpful assistant."},
+        {"role": "user", "content": "안녕하세요."},
+    ]
+
+inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
+
+output = model.generate(
+    **inputs,
+    max_new_tokens=2048,
+    # do_sample=True, # Optional
+    # top_p=0.95, # Optional
+    # temperature=0.6, # Optional
+    # repetition_penalty=1.1, # Optional
+)
+
+response = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
+print(response)
+```
+
+## 📊 Model Performance Comparison
+Performance scores across three Korean language benchmarks (KMMLU, ko_best, pawsx_ko).
+
+| Model                          | KMMLU (0-shot) | ko_best (5-shot) | pawsx_ko |
+|-------------------------------|----------------|------------------|----------|
+| Phi-4-mini-instruct           | 0.3161         | 0.6341           | 0.5300   |
+| kanana-1.5-2.1b-instruct-2505 | 0.1577         | 0.7165           | 0.5070   |
+| EXAONE-3.5-2.4B-Instruct       | 0.3071         | 0.6496           | 0.5655   |
+| siwon-mini-instruct-0626      | 0.3387         | 0.5576           | 0.5485   |
+
+
+## 📌 Caution
+* Commercial use is strictly prohibited.
+
+* This model is intended for research and educational use only.
+
+* Redistribution or use in commercial products or services is not allowed.
+
+
+## ✍️ Acknowledgments
+* Base model: microsoft/Phi-4-mini-instruct
+* Special thanks to the open-source community for instruction-tuning resources and Korean language corpora.
+
+
+## 🙏 Feedback & Contributions
+We welcome any feedback to improve the model’s performance, usability, and alignment with Korean instruction tasks.
+If you encounter any issues or have suggestions, please feel free to open an issue on the Hugging Face model page.
+
+Your input is greatly appreciated and will help us enhance the model further.