155 lines
3.3 KiB
Markdown
155 lines
3.3 KiB
Markdown
---
|
||
library_name: transformers
|
||
language:
|
||
- en
|
||
base_model:
|
||
- Qwen/Qwen3-4B
|
||
datasets:
|
||
- Mindie/Summary_article_KSS_style_dataset
|
||
---
|
||
|
||
```markdown
|
||
> Instruction-tuned model with LoRA can enforce structured output without losing general knowledge.
|
||
|
||
# KSS-Format Instruction-Tuned Model (LoRA)
|
||
|
||
## 📌 Model Description
|
||
|
||
This model is instruction-tuned to generate summaries in a structured format:
|
||
|
||
Subject:
|
||
Keywords:
|
||
Summary:
|
||
|
||
The goal of this project is to enforce output format while preserving the base model’s general knowledge and reasoning ability.
|
||
|
||
---
|
||
|
||
## 🎯 Key Features
|
||
|
||
- ✅ Structured summary generation (KSS-style format)
|
||
- ✅ Instruction-following behavior
|
||
- ✅ Knowledge preservation after fine-tuning
|
||
- ✅ Robust across both short and long inputs
|
||
|
||
---
|
||
|
||
## 🧠 Base Model
|
||
|
||
- Base model: Qwen3-4B
|
||
- Fine-tuning method: LoRA (Low-Rank Adaptation)
|
||
|
||
---
|
||
|
||
## 🏗️ Training Details
|
||
|
||
### Dataset
|
||
|
||
- Total samples: 596
|
||
- Contrastive dataset:
|
||
- Instruction data (format enforced)
|
||
- Non-instruction data (free-form output)
|
||
|
||
### Data Sources
|
||
|
||
- GPT-generated summaries (short-form)
|
||
- Base model-generated summaries (long-form)
|
||
- CNN article dataset (961 samples used for label generation)
|
||
|
||
---
|
||
|
||
## ⚙️ Training Setup
|
||
|
||
- Method: LoRA fine-tuning
|
||
- Objective:
|
||
- Learn when to apply structured format
|
||
- Avoid overfitting to instruction-only behavior
|
||
|
||
---
|
||
|
||
## 📊 Evaluation
|
||
|
||
### 1. Format Adherence
|
||
|
||
- Evaluated on 50 samples
|
||
- Result:
|
||
- High consistency in following required format
|
||
|
||
---
|
||
|
||
### 2. Knowledge Preservation (MMLU)
|
||
|
||
- Evaluation method: Logit-based scoring
|
||
- Samples: 20 per subject
|
||
|
||
| Model | Score |
|
||
|--------------|------|
|
||
| Base Model | 0.725 |
|
||
| Tuned Model | 0.724 |
|
||
|
||
👉 No significant performance degradation observed
|
||
|
||
---
|
||
|
||
## 💡 Key Insight
|
||
|
||
Instruction tuning can enforce output structure **without degrading model knowledge**,
|
||
when using a carefully designed LoRA fine tuning setup.
|
||
|
||
---
|
||
|
||
## 🧪 Example Usage
|
||
This model is finetuned on non-thinking mode. non-thinking mode is recommended
|
||
```python
|
||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||
|
||
#model calling
|
||
model_name = "Mindie/Qwen3-4b-kss-style-tuning"
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
model_name,
|
||
torch_dtype=torch.bfloat16,
|
||
device_map="auto"
|
||
)
|
||
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
||
|
||
#generation
|
||
prompt = text
|
||
messages = [
|
||
{"role": "user", "content": f'Use KSS style Summaries: {prompt}'}
|
||
]
|
||
text = tokenizer.apply_chat_template(
|
||
messages,
|
||
tokenize=False,
|
||
add_generation_prompt=True,
|
||
enable_thinking=False # Switches between thinking and non-thinking modes. Default is True.
|
||
)
|
||
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
||
|
||
# conduct text completion
|
||
generated_ids = model.generate(
|
||
**model_inputs,
|
||
max_new_tokens=200
|
||
)
|
||
output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
|
||
content = tokenizer.decode(output_ids, skip_special_tokens=True).strip("\n")
|
||
print(content)
|
||
|
||
###Ouptut foramt
|
||
Subject:
|
||
Keywords:
|
||
Summary:
|
||
|
||
|
||
## 🔗 Project Links
|
||
|
||
- GitHub Repository: https://github.com/byeongmin1/qwen3-4b-lora-instruction-tuning
|
||
|
||
## 📖 Training Details
|
||
|
||
This model was trained using a full pipeline including:
|
||
- data generation
|
||
- filtering
|
||
- instruction tuning (LoRA)
|
||
- Logit Evaluation (MMLU)
|
||
|
||
For full implementation details, please refer to the GitHub repository. |