Qwen3-4b-kss-style-tuning/README.md

library_name, language, base_model, datasets

library_name	language	base_model	datasets
transformers	en	Qwen/Qwen3-4B	Mindie/Summary_article_KSS_style_dataset

> Instruction-tuned model with LoRA can enforce structured output without losing general knowledge.

# KSS-Format Instruction-Tuned Model (LoRA)

## 📌 Model Description

This model is instruction-tuned to generate summaries in a structured format:

Subject:
Keywords:
Summary:

The goal of this project is to enforce output format while preserving the base model’s general knowledge and reasoning ability.

---

## 🎯 Key Features

- ✅ Structured summary generation (KSS-style format)
- ✅ Instruction-following behavior
- ✅ Knowledge preservation after fine-tuning
- ✅ Robust across both short and long inputs

---

## 🧠 Base Model

- Base model: Qwen3-4B
- Fine-tuning method: LoRA (Low-Rank Adaptation)

---

## 🏗️ Training Details

### Dataset

- Total samples: 596
- Contrastive dataset:
  - Instruction data (format enforced)
  - Non-instruction data (free-form output)

### Data Sources

- GPT-generated summaries (short-form)
- Base model-generated summaries (long-form)
- CNN article dataset (961 samples used for label generation)

---

## ⚙️ Training Setup

- Method: LoRA fine-tuning
- Objective:
  - Learn when to apply structured format
  - Avoid overfitting to instruction-only behavior

---

## 📊 Evaluation

### 1. Format Adherence

- Evaluated on 50 samples
- Result:
  - High consistency in following required format

---

### 2. Knowledge Preservation (MMLU)

- Evaluation method: Logit-based scoring
- Samples: 20 per subject

| Model        | Score |
|--------------|------|
| Base Model   | 0.725 |
| Tuned Model  | 0.724 |

👉 No significant performance degradation observed

---

## 💡 Key Insight

Instruction tuning can enforce output structure **without degrading model knowledge**,  
when using a carefully designed LoRA fine tuning setup.

---

## 🧪 Example Usage
This model is finetuned on non-thinking mode. non-thinking mode is recommended
```python
from transformers import AutoTokenizer, AutoModelForCausalLM

#model calling
model_name = "Mindie/Qwen3-4b-kss-style-tuning"
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

#generation
prompt = text
messages = [
    {"role": "user", "content": f'Use KSS style Summaries: {prompt}'}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False # Switches between thinking and non-thinking modes. Default is True.
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

# conduct text completion
generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=200
)
output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
content = tokenizer.decode(output_ids, skip_special_tokens=True).strip("\n")
print(content)

###Ouptut foramt
Subject:
Keywords:
Summary:


## 🔗 Project Links

- GitHub Repository: https://github.com/byeongmin1/qwen3-4b-lora-instruction-tuning

## 📖 Training Details

This model was trained using a full pipeline including:
- data generation
- filtering
- instruction tuning (LoRA)
- Logit Evaluation (MMLU)

For full implementation details, please refer to the GitHub repository.