54 lines
2.0 KiB
Markdown
54 lines
2.0 KiB
Markdown
---
|
|
license: apache-2.0
|
|
base_model: unsloth/Qwen2.5-1.5B
|
|
tags:
|
|
- unsloth
|
|
- lora
|
|
- qlora
|
|
- fine-tuned
|
|
- hr-policy
|
|
language:
|
|
- en
|
|
library_name: transformers
|
|
pipeline_tag: text-generation
|
|
---
|
|
|
|
# Northwind HR Policy Assistant — DPO-Aligned
|
|
|
|
A domain-specific assistant for **HR Policy**, fine-tuned from
|
|
[`unsloth/Qwen2.5-1.5B`](https://huggingface.co/unsloth/Qwen2.5-1.5B) using
|
|
**Unsloth** with LoRA/QLoRA. Format of this repo: **merged**.
|
|
|
|
## Training pipeline
|
|
1. **Non-instruction fine-tuning** — domain adaptation on raw hr policy text.
|
|
2. **Instruction fine-tuning (SFT)** — supervised tuning on instruction→response pairs.
|
|
3. **DPO alignment** — preference tuning for safer, more professional answers.
|
|
|
|
## Hyperparameters
|
|
- LoRA rank / alpha / dropout: 16 / 16 / 0.0
|
|
- Quantization: 4-bit QLoRA
|
|
- SFT learning rate: 0.0002; DPO learning rate: 5e-06; DPO beta: 0.1
|
|
- Effective batch size: 8
|
|
- Max sequence length: 1024
|
|
|
|
## Usage
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
|
|
repo = "meghaGenAI/northwind-hr-policy-dpo-merged"
|
|
tok = AutoTokenizer.from_pretrained(repo)
|
|
model = AutoModelForCausalLM.from_pretrained(repo)
|
|
|
|
msgs = [{"role": "system", "content": "You are the Northwind HR Policy Assistant. Answer user questions about hr policy clearly, accurately, and professionally. If something is outside your scope, say so and point the user to the right team."},
|
|
{"role": "user", "content": "Your question here"}]
|
|
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
|
|
print(tok.decode(model.generate(inputs, max_new_tokens=200)[0]))
|
|
```
|
|
|
|
## System prompt
|
|
> You are the Northwind HR Policy Assistant. Answer user questions about hr policy clearly, accurately, and professionally. If something is outside your scope, say so and point the user to the right team.
|
|
|
|
*Built with [Unsloth](https://github.com/unslothai/unsloth). This model is a
|
|
domain-specific assistant; verify critical answers against authoritative sources.*
|