25 lines
841 B
Markdown
25 lines
841 B
Markdown
|
---
|
||
|
|
base_model: Qwen/Qwen2.5-1.5B-Instruct
|
||
|
|
tags:
|
||
|
|
- safe-flat-lora
|
||
|
|
- baseline
|
||
|
|
- lora
|
||
|
|
- qwen
|
||
|
|
license: apache-2.0
|
||
|
|
---
|
||
|
|
|
||
|
|
# mahdiboughrous/safe-flat-lora-baseline-qwen2.5-1.5b-instruct-merged
|
||
|
|
|
||
|
|
Part of the **Safe-Flat-LoRA** project (the Safety-Flatness Paradox in 4-bit
|
||
|
|
quantization of LoRA-fine-tuned LLMs -- see the project blueprint for the
|
||
|
|
full motivation and methodology).
|
||
|
|
|
||
|
|
- **Base model:** [`Qwen/Qwen2.5-1.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct)
|
||
|
|
- **Method:** `baseline` (plain LoRA fine-tune, used as the pre-Flat-LoRA baseline)
|
||
|
|
- **Artifact:** merged
|
||
|
|
- **Fine-tuning task:** SQL generation (`b-mc2/sql-create-context`), used as a
|
||
|
|
downstream-task proxy to study how post-training 4-bit (NF4) quantization
|
||
|
|
shifts safety-refusal behavior after LoRA fine-tuning.
|
||
|
|
|
||
|
|
Base model Qwen2.5-1.5B-Instruct is Apache-2.0.
|