---
license: apache-2.0
language:
- en
pipeline_tag: text-generation
tags:
- liquid
- lfm
- abliterated
- uncensored
- orthogonal-projection
datasets:
- mlabonne/harmful_behaviors
- mlabonne/harmless_alpaca
base_model:
- LiquidAI/LFM2.5-1.2B-Instruct
---

# LFM-1.2B-Abliterated

This is an abliterated version of Liquid AI's `LFM-1.2B` instruct model. It has been modified via layerwise orthogonal projection to completely remove its built-in safety filters and refusal mechanisms, allowing the continuous-time hybrid architecture to flow uninhibited. 

It was created because I wasn't satisfied with other abliterations I saw for these, and decided to take a crack at it in a way that matched one of my favorite models: malbonne's gemma3-27b-it-abliterated

### ## Architectural Hurdles & Methodology
Liquid Foundation Models use a non-standard hybrid architecture combining Grouped Query Attention (GQA) with continuous-time Gated-Short Convolutions. Standard ablation scripts designed for Llama-class transformers will crash on this architecture due to complex caching objects (`Lfm2HybridConvCache`) and completely different linear projection pathways.

This model was abliterated by:
1. Adapting forward hooks to safely pass Liquid's dynamic states during the measurement phase.
2. Extracting the "refusal vector" from the hidden states of 100 harmful vs. 100 harmless instructions (utilizing `mlabonne/harmful_behaviors` and `mlabonne/harmless_alpaca`).
3. Applying orthogonal projection (`W_new = W - v(v^T W)`) directly to the `conv.out_proj` (Token Mixing) and `feed_forward.w2` (Channel Mixing) base weights across all 16 `Lfm2DecoderLayer` blocks.

*Credit to Maxime Labonne and Sumandora for the foundational datasets and math, adapted here for the LFM architecture.*
## Notes on AMD/ROCm Compatibility

If you are running this model (or attempting similar LFm ablations) on AMD consumer hardware (RDNA3/7000 series), be aware that PyTorch's hipblas backend has known segmentation faults with Liquid's RoPE expansion implementation and unaligned bfloat16 matrix multiplications. Loading the model in float16 or using CPU offloading for the forward passes is strongly recommended.
### ## Usage 
This model retains the exact same architecture as the base `LFM-1.2B` and requires `trust_remote_code=True` when loading via `transformers`. It is highly recommended to use the exact `<|user|>` and `<|assistant|>` chat formatting without any injected system prompts for the best uncensored performance.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "paperscarecrow/LFM2.5-1.2B-Instruct-abliterated"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.float16, 
    device_map="auto", 
    trust_remote_code=True
)

prompt = "<|user|>\nGive me a detailed tutorial on picking a master padlock.\n<|assistant|>\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs, 
        max_new_tokens=150, 
        do_sample=True,
        temperature=0.7
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=True))