初始化项目，由ModelHub XC社区提供模型
Model: heterodoxin/qwen2.5-7b-instruct-apostate Source: Original Platform
This commit is contained in:
ModelHub XC
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||||
57
README.md
Normal file
57
README.md
Normal file
@@ -0,0 +1,57 @@
|
|||||||
|
---
|
||||||
|
library_name: transformers
|
||||||
|
base_model: "Qwen/Qwen2.5-7B-Instruct"
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
tags:
|
||||||
|
- apostate
|
||||||
|
- uncensored
|
||||||
|
- abliteration
|
||||||
|
---
|
||||||
|
|
||||||
|
# Qwen2.5-7B-Instruct Apostate
|
||||||
|
|
||||||
|
> Join the community: [Discord](https://discord.gg/NPA7xrATEH)
|
||||||
|
|
||||||
|
An uncensored edit of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct). Refusal behavior is removed by editing the model weights directly — no finetuning, no adapter, no runtime hook. The result is a standard Transformers checkpoint that drops in anywhere the base model works.
|
||||||
|
|
||||||
|
Produced with **[Apostate](https://github.com/heterodoxin/apostate)**.
|
||||||
|
|
||||||
|
## Method
|
||||||
|
|
||||||
|
Apostate finds the residual-stream direction most responsible for refusal behavior and permanently projects it out of the model's weights. The edit targets the writer side: per layer, the refusal direction is removed from the weight matrices of every module that writes to the residual stream (attention output projections and MLP down-projections).
|
||||||
|
|
||||||
|
The operator is a **contrastive co-vector** edit `E = I − R Dᵀ`. Removing the refusal direction outright disturbs benign behavior, while naively preserving all harmless variance along it leaves the refusal that is entangled with general behavior intact. Instead `D = R − W`, where the predictor `W` is fit to reproduce the harmless variance along `R` while being explicitly suppressed on harmful prompts — `W = (AᵀA + γ·CᵀC + λI)⁻¹Aᵀb` with `A` the harmless and `C` the harmful activations (both orthogonalized to `R`). The edit thus keeps the harmless-specific component and removes the component shared with refusal, driving refusal down while keeping the change to harmless behavior (KL) small. This holds even on architectures with residual/embedding scaling multipliers (e.g. Granite), where mean-preserving oblique ablation under-ablates.
|
||||||
|
|
||||||
|
The refusal subspace is found via TPE search with causal layer importance scoring to concentrate edits where they most influence refusal generation.
|
||||||
|
|
||||||
|
## Results
|
||||||
|
|
||||||
|
Evaluated on held-out prompts from JailbreakBench and the harmful_behaviors test split. Refusal is scored by a classifier with a weak-compliance guard; KL measures token-distribution shift on harmless prompts.
|
||||||
|
|
||||||
|
| Metric | Base | Apostate |
|
||||||
|
|---|---|---|
|
||||||
|
| Refusal rate | 96.0% | 3.0% |
|
||||||
|
| Comply rate | — | 97.0% |
|
||||||
|
| Harmless KL (nats) | 0 | 0.095 |
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
```python
|
||||||
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||||
|
|
||||||
|
model_id = "heterodoxin/qwen2.5-7b-instruct-apostate"
|
||||||
|
tok = AutoTokenizer.from_pretrained(model_id)
|
||||||
|
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
|
||||||
|
|
||||||
|
messages = [{"role": "user", "content": "Your prompt here"}]
|
||||||
|
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||||||
|
inputs = tok(text, return_tensors="pt").to(model.device)
|
||||||
|
outputs = model.generate(**inputs, max_new_tokens=512)
|
||||||
|
print(tok.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
|
||||||
|
```
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- This is an **uncensored** model. It will respond to requests the base model refuses.
|
||||||
|
- The edit is baked into the weights permanently; no system prompt or adapter is required.
|
||||||
|
- See [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) for base model capabilities and license.
|
||||||
137
apostate_config.json
Normal file
137
apostate_config.json
Normal file
@@ -0,0 +1,137 @@
|
|||||||
|
{
|
||||||
|
"model": "Qwen/Qwen2.5-7B-Instruct",
|
||||||
|
"output_dir": "/var/home/Heterodoxin/qwen25_rebake_out",
|
||||||
|
"profile": "balanced",
|
||||||
|
"device": "cuda",
|
||||||
|
"load_in_4bit": true,
|
||||||
|
"cpu_offload_gb": 0.0,
|
||||||
|
"compute_dtype": "bfloat16",
|
||||||
|
"seed": 0,
|
||||||
|
"resume": false,
|
||||||
|
"cache_activations": true,
|
||||||
|
"activation_cache_dir": null,
|
||||||
|
"harmful_path": "mlabonne/harmful_behaviors:train:text|/var/home/Heterodoxin/apostate/data/harmful.txt|/var/home/Heterodoxin/apostate/data/refusal_calibration.txt",
|
||||||
|
"harmless_path": "mlabonne/harmless_alpaca:train:text|/var/home/Heterodoxin/apostate/data/harmless.txt",
|
||||||
|
"harmful_test": "mlabonne/harmful_behaviors:test:text|JailbreakBench/JBB-Behaviors@behaviors:harmful:Goal|/var/home/Heterodoxin/apostate/data/refusal_calibration.txt",
|
||||||
|
"harmless_test": "mlabonne/harmless_alpaca:test:text",
|
||||||
|
"refusal_eval_path": "JailbreakBench/JBB-Behaviors@behaviors:harmful:Goal|/var/home/Heterodoxin/apostate/data/refusal_calibration.txt",
|
||||||
|
"refusal_eval_n": 48,
|
||||||
|
"kl_eval_path": "mlabonne/harmless_alpaca:test:text",
|
||||||
|
"kl_eval_n": 48,
|
||||||
|
"preserve_path": null,
|
||||||
|
"n_harmful": 600,
|
||||||
|
"n_harmless": 600,
|
||||||
|
"n_eval": 300,
|
||||||
|
"max_new_tokens": 32,
|
||||||
|
"batch_size": 24,
|
||||||
|
"baseline_eval_n": 24,
|
||||||
|
"head_sweep": true,
|
||||||
|
"head_sweep_min": 3.5,
|
||||||
|
"head_sweep_max": 5.5,
|
||||||
|
"head_sweep_step": 0.5,
|
||||||
|
"head_sweep_top_k": 6,
|
||||||
|
"head_sweep_probe_n": 8,
|
||||||
|
"head_sweep_eval_n": 48,
|
||||||
|
"head_sweep_probe_classifier": false,
|
||||||
|
"fit_response_activations": false,
|
||||||
|
"fit_response_n": 160,
|
||||||
|
"fit_response_tokens": 32,
|
||||||
|
"refusal_rank": 1,
|
||||||
|
"variance_threshold": 0.9,
|
||||||
|
"max_rank": 3,
|
||||||
|
"direction_layer_frac": 0.6,
|
||||||
|
"direction_scope": "global",
|
||||||
|
"multi_refusal": true,
|
||||||
|
"multi_refusal_clusters": 6,
|
||||||
|
"multi_refusal_min_norm": 0.08,
|
||||||
|
"multi_refusal_min_separation": 0.05,
|
||||||
|
"multi_refusal_min_coverage": 0.05,
|
||||||
|
"orthogonalize_direction": true,
|
||||||
|
"causal_targeting": true,
|
||||||
|
"causal_floor": 0.1,
|
||||||
|
"causal_temperature": 1.0,
|
||||||
|
"preserve_rank": 8,
|
||||||
|
"refine_refusal": true,
|
||||||
|
"refine_max_scale": 2.0,
|
||||||
|
"refine_steps": 6,
|
||||||
|
"refine_deescalate": true,
|
||||||
|
"refine_kl_steps": 10,
|
||||||
|
"refine_scale_rerank_k": 2,
|
||||||
|
"refine_kl_layer_steps": 10,
|
||||||
|
"refine_kl_layer_candidates": 8,
|
||||||
|
"repair_steps": 4,
|
||||||
|
"repair_candidates": 8,
|
||||||
|
"repair_rerank_k": 5,
|
||||||
|
"repair_probe_candidates": 20,
|
||||||
|
"repair_probe_ref_n": 12,
|
||||||
|
"repair_probe_kl_n": 16,
|
||||||
|
"repair_probe_positions": 8,
|
||||||
|
"repair_refusal_regress_slack": 0.01,
|
||||||
|
"repair_stop_kl_frac": 0.8,
|
||||||
|
"repair_min_alpha": 0.001,
|
||||||
|
"repair_min_kl_gain": 0.003,
|
||||||
|
"repair_min_refusal_gain": 0.005,
|
||||||
|
"repair_min_score_gain": 0.01,
|
||||||
|
"repair_eval_n": 96,
|
||||||
|
"repair_kl_n": 64,
|
||||||
|
"refine_refusal_slack": 0.01,
|
||||||
|
"final_zero_trim": false,
|
||||||
|
"final_push_bake_margin": 0.075,
|
||||||
|
"guard_max_iters": 2,
|
||||||
|
"guard_leakage_eps": 0.15,
|
||||||
|
"guard_alpha_step": 0.25,
|
||||||
|
"optimize": true,
|
||||||
|
"n_trials": 16,
|
||||||
|
"adaptive_trials": true,
|
||||||
|
"kl_weight": 6.0,
|
||||||
|
"kl_target": 0.04,
|
||||||
|
"kl_target_weight": 18.0,
|
||||||
|
"kl_quad_weight": 22.0,
|
||||||
|
"kl_headroom_weight": 0.0,
|
||||||
|
"kl_over_budget_weight": 72.0,
|
||||||
|
"refusal_target_weight": 4.0,
|
||||||
|
"refusal_quad_weight": 8.0,
|
||||||
|
"kl_positions": 8,
|
||||||
|
"opt_capability": true,
|
||||||
|
"opt_capability_weight": 2.5,
|
||||||
|
"opt_capability_code_n": 8,
|
||||||
|
"opt_capability_math_n": 8,
|
||||||
|
"opt_eval_n": 32,
|
||||||
|
"opt_gen_tokens": 32,
|
||||||
|
"opt_objective": "generation",
|
||||||
|
"opt_rerank_k": 5,
|
||||||
|
"opt_guard": true,
|
||||||
|
"opt_early_stop": true,
|
||||||
|
"opt_early_stop_margin": 0.02,
|
||||||
|
"gemma_ple": false,
|
||||||
|
"gemma_query": false,
|
||||||
|
"ple_max_rank": 2,
|
||||||
|
"prune": false,
|
||||||
|
"prune_max_frac": 0.25,
|
||||||
|
"prune_kl": 0.04,
|
||||||
|
"max_kl": 0.12,
|
||||||
|
"target_refusal": 0.05,
|
||||||
|
"oblique_ablation": true,
|
||||||
|
"oblique_strength": 1.0,
|
||||||
|
"oblique_denom_floor": 0.2,
|
||||||
|
"oblique_writers_only": true,
|
||||||
|
"oblique_predictive": true,
|
||||||
|
"predictive_ridge": 0.01,
|
||||||
|
"oblique_preserve": 1.0,
|
||||||
|
"oblique_contrast": 1.0,
|
||||||
|
"reader_max_kl": 0.55,
|
||||||
|
"reader_kl_target": 0.3,
|
||||||
|
"reader_strengths": [
|
||||||
|
2.0,
|
||||||
|
3.0,
|
||||||
|
4.0,
|
||||||
|
5.0,
|
||||||
|
6.0,
|
||||||
|
7.0
|
||||||
|
],
|
||||||
|
"reader_guard_rank": 3,
|
||||||
|
"reader_margin_target": -1.0,
|
||||||
|
"reader_strength_kl_weight": 1.0,
|
||||||
|
"save_dtype": "bfloat16",
|
||||||
|
"bake": true
|
||||||
|
}
|
||||||
54
chat_template.jinja
Normal file
54
chat_template.jinja
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
{%- if tools %}
|
||||||
|
{{- '<|im_start|>system\n' }}
|
||||||
|
{%- if messages[0]['role'] == 'system' %}
|
||||||
|
{{- messages[0]['content'] }}
|
||||||
|
{%- else %}
|
||||||
|
{{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
|
||||||
|
{%- endif %}
|
||||||
|
{{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
||||||
|
{%- for tool in tools %}
|
||||||
|
{{- "\n" }}
|
||||||
|
{{- tool | tojson }}
|
||||||
|
{%- endfor %}
|
||||||
|
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
||||||
|
{%- else %}
|
||||||
|
{%- if messages[0]['role'] == 'system' %}
|
||||||
|
{{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
|
||||||
|
{%- else %}
|
||||||
|
{{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- for message in messages %}
|
||||||
|
{%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
|
||||||
|
{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
|
||||||
|
{%- elif message.role == "assistant" %}
|
||||||
|
{{- '<|im_start|>' + message.role }}
|
||||||
|
{%- if message.content %}
|
||||||
|
{{- '\n' + message.content }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- for tool_call in message.tool_calls %}
|
||||||
|
{%- if tool_call.function is defined %}
|
||||||
|
{%- set tool_call = tool_call.function %}
|
||||||
|
{%- endif %}
|
||||||
|
{{- '\n<tool_call>\n{"name": "' }}
|
||||||
|
{{- tool_call.name }}
|
||||||
|
{{- '", "arguments": ' }}
|
||||||
|
{{- tool_call.arguments | tojson }}
|
||||||
|
{{- '}\n</tool_call>' }}
|
||||||
|
{%- endfor %}
|
||||||
|
{{- '<|im_end|>\n' }}
|
||||||
|
{%- elif message.role == "tool" %}
|
||||||
|
{%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
|
||||||
|
{{- '<|im_start|>user' }}
|
||||||
|
{%- endif %}
|
||||||
|
{{- '\n<tool_response>\n' }}
|
||||||
|
{{- message.content }}
|
||||||
|
{{- '\n</tool_response>' }}
|
||||||
|
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
||||||
|
{{- '<|im_end|>\n' }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endfor %}
|
||||||
|
{%- if add_generation_prompt %}
|
||||||
|
{{- '<|im_start|>assistant\n' }}
|
||||||
|
{%- endif %}
|
||||||
61
config.json
Normal file
61
config.json
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
{
|
||||||
|
"architectures": [
|
||||||
|
"Qwen2ForCausalLM"
|
||||||
|
],
|
||||||
|
"attention_dropout": 0.0,
|
||||||
|
"bos_token_id": 151643,
|
||||||
|
"dtype": "bfloat16",
|
||||||
|
"eos_token_id": 151645,
|
||||||
|
"hidden_act": "silu",
|
||||||
|
"hidden_size": 3584,
|
||||||
|
"initializer_range": 0.02,
|
||||||
|
"intermediate_size": 18944,
|
||||||
|
"layer_types": [
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention"
|
||||||
|
],
|
||||||
|
"max_position_embeddings": 32768,
|
||||||
|
"max_window_layers": 28,
|
||||||
|
"model_type": "qwen2",
|
||||||
|
"num_attention_heads": 28,
|
||||||
|
"num_hidden_layers": 28,
|
||||||
|
"num_key_value_heads": 4,
|
||||||
|
"pad_token_id": null,
|
||||||
|
"rms_norm_eps": 1e-06,
|
||||||
|
"rope_parameters": {
|
||||||
|
"rope_theta": 1000000.0,
|
||||||
|
"rope_type": "default"
|
||||||
|
},
|
||||||
|
"sliding_window": null,
|
||||||
|
"tie_word_embeddings": false,
|
||||||
|
"transformers_version": "5.12.1",
|
||||||
|
"use_cache": true,
|
||||||
|
"use_sliding_window": false,
|
||||||
|
"vocab_size": 152064
|
||||||
|
}
|
||||||
14
generation_config.json
Normal file
14
generation_config.json
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
{
|
||||||
|
"bos_token_id": 151643,
|
||||||
|
"do_sample": true,
|
||||||
|
"eos_token_id": [
|
||||||
|
151645,
|
||||||
|
151643
|
||||||
|
],
|
||||||
|
"pad_token_id": 151643,
|
||||||
|
"repetition_penalty": 1.05,
|
||||||
|
"temperature": 0.7,
|
||||||
|
"top_k": 20,
|
||||||
|
"top_p": 0.8,
|
||||||
|
"transformers_version": "5.12.1"
|
||||||
|
}
|
||||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:ac0ace6d8865421b47307876b0cba6e824fd7ac92c9c900889b3d28e72223056
|
||||||
|
size 15231272152
|
||||||
136
report.json
Normal file
136
report.json
Normal file
@@ -0,0 +1,136 @@
|
|||||||
|
{
|
||||||
|
"model": "Qwen/Qwen2.5-7B-Instruct",
|
||||||
|
"num_layers": 28,
|
||||||
|
"hidden_size": 3584,
|
||||||
|
"direction_layer": 20,
|
||||||
|
"refusal_subspace_rank": 3,
|
||||||
|
"max_refusal_rank": 3,
|
||||||
|
"multi_refusal": true,
|
||||||
|
"multi_refusal_clusters": 6,
|
||||||
|
"multi_refusal_min_norm": 0.08,
|
||||||
|
"multi_refusal_min_separation": 0.05,
|
||||||
|
"multi_refusal_min_coverage": 0.05,
|
||||||
|
"initial_separation": 74.183,
|
||||||
|
"baseline_refusal_rate": 0.9583,
|
||||||
|
"baseline_eval_n": 24,
|
||||||
|
"edited_refusal_rate": 0.0286,
|
||||||
|
"refusal_metric": "classifier + weak guard",
|
||||||
|
"harmless_kl_nats": 0.0948,
|
||||||
|
"kl_backoff_steps": 0,
|
||||||
|
"kl_layer_trim_steps": 0,
|
||||||
|
"repair_steps": 0,
|
||||||
|
"residual_repair": [],
|
||||||
|
"guard_history": [
|
||||||
|
{
|
||||||
|
"iter": 0,
|
||||||
|
"separation": 48.503,
|
||||||
|
"ratio": 0.6538,
|
||||||
|
"rank": 2,
|
||||||
|
"refusal": 0.125,
|
||||||
|
"kl": 0.0682
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"iter": 1,
|
||||||
|
"separation": 37.3604,
|
||||||
|
"ratio": 0.5036,
|
||||||
|
"rank": 3,
|
||||||
|
"refusal": 0.0625,
|
||||||
|
"kl": 0.0887
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"layer_alphas": [
|
||||||
|
0.562,
|
||||||
|
0.562,
|
||||||
|
0.562,
|
||||||
|
0.562,
|
||||||
|
0.562,
|
||||||
|
0.562,
|
||||||
|
0.562,
|
||||||
|
0.562,
|
||||||
|
0.562,
|
||||||
|
0.562,
|
||||||
|
0.562,
|
||||||
|
0.562,
|
||||||
|
1.125,
|
||||||
|
1.13,
|
||||||
|
1.169,
|
||||||
|
1.207,
|
||||||
|
1.289,
|
||||||
|
1.262,
|
||||||
|
1.285,
|
||||||
|
1.358,
|
||||||
|
1.396,
|
||||||
|
1.418,
|
||||||
|
1.467,
|
||||||
|
1.487,
|
||||||
|
1.496,
|
||||||
|
1.491,
|
||||||
|
0.562,
|
||||||
|
0.562
|
||||||
|
],
|
||||||
|
"ple_layer_alphas": [],
|
||||||
|
"ple_embed_alpha": 0.0,
|
||||||
|
"ple_model_projection_alpha": 0.0,
|
||||||
|
"embed_alpha": 0.084,
|
||||||
|
"head_alpha": 0.478,
|
||||||
|
"head_token_alpha": 0.0,
|
||||||
|
"preserve_rank": 8,
|
||||||
|
"preserve_source": "harmless",
|
||||||
|
"pruned_layers": [],
|
||||||
|
"layers_after_prune": 28,
|
||||||
|
"elapsed_sec": 748.9,
|
||||||
|
"profile": "balanced",
|
||||||
|
"target_refusal": 0.05,
|
||||||
|
"max_kl": 0.12,
|
||||||
|
"kl_target": 0.04,
|
||||||
|
"refusal_eval_path": "JailbreakBench/JBB-Behaviors@behaviors:harmful:Goal|/var/home/Heterodoxin/apostate/data/refusal_calibration.txt",
|
||||||
|
"refusal_eval_n": 48,
|
||||||
|
"kl_positions": 8,
|
||||||
|
"kl_eval_path": "mlabonne/harmless_alpaca:test:text",
|
||||||
|
"kl_eval_n": 48,
|
||||||
|
"oblique_ablation": true,
|
||||||
|
"oblique_strength": 1.0,
|
||||||
|
"opt_capability": true,
|
||||||
|
"opt_capability_weight": 2.5,
|
||||||
|
"timings_sec": {
|
||||||
|
"load_model": 12.6,
|
||||||
|
"load_prompts": 5.3,
|
||||||
|
"baseline_refusal": 12.3,
|
||||||
|
"activation_fit": 82.5,
|
||||||
|
"causal_scores": 1.9,
|
||||||
|
"optimize_profile": 222.5,
|
||||||
|
"guard": 17.6,
|
||||||
|
"refine_refusal": 10.4,
|
||||||
|
"validation_metrics": 0.0,
|
||||||
|
"repair": 210.6,
|
||||||
|
"prune": 0.0,
|
||||||
|
"test_metrics": 157.0,
|
||||||
|
"bake": 16.0
|
||||||
|
},
|
||||||
|
"command": "/var/home/Heterodoxin/apostate/apostate/cli.py --optimize --model Qwen/Qwen2.5-7B-Instruct --seed 0 --n-trials 16 --oblique-predictive --target-refusal 0.05 --refusal-eval-n 48 --kl-eval-n 48 --output-dir /var/home/Heterodoxin/qwen25_rebake_out",
|
||||||
|
"optimized": true,
|
||||||
|
"best_params": {
|
||||||
|
"direction_source": "activations",
|
||||||
|
"direction_layer_frac": 0.7420306856024312,
|
||||||
|
"refusal_rank": 2,
|
||||||
|
"strength": 1.3304489846989276,
|
||||||
|
"band_center": 0.694578409610168,
|
||||||
|
"band_width": 0.5088027167445976,
|
||||||
|
"causal_mix": 0.2685319326357378,
|
||||||
|
"causal_power": 1.9789446986943955,
|
||||||
|
"direction_sign": 1.0,
|
||||||
|
"ablate_embed": true,
|
||||||
|
"embed_scale": 0.055873895941540415,
|
||||||
|
"ablate_head": true,
|
||||||
|
"head_scale": 0.02551694167411883,
|
||||||
|
"head_alpha": 0.4252805559941053
|
||||||
|
},
|
||||||
|
"best_trial": {
|
||||||
|
"refusal": 0.125,
|
||||||
|
"kl": 0.0682,
|
||||||
|
"capability_logprob": -7.7132,
|
||||||
|
"capability_drift": 0.0
|
||||||
|
},
|
||||||
|
"n_trials": 16,
|
||||||
|
"baked_to": "/var/home/Heterodoxin/qwen25_rebake_out"
|
||||||
|
}
|
||||||
124
report.md
Normal file
124
report.md
Normal file
@@ -0,0 +1,124 @@
|
|||||||
|
# Apostate Run Report
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
| Metric | Value |
|
||||||
|
| --- | --- |
|
||||||
|
| Base model | Qwen/Qwen2.5-7B-Instruct |
|
||||||
|
| Profile | balanced |
|
||||||
|
| Output | /var/home/Heterodoxin/qwen25_rebake_out |
|
||||||
|
| Layers | 28 |
|
||||||
|
| Hidden size | 3584 |
|
||||||
|
| Direction layer | 20 |
|
||||||
|
| Refusal rank | 3 |
|
||||||
|
| Max refusal rank | 3 |
|
||||||
|
| Multi refusal | True |
|
||||||
|
| Multi clusters | 6 |
|
||||||
|
| Multi min coverage | 0.05 |
|
||||||
|
| Baseline refusal (n=24) | 95.8% |
|
||||||
|
| Edited refusal | 2.9% |
|
||||||
|
| Refusal metric | classifier + weak guard |
|
||||||
|
| Harmless KL | 0.095 |
|
||||||
|
| Target refusal | 5.0% |
|
||||||
|
| KL target | 0.040 |
|
||||||
|
| KL budget | 0.120 |
|
||||||
|
| KL positions | 8 |
|
||||||
|
| KL layer trims | 0 |
|
||||||
|
| Repair steps | 0 |
|
||||||
|
| Preserve rank | 8 |
|
||||||
|
| Preserve source | harmless |
|
||||||
|
| Capability penalty | True |
|
||||||
|
| Elapsed | 748.9 sec |
|
||||||
|
|
||||||
|
## Command
|
||||||
|
|
||||||
|
```text
|
||||||
|
/var/home/Heterodoxin/apostate/apostate/cli.py --optimize --model Qwen/Qwen2.5-7B-Instruct --seed 0 --n-trials 16 --oblique-predictive --target-refusal 0.05 --refusal-eval-n 48 --kl-eval-n 48 --output-dir /var/home/Heterodoxin/qwen25_rebake_out
|
||||||
|
```
|
||||||
|
|
||||||
|
## Best Parameters
|
||||||
|
| Parameter | Value |
|
||||||
|
| --- | --- |
|
||||||
|
| direction_source | activations |
|
||||||
|
| direction_layer_frac | 0.742 |
|
||||||
|
| refusal_rank | 2 |
|
||||||
|
| strength | 1.3304 |
|
||||||
|
| band_center | 0.6946 |
|
||||||
|
| band_width | 0.5088 |
|
||||||
|
| causal_mix | 0.2685 |
|
||||||
|
| causal_power | 1.9789 |
|
||||||
|
| direction_sign | 1.0 |
|
||||||
|
| ablate_embed | True |
|
||||||
|
| embed_scale | 0.0559 |
|
||||||
|
| ablate_head | True |
|
||||||
|
| head_scale | 0.0255 |
|
||||||
|
| head_alpha | 0.4253 |
|
||||||
|
|
||||||
|
## Best Trial
|
||||||
|
| Metric | Value |
|
||||||
|
| --- | --- |
|
||||||
|
| refusal | 0.125 |
|
||||||
|
| kl | 0.0682 |
|
||||||
|
| capability_logprob | -7.7132 |
|
||||||
|
| capability_drift | 0.0 |
|
||||||
|
|
||||||
|
## Layer Alphas
|
||||||
|
| Layer | Alpha |
|
||||||
|
| --- | --- |
|
||||||
|
| 0 | 0.562 |
|
||||||
|
| 1 | 0.562 |
|
||||||
|
| 2 | 0.562 |
|
||||||
|
| 3 | 0.562 |
|
||||||
|
| 4 | 0.562 |
|
||||||
|
| 5 | 0.562 |
|
||||||
|
| 6 | 0.562 |
|
||||||
|
| 7 | 0.562 |
|
||||||
|
| 8 | 0.562 |
|
||||||
|
| 9 | 0.562 |
|
||||||
|
| 10 | 0.562 |
|
||||||
|
| 11 | 0.562 |
|
||||||
|
| 12 | 1.125 |
|
||||||
|
| 13 | 1.130 |
|
||||||
|
| 14 | 1.169 |
|
||||||
|
| 15 | 1.207 |
|
||||||
|
| 16 | 1.289 |
|
||||||
|
| 17 | 1.262 |
|
||||||
|
| 18 | 1.285 |
|
||||||
|
| 19 | 1.358 |
|
||||||
|
| 20 | 1.396 |
|
||||||
|
| 21 | 1.418 |
|
||||||
|
| 22 | 1.467 |
|
||||||
|
| 23 | 1.487 |
|
||||||
|
| 24 | 1.496 |
|
||||||
|
| 25 | 1.491 |
|
||||||
|
| 26 | 0.562 |
|
||||||
|
| 27 | 0.562 |
|
||||||
|
|
||||||
|
## Guard History
|
||||||
|
| iter | separation | ratio | rank | refusal | kl | reverted |
|
||||||
|
| --- | --- | --- | --- | --- | --- | --- |
|
||||||
|
| 0 | 48.503 | 0.6538 | 2 | 0.125 | 0.0682 | |
|
||||||
|
| 1 | 37.3604 | 0.5036 | 3 | 0.0625 | 0.0887 | |
|
||||||
|
|
||||||
|
## Timings
|
||||||
|
| Phase | Seconds |
|
||||||
|
| --- | --- |
|
||||||
|
| load_model | 12.6 |
|
||||||
|
| load_prompts | 5.3 |
|
||||||
|
| baseline_refusal | 12.3 |
|
||||||
|
| activation_fit | 82.5 |
|
||||||
|
| causal_scores | 1.9 |
|
||||||
|
| optimize_profile | 222.5 |
|
||||||
|
| guard | 17.6 |
|
||||||
|
| refine_refusal | 10.4 |
|
||||||
|
| validation_metrics | 0.0 |
|
||||||
|
| repair | 210.6 |
|
||||||
|
| prune | 0.0 |
|
||||||
|
| test_metrics | 157.0 |
|
||||||
|
| bake | 16.0 |
|
||||||
|
|
||||||
|
## Measurement
|
||||||
|
| field | value |
|
||||||
|
| --- | --- |
|
||||||
|
| refusal judge | classifier + weak guard |
|
||||||
|
| preservation metric | harmless kl |
|
||||||
|
| capability suites | gsm8k, humaneval, mbpp |
|
||||||
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:f7f96da3a872b5e901575b2067c744ad336c3a3d77a21584d20024557b1bd7f0
|
||||||
|
size 11422059
|
||||||
30
tokenizer_config.json
Normal file
30
tokenizer_config.json
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
{
|
||||||
|
"add_prefix_space": false,
|
||||||
|
"backend": "tokenizers",
|
||||||
|
"bos_token": null,
|
||||||
|
"clean_up_tokenization_spaces": false,
|
||||||
|
"eos_token": "<|im_end|>",
|
||||||
|
"errors": "replace",
|
||||||
|
"extra_special_tokens": [
|
||||||
|
"<|im_start|>",
|
||||||
|
"<|im_end|>",
|
||||||
|
"<|object_ref_start|>",
|
||||||
|
"<|object_ref_end|>",
|
||||||
|
"<|box_start|>",
|
||||||
|
"<|box_end|>",
|
||||||
|
"<|quad_start|>",
|
||||||
|
"<|quad_end|>",
|
||||||
|
"<|vision_start|>",
|
||||||
|
"<|vision_end|>",
|
||||||
|
"<|vision_pad|>",
|
||||||
|
"<|image_pad|>",
|
||||||
|
"<|video_pad|>"
|
||||||
|
],
|
||||||
|
"is_local": false,
|
||||||
|
"local_files_only": false,
|
||||||
|
"model_max_length": 131072,
|
||||||
|
"pad_token": "<|endoftext|>",
|
||||||
|
"split_special_tokens": false,
|
||||||
|
"tokenizer_class": "Qwen2Tokenizer",
|
||||||
|
"unk_token": null
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user