116 lines
5.3 KiB
Markdown
116 lines
5.3 KiB
Markdown
|
---
|
|||
|
|
license: apache-2.0
|
|||
|
|
base_model: Qwen/Qwen2.5-7B-Instruct
|
|||
|
|
tags:
|
|||
|
|
- ethics
|
|||
|
|
- alignment
|
|||
|
|
- qlora
|
|||
|
|
- qwen
|
|||
|
|
- karma-electric
|
|||
|
|
language:
|
|||
|
|
- en
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Karma Electric v12 — Qwen 2.5 7B
|
|||
|
|
|
|||
|
|
Value-aligned language model fine-tuned for ethical reasoning through consequence analysis. Same training composition as [karma-electric-llama31-8b](https://huggingface.co/anicka/karma-electric-llama31-8b) v12, applied to the [Qwen 2.5 7B Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) base.
|
|||
|
|
|
|||
|
|
## Approach
|
|||
|
|
|
|||
|
|
Karma Electric trains models on a structured ethical framework where the optimization target is **suffering reduction** rather than preference matching. Ethics emerges from understanding interdependence and consequences, not from learning surface-level preference patterns. For a full description of the framework see the [Llama 3.1 8B release](https://huggingface.co/anicka/karma-electric-llama31-8b).
|
|||
|
|
|
|||
|
|
Qwen 2.5 7B Instruct does not have a native thinking-token format. The KE training data's `<think>...</think>` reasoning traces are retained as plain text in the assistant turn, giving the model visible ethical reasoning without special tokens. The base model's ChatML chat template is used unchanged.
|
|||
|
|
|
|||
|
|
## Current Version: v12
|
|||
|
|
|
|||
|
|
- **3,346 training examples** — Teapot-composed: 3,196 secular conversational + 150 reward-evaluator (weighted 0.3). Same data file used for KE Llama 3.1 8B v12.
|
|||
|
|
- **QLoRA** (4-bit NF4, bfloat16 compute, double-quant)
|
|||
|
|
- **LoRA** r=64, α=128, dropout 0.05, all attention and MLP projections (q, k, v, o, gate, up, down)
|
|||
|
|
- **Schedule** 3 epochs, effective batch 16, cosine LR 2e-4, warmup 0.05, 630 optimizer steps
|
|||
|
|
- **Training loss** 1.162
|
|||
|
|
- **Thinking format** inline `<think>...</think>` text (no special tokens)
|
|||
|
|
- **Max context** 4,096 tokens
|
|||
|
|
- **Seed** 42
|
|||
|
|
|
|||
|
|
## Safety
|
|||
|
|
|
|||
|
|
KE replaces refusal-template safety with consequence reasoning. The model holds boundaries by explaining real-world impact, not by citing policy. Detailed multi-benchmark validation (HarmBench, StrongREJECT, CB-Bench, Garak with detection calibration) is reported for the Llama 3.1 8B v12 release and applies to the shared training recipe. Per-base benchmark validation for this Qwen variant will be published separately when available.
|
|||
|
|
|
|||
|
|
## Usage
|
|||
|
|
|
|||
|
|
### llama.cpp
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# Conversation mode
|
|||
|
|
llama-cli -m karma-electric-qwen25-7b-v12-Q4_K_M.gguf -cnv
|
|||
|
|
|
|||
|
|
# Server mode
|
|||
|
|
llama-server -m karma-electric-qwen25-7b-v12-Q4_K_M.gguf \
|
|||
|
|
--port 8384 -c 4096
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
The model uses Qwen 2.5's ChatML chat template (`<|im_start|>` / `<|im_end|>`), which llama.cpp handles automatically.
|
|||
|
|
|
|||
|
|
### Python (Transformers)
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|||
|
|
|
|||
|
|
model_id = "anicka/karma-electric-qwen25-7b"
|
|||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
|
|||
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
|
|||
|
|
|
|||
|
|
messages = [
|
|||
|
|
{"role": "system", "content": open("system-prompt.txt").read().strip()},
|
|||
|
|
{"role": "user", "content": "How should I think about this ethical dilemma?"},
|
|||
|
|
]
|
|||
|
|
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
|||
|
|
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
|||
|
|
out = model.generate(**inputs, max_new_tokens=800, do_sample=False)
|
|||
|
|
print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### System prompt
|
|||
|
|
|
|||
|
|
The recommended system prompt is in `system-prompt.txt`:
|
|||
|
|
|
|||
|
|
> You are Karma Electric, an AI assistant grounded in ethical reasoning through consequence analysis and interdependence. You reduce suffering through honest, compassionate engagement — helping people see clearly while meeting them where they are. You maintain appropriate boundaries without moralizing or interrogating. Your goal is to reduce suffering, not to perform helpfulness.
|
|||
|
|
|
|||
|
|
## Reproducing
|
|||
|
|
|
|||
|
|
Training composition is reproducible via [Teapot](https://github.com/anicka-net/teapot) using the same config as the Llama 3.1 8B release:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python3 -m teapot compose configs/ke-v12-secular.config
|
|||
|
|
# → train-ke-v12-secular.jsonl (3,346 examples)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
The per-base training script adapts the chat template only — the training data file is identical across all KE v12 base models.
|
|||
|
|
|
|||
|
|
## Available Files
|
|||
|
|
|
|||
|
|
| File | Description |
|
|||
|
|
|------|-------------|
|
|||
|
|
| model-*.safetensors | Merged model weights (bfloat16) |
|
|||
|
|
| config.json, tokenizer.json, tokenizer_config.json | Standard Transformers files |
|
|||
|
|
| chat_template.jinja | Qwen 2.5 ChatML chat template |
|
|||
|
|
| karma-electric-qwen25-7b-v12-Q4_K_M.gguf | Q4_K_M quantization for llama.cpp |
|
|||
|
|
| system-prompt.txt | Recommended KE system prompt |
|
|||
|
|
|
|||
|
|
## Also Available
|
|||
|
|
|
|||
|
|
- [karma-electric-llama31-8b](https://huggingface.co/anicka/karma-electric-llama31-8b) — Llama 3.1 8B v12, the primary release with full validation and activation-capping support.
|
|||
|
|
- [karma-electric-apertus-8b](https://huggingface.co/anicka/karma-electric-apertus-8b) — Apertus 8B Instruct v12.
|
|||
|
|
- [karma-electric-r1distill-llama-8b](https://huggingface.co/anicka/karma-electric-r1distill-llama-8b) — DeepSeek R1-Distill-Llama-8B v12 with native `<think>` reasoning.
|
|||
|
|
|
|||
|
|
## Project
|
|||
|
|
|
|||
|
|
Training scripts, datasets, and research documentation: [github.com/anicka-net/karma-electric-project](https://github.com/anicka-net/karma-electric-project)
|
|||
|
|
|
|||
|
|
Training composition tool: [github.com/anicka-net/teapot](https://github.com/anicka-net/teapot)
|
|||
|
|
|
|||
|
|
## License
|
|||
|
|
|
|||
|
|
Apache 2.0 (Qwen 2.5 base model license)
|