deepseek-governed-no-amnesia/README.md

---
license: mit
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
tags:
  - deepseek
  - experimental
  - research
library_name: transformers
pipeline_tag: text-generation
---

CODE: https://github.com/frank-morales2020/AST/blob/main/DEEPSEEK_PRIME_ANCHORE_LLM.ipynb


# deepseek-governed-no-amnesia

An experimental repository pairing **DeepSeek-R1-Distill-Qwen-7B** with a
"Prime-Anchored Spectral Governor" artifact. This card describes exactly what
the repository contains and what the accompanying run did and did not do.

## What this repository is

- The model weights are **identical to the base model**,
  [`deepseek-ai/DeepSeek-R1-Distill-Qwen-7B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B).
  The governor did not modify them.
- Attached is `governor_state.pt`, which records the governor configuration
  (anchor primes, threshold, cached anchor rows, gate statistics, and a
  SHA-256 signature of the prime-indexed embedding rows).
- `VERIFICATION.txt` records the run details.

## What the governor does (and doesn't)

The governor evaluates a gate on each training step and pins six
prime-indexed embedding rows (`[2, 3, 5, 7, 11, 13]`) to their original
values. In the run that produced this repository, the optimizer step was a
**no-op** (`STEP_GATED_NO_MUTATION`), so the model was not trained or
fine-tuned.

As a result:

- The prime-indexed rows are unchanged (anchor signature matches).
- The full model is byte-for-byte the base model.
- **This run does not demonstrate** training, fine-tuning, or that any form
  of catastrophic forgetting was prevented, because the model was not
  modified. The repository name reflects the project's intent, not a
  measured property of this checkpoint.

## Intended use

Research and experimentation with the governor tooling. For general use,
loading the base model directly is equivalent and avoids downloading a
duplicate copy of the weights.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

REPO_ID = "frankmorales2020/deepseek-governed-no-amnesia"

tokenizer = AutoTokenizer.from_pretrained(REPO_ID, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    REPO_ID, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True
)
model.eval()

prompt = "Explain why prime numbers are important in cryptography."
text = tokenizer.apply_chat_template(
    [{"role": "user", "content": prompt}],
    tokenize=False, add_generation_prompt=True,
)
enc = tokenizer(text, return_tensors="pt").to(model.device)

out = model.generate(
    **enc,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    repetition_penalty=1.3,
    no_repeat_ngram_size=3,
    pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))
```

> This is a DeepSeek R1 distill (reasoning) model. Feed prompts through the
> chat template, and expect a `<think> ... </think>` reasoning trace before
> the final answer. To keep only the final answer, split the output on
> `</think>`.

## Inspecting the governor artifact

```python
from huggingface_hub import hf_hub_download
import torch

state = torch.load(
    hf_hub_download(REPO_ID, "governor_state.pt"),
    map_location="cpu", weights_only=False,
)
print(state["primes"], state["LAMBDA_12"], state["anchor_signature"], state["stats"])
```

## License

Released under the MIT license, inherited from the base model.

## Acknowledgements

Base model: DeepSeek-R1-Distill-Qwen-7B by DeepSeek-AI.