初始化项目,由ModelHub XC社区提供模型
Model: frankmorales2020/deepseek-governed-no-amnesia Source: Original Platform
This commit is contained in:
112
README.md
Normal file
112
README.md
Normal file
@@ -0,0 +1,112 @@
|
||||
---
|
||||
license: mit
|
||||
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
|
||||
tags:
|
||||
- deepseek
|
||||
- experimental
|
||||
- research
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
CODE: https://github.com/frank-morales2020/AST/blob/main/DEEPSEEK_PRIME_ANCHORE_LLM.ipynb
|
||||
|
||||
|
||||
# deepseek-governed-no-amnesia
|
||||
|
||||
An experimental repository pairing **DeepSeek-R1-Distill-Qwen-7B** with a
|
||||
"Prime-Anchored Spectral Governor" artifact. This card describes exactly what
|
||||
the repository contains and what the accompanying run did and did not do.
|
||||
|
||||
## What this repository is
|
||||
|
||||
- The model weights are **identical to the base model**,
|
||||
[`deepseek-ai/DeepSeek-R1-Distill-Qwen-7B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B).
|
||||
The governor did not modify them.
|
||||
- Attached is `governor_state.pt`, which records the governor configuration
|
||||
(anchor primes, threshold, cached anchor rows, gate statistics, and a
|
||||
SHA-256 signature of the prime-indexed embedding rows).
|
||||
- `VERIFICATION.txt` records the run details.
|
||||
|
||||
## What the governor does (and doesn't)
|
||||
|
||||
The governor evaluates a gate on each training step and pins six
|
||||
prime-indexed embedding rows (`[2, 3, 5, 7, 11, 13]`) to their original
|
||||
values. In the run that produced this repository, the optimizer step was a
|
||||
**no-op** (`STEP_GATED_NO_MUTATION`), so the model was not trained or
|
||||
fine-tuned.
|
||||
|
||||
As a result:
|
||||
|
||||
- The prime-indexed rows are unchanged (anchor signature matches).
|
||||
- The full model is byte-for-byte the base model.
|
||||
- **This run does not demonstrate** training, fine-tuning, or that any form
|
||||
of catastrophic forgetting was prevented, because the model was not
|
||||
modified. The repository name reflects the project's intent, not a
|
||||
measured property of this checkpoint.
|
||||
|
||||
## Intended use
|
||||
|
||||
Research and experimentation with the governor tooling. For general use,
|
||||
loading the base model directly is equivalent and avoids downloading a
|
||||
duplicate copy of the weights.
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
import torch
|
||||
|
||||
REPO_ID = "frankmorales2020/deepseek-governed-no-amnesia"
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(REPO_ID, trust_remote_code=True)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
REPO_ID, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True
|
||||
)
|
||||
model.eval()
|
||||
|
||||
prompt = "Explain why prime numbers are important in cryptography."
|
||||
text = tokenizer.apply_chat_template(
|
||||
[{"role": "user", "content": prompt}],
|
||||
tokenize=False, add_generation_prompt=True,
|
||||
)
|
||||
enc = tokenizer(text, return_tensors="pt").to(model.device)
|
||||
|
||||
out = model.generate(
|
||||
**enc,
|
||||
max_new_tokens=512,
|
||||
do_sample=True,
|
||||
temperature=0.7,
|
||||
top_p=0.9,
|
||||
repetition_penalty=1.3,
|
||||
no_repeat_ngram_size=3,
|
||||
pad_token_id=tokenizer.eos_token_id,
|
||||
)
|
||||
print(tokenizer.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
> This is a DeepSeek R1 distill (reasoning) model. Feed prompts through the
|
||||
> chat template, and expect a `<think> ... </think>` reasoning trace before
|
||||
> the final answer. To keep only the final answer, split the output on
|
||||
> `</think>`.
|
||||
|
||||
## Inspecting the governor artifact
|
||||
|
||||
```python
|
||||
from huggingface_hub import hf_hub_download
|
||||
import torch
|
||||
|
||||
state = torch.load(
|
||||
hf_hub_download(REPO_ID, "governor_state.pt"),
|
||||
map_location="cpu", weights_only=False,
|
||||
)
|
||||
print(state["primes"], state["LAMBDA_12"], state["anchor_signature"], state["stats"])
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
Released under the MIT license, inherited from the base model.
|
||||
|
||||
## Acknowledgements
|
||||
|
||||
Base model: DeepSeek-R1-Distill-Qwen-7B by DeepSeek-AI.
|
||||
Reference in New Issue
Block a user