初始化项目，由ModelHub XC社区提供模型

Model: dormantx/Qwen2.5-1.5B-Instruct-NLA-L18-av
Source: Original Platform

README.md

@@ -0,0 +1,73 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen2.5-1.5B-Instruct
+library_name: transformers
+tags:
+  - interpretability
+  - natural-language-autoencoder
+  - activation-verbalizer
+  - qwen2.5
+---
+
+# Qwen2.5-1.5B-Instruct NLA L18 — AV (Activation Verbalizer)
+
+The AV half of a Natural Language Autoencoder trained on the layer-18 residual stream of
+`Qwen/Qwen2.5-1.5B-Instruct`. An activation is L2-normalised, scaled by
+`injection_scale`, and spliced into the embedding slot of the single injection token in a fixed
+prompt; the model then generates a free-text description of what the activation encodes.
+
+This repo mirrors the layout of
+[kitft/Llama-3.3-70B-NLA-L53-av](https://huggingface.co/kitft/Llama-3.3-70B-NLA-L53-av):
+full merged weights at the root (LoRA already folded in — load with plain
+`AutoModelForCausalLM`, no PEFT needed) and an `nla_meta.yaml` sidecar (schema v2) at the root.
+The AR half lives in the sibling repo
+[dormantx/Qwen2.5-1.5B-Instruct-NLA-L18-ar](https://huggingface.co/dormantx/Qwen2.5-1.5B-Instruct-NLA-L18-ar).
+The original LoRA-adapter release (both halves in one repo, `av/` + `ar/` subdirs) is
+[dormantx/Qwen2.5-1.5B-Instruct-NLA-L18](https://huggingface.co/dormantx/Qwen2.5-1.5B-Instruct-NLA-L18);
+weights here are numerically identical to that release
+(merged-vs-adapter greedy generations match exactly).
+
+Notes for loaders expecting the 70B layout:
+
+- Qwen2.5-1.5B ties embeddings (`tie_word_embeddings: true`), so there is no separate
+  `lm_head.weight` key in the shards; `from_pretrained` re-ties automatically.
+- The AV prompt template in `nla_meta.yaml` uses the `{injection_char}` placeholder; the
+  injection token is `<|image_pad|>` (id 151655), a registered special token that tokenizes
+  atomically in context.
+- `mse_scale: 1.0` — AR targets in this pipeline are L2-normalised to unit norm, not
+  `sqrt(d_model)`.
+
+## Usage
+
+```python
+import torch, torch.nn.functional as F, yaml
+from huggingface_hub import hf_hub_download
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+repo = "dormantx/Qwen2.5-1.5B-Instruct-NLA-L18-av"
+meta = yaml.safe_load(open(hf_hub_download(repo, "nla_meta.yaml")))
+tok = AutoTokenizer.from_pretrained(repo)
+av = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16).cuda().eval()
+
+prompt = meta["prompt_templates"]["av"].format(injection_char=meta["tokens"]["injection_char"])
+ids = tok(prompt, add_special_tokens=False)["input_ids"]
+slot = ids.index(meta["tokens"]["injection_token_id"])
+emb = av.get_input_embeddings()(torch.tensor(ids).cuda()[None]).clone()
+
+act = ...  # a raw layer-18 residual-stream activation, i.e. hidden_states[18], shape [1536]
+emb[:, slot] = F.normalize(act, dim=-1).to(emb.dtype) * meta["extraction"]["injection_scale"]
+out = av.generate(inputs_embeds=emb,
+                  attention_mask=torch.ones(emb.shape[:2], device=emb.device),
+                  max_new_tokens=32, do_sample=False)
+print(tok.decode(out[0], skip_special_tokens=True))
+```
+
+Activations must come from `hidden_states[18]` of the base model (output of block 18, before the
+final norm) and be passed raw — the injection step does the normalising and rescaling.
+
+## Provenance
+
+Pipeline and honesty-check methodology build on
+[kitft/natural_language_autoencoders](https://github.com/kitft/natural_language_autoencoders),
+[kameshkanna/nla-train](https://github.com/kameshkanna/nla-train), and
+[SolshineCode/nla-gemma-4-e2b](https://github.com/SolshineCode/nla-gemma-4-e2b).