初始化项目，由ModelHub XC社区提供模型

Model: ceselder/nanonla-l24-av-qwen3-8b
Source: Original Platform

README.md

@@ -0,0 +1,114 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen3-8B
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+  - interpretability
+  - natural-language-autoencoder
+  - nla
+  - activation-verbalizer
+  - qwen3
+datasets:
+  - ceselder/qwen3-8b-nla-L24-finefineweb-100k
+language:
+  - en
+---
+
+# nanoNLA — Qwen3-8B Activation Verbalizer (Layer 24, Karvonen injection)
+
+The **activation-verbalizer (AV)** half of a
+[Natural Language Autoencoder](https://transformer-circuits.pub/2026/nla/index.html)
+for **Qwen3-8B**, layer **24**. Hand it a residual-stream activation vector and it
+describes, in natural language, what that activation represents.
+
+This is the **AV-SFT** checkpoint — the supervised warm-start stage (before RL) —
+trained with the **Karvonen norm-matched injection** formula.
+
+- 🧠 **Code / training pipeline:** [github.com/ceselder/nanoNLA](https://github.com/ceselder/nanoNLA)
+- 📚 **Docs:** [design](https://github.com/ceselder/nanoNLA/blob/main/docs/design.md) ·
+  [inference](https://github.com/ceselder/nanoNLA/blob/main/docs/inference.md) ·
+  [setup](https://github.com/ceselder/nanoNLA/blob/main/docs/setup.md) ·
+  [Qwen3-8B run notes](https://github.com/ceselder/nanoNLA/blob/main/docs/qwen3_8b_run.md)
+- 🗂️ **Warm-start dataset:** [ceselder/qwen3-8b-nla-L24-finefineweb-100k](https://huggingface.co/datasets/ceselder/qwen3-8b-nla-L24-finefineweb-100k)
+- 📄 **Paper:** *Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations*
+  (Fraser-Taliente et al., Transformer Circuits 2026) —
+  [paper](https://transformer-circuits.pub/2026/nla/index.html) ·
+  [blog](https://www.anthropic.com/research/natural-language-autoencoders)
+
+---
+
+## What this is
+
+A Natural Language Autoencoder is a pair of fine-tuned LMs that map residual-stream
+activations to language and back:
+
+| direction | model | mechanism |
+|---|---|---|
+| activation → text | **AV (this model)** | inject the vector at a marker token, autoregress an explanation |
+| text → activation | AR | truncated K+1-layer copy of the base + `Linear(d, d)` readout (reconstruction) |
+
+This repo hosts the **AV** for Qwen3-8B @ layer 24.
+
+## ⚠️ This checkpoint uses the Karvonen injection formula
+
+It was trained with the **Karvonen norm-matched additive injection** at the marker
+token (a layer-1 residual-stream hook):
+
+```
+h'_p = h_p + ‖h_p‖ · v / ‖v‖
+```
+
+where `v` is the layer-24 activation you want verbalized. **Serve it with the same
+injection** — *not* the paper's embedding-replacement default. In the
+[nanoNLA](https://github.com/ceselder/nanoNLA) code this is the Karvonen path; see
+[`docs/qwen3_8b_run.md`](https://github.com/ceselder/nanoNLA/blob/main/docs/qwen3_8b_run.md).
+The marker token, injection/MSE scales, and `d_model` are all read from the run's
+`nla_meta.yaml` sidecar — never hardcode them.
+
+## Usage
+
+See [`docs/inference.md`](https://github.com/ceselder/nanoNLA/blob/main/docs/inference.md)
+for the full harness (marker detection, neighbor-anchoring, the Karvonen hook). Sketch:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+model = AutoModelForCausalLM.from_pretrained("ceselder/nanoNLA", torch_dtype="bfloat16")
+tok   = AutoTokenizer.from_pretrained("ceselder/nanoNLA")
+# 1. register the Karvonen layer-1 injection hook with your layer-24 activation `v`
+# 2. put the marker token in the prompt (neighbor-anchored)
+# 3. generate the explanation
+# (full, correct hook + prompt template live in the nanoNLA repo)
+```
+
+## Training
+
+- **Base:** [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) — fine-tuned; merged bf16 weights provided here
+- **Layer:** 24 (residual stream)
+- **Stage:** AV-SFT (warm-start), 1000 steps
+- **Injection:** Karvonen norm-matched additive (layer-1 hook)
+- **Warm-start labels:** explanations over
+  [qwen3-8b-nla-L24-finefineweb-100k](https://huggingface.co/datasets/ceselder/qwen3-8b-nla-L24-finefineweb-100k)
+- **Peak LR:** 3e-5 (cosine)
+
+Exact recipe (rank/scales/hparams) and eval numbers are in the
+[nanoNLA repo](https://github.com/ceselder/nanoNLA) and
+[`docs/qwen3_8b_run.md`](https://github.com/ceselder/nanoNLA/blob/main/docs/qwen3_8b_run.md).
+
+## Caveats
+
+Research checkpoint, not a product. This is the **SFT** stage (pre-RL), so the AV
+produces plausible-but-imperfect explanations. Whether an NLA's explanations are
+*faithful* to the underlying computation is an open research question — see the paper.
+
+## Citation
+
+```bibtex
+@article{frasertaliente2026nla,
+  title   = {Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations},
+  author  = {Fraser-Taliente, others},
+  journal = {Transformer Circuits Thread},
+  year    = {2026},
+  url     = {https://transformer-circuits.pub/2026/nla/index.html}
+}
+```